YHK - yet another hacker news clone

After we started excessively documenting and extracting skills everyone has stopped complaining about running out of tokens, because agents stopped having to reconstruct the full context each time from scratch. Harnesses like claude code also push the model to aggressively keep this documentation in sync so there's little concern about drift.

The worst thing to do from a token usage pov is to give a model a vague open ended prompt because they are so scared to be wrong that they'll waste a ton of tokens "thinking" through the issue and verifying everything. Whereas they almost trust skills blindly and skip all this unnecessary work.

> I believe we'll see a shift to more organizations building their own harnesses with model routing logic to get central control over who can use what AI and for what.

This is what happened with cloud computing. People with expertise wrote software around the software creating machine because just giving naked compute to people en-masse didn’t actually achieve anything valuable. Likewise all of the talk of enterprise agents etc, they’re all custom software that wraps around a reasoning model.

This summer my team basically shut down when we hit a usage cap because we had recently pushed a decent portion of our devops workflow into agent skills. It had been done in such a way that it was difficult for humans to navigate. Relevant scripts turned out to be buggy and poorly documented, and nobody realized because these harnesses that are tuned to be absurdly tenacious about searching for workarounds had been quietly burning heaps of tokens on muddling through instead of raising any alerts about the horribly broken state of the system.

I do agree that home (or at least independently) grown harnesses might be the next logical step. Harness vendors who charge by the token have an inescapable conflict of interest here. Moderate, well-governed LLM usage isn't good for their revenue.

(Couldn't even get this to happen in a 30 person team...)

I use these models all day long, experiment, and have no clue how to choose that.

It just showed up one day in the interface with no explanation or guidance. Its use is mysterious, its effects unclear except through intensive experimentation, and to this day it mostly seems "how many bad decisions will Claude go forward with when it finally dumps out screenfuls of text instead of getting better guidance early on" though it's certainly not a guarantee on anything.

These models are being released at breakneck speed even before their creators know how to use them. It's a big project of collective discovery to figure out what they are doing and how to use them.

Shouldn't software engineers have informed opinions on databases, programming languages, frameworks, etc.?

Right, the problem is frontier models can do ~all of that pretty well, a lot better than they could 9 months ago. Luna 6 can do most of it practically free and instantaneously. So, your quote is likely what's increasingly being said in c-suite meetings as they decide to start mass layoffs.

The level of appropriate LLM usage in my opinion? Ask Gemini some questions when you need to search the web then read the sources it gives you.

LLM written code still has the problem IBM identified. A computer cannot be heald responsible, and so it cannot be allowed to make decisions. That applies to executive management AND software engineering.

Of course that requires working in an industry, like aerospace, where engineers are (usually, Boeing not counting) heald accountable.

All while suffering from the increased the mental load/lack of focus from even more workload. "You've got AI, you should be able to get it done today, right?"

Everyone has a great researcher piped to their desk now. You can have it follow the Twitter zeitgeist for you, you don't have to do it yourself.

edit: grammar

We're sorta in the age of alchemy. Lots of cranks out there, but there are real recipes.

- full autonomous camp

- developer augmentation camp

- no-ai camp

The trouble I see with all of it is that the future seems unpredictable at the moment. The costs related to AI are low enough at the moment that full autonomous seems to be possible, but we have reasons to believe that costs will rise significantly, which may change that calculus. The no-AI camp is in ostrich mode, and is betting on this all going away once the bubble pops. The developer augmentation camp treats it like just another tool, which is somewhere in the middle.

The trouble is that even if there is a clear advantage today, the ground truth of costs built in is probably not stable.

If you don't have at least some workloads doing that you're missing out on the biggest wins from the current phase of the technology.

If i had to guess upwards of 80 percent in corporate is "useless stuff".

The whole point of AI is for companies to invest less resources into employees. There's no way they start spending money on training people now, when that was already something that made executives roll their eyes before AI.

Alternate take: it takes a lot of up front effort to ensure when you "max out tokens" the work is valuable.

The same thing goes for token use operating business processes (vs. building software that runs business processes): Let's imagine buying $250k tokens per year to operating some business processes - how much human effort is needed to ensure that level of spend is valuable? I'm using a number that could be "we could hire a skilled human at that level" as that changes the feel of the question.