YHK - yet another hacker news clone

On one hand, yeah, I agree.

On the other hand, using AI to trawl through all the grunt work, experiments, and well trodden areas means I can crawl up pretty close to that giants shoulder and spend way less effort myself answering the actually important questions so I can stand on it.

(This isn't supposed to sound self aggrandising - the giants whose shoulders I'm standing on aren't very tall, and they probably have bigger brothers and sisters, but the idiom was too good of a match to pass up using)

Using Fabel to improve some non-AI ML models a while back, I had a similar result - it could implement perfectly any suggestion I made, but it could not work autonomously due to making very poor suggestions itself, or getting distracted by the wrong things, making dumb assumptions, failing to see the big picture, etc.

These seem pretty easy to solve problems though if I could just afford 100 or 1000 agents. Let them explore and compete and work out their errors for themselves.

At a practical level though, I do a lot more 'bonus' R&D work now by just keeping the agents running and testing ideas in the background.

LLMs are great at doing work quickly and they don’t say something is impossible or discourage you, so you can really tackle problems that previously would have taken years of work.

They aren’t capable of generating truly innovative ideas though. Ask it to come up with a name, it will come up with the same weird slop names over and over. Always gives people the surname Okafor. Because it works by generating probable answers, it can only generate probable names.

I've also found it very hard to leave AI agents up to their own devices for long periods and get much productive benefit from it.

I suppose it's possible that frontier labs have internal models that are so far beyond that autonomous research becomes possible, but I'm not sure that's it.

It seems to me like these models are like a very competent, important part of some hypothetical digital brain, but alone they don't have all the facilities they need to be useful, like a genius with severe ADHD and short-term memory loss.

For anything where quality isn't objective in a clear-cut way, very intentional context management is needed to get good outputs. As context size expands, performance degrades. As the number of tasks increases, performance degrees. As the number of independent decisions based on subjective criteria increases, performance degrees.

If you can get a harness that makes sure every decision is made one at a time, focused with only the context it needs, things work great. But making those harnesses is still hard, a lot of work, and something that agents seem to have no clue how to help you with.

Innovation comes out of the human experimenting with solutions and learning new things that the llm knows.

It’s a dance, not an orchestration.

Llms are far from being ingenious or innovative. The main way they can surprise you today is pulling in information that’s new TO YOU - I impressed with the algorithms they pop out sometimes simply because never formally studied compsci. But when it treads on my home turf I find it very basic, and for bigger picture stuff like system architecture its terrible, by its nature it doesn’t have a “vision” of how the whole system fits together

I'm old enough to have realized that I've done plenty of R&D in my life that wasn't formally named that. R&D is about the substance, the mode, the impact, not the names or titles.

Part of the problem here is how people use these models and expectations are formed.

If you want autonomous work of such systems you can not expect that: a) each reasoning step will make sense and be correct; b) results will come very quickly - especially in the first stages of the project.

I have very positive experience in non-trivial problems with using AI as an autonomous researcher or engineer. But it worked since an agent had a separated, fully dedicated environment and enough time to build up (emerge) some strong patterns of work. Early days were underperforming and slow progress. Now it is overperforming and frequently surprising me with new stuff.

anything you prompt is such that instruction fine-tuning means you're actually the idiot