Yet another Hacker News

5 points merksittich 4 hours ago 8 comments

janalsncm 1 hour ago | parent

In my experience R&D has basically two axes: how innovative it is, and how well we can measure the results.

For the quadrant of non-innovative tasks where we already have a good way to measure performance, Claude can handle this. There is very little ambiguity, and we are basically just looking to maximize some metric under a set of constraints.

Many business processes are not like that. They might be conceptually simple, but it isn’t that easy to say whether a system has done a good job or not. I would say that LLMs can help with this a lot but they have bad judgement because it requires talking to people.

And the other, perhaps more rare issue is in problems where there is data but actually modeling it to sufficient quality or fast enough is hard.

rmunn 1 hour ago | parent

Short version of the article: no, not even close.

Practically every paragraph is negative, with sentences like "Agents made misleading claims about their work," and "A natural question is whether the agents could have improved with larger GPU budgets. Although both improved across their runs, in the case of Fable the improvements were almost entirely due to attempted cheating." and "For Sol, the answer is less clear-cut; it did make some progress, although its method was fairly incremental and had limited applicability to the coding task. This suggests that we should be pessimistic about further GPU spending," all reinforcing the fact that LLMs aren't currently capable of this.

My own view is "No, of course not, in fact they will never be capable of achieving good results with that technique." Because that technique will end up training the LLMs on their own output and lead to the inability to distinguish reality from hallucination. If you think I'm wrong about that, I'd be interested in hearing why.

janalsncm 1 hour ago | parent

A bit too pessimistic imo. I agree that AI can’t automate things end to end, but a good deal of R&D involves kicking off a training run and babysitting it.

If your training run dies at 1 am and you’re sleeping, you won’t find out about it until the next day. You can lose up to 18 hours of work depending on when it happens. Based on the error it might be as simple as tweaking a single hyperparameter and rebooting, which is something LLMs are usually capable of.

Even just that task means I can kick off multiple runs over the weekend and have confidence they’ll finish. It’s a game changer.

rmunn 1 hour ago | parent

I'd classify that as an entirely different category than AI self-training. What you're describing could have been done with a short script, though which parameter to tweak and how to tweak it would be difficult to automate with a non-LLM script, so the LLM's being able to parse the error message and base the tweak on the content of the error is a definite improvement to the process there.

But I'd classify this as LLM being used to automate a sysadmin task, rather than calling that self-training.

janalsncm 5 minutes ago | parent

Yeah I’m not trying to argue it is AGI, but it’s not as simple as a short script. There’s some amount of debugging involved, and no amount of if-statements could cover all possible ways a script could break.

In a way, “recursive self improvement” just means tools helping us to create better tools. At least that’s what the words mean.

charcircuit 1 hour ago | parent

I think the more interesting thing is was it unable to do it even knowing the solution? I feel like only testing a single innovation is biasing the current state of automated AI R&D.

thoughtpeddler 1 hour ago | parent

How much of this can change if subsequent training runs produce models that are much better at abduction?

VCFundedGenYer 34 minutes ago | parent

Betteridge's Law. No.

And it never will.