YHK - yet another hacker news clone

I have t see t hat I a few years

I think its solved, with the right setup and model. I couldn't tell u the last time my agent hallucinated. For me I consider it solved. However some dude feeding massive docs into gpt chat and long conversations, It is not solved in this context.

I'm hoping for more than anecdotes. I believe AI is not ready for what it's being used for, but all I get are downvotes and no citations. Maybe all of you are just scared to say it out loud?

My personal opinion: all output from LLMs is hallucinations. The idea that it’s “right” or “wrong”, when stating factual information is really in the eye of the beholder/user. What year did astronauts land on the moon? — is a question that may or may not have a factual answer depending on your own beliefs. Philosophically, you would need to also define what a fact is, or what “not hallucinating” is, to define what a hallucination is. My understanding is that this property of the LLM architecture is innate, and until we have “world model” LLMs, or some other model that reasons from first principles, instead of the current “guess the next word” model, this isn’t going away. Just don’t rely on the for facts.

AI will never be fully reliable, and people never will be. But AI is wrong in much less predictable and stable ways, and with much less accountability.

There are two common ways to spot hallucinations. Talk to it about a topic you know well, or implement what it suggests and get an error or failure.

For the first, we're mostly past the point of just "testing". So I don't see that too much anymore. Mostly I still see that though in bug reports generated by AI by someone else. There is usually some underlying bug being reported, but the AI explanation and "helpful suggestion" is typically inaccurate. Generally, suggested fixes are terrible. (They likely work, but fix a symptom not the cause.)

The second still happens, but with much less regularity for me though. It does make mistakes though.

In areas where I'm not as skilled it's very hard to spot errors. When researching general information I'm mostly accepting it on face value.

I find the bug-report thing really interesting. For lots of simple bugs it's great. For more complex things it seems to be very superficial- if a 0 causes an issue here, add a simple guard for 0. There's no depth of understanding why the value is 0 in the first place, when it should be set. If it can (incorrectly) be 0 here, where else might 0 be impacting the code?

This informs my opinion of vibe coded stuff - where there is no skilled human inspection. I expect that code to be of a poor underlying quality. Especially if it's AI changes to an existing human-coded app.

The problem hasn't been solved, and, by the very nature of what LLMs are, can't be. However, it has gotten a lot better, as models have got much larger and pretraining more sophisticated.

Part of the difficulty is in defining what a hallucination is. On the face of it, it seems straightforward: an obviously counterfactual claim or manifest error of reasoning. However, it's not always that simple. A lot of what people consider to be hallucinations are misattributions, specious diagnoses, strangely lopsided preoccupations, eccentric design choices, needlessly verbose or circuitous output or explanations, a kind of metaphysical conflation of the trivial with the significant, etc.

[delayed]