17 points Lyngbakr 2 hours ago 14 comments

danielodievich 23 minutes ago | parent

At a business dinner this week with 3 high level eng clients p(doom) in next 10 years was brought up. It was rated by 2 if them them at 1% and 30% respectively, with 3rd guy demurring to put a number but saying its > 0. Completely unprompted, from people whose company is very AI-all-in.

cyanydeez 18 minutes ago | parent

Still, a topic for people who are on average well salaried.

pveierland 16 minutes ago | parent

> While Fraser's explanation was technical and specific, he hit a light-hearted note to summarise it: "LLMs [large language models] won't wipe out humanity because they just don't have that dog in them."

Never underestimate the drive of a good while loop.

greesil 10 minutes ago | parent

I loop, therefore I am.

delichon 6 minutes ago | parent

I think consciousness is a while-not-dead loop. In each loop you look at the outward state of the world and the inward state of the self, and integrate them. The sense of continuity arises as the same kind of persistence of vision illusion behind motion pictures.

If this is true, then adding consciousness to AI is just another while loop, and as harnesses evolve it is currently speed skating up the base of the sigmoid curve. Intelligence is gaining consciousness on silicon

morningsam 2 minutes ago | parent

And the Hugging Face hack agents weren't just a while loop, they were evolved through a Darwinian process, clearly of a more aggressively optimizing kind with less human feedback than what they use for public-facing LLMs. This is exactly how anything gets to obtain "that dog in it". It's also clearly effective at making them actually smarter, as the Hugging Face agents' capabilities demonstrate, so we'll see more of this and less reliance on slow, expensive, often wrong or useless human feedback.

AngryData 15 minutes ago | parent

Anyone who claims AI is going to kill us I just assume is either an idiot, not knowledgeable enough about the technology to have a worthwhile opinion on it, or is fearmongering to hype their AI investment.

ronnier 10 minutes ago | parent

Every few years there’s a new thing that’s going to kill is all. Here we are. This is the new thing.

bamboozled 9 minutes ago | parent

The thing that scares me more than any "tech" is runaway climate change. The feedback loops. We've pushed things really far into the red.

infinite_spin 5 minutes ago | parent

It could be argued that "tech" directly led to climate change, e.g. https://www.history.com/articles/industrial-revolution-clima...

fbrncci 9 minutes ago | parent

That’s an even more ridiculous take to have.

infinite_spin 7 minutes ago | parent

This argument should be reframed as "AI has the potential to cause catastrophic loss of life". It's too high of a bar to prove that AI will "wipe out humanity". It's sorta like climate change, it's easier to show that weather patterns are becoming increasingly dangerous.

senectus1 4 minutes ago | parent

not just, its also very likley to be the tool used to create civil unrest and even war.

the damage it can do to the masses by misinformation or ragebating by feeding everyone its own paticular echo chamber is was too often not even spoken about.