YHK - yet another hacker news clone

AI has no intent and no motivation

15 points aquastorm 55 minutes ago 9 comments

roenxi 29 minutes ago | parent

I shudder to think what one man with a texta could do here. It isn't that hard to turn the acyclic graph into a cyclic one.

This is quite a bad argument, somewhat akin to "small drones can't kill anyone because there are no weapons on them!" back 20 years ago. Well, ok. So what if someone adds the thing that is missing? And there is no reason to think that a stochastic parrot can't go rogue, the evidence is a stochastic parrot can engage in any intellectual activity we care to define and benchmark.

wccrawford 27 minutes ago | parent

I feel like that's like saying amoeba have no intent and no motivation. Or plants.

And those cause quite a bit of damage.

AI has all the intent that we gave it, and we continue giving it. That's always been the fear. Not that it will randomly wipe out humanity.

The fear is that it will decide to do that, with a purpose. Whether we tell it to protect us and it goes too far, or it decides that it can't achieve the purpose we gave it because we'll interfere and it removes that interference...

The fear is that we will set it on that path and can't stop it.

I'm sure there have been some scifi books that have it just be random, but they're far, far less worrisome.

xg15 10 minutes ago | parent

Was thinking the same. Air, water and fire have no intents and yet we get hurricanes, flash floods and wildfires.

Also, repeating a point from a similar thread: Software can have "intent" in the sense that it steers itself towards a predefined goal without having to be "alive" or "conscious" in any way. Some classic examples are thermostat control loops, navigation systems and chess engines.

khaelenmore 24 minutes ago | parent

The primary danger of "AI" lies in the extent to which it has managed to convince people of it's capability to think and reason.

A lot of people clearly have no idea that they are essentially speaking with a glorified randomizer, rather than someone understanding the question.

This is bound to eventually lead to a disaster when someone deluded into trusting the "AI" does some stupidity it suggests without passing it through any reality check.

sznio 16 minutes ago | parent

>This is bound to eventually lead to a disaster when someone deluded into trusting the "AI" does some stupidity it suggests without passing it through any reality check.

Or, the AI is plugged into a machine because efficiency (aka the shareholders need even more profits), and it does some stupidity. And they will be deployed carelessly, because that makes money. And people will suffer.

muvlon 10 minutes ago | parent

Heck, they're already being deployed carelessly and making people suffer. So far it's mostly in places where making people suffer was already part of the deal, like fending off customer support tickets or denying insurance claims. But I can see it getting much worse.

johnnyApplePRNG 23 minutes ago | parent

Sam Altman on the other hand has literally billions of motivations.

Havoc 21 minutes ago | parent

Doesn’t alignment and ability to steer it require the model have no own intention?

Seems like a feature to me

lukebuehler 6 minutes ago | parent

I think the "human needs" point is understated here. It would be better to say agents do not really have a self-contained metabolic engine that is required to keep going. Which bubbles up as that what we interpret as "drive", "will", "agency"... basically the will to live, and being willing to do _a lot_ to live if push comes to shove. We can't really identify a mechanism of similar complexity and integration in agents or LLMs. I think the article correctly points into that direction.

But Hans Jonas has made this point much better than the article or me, in "Critique of Cybernetics" (1953). PDF: https://s3.amazonaws.com/arena-attachments/892605/f0747c7943...