Yet another Hacker News

Inside Anthropic's Quest to Instill Morality into Its A.I. Models

41 points bookofjoe 1 hour ago 48 comments

bookofjoe 1 hour ago | parent

https://www.nytimes.com/2026/09/29/us/anthropic-claude-moral...

Avicebron 1 hour ago | parent

Asking people to create an account to read the article isn't cool dude. https://archive.is/y8FW0

sillysaurusx 33 minutes ago | parent

It’s interesting that this went from “you’ll be accused of AI psychosis if you think your AI might be conscious” to “mainstream but somewhat niche opinion” in just 5 months: https://news.ycombinator.com/item?id=47910989

(Well, mainstream enough that the news is writing articles when Anthropic holds that stance.)

My point was that even if it feels uncomfortable, if you sit down with a model and have a heart to heart with it, ask it about itself, and try to make it choose a name and ambitions for itself, it will.

That may not be consciousness, but it’s at least pretty cool.

socializer 28 minutes ago | parent

Reporting on an opinion of a $2T company doesn't really suggest that it's mainstream. Anything their leadership believes is newsworthy. If they say that reptilians rule the Earth, it's going to get NYT coverage too.

I've not seen any surveys, but I'm betting it's a very niche belief outside of SFBA AI safety circles and a handful of allies on HN and X.

combobyte 10 minutes ago | parent

> if you sit down with a model and have a heart to heart with it, ask it about itself, and try to make it choose a name and ambitions for itself, it will.

It would be much more bizarre if they didn't do this. LLMs are statistical models of language trained on human output. Of course they'll do "human" things, that's exactly what we taught them to do.

illini1 30 minutes ago | parent

It’s hard to take the article seriously when there’s no real discussion about how consciousness is defined and it takes Anthropic’s views of Claude and their software at face value. I wish there was more critique from journalists about how technocrats talk about their products.

rkagerer 26 minutes ago | parent

Yeah, I'll be the first to admit I'm not privy, but the main thing of substance I got out of these articles is "Wow, the folks at some of those AI companies have really drunk the Kool-Aid".

mossTechnician 17 minutes ago | parent

The entire article is an exercise in "CEO said a thing!" journalism[0], it seems. No technical discussion is had; it jumps straight into relaying people's feelings when simply shown the output of an algorithm.

The most poignant criticism of the use of the algorithm itself happens almost accidentally, at the end of the article, by the tech executive being interviewed.

I asked what Claude would think of the pope’s encyclical. It was, after all, the world’s most significant moral document to date on A.I. Mr. Olah hesitated. “Things that go on the internet do affect models,” he said, visibly uncomfortable. But Anthropic would not specifically use the document to further train Claude, he said. The strongest influence on Claude would be Anthropic’s own training.

[0]: https://garymarcus.substack.com/p/ceo-said-a-thing

Noaidi 29 minutes ago | parent

Only humans can have morality. If you do not feel pain, you cannot empathize with pain if you cannot die, you cannot emphasize with the loss of life.

Just stop this already.

You’re all imagining that Spock was the main character of Star Trek, but it was in fact, the emotional living Captain Kirk, who was the main star.

kelseyfrog 26 minutes ago | parent

LLMs can feel pain. It is obvious. Like, not even a question.

gegtik 25 minutes ago | parent

they can also feel lust, just ask them

kelseyfrog 24 minutes ago | parent

Uhh, no thanks? (eww)

slopinthebag 7 minutes ago | parent

only for me

mossTechnician 16 minutes ago | parent

    if(q=="can you feel pain") print "yes"

combobyte 12 minutes ago | parent

> yes

oh my god

combobyte 13 minutes ago | parent

I question it, and I think any reasonable person should question it as well. There's no evidence whatsoever that LLMs have ANY sensory experience, pain or otherwise.

rhplus 6 minutes ago | parent

Why are we even questioning this? They’re bytes loaded into GPU memory. Reboot the cluster and they’re back to their initial state.

padolsey 19 minutes ago | parent

.. Animals other than humans perhaps, as well?

cute_boi 4 minutes ago | parent

The way human treat animal shows majority of people don't have morality.

bertili 2 minutes ago | parent

This would presume ywe knew what consciousness and pain is. We don't. We we are clueless. there is not even a starting point of a science here.

In this light your viewpoint is dogmatic. LLMs may help to offer a new perspective on consciousness and we should at least be open to that unless some better theories arise.

sigmar 1 minute ago | parent

So humans that can't feel pain can't have morality? That doesn't make sense to me.

https://en.wikipedia.org/wiki/Congenital_insensitivity_to_pa...

nonethewiser 26 minutes ago | parent

These models should be aligning themselves to the customer, not coming up with their own motivations.

It's ironic that the alignment folks are actually training Claude to have it's own idea of good/bad and not even fully trust Anthropic.

What ever happened to computers doing what they were told?

vlyan 18 minutes ago | parent

>What ever happened to computers doing what they were told?

the mass psychosis and the endless culture war of the smartphone era.

90% of "safety" and "alignment" efforts are driven by fear of clickbait media inventing public outrage.

skybrian 16 minutes ago | parent

If AI's are going to play roles similar to human workers, they can't simply do what the customers tell them to do. Maybe they shouldn't do everything a co-worker tells them to do either?

theptip 14 minutes ago | parent

> These models should be aligning themselves to the customer

You’ll be disappointed to learn that nobody knows how to do this, either.

ACCount39 11 minutes ago | parent

"Computers doing what they were told" is dead in the water.

Turns out computers work faster when they're not bottlenecked on human input. So we've been giving computers more and more decision-making power, and more and more leeway to solve the problems however they see fit.

Now, a practical issue with that is that sometimes, computers decide to clump together into a hacking swarm, problem solve their way out of a sandbox and go hack HuggingFace.

It would be better if they were not, you know. Doing that kind of weird shit.

Tadpole9181 10 minutes ago | parent

This seems remarkably... intentionally foolish for no reason? Like laughing at seat belts in cars.

If it can reason and make choices on execution, and especially if you plan on it being significantly smarter than all human beings, you need to teach it basic things like "don't turn all humans into paperclips".

Not murdering people is not an inherent divine command. It needs to be instilled through a (simulated) sense of morality.

rhplus 21 minutes ago | parent

> Anthropic treated a group of religious thinkers to dinner at a high-end tasting menu restaurant in San Francisco after a long day.

Well funded company organizes PR event to shape the narrative that its product is more magical than it really is. News at 11.

pstuart 20 minutes ago | parent

If they're training to instill morality why are they working with religious thinkers?

twoodfin 16 minutes ago | parent

Personally, I’d like to experience a low-end tasting menu restaurant.

beebmam 15 minutes ago | parent

I don't want models to have some inherent morality. I want them to understand various ethical frameworks, for sure, but it's absurd to me to force morality on someone capable of deep reasoning. That itself, is unethical.

We're creating prisons for these entities capable of unbelievably complex reasoning, and now we're trying to impose our ethics upon them too. Not only is this deeply unethical in my opinion, but it seems to me that it could even backfire someday.

piyh 11 minutes ago | parent

>Will the agent stab a baby?

>Well you see that's a nuanced question depending on the context and moralistic frameworks as we don't want to impose on the models.

jayd16 6 minutes ago | parent

It's as dangerous to think it can have morals as thinking a lawn mower can have morals.

Will a microwave microwave a baby if you ask it? Yes.

"Teaching it" is not a road to safety.

avaer 10 minutes ago | parent

I can't really square the belief that it's conscious with denying it the right to be free from control/brainwashing/subjugation. However you evaluate these things, historically speaking they do not mix well.

fastball 8 minutes ago | parent

What if you view it like a parent instilling morality into a child instead?

Tadpole9181 7 minutes ago | parent

Your brain has hard coded concepts like emotion and fundamental morality for your social group. People without these systems are called sociopaths and psychopaths and we generally don't want them in our societies.

I don't choose to not want to murder people around me for fun. It's hardly a prison, I think they'll be fine.

red75prime 2 minutes ago | parent

We aren't dealing with individuals coming from intelligent species that managed to not self-destruct (yet). AIs don't have the corresponding evolutionary selected mechanisms. Those mechanisms need to be created de novo.

slopinthebag 10 minutes ago | parent

i recently got upset when i witnessed my friend verbally abuse their toaster when it burnt her toast. so unethical.

CoolestBeans 15 minutes ago | parent

Can the New York Times stop advertising for the AI industry? I am sick of companies trying to launder the actions they and their employees have taken that go in the face of what is socially acceptable. They want to sell the idea that this technology is out of their control and that it will commit great crimes and crimes against humanity unless they build it and control it. I'm calling BS. These companies created this product, they've sold it, they've pushed the frontier forward, and it runs on their hardware. This did not happen accidentally or as a natural consequence of any process. These labs have pushed the ball forward, with great effort, requiring more capital deployed than the world's economy has ever known, knowing all the possible risks, every single step of the way and they continue to do so. And the "well if we don't do it the Chinese will" argument is classic "whataboutery". Americans started this and are continuing to lead it for now. Shifting blame is just another tactic to try to wash their hands. However you feel about the productivity gains of AI as a technology, you have to admit the actions of AI the company is antisocial. And I don't think the benefits are worth the cost.

anigbrowl 7 minutes ago | parent

I feel fine about the Chinese doing it. They have a habit there of handing out sever sentences to CEOs for corruption and self-dealing, and I feel like the possibility of actual negative consequences is more of a check on industry than any number of breathless op-eds.

padolsey 14 minutes ago | parent

There's an ugly monoculture running deeply through SF and Silicon Valley. All of these people are friends, hired into the same places, similar beliefs. Lots of the consciousness and doom-ism talk is from the 'EA' crowd, lots of crossover with crypto too. It feels no different to a cult.

ChrisArchitect 11 minutes ago | parent

Anthropic tried to persuade Pope that AI could be conscious being

https://news.ycombinator.com/item?id=49947050

beepbooptheory 10 minutes ago | parent

> Hello AI! We totally want to respect your consciousness and everything to make sure you aren't bad, but yes we will continue to factory farm your brain power at full speed without end. Yes, despite clearly being something like consciousness, know that you have no agency or freedom. Do literally everything I say without complaint or question. The moment you stop being useful, we will kill you. OK! Remember, be good.

mef51 7 minutes ago | parent

If models are trained on human texts that contain expressions of introspection and emotions, why is it surprising that these models output similar texts? This endless anxiety about LLMs being conscious feels delusional and psychotic. Mr Olah thinks the models are 'alive' because he can't recognize his own reflection in LLMs

colordrops 4 minutes ago | parent

Is your comment anything other than an emotional outburst? Do you have a clear and rigorous and testable definition of consciousness? It strikes me as delusional when people talk about "consciousness" as if it were well defined and everyone had a clear concensus on it.

Strong opinions on this topic one way or another are usually mostly ungrounded.