Yet another Hacker News

Altman tells staff OpenAI is open to slowing AI development

20 points signa11 1 hour ago 42 comments

loose-cannon 49 minutes ago | parent

I don't understand. They're slowing the development because they want to or because they have to? Because of ethical concerns? Or because of economics factors that prevent them from delivering results?

Since when have ethical concerns slowed the pace of technology?

malcolmwhite 44 minutes ago | parent

I suspect it's all about the last bullet point:

> OpenAI said on Wednesday that it was pushing for mandatory national AI safety requirements in the United States, citing concerns that advanced AI systems could accelerate its own development.

Insanity 23 minutes ago | parent

They can just have codex develop part of its next version and oh look, it’s accelerating itself! So scary. </s>

deagle50 44 minutes ago | parent

Because it serves as a justification for regulation and government funding.

amelius 28 minutes ago | parent

No, Altman is saying this exactly because he wants to avoid regulation.

This is a common tactic in any industry. Say that you will self-regulate, but then a decade or so later everybody wonders why anyone thought it was a good idea to allow an industry to make their own rules.

JumpCrisscross 41 minutes ago | parent

> hey're slowing the development

They're doing no such thing. Altman is saying words ahead of the midterms.

ahartmetz 35 minutes ago | parent

My guess is: progress is stagnating, so they look best if they pretend to slow down intentionally while continuing to work at full speed.

Insanity 24 minutes ago | parent

My money is on this as well. Their model improvements are getting more marginal over time, they have no moat, Anthropic is moving past them and apart from software engineering and a few tech fields LLMs fail to deliver business value.

So the “slowing down” narrative is a convenient story to spin.

lowsong 18 minutes ago | parent

They don't deliver business value in software engineering either. It gives the _illusion_ of increased productivity, but a machine that drives your engineers to burnout in exchange for more code shipped that's still bottlenecked at all the other stages of software development is not a winning strategy.

The entire AI market is propped up on the hope that the models will magically solve all of these problems, and now that even Altman is realising they won't he needs the slowdown to stave of the crash for as long as he can.

coffeefirst 22 minutes ago | parent

Yes. They will reach an agreement not to pursue RSI for “safety.” That RSI is utter nonsense in the first place will go unnoticed.

glimshe 43 minutes ago | parent

The problem is that even if the majority of labs decide to slow down, there will be one lab, somewhere, with dark money moving forward. The military wouldn't let that chance pass by.

JumpCrisscross 42 minutes ago | parent

> there will be one lab, somewhere, with dark money moving forward

If this is really an existential threat, that's better than fifty of them.

nba456_ 32 minutes ago | parent

No it's not. It's worse. Do you have any idea how much power that one lab would have?

henry2023 29 minutes ago | parent

How is worse?

jacquesm 25 minutes ago | parent

If they manage to leap ahead they can effectively kill all competition, present and future and you'll be at their mercy.

henry2023 8 minutes ago | parent

So the solution is to make the company that can’t avoid sub-AGI models from going rouge to try RSI?

Also, AI performance has proven to be a function of power and compute, and right now most of the compute lies between two companies rushing towards RSI not because they are afraid someone is going to do this before but because they are the ones who wants to build it.

JoshTriplett 39 minutes ago | parent

They will if it would be against international treaties to do otherwise, mutually enforced by every other world power.

dgellow 35 minutes ago | parent

We don’t even need something that powerful, you can control the hardware manufacturers and distributors. If you control GPUs, HBM, and the hardware needed to cool all that down and provide the power, that’s enough to make it very hard to scale up the risk.

You cannot really hide an AI datacenter consuming GW of power for too long.

nba456_ 31 minutes ago | parent

Except no one country controls all that. And if you put limits on it, other countries will speed up doing it themselves.

JoshTriplett 16 minutes ago | parent

If you stop going "nobody will do anything actually effective", and start going "how do we do something actually successful", any number of successful ideas start readily arising. It is possible to stop. Not easy, but possible.

JumpCrisscross 32 minutes ago | parent

> international treaties to do otherwise, mutually enforced by every other world power

This does not exist.

JoshTriplett 30 minutes ago | parent

Look at how we regulate nuclear weaponry. Consider how much more widespread it would be if we hadn't.

petcat 21 minutes ago | parent

Nuclear weaponry is effectively regulated because the nuclear powers have agreed that nobody else should be allowed to have the capability.

So this is basically suggesting that USA and China should work together to ensure that they are the only ones allowed to develop and maintain capable AI. And between the two of them they can carve up and techno-colonize the rest of the world.

JoshTriplett 18 minutes ago | parent

We've also agreed, for instance, not to do certain types of testing anymore. Just as we could agree to stop training even larger frontier models.

jacquesm 16 minutes ago | parent

Except...

(From WP)

India

In 1998, India said it would only sign the treaty if the United States presented a schedule for eliminating its nuclear stockpile, a condition the United States rejected.[18]

Israel

In 2016, Israeli Prime Minister Benjamin Netanyahu said that its ratification was dependent upon "the regional context and the appropriate timing".[19]

United States

The United States has signed the CTBT, but not ratified it; there is ongoing debate whether to ratify the CTBT.

https://en.wikipedia.org/wiki/List_of_parties_to_the_Compreh...

jacquesm 18 minutes ago | parent

They made a movie about that.

Between:

   "Iron Sky"
   "Whoops Apocalypse" 
   "Wag The Dog"

and

   "Idiocracy"

You'd think we were amply warned.

threatofrain 19 minutes ago | parent

And what framework would there be for international cooperation? Without that element, this would just be self-destruction in a global race.

1dom 9 minutes ago | parent

Maybe we're about to test the scifi trope that a common enemy - like a superior invading alien life form - is the only thing that could unite all of humanity.

dgellow 38 minutes ago | parent

> In August, OpenAI paused much of its model development for two weeks to bolster its defenses after its AI agents escaped containment and hacked open-source platform Hugging Face.

That’s one sneaky way to remove OpenAI responsibility…

nba456_ 33 minutes ago | parent

How does that imply anything like that?

wernerb 32 minutes ago | parent

Meta in their lawsuit negotiated 5B to be paid out only if other market players agree on implementing safeguards. In the same vein OpenAI can slow down competition, lock out opensource AI.

> OpenAI said on Wednesday that it was pushing for mandatory national AI safety requirements in the United States, citing concerns that advanced AI systems could accelerate its own development.

As well, the weaponisation of AI is already there, open ai can switch off safeguards at will. A massive swarm was allocated to solving Millenium problems in a few days. Imagine the US having such a weapon, they want to ensure other nations cannot have this kind of access. I imagine the US might be very interested in stifling globaal Innovation and be sympathetic to OpenAI for these reasons.

simonw 21 minutes ago | parent

Timing is pretty good for OpenAI right now, given GPT-6 Astra is out and appears to match or even beat Claude Fable 5.1.

They might not be as interested in slowing down if Anthropic currently had a significantly better model than them!

oinoom 12 minutes ago | parent

I’m curious why you use this language of “appears to match or even beat”, seemed pretty clear from the benchmarks that it’s well ahead of Fable. Do you doubt the reporting?

petesergeant 9 minutes ago | parent

I doubt the reporting: I've found it to suck for productive work. Gone back to using Sol.

oinoom 8 minutes ago | parent

You know… I agree. I haven’t found it to be as great as reported.

s3p 8 minutes ago | parent

[delayed]

simonw 3 minutes ago | parent

I'm using both extensively at the moment and I've not yet found that either is demonstrably better at specific tasks than the other. They're both beasts!

(I've not had Fable do as much work in Blender as Astra yet though, and it's quite possible that Astra has Fable beat on 3D modeling.)

skerit 17 minutes ago | parent

All they're saying to the other players is: let's lock it down and slice up the market between us.

Claiming Skynet is just around the corner is a great way to get the public to believe only these huge companies can harness AI, and how all open-source models should be outlawed.

petcat 7 minutes ago | parent

There are no open source models, at least not useful ones. Open weight is not the same as open source. Open weight models are still just inscrutable binary blobs that you can (theoretically) run on your own computer instead of through a SAAS web app. The open weight model labs don't even provide a high level catalog or any description whatsoever about what went into the training data.

This is not open source and we really should stop conflating the two things.

seydor 16 minutes ago | parent

Is he trying to find a cop-out for his diminishing returns?

gizajob 14 minutes ago | parent

Absolutely this. Flip the idea on the head to reveal where he’s at coming out with such statements.

signa11 6 minutes ago | parent

translation: we have hit the wall.