30 points dangelosaurus 58 minutes ago 20 comments

dangelosaurus 53 minutes ago | parent

This is my team’s work. I’m happy to try to answer questions or take questions back to the team.

And we're hiring! https://openai.com/careers/search/?q=cyber

BoppreH 42 minutes ago | parent

It's a bittersweet result, like finding asbestos in your house. I'd rather know than not, but I would much prefer not having it in the first place.

I understand this work was based on a dated snapshot. Will there be any continuous effort, scanning new PRs as they land, to prevent new weaknesses from being added?

ky1ebot1337 12 minutes ago | parent

(I am the author the pipeline, just created an account to answer some questions :D) My goal is to work towards a vuln-free Internet. What I observed is that stronger models can find many more vulnerabilities that weaker models cannot. If compute permits (there are other projects to secure besides Linux), I plan to scan Linux again with every newer generation of cyber models.

farisght 31 minutes ago | parent

I wish these blog posts included statistics on the categories of deficiencies that cause these vulnerabilities, and proposed measures to avoid them

A system that admits THOUSANDS of such vulnerabilities is certainly flawed in several fundamental aspects

jethro_tell 8 minutes ago | parent

It sounds like they can’t go through them all enough to have that data yet.

konaraddi 26 minutes ago | parent

As models get more advanced and generate an increasing throughput of increasingly higher quality results, how are you determining when to trust its results and when human review is needed on what?

E.g., 948 results is a high number to review in a timely manner unless your team is very large or spending several months combing through the results

EDIT: nvm i shouldve kept reading, answered

> At this scale, manually investigating every report is impractical for our research team. We therefore focused on validating findings that a) were reachable by a low-privileged user, b) required no special capabilities, and c) produced measurable evidence of memory corruption.

zahlman 32 minutes ago | parent

> We've developed an autonomous vulnerability discovery, exploitation, and patching pipeline. In one run with our cyber model, it validated and generated patches for 948 local privilege escalation vulnerabilities in Linux at an API cost of $1.2 million.

Imagine if open-source organizations other than the Linux Foundation (and Mozilla I guess) could throw $1.2M of API-rate LLM usage at security.

getcrunk 29 minutes ago | parent

> For the near and medium term, having access to a low-privileged account on an operating system, that isn’t up-to-date on its latest available patches should be considered functionally equivalent to root access for a well-resourced threat actor.

*buries head in sand*

*cries*

rubinlinux 10 minutes ago | parent

This has been true for DECADES.

Plasmoid 22 minutes ago | parent

How did we get to the point where there are a thousand privilege escalation exploits in the kernel?

Is the kernel simply too complex to reason about? Is it an underlying flaw in the security model or language implementation?

Should we assume that all software is more compromised than the kernel?

antonvs 18 minutes ago | parent

> How did we get to the point where there are a thousand privilege escalation exploits in the kernel?

We developed AIs that can find them.

p-e-w 11 minutes ago | parent

How can it be surprising that a software made from 40+ million lines of memory-unsafe code has hundreds of critical bugs?

hoppp 10 minutes ago | parent

There are always more left.

0xbadcafebee 5 minutes ago | parent

[delayed]

bombcar 8 minutes ago | parent

The dirty secret is multi-user died in the 90s and privilege escalation has basically been considered table-stakes since then.

mikestylz 6 minutes ago | parent

> Should we assume that all software is more compromised than the kernel?

Absolutely, the front door has always been unlocked. We were previously protected by the economics of exploit detection favoring the developers, but now we have entered a new world.

The issues arise almost everywhere we look. It's frustrating because the user experience in edge systems has barely changed in the last fifteen years. We could have rode the advances in semiconductor manufacturing to embed more layers of protection such as CHERI while slightly slowing down the rate of performance improvements. Instead we have only marginally upgraded security and largely added features which the users scarcely notice.

I am hopeful about the long term prospects of formal verification, but having humans reviewing the proof statements and confirming they match our desired properties will take many years. Meanwhile, AI assisted hacking is an urgent issue. So we will have to delegate implementation, proof specification, and proof development to the AIs in the short term. Maybe that goes well, but who knows. Interesting times.

nas 13 minutes ago | parent

I've always considered local escalations to be possible in Linux (i.e. having user access almost certainly means you can get root). However, 948 vulnerabilities is kind of shocking. Hopefully we are getting to the end of the part where it "gets worse before it gets better".

tamimio 12 minutes ago | parent

These are becoming more and more marketing materials than actual findings, because OAI probably hit the wall in making their models smarter (just like how google try to specialize their models in legal or whatever), but they need to remain relevant in the news. Sol 6.1 sucked so bad in reasoning that either it’s actually a bad model or users behind the scenes get degraded outcome while using all the resources for these marketing stunts. Stop, it’s embarrassing, make better models or be gone, at least we will get cheaper RAM to advance the development of open weights models.

hoppp 11 minutes ago | parent

Well at least they are doing public service while hacking the planet.