65 points md224 1 hour ago 59 comments

ComplexSystems 44 minutes ago | parent

I can't relate to this at all. AI models will surely get better at writing "enjoyable proofs," but for now the situation is what it is. You're passionate about this problem, right? But you don't want to do the work to understand the result? Fine. There's a new generation of younger, hungry mathematicians that are highly interested in figuring out why the result is true and I am sure they'd be happy to wade through it and spoon-feed you the answer instead. Maybe they should be running things.

bamboozled 41 minutes ago | parent

So OpenAI should be able to flood the world with AI pollution and ask scientists and mathematicians to wade through it all and tell us if there is any sense in it, then sit back and wait for them to report in?

Nice idea.

ComplexSystems 38 minutes ago | parent

Yes, they should. They have invented a magic button that can tell you the long-awaited answers to the burning mathematical questions that you've spent your life researching. The caveat is that the technology is still new, so the explanations "are not fun to read" like set theory papers usually are (lol). If you don't think that's a worthwhile tradeoff, that's your call, but it sure as hell isn't everyone's.

advael 33 minutes ago | parent

I don't think the claim is "they definitely have an oracle that solves the problem, and I reject it because it's hard to read". The claim is "OpenAI claims to have used an oracle to solve the problem. The proof is very difficult to read, and to even know if it does or not, we have to go through it with a fine-toothed comb, but they're going around claiming they definitely solved the problem (or at least getting press that claims that which they aren't pushing back against) and this might convince the people who sign grants even if it isn't true"

People who are invested in the idea that we've invented a general intelligence, now, which includes all these companies that are literally financially invested in this claim they are making, will tend to believe that its results can already be trusted in domains like this. Some mathematicians seem to believe some of the proofs written by their models, and some, like this one, don't. I do think it's valid for an expert to push back against the claim that the best use of their time right now is to verify the poorly written work of everyone who's claimed to solve the problem

famouswaffles 28 minutes ago | parent

Over the year, nothing they've released with a lean proof attached has turned out false (That's sort of the entire point. It's not impossible but it's really difficult). There's a reason most mathematicians, including the ones vehemently against OpenAI's dumping are not arguing the results are secretly false or have a high potential to be. And indeed, if that were the case, it would quickly become apparent and all this worry about grant signers would vanish into the wind. It's very easy to ignore nonsense. The problem is that it isn't nonsense.

bamboozled 32 minutes ago | parent

The problem isn't just that the papers aren't fun to read. The problem is that a lot of the research that goes into solving these issues leads to other discovers, new fields to explore and people have to develop new approaches to solve them. The other part of it is, the quality and the enjoyment of working on these problems leads people to find new and other interest problems to work on.

If you just strip mine the answers and Sam Altmans magic button solves 100/100 problems, what's next? Who is left to come up with a new interesting question for the magic button to solve?

Lastly, life and the present moment is all there is, if there is no enjoyment in anything we do, then what's the point of all the "living for ever" Altman et al want to achieve.

We will live forever to read boring papers generated by LLMs? Literally sounds like an eternal hell.

vikramkr 23 minutes ago | parent

Nuclear fusion is already proven by the universe to be a viable energy source by the fact that the sun exists but people still work on understanding and taking it and developing new approaches to accomplish it. People didn't stop experimenting with and developing programming languages because technically they're all turning complete and the first one was "enough." Y'all will be fine - every JavaScript framework that exists is someone looking at a theoretically correct and complete solution and deciding actually it sucks and they could do better. "I want to understand xyz but the proof is trash and I think it's ugly" will be plenty motivation for a lot of people to work on it.

inference-god 14 minutes ago | parent

How is the journey to understand fusion related to not wanting to spent your limited time on earth wading through AI slop?

ComplexSystems 10 minutes ago | parent

Here's a guy who's made a "beyond n log n" tracker: https://x.com/aurel_pr/status/2108214135179944096

He's tracking the community progress on sub-n log n multiplication. OpenAI started with 1 - 1.63e-55. The result has been now improved on 115 times, and the current record is "rohanarun"'s 1 - 9.87e-5. I'm sure by tomorrow it'll have improved again.

Does this look like people aren't having fun? Does it look like they aren't discovering stuff? It looks like it's spurred a cascade of interesting community activity. It doesn't really seem much different from what happened with the twin primes conjecture. Isn't that supposed to be the point of all this?

buriram 15 minutes ago | parent

I don't understand. Why is this the onus of scientists and PhDs to review whatever results OpenAI had dumped out? If OpenAI had produced incomprehensible papers, surely any journals would just reject it, or demand the author to do a complete rewrite? Unless we are talking about a race to solve problems, which PhDs are afraid that they had been scooped up on?

fbrncci 34 minutes ago | parent

What else should they do? See these models get smarter and smarter, somewhat-solve things but only to the tune of 90% what mathematicians (or experts in any other field) would deem acceptable, and then gate keep the findings for the next few years going through peer review and paywalled journals? I for one welcome the flood, bring on more in every possible industry and see where all that progress lands up. Sure it will upset a lot. A lot of things also upset the luddites.

bamboozled 30 minutes ago | parent

Are you going to be doing the work to verify the results ? Will you just expecting other people to wade through the flood and reap the benefits later on?

fbrncci 4 minutes ago | parent

Nobody is forced to verify the results. I am honestly not expecting anything other than AI to get smarter and smarter and people who are motivated and interested enough to pick up after it; and potentially reap all the long term benefits ahead of those who aren’t (seeing this happening with software development in my own field).

XenophileJKO 25 minutes ago | parent

Exactly.. this is an opportunity for people to pick up where the model left off and run with it. Like you don't have to, nobody is forcing you to.

However, there are always smarter, hungrier people out there and this is a buffet.

Some output is going to be wrong or incomplete. I am willing to bet even those have nuggets that can be used elsewhere.

LPisGood 32 minutes ago | parent

They should certainly be allowed to share their findings. No one is forcing scientists and mathematicians to review the findings in general. It’s just the case that the findings are of such such a quality that it would not make sense to ignore them wholesale

Arainach 23 minutes ago | parent

> No one is forcing scientists and mathematicians to review the findings in general.

This is like "no one is forcing software engineers to use AI tooling" or "no one is forcing you to show your ID in the airport" or "no one is forcing you to own a car in your small midwestern city" - there can be no law requiring something and the practical consequences of not doing so can be so painful that you're effectively forced anyway.

skeledrew 31 minutes ago | parent

No, they don't have to ask. Those interested enough will jump at the opportunity, even if it's just to be "one of the first to get it".

latentsea 24 minutes ago | parent

You know, I kinda relate to the feeling of not wanting look at those outputs if I think of it from a layman's perspective.

I just imagined that instead of math papers, they released 700+ feature length films, and the only way to tell if one of them is any good is to watch it in its entirety.

That feels pretty unappealing to me.

I know it's the same for human made films, so what's the difference right? But those are good enough most of the time that it's a decent bet, and the people that made them had real skin in the game.

Contrast that with something made by a nondeterministic slop machine with no skin in the game where small details can be off in a way that's jarring. Right out the gate I have an aversion to committing that much time to something that very well may waste it.

rbehrends 28 minutes ago | parent

The problem here is that while you can call what OpenAI does "mathematics", I would hesitate to call it science. Science as a process of acquiring and developing knowledge within a domain involves a lot more than just dumping unfinished work on the scientific community. Among other things, it involves developing frameworks and understanding of the domain, formulating questions, creating results in a fashion suitable for verification/testing/replication, and relating these results to and integrating them with that edifice.

judge2020 23 minutes ago | parent

Kinda sounds like computer science vs developing - in the sense that people with a master's in CS and are dedicated to the craft will write wonderfully artistic software, while it doesn't actually take a love for the process to write code and get hired at some tech company (even less so now with agentic development).

rbehrends 17 minutes ago | parent

Not what I am getting at. In fact, the problem I am getting at is the abolition of existing scientific and engineering principles and processes without a replacement and applies to programming with agents as well. This is not about artistry, but about building durable things.

ComplexSystems 18 minutes ago | parent

Have you looked around recently? Because there are plenty of mathematicians that are excited to read and learn about all of the new results. They've improved on the sub-n log n result and have even made a web site to track progress on it: https://beyond-n-log-n.netlify.app/

It looks like people are enjoying themselves, having fun with the new results, and generally doing all of the things you say "science" is supposed to be about. So what's the problem?

rbehrends 11 minutes ago | parent

I didn't say it is useless. Consider Ramanujan, whose work gave rise to a lot of interesting math, even (and sometimes, especially) the parts that lacked proofs or had other gaps (probably because it was obvious to him). But a singular genius, whether a person or a machine, does not science make.

SpicyLemonZest 28 minutes ago | parent

The question is why the situation is what it is. Did OpenAI publish a large volume of unreadable proofs because that was their best attempt to contribute to the field of mathematics? Or does OpenAI feel that it’s more profitable for them if people come to see mathematics as something that’s less focused on understanding and more focused on using AI to generate proofs?

conformist 16 minutes ago | parent

Yes, I guess it matters a lot whether this was quite close to the best they could do or the best they could do given specific resource constraints or whether they just didn’t bother to try doing better (eg to invest more tokens into readable papers)?

fn-mote 10 minutes ago | parent

> I guess it matters a lot whether this was quite close to the best they could do

As the author of the post points out, there is no way this is “the best they could do”. It’s a write up that didn’t involve someone with the math + communication skills required to clearly explain the result.

senorcrab 12 minutes ago | parent

You don't know what you're talking about. Just because it was created by an LLM, and verified in Lean, does not make it true. The whole point of writing a proof is for it to be understandable.

conformist 12 minutes ago | parent

How about the scenario where they are already better if you give them more time/tokens/…? Would it be a more relatable concern in that case?

transitivebs 43 minutes ago | parent

openai will take expert responses like this and improve the next set of papers

it won't be long before there's no more low hanging fruit like this to complain about, and the writing / explanations of the results are superhuman as well

separately, i really liked the author's denial-of-service analogy. super useful practical framing

bamboozled 42 minutes ago | parent

Look forward to seeing an LLM write something well, that will truly be a breakthrough in the field.

jdw64 36 minutes ago | parent

Is a proof that cannot be understood worthless? How would this be framed philosophically?

In fact, the academic system is a kind of worldview created by humans. And as it is shared and the community grows, the problem will gradually become more complex. Because when a discipline develops sufficiently, just as in a mine where rich veins are easy to extract early on but become very hard to extract once much has been dug out... in that sense, as things gradually become more complex, once a certain threshold is reached, won't scholarship surpass the limits of human understanding? Of course, scholarship is entirely for humans, but at some point the system itself may face its limits, and then wouldn't it again reduce the existing normalized minimum within that discipline and establish a new normalization of a new logical system?

In my view, perhaps for very complex work like today, AI will do it, and then there will be work that normalizes and further simplifies the results of that AI. Then, coming back to the human fold, if humans create the initial skeleton, the LLM will learn that again and it will become complex work again, and won't this create a continuing cycle?

I think verification and understanding can be separated. If the proof targets a correctly formalized proposition and passes a reliable proof checker, isn't it valuable? We have obtained knowledge justified as true, but there is simply no new theory that understands that knowledge. As was the case with the Four Color Theorem...

I am always curious what shape the newly compressed new discipline will take. At that time, I hope even people like me, who are intellectually behind, will be able to learn that discipline.

computerex 34 minutes ago | parent

Wow talk about sour grapes! No one is forcing you or any other mathematician at gun point to engage with this release at all. Ridiculous drivel.

samuelknight 32 minutes ago | parent

At first I thought this was going to be more Luddite babble, but it makes a good point. OpenAI isn't contributing if they are make unreadable papers. They should use a little more of their compute to nail interpretability. The difficulty will only get worse as AI plow deeper into the frontier and produce increasingly alien looking output. I suspect it's a workflow issue. If not, it's a bad oversight if the current generation of models are capable of making mathematical breakthroughs but can't explain how they build on existing frameworks.

nba456_ 27 minutes ago | parent

If you don't think they have contributed anything then you can safely ignore them.

flowerlad 32 minutes ago | parent

> OpenAI drops some hundreds of "solutions", incomprehensibly written "solutions", and we are all expected to jump on them and what? Appreciate their contributions?

This is a strawman. OpenAI didn't say they are expecting all mathematicians to read the solutions, incomprehensible or not.

So mathematicians are upset with OpenAI for solving "their" math problems. Software engineers are even more affected by AI, yet mathematicians seem to be reacting more strongly. I don't get why.

mcshicks 31 minutes ago | parent

"not at the Lean code, since I know very little of the actual usage of Lean, and that code was enormous"

This part I don't understand. Not that anyone should read the entire Lean code of any proof, but if the statement of the theorem to be proven in lean seems to be correct, then I would think there would be at least some interest if in fact there was a formal proof (which might or might not correspond to the written proof) of something I was working on. That to me would be interesting. Or you are saying you doubt the validity of the formal proof, which would also be interesting. But saying it is of no consequence doesn't make any sense to me.

bluepeter 30 minutes ago | parent

I've seen my entire profession vanish overnight due to AI... well, not vanish. But, yeah, software development is WAYYYY different. And I couldn't be happier. I think it's amazing. I see the productivity boost. Even if it means I can't add nearly as much value as I used to.

Sorry, but I don't get why mathematicians are so upset. Like, just accept the knowledge and insights and acceleration in your field! If it isn't "fit for human consumption" because an AI produced, okay... it soon will be explained ELI5 by even better models.

za_creature 27 minutes ago | parent

You will own nothing and you will be happy.

drusepth 17 minutes ago | parent

I don't see how this quote is relevant here.

Most software developers (or people in any field, working for anyone) rarely own anything they do at work.

And the entrepreneurs running their own companies (which there's an explosion of atm largely because of AI) do indeed "own" the higher-level products and things they're producing, even if AI writes the code.

What is supposed to have changed?

heyitsdaad 29 minutes ago | parent

Very sensible comments. It is along the lines of the fury I get when I am confronted with an 11 page dump of an issue analysis created by an AI agent that makes no sense but I have to go through because customer shared it.

If you did’t bother to write it, I shouldn’t be bothered to read it.

Perhaps AI agents can have their own publications and magazines where they are the chairs and associate editors and reviewers.

red75prime 7 minutes ago | parent

> Perhaps AI agents can have their own publications and magazines where they are the chairs and associate editors and reviewers.

AI agents are able to coordinate without all this bureaucracy. But, yeah, a spoon-feeding department in their setup should be a must.

sigmar 28 minutes ago | parent

>OpenAI drops some hundreds of "solutions", incomprehensibly written "solutions", and we are all expected to jump on them and what? Appreciate their contributions? Why?

...

>So, no, I will not be sending Sam Altman a bottle of whisky anytime soon, nor I am planning on spending my time reading through that paper and trying to make sense of it.

Think about a hypothetical circumstance where we get radio communication with some aliens on another planet. They send over tons of math to help us advance our tech, we know the math they're sending us is correct, but their explanations are really hard to work through because they aren't humans and the math is so different from anything we've done. Should we whine about the results they sent to us and refuse to engage with it?

conformist 20 minutes ago | parent

In this hypothetical case funding agencies would probably happily pay for “alien maths translation grants” and you could write papers about advances in that field and get jobs etc. So arguably for better or worse it would create a specialised academic cottage industry that somehow interfaces with the rest of maths.

agnishom 24 minutes ago | parent

Very well written summary of the current situation

Imagine being someone who is working on one of these problems. You have no good guarantee that the problem was solved, but you will have the horrible homework of reading the AI slop. Also, if you do have something interesting to say about the problem, people will have less enthusiasm about it now

positron26 21 minutes ago | parent

The gradient into formalisms is steep and the bottom is deep. These are the worst results anyone will ever see again. The bottom of the internet is also deep, but won't be around much longer.

warkdarrior 23 minutes ago | parent

Love this!

> This is why I generally avoid using AI for mathematics (I am happy to ask LLMs to consolidate information for me, or to generate a useful infographic, or to proof read an email, etc.)

In other words, the author is OK with using LLMs to replace data analysts (that could consolidate information), to replace graphic designers (that could generate infographics), and to replace editors (that could proofread an email). But don't you dare use LLMs in their mathematics.

creato 10 minutes ago | parent

I don't know about this person, but for me and I'm guessing the average professional in basically any field, before LLMs, they simply ignored information that needed summarizing, made their own crappy slides, and didn't proofread their emails.

airtnp 22 minutes ago | parent

I would admit seeing more mathematicians irritated is a good popcorn show.

For what's worth it, this batch of results are not very tight intentionally by OpenAI and promising mathematicians already started to consume them and improve the results, while someone is still complaining on it.

inference-god 17 minutes ago | parent

Reading though these comments I truly understand why the hardest part of my career in "IT" has been personalities.

The lack of emotional maturity and empathy is very on par with my experience thus far.

vikramkr 17 minutes ago | parent

This whole "oh no this problem is solved now who would ever want to work on it" thing is quite funny. Mathematicians, let me introduce you to something called bike shedding. Folks proved 1s and 0s are turing complete decades ago and yet we have 27 new JavaScript web frameworks every week (or every day or minute now with ai). Y'all will be fine. Thinking a solution is ugly and that you could do better is also a perfectly good motivator and most of the sciences and engineering are "ok, so we know xyz to be true about the world because like, I'm looking at it, but wtf is going on." The theorems were true false or otherwise before some random openai model solved them. And if you don't understand the proof nothing of significance has changed except you've got a bit of a hint now.

drivebyhooting 13 minutes ago | parent

If he doesn’t like the paper’s structure he can just prompt gpt pro to review and amend.

I know the model that produced these proofs is still private, but it’s worth a shot tackling the proofs with the current consumer-available frontier.

cs_throwaway 13 minutes ago | parent

There is no nice way to tell someone that you’ve scooped them, and this is industrial scale scooping.

A few papers have been retracted, but it looks like many are withstanding intense scrutiny. Lean is making the results more likely to be correct, but I think making them harder to understand.

The world has changed and you’ll know a math department is making a serious attempt to adapt when it teaches a required Lean course in freshman year.

nxobject 4 minutes ago | parent

In short, we’ve reinvented cranks sending unsolicited, poorly written putative proofs