YHK - yet another hacker news clone

> Before injecting this cocktail into your bloodstream, would you want to know that there is at least one human cancer researcher who understands the mechanism behind this cure.

Tao is out of his lane; lots of medicines don't have understood mechanisms. We don't even have the mechanisms behind general anesthesia nailed down; do you want to forgo it when the docs cut you open to remove your tumors?

It is nice to see some sanity back in the conversation!

I'm still not sure about math-2.0 (humans+AI will make fundamentally more progress):

- AI and computer usage take a mental toll on humans and humans will overlook radical improvements.

- AI may be good at finding useless things like "P==NP, but the complexity is O(n**4242424242424242)". In other words, useless.

- Humans become formalists and lose traditional sources of inspiration. Maybe interacting with Lean should be left to specialists, but not to creative blackboard mathematicians.

- AI exposure will further intellectual conformity, more than the Internet did.

As to the last point, a lot of progress (real, not measured in publications) seems to have been made when communication was slower and there were several different schools and approaches.

> Before injecting this cocktail into your bloodstream, would you want to know that there is at least one human cancer expert who understands the mechanism behind this cure?

No. I'm gonna die, my man.

This guy might have the highest IQ on the planet, but it's clear he hasn't spent much time around average people.

This argument is so silly against reality, and already, almost nobody understands the things they put in their body to any significant degree, beyond the effect produced.

Will I take a cancer cure that no human understands? Yes. And so will billions of others. Just needs to cure cancer, that's nearly the only requirement.

page 20 ("A thought experiment on alignment and understanding") is perhaps not likely to quite induce the reaction in most people that Mr. Tao expected.

> Before injecting this cocktail into your bloodstream, would you want to know that there is at least one human cancer expert who understands the mechanism behind this cure?

> Or a human mathematician who understands the mathematical model used to locate the cocktail?

If we're being fair to AI - it contains collective knowledge from all fields, which means it's probably less likely to miss something that a human would.

The cancer arguement on slide 20 is pretty weak. I first need to be alive in the long term to worry about the long term effects. If I had terminal cancer I'd gladly take an AI developed 'cure'.

There are still a lot of treatments/medicines in medical science where we dont know 100% the real reason as to why it does what it does but we still prescribe them because the intended effect is what we are interested in.

Man with a cozy and soon to be irrelevant sinecure grasping at straws.

It turns out research math was puzzling solving and we rewarded idiot savants.

I just watched Primeagen’s video on this and Tao’s point is that juniors no longer have the path of solving a proof to earn Field’s medals. He also argues that the community part is being hurt by AI discovering proofs because in the past people used to get invite to talk and collaborate. Now all that is being taken away. The community must adapt because Pandora’s box cannot be closed.

Prime also mentioned that software development is different. In Software development, the product is what you’re building towards, so the means to get there can be disrupted without the industry being cannibalized.

In math research, the process is the product. You take away the researching part and not much is left. But my question is, these math proofs OpenAI released, will math shift to actually using the proofs to change the world instead of just finding new ones?

I'm learning so much about specific vs. generalized intelligence through this entire ordeal.

> Using AI to find and highlight new principles, methods, or insights, rather than merely new proofs.

I would have assumed that, by producing new proofs, the AI has either validated existing principles, or discovered new ones? Isn't that worth studying?

Are mathematicians complaining that reviewing AI's proofs is not as fun as writing your own? Try being a programmer... welcome to our world!

If AI lacks imagination and is not discovering new principles, then it's doing us a favour: it's crossing out the problems that don't need new principles. So the problems/conjectures that are still left are the more interesting ones.

Has the world gone mad? In which universe would you get a mathematical model to produce a tonic of random chemicals that optimize said mathematical model, put that through stage 3 testing, and only THEN ask if you want to inject this into yourself. My Guy, you just did clinical testing on fucking humans for phase 3, it's a little late to consider the moral ramifications of the analysis-experiment dichotomy.

The real example is that you ask ChatGPT for a novel cancer cure, and it spit out some random chemicals, probably including bleach. Do you inject that into cancer patients that have no other hope? No, you don't. You absolute charlatan.

Slide 12 and 13 sum up my conclusions from the Oct 8 100+ reactions to 100+ solutions. Several reported OpenAI drop included answers put a wrap on problems they had been working for years. Several said that they have to completely rewrite grant requests that they had just submitted. Others described learning of the solutions problems they had been working on like losing an old friend or a lover. The general sentiment was to bemoan the loss of a field, as if it would have been better be born in the early 20th century and conclude their career arc before reaching this point. My take away is that the field needs to get it through their heads that their old problems are no longer ambitious, and that their job in the short term is to find the new frontier.

One idea Terrance Tao conjectures, which is highly doubtful, is that spamming the AI button will solve open problems without producing insightful new methods. But the OpenAI drop would seem to disprove this. The sub O(nlogn) proof for DFT for example violated very old human assumptions. Decades of work in the field was incremental progress on sub optimal method that nobody questioned hard enough. More generally, we should always be able to go back to a super-human AI and say, "Attack this problem, but don't use a method tried before."

From the slide Beyond Problem Solving:

"Reaching these lighthouses [resolutions of open problems] prematurely by automated tools can disrupt the exploration of the paths not taken, and sterilize the surrounding field."

This crucial issue is centered in mathematician psychology and the incentive structure of academic/institutional mathematics worldwide. For mathematics to flourish going forward, we will need to realign our brains to think differently about the nature of mathematical progress. And we need to reorient our institutional incentive structures towards the promotion of meaningful mathematical progress itself rather than targeting proxies that are no longer faithful.

Regardless of the precise nature or the causes of the "sterilization" Tao refers to, we (the mathematics community) can only rely on ourselves to repair it. Though, since it will involve fundamental change at the level of ossified academic institutions with many stakeholders and divergent vested interests, any such repair will be slow, frustrating, controversial, and lacking any guarantee of success.

Just a ~~few~~ ton of things (sorry!), with the upfront caveat that Tao is a hero who's trying his damndest:

1. The use of semi-ugly slides to communicate this is just perfect and quite heartwarming, but it does highlight my main criticism of the mathstadon version of this thesis: he's myopically focused on mathematics as he has practiced it, rather than mathematics as a ~2400y old academy. Like, "stable for almost a century" sounds impressive, but should be a pretty obvious red flag in hindsight!

2. Glossing over "objective verifiability" feels like another place where he's ignoring a ton of relevant philosophy for no clear reason -- yes, mathematics is the only academy based in pre-conscious cognitive facts about our processing of time and space, but that's not the end of the story on "objectively verifiable". To say the least! He hedges with "broad consensus" which doesn't need to be absolute, but that seems to be not only dismissing a highly relevant question, but even implying that he might be unaware of it. I doubt he is, but still: not great.

3. Who is this for...? Why is an explanation of Lean needed in a talk given at CalTech? I suppose he's welcoming his role as a bit of an influencer, there?

4. Re:the focus-on/centrality-of 'highly digitizable' as a unique class of task that applies to mathematics in particular, I must sadly trot out the increasingly-common trop: Yudkowsky called it... https://intelligence.org/files/IEM.pdf

5. "the space of mathematical problems remains infinite" is, again, ignoring really important philosophy around academies as social structures, built for human means. Mathematics is only infinite if we decide that all knowledge is useful (the quintessential example being 'counting the grains of sand on a beach'). Not really important in the first place, but another worrying case of the above.

6. Problem solving is the goal of mathematics; he has a completely valid point here (that we shouldn't throw AIs at unsolved problems in bulk and thus lose human expertise), but it's obscured by the use of "[open] problem" being a too-technical one. IMHO. Slide 16 fails to disabuse me of this notion.

7. Slide 18 is describing the differences between functions and systems, and is arguably even talking about assemblages.

8. If we're gonna explain lean up-front, it feels like a baffling choice to throw the "maybe AI will solve cancer but use it to secretly plot to kill us all" slide in there. It's also already lead to misunderstandings and backlash on Reddit, where 'yes we want to not die of cancer!' is a pretty convincing counterpoint (if a ultimately a subtle strawman, ofc). It's also quite distinct from the rest of the talk.

As always, the best part of any Tao publication is his ability to inspire and rally and organize. I think Math 5.0 will indeed be a matter for creativity! Hopefully the IE levels off before we cease to be helpful in that capacity...

Another example that people should not use medical analogies to illustrate a side point: Discussion boards will focus on the completely irrelevant side point to drown out the renewed AI caution that they do not want to hear.

> The future of mathematics — “Math 2.0” — will require both expanding the research frontier, while simultaneously decentering the traditional role of problem solving.

Seems to be the crux of the argument, but "use your imagination" isn't a great thing to tell people who are looking at degree irrelevancy, concerned about getting tenure or a research position. How do we measure if someone is a good mathematician or not, if they are one of the sanctioned few who get access to the biggest AIs?

Putting aside the cancer question: I still don't understand how TT seems to be fixated on what models can do today instead of tomorrow. It's realistic and even conceivable that the models will also become better at explaining and presenting proofs too. Maybe he's not emotionally ready to accept that there may not be a future where his (and to some extent, my) skills are relevant and valued. It breaks my heart. I hope I'm wrong, but it feels like we've run out of higher ground to run to.

I think a better analogy would be conducting a marathon in a fog. If you can't see the path of the proof, how do you know it is completely true in all scenarios? If you can't see whether the AI runner ran through every part of the race, how do you know it didn't draw hallucinated shortcuts in the parts where humans can't see? Or worse, create obscurity and blow smoke to hide the shortcut section? If the same AI was to guide the last living humans to a star, because it found a path clear of danger, could you trust it to get in that ship? Or did it just forgot mentioning an asteroid belt the ship is not built to navigate? Truth is verifiable truth that multiple parties can agree upon. Can you trust with your life something you can't verify?

It's quite noticeable how when Terence tao was sounding pro-AI the sentiment was much more positive and he was being held up as an authority to listen to. Then he puts out some limitations of AI in a presentation about how to work with it as an expert and suddenly on HN he is just some out of touch killjoy trying to hold back progress etc.

I work in drug discovery and AI and his slide about cancer medicine isn't very well informed. We have models and narratives about how drugs work, but they are woefully incomplete.

I ask nearly every doctor/researched in the medical field: if you had a AI-created drug that tremendously improved cancer treatment outcomes for your patient, would you hesitate to prescribe it because nobody understood how it worked? I have yet to hear "yes, I would hesitate", most people say "it would be cruel to deny a person a treatment that worked".

I think Tao is focusing too much on second order effects of AI on math and other fields; humans are terrible at reasoning about second order effects, especially ones that are happening dynamically, in real time, using the most advanced mathematical models the world has yet created.

>" “Ablation studies”: taking an already proved theorem and seeing whether it can still be proved after

removing some key theories or inputs

(e.g., finding an elementary proof for a result currently only provable by non-elementary means)."

As usual, Tao is brilliant in all that he researches, all that he writes about.

I chose the above statement (which is brilliant, in and of itself!) to comment on, because it leads to the following idea:

There there exists, or should exist, a dependency map in the fields of not only Mathematics, but also of Computer Programs/Software, Engineering, and even a seemingly non-related field: The Law...

In other words, how do we get from the simplest of axioms or foundational things (aka "first principles", aka "self-evident truths") to much more complex entities?

In Law for example, how do we go from the simplest of historical legal constructs to the most complex of the most complex Supreme Court cases?

You see, there is, or should be a map, you could call it a dependency map, you could call it a dependency graph, which shows more and more abstract/complex mechanisms/things/assertions/statements/truths/functions which is mapped back to , that is, dependent on various chains, various stackings, various "stacks" of simpler ones.

In Engineering, for example, how do we get from the simplest of machines to the most complex of machines? What simpler machines and/or sub-components (aka "dependencies", aka "subcomponents") are required to build it, and how do those simpler machines work, and what's the dependency graph or map for their subcomponents?

More generalized, if we have something of complexity, then how do we get there, step by step, from individual subcomponents, individual inputs, individual proofs, individual software systems, step by step?

What is the map of those dependencies?

Note that in some systems, Math proofs, for example, there may be different paths which can be traversed to get to the same destination.

Ablation Studies could be thought of in Travel, in Geography as "if I cannot take one, or a specific set of routes to get to a place, can I still get there?"

A simple example would be in Google Maps, where you'd like to drive somewhere, but you'd like to avoid tolls. Is the route still traversable while avoiding tolls? Well, that's an example of one constraint. In Ablation Studies, you might wish to remove a bunch of routes with whatever criteria or characteristics , i.e. muddy roads, roads that have characteristic X, roads that do not have characteristic Y, etc., etc.

Getting back to Math, specifically proofs, it would be great to create a dependency map/graph of all of them, and then try removing inputs (aka, paths to them, dependencies on other mathematical proofs/objects that they may have) and see if they are still reachable.

In software, when we desire the tightest, cleanest, source code, the above is related to refactoring.

In the future, I'd love to see dependency maps/graphs (call them whatever you will) for not just Mathematical Proofs (although I'd love to see that too!), but also in such diverse subjects as Science, Engineering, Programming/CS, and even the Law!

Because they should exist in all of those subjects!

Anyway, another great piece of work by Terrence Tao!