数学遭遇的数学风暴正席卷人工智能发展

Many thanks to Max Farrens, Gabe Fried, and Thomas Larsen for pairing with me through significant parts of writing and researching this essay. You can watch or listen to me narrate this essay on YouTube, Spotify, and Apple Podcasts.If you asked me earlier this year how far away we are from superintelligence, I’d probably have said at least the better part of a decade. But over the last three months, I’ve updated my views a ton. I’ve been debating people about this question, both on and off the podcast. And I’ve been watching the progress of AI on verifiable domains and thinking about what that implies for AI research itself.I’ve now come to the view that we’re only a couple of years away, and possibly only a couple of months away, from AIs that are as good as the best humans at AI research. I’ve also come to believe that automating AI research will compress so much progress into a single year that, soon thereafter (so by 2030, but plausibly as early as next year), we’ll have AIs that are at least as good as the best human experts in every field. I know this is a wild claim, and it’s one that I myself didn’t believe even a couple of months ago. Let me walk you through how my thinking has changed.Before we get into more speculative arguments, I want to point out that we have a lot of evidence that AI is already dramatically speeding up further AI progress at the frontier.For example, METR, which is an independent evaluation organization, concluded after three months embedded inside Anthropic that Anthropic’s overall rate of progress on well-scoped tasks is 2x what it was a year ago, and that a “majority of this acceleration today is likely attributable to Anthropic’s extensive use of AI to do R&D.”OpenAI published a blog post about this in September, where they showed that the heaviest 10% of AI users among OpenAI’s researchers went from using around $1,000 a day of internal coding agents at the beginning of the summer to over $7,000 a day by the end of August.In September, Anthropic put out a similar post on its own AI uplift. They reported that, as of August, Claude “leads” more than 26% of Anthropic’s AI R&D work—up from under 1% as recently as February.1Math is a preview of what will happen to AI researchWhen I think about how big a deal it could be if AIs automate AI research, and how soon that could happen, a really important update for me has been just seeing how quickly AI capabilities in mathematics have exploded. Current AIs are slicing through well-scoped problems in mathematics like a hot knife through butter. It was only two years ago that I asked Dario why LLMs aren’t able to use their vast knowledge across so many different fields in order to connect different ideas and make new discoveries. And lo and behold, here we are: In a wholly verifiable domain like mathematics, AIs are doing this search-and-match discovery to such a great extent that hundreds of important results (which the top mathematicians and computer scientists in the world had spent decades trying to resolve) are being churned out as side quests to hype up future model releases.It’s crazy how fast we went from, in 2023, “This model can’t reliably add four-digit numbers,” to, in 2024, “This model can barely solve any problems in a high school math competition,” to, in 2025, “Oh, it can win a gold medal at the International Mathematical Olympiad,” to, earlier this year, “Wow, it’s solving open Erdős problems, but maybe just because no human has attempted to solve these particular problems, and a similar solution was already in the literature,” to, about a month ago, “Okay, it fucking solved a Millennium Prize problem in 88 hours.”There’s a line in the novel East of Eden: “And there is one sure thing about the fall of gods: they do not fall a little; they crash and shatter.”It’s true that progress has skewed disproportionately towards the verification-laden parts of math, like finding counterexamples (rather than coming up with new theories, say). Toby Ord posted a great essay about this recently. He writes, “While current AI systems are starting to automate mathematical proof, there is currently very little evidence that they can ask powerful questions or find profound new fields of mathematical enquiry.”But another way to say the same point would be: Current AI systems have solved a Millennium Prize problem, all without the ability to ask powerful questions or find profound new fields of mathematical inquiry.And this ability to solve well-scoped problems is going to be an even bigger deal for machine learning research than for mathematics, broadly construed. That’s because the goal of mathematics research is not just to prove or disprove long lists of conjectures, but to extend human understanding of mathematical concepts. Coming up with the idea of topology, group theory, or the Cartesian grid matters far more than knowing exactly how many one-unit-apart pairs you can make from n dots. We mainly care about such abstruse questions insofar as they might elicit new mathematical theories.But in AI research, the priorities are exactly reversed. AI labs mainly care about how any new algorithmic recipe empirically makes training more efficient or elicits smarter models. They don’t care about theory for its own sake, and all the progress we’ve made in deep learning over the last decade and a half has not required much new theory anyway. The equivalent of grinding through random conjectures would be enormously valuable in machine learning in a way that it just isn’t in mathematics.AI research is actually surprisingly similar to the type of mathematical progress that current AIs are disproportionately good at: unlocking a ton of low-hanging fruit by applying a combination of incredible persistence and a superhuman capacity to connect ideas from lots of adjacent domains.For example, let’s talk about the transformer architecture. My friend Sholto Douglas, who is an AI researcher, has a perennially half-finished blog post where he explains how you could have invented the transformer two years earlier than people actually did. Maybe this essay will get him to quit stalling and complete it, but in the meantime, I’ll summarize a bit of the discussion I had with him about the contents of that essay.Before the transformer paper was published in 2017, many researchers had already identified attention as a way to better parallelize training and communicate information across longer sequences. Chris Olah, who was an active researcher then, has said: “At the time, my sense is that the transformer architecture was a natural combination of many ideas (attention, residual networks) which had independently been effective, taken to quite aggressive lengths.” In other words, none of the core pieces were a surprise to researchers working in the field.So then why didn’t people invent the transformer sooner? One reason was that this new architecture was only obviously better than the state of the art once you connected half a dozen additional tweaks and innovations. And though all of these had been in the literature for years,2 no one knew how to combine the puzzle pieces. Doing so required research taste, of course, but also tons and tons of small experiments.But what I’ve just described above—persistently and intelligently matching dozens of different combinations of ideas—is exactly the kind of capability that AIs have shown themselves disproportionately skilled at in their contributions to mathematics.This is all to say, even the transformer, which is arguably the biggest breakthrough in ML since AlexNet kicked off the deep learning revolution in 2012, is the kind of thing that Claude 7 could invent if Chris Olah asked it to try hard to get this attention thing to work better.I want to clarify that some parts of AI research are, in fact, less verifiable than math. That’s why OpenAI hasn’t fired all of its researchers, despite having a model that can autonomously solve Millennium Prize problems. Bu