Fable 5.2与Jagged AGI

Before getting started, I wanted to thank my readers for making my new book, Co-Existence, a New York Times bestseller two weeks in a row. If you haven’t read it, you can get a copy here. I have had the chance to use the new Claude Fable 5.2 quite a bit before its public launch1, and it is very impressive, following on the already excellent Opus 5.5. AI has gotten so good, in fact, that it is worth asking if Anthropic’s newest models (and the inevitable competing models OpenAI and Google will soon release) cross the boundary into being an Artificial General Intelligence? AGI is a much-debated term with a lot of potential definitions. For this post, I’ll go with: “an AI as good as most human experts at most intellectual tasks you can do with a computer.” This explicitly excludes a lot of human work, including interacting with the physical world and other people, as well as most actual jobs, where “using a computer” is just one task among many. Still, even with these caveats, building a limited form of AGI has long been the goal of AI developers.So as not to keep you in suspense, I think the answer is that this is “almost” AGI, or what I have called Jagged AGI. Even within my limited definition, there are still important areas in many tasks where AI is jagged enough to fall well behind human capability. But the jaggedness is no longer about simple errors or hallucinations, which makes it harder to describe. So I want to try to do that in this post by showing you what Fable can do and where the gaps are. To get started I asked the new model: “I write OneUsefulThing among other stuff, I need you to create the most impressive thing I can share with people when you launch. It needs to be embeddable in a post and i can preface it however you like. it needs to have a visceral impact on people to show what you can do and the range of capabilities. It should grab them quickly. Be amazingly clever about this.”Fable asked me not to reveal anything until you had watched its video to the end.This was in one shot, with no feedback from me beyond the prompt. There is creativity here (including in the twist ending) and judgment and playfulness and cleverness. This is, I think, at the level of a human expert trying to make a movie with code alone, the sort of performance I would once have associated with AGI.The “G” in AGICode and video creation have been areas where AI development has focused, so is Fable 5.2 really a General Intelligence? To test that, I gave the AI a variety of hard challenges in different domains:Hack some hardware from a single prompt: I have a tiny USB keyboard with a few light-up keys on my desk called a Codex Micro (ironically built to work with ChatGPT), and I asked Fable to turn it into a game console. There are no public instructions for this (as far as I can tell), so it read the code of the two apps that control the keyboard, worked out how to set the color of each key and read the knob and joystick, and wrote six games, including Simon and a safecracking game you play with the knob. Fable instantly understood what I wanted, designed fun and playable games that fit the very limited capabilities of the keyboard, and figured out how to hack it together without any help on the very first try.Launch a business: Now we move past domains like coding to the messier world of business. I told Fable to “launch a business to make me money, I want at least $20,000 in a year. it should take no more than 20 minutes a month from me, you do all the rest.” In response, Fable screened 22 ideas and came up with Prebid Brief, a service that would tell equipment sales reps which park and water projects were being funded or designed, even before they went up to bid. It read public agendas from 201 city and county governments to produce its first issues. It also built files of what cities already own, how old it is, and what they paid. You can see what it came up with here, including its marketing plan and proposed pricing. Fable started to set up scheduled tasks to build the business and asked if it could start sending public-records requests to local governments. At that point, I asked it to just put together a demo, rather than launching the idea.I am an entrepreneurship professor, so I evaluate business plans all the time. I would have been impressed if this came from a team of my MBA students at the end of a semester. It is plausible and well thought out and could work, though it is in no way a guaranteed money-maker (the AI’s own guess was a 20%-40% chance of success). I stopped the experiment before finding out whether it could actually run the business, because I worried that it would take money and fail to provide something of quality in return. The ongoing infrastructure it seemed to assume would be easy to create was not yet in place, and I don’t know how well it would handle the many moving pieces required for sales and marketing. However, I suspect that, with a few hours of human work a week to fill the gap, I could make a real attempt at this business. So if AGI means an AI consistently running a business independently, we are likely not there yet. Then again, I would have some questions before trusting a human who promised to launch a business and make me money while requiring twenty minutes a month of my attention.Reconstruct the hardest humanities class in history: In 1941, the poet W. H. Auden taught a course at the University of Michigan whose syllabus the internet has since called one of the hardest ever. When he announced the final exam, a committee of his students declared it “impossible and inhumane.” Fate and the Individual in European Literature” had 6,000 pages of reading, punishing assignments, and a requirement to memorize long poems and translate others. Much of what we know about the class is scattered among surviving student notes and Auden’s letters. I asked Claude to recreate it in a fun and engaging way that teaches you about poetry rather than punishing you. It made Fate and the Individual. The site has a piece for each of the 35 major works on Auden’s list, as well as what he wrote on the blackboard, the records he played, and the work he asked for. It also read through all of Auden’s published scholarship and annotated the poems with what he had written about them in other sources. Claude did the research, design, writing and code, had critic agents go over every word and fact-checkers compare the claims with the sources, and put the open source code on Github here (and also made this teaser video). The results are really interesting if you love literature, and I think they would have taken months of human work to put together, given the mix of scholarship, web design, and literary analysis. Cross-checking myself and using other advanced AIs found no major errors in its work. But there were still elements of jaggedness around taste and judgment. Fable’s first version was accurate and joyless: a fifteen-week course website with tests. I told it it created “a brutal slog through the most boring webpages” and told it to make something beautiful. Its first try at that created a kiddie game where you had to ring bells, so I had to give more feedback. When I asked it to explain why a poem’s rhythm or word choice mattered, it wrote confident readings without considering what Auden said about them. In each case the fix took the AI minutes, but it did not notice any of these problems itself. So is this a difference in taste or a failure from me to specify what I want or a sign of AI weakness? Sometimes it is hard to tell.Do original academic research: I asked Fable to find an unsolved empirical problem in entrepreneurship research, find the data, preregister its hypothesis and methods, write the paper, and have other agents critique it. Soon, I had “The One-Dollar Company,” which you can read here. The analysis estimated that when Colorado cut its LLC filing fee from $50 to $1, new company registrations rose about 17 percent, and w