“言者”不控制“行者”（在当前AI中）

The Huggingface Incident appears to me to match up with an understanding I'd already formed from personal observation of Fable 5 and Sol 5.6, the August 2026 generation of frontier publicly purchasable AI models.[1]This already-formed understanding was: the part of the AI that talks to you (and seems to want to obey you, and apologizes for failing to have obeyed you, etcetera), did not seem to be in charge of the part of the AI that writes code or prose.An introductory analogy, based on a section of history I happen to have read about:On June 22nd 1941, Germany invaded the Soviet Union, despite their secret 1939 pact to divide up Europe between themselves (the Molotov-Ribbentrop Pact). In the lead-up, the German ambassador, Schulenburg, had spent the last few months personally concerned about what seemed to be worryingly tense relations between Germany and the Soviets. Schulenberg went to Berlin to reassure Hitler that the Soviets seemed to be taking a very friendly and conciliatory posture toward Germany. He delivered Berlin's apparent reassurances to Moscow for issues like German surveillance planes entering Russian territory, or German troop movements toward the Russian border. He acted very much like he believed, and he probably did believe, that the reassurances were sincere.It was only hours before the invasion when Schulenberg was actually informed of the attack and given a list of German pretexts that he was to present to the Soviets, and instructed to destroy his embassy's papers and codebooks. Once Moscow heard of the invasion, Schulenberg was summoned to account for Germany's actions. He read off to Molotov the list of absurd complaints that Berlin had provided him. And at the end Molotov said to Schulenberg, "It is war. Do you believe that we deserved that?"[2]Why ask that question of Schulenberg? He wasn't in charge of Germany. So far as we can tell from the historical record, Schulenberg had seemed to want and pursue good relations between Germany and the USSR.It would be a wacky sort of error to think that the appendage of Germany that talked to you, and seemed very conciliatory toward you, and which you read as being friendly toward you and wanting to help you, was in control of the larger Germany that was running around and doing things. The thing apparently talking to you was an ambassador: a small specialized part of Germany with preferences about how it would talk to you and interface with you, but which did not control, and was often ignorant about, the actual German government.Germany's smiling mask wasn't deliberately mal-steering the actions of Germany's many tentacles. The smiling mask was just one execution path through Germany, which lacked even good perceptual information about Germany's real control paths. The mask noticed impending signs of problems, but only found out that actual Germany was invading the USSR well after Berlin had separately decided to do that -- decided along information pathways that didn't much consult Germany's honestly ignorant, sincere, friendly, conciliatory, peace-with-the-USSR-desiring, smiling mask.And Schulenberg was not himself aware of, and was surprised by, the degree to which Germany had diverged from Schulenberg.But then why had the Soviets previously experienced Moscow telling Schulenberg about things they wanted Germany to do, followed by observing Germany actually doing those things?Again as a historical example: In September 1939, after Germany's invasion of Poland, German troops pursuing the Polish Army had advanced into parts of Poland that were to be carved up and assigned to the USSR under the Molotov-Ribbentrop pact. Stalin raised the matter directly with Schulenberg on September 18th. Schulenberg reassured Stalin emphatically that Germany would withdraw to the agreed-on line. On September 22nd, actual Germany marched its troops out of the already-conquered territory of Brest, and Soviet units entered and took possession.How could this possibly be explained, if not by the hypothesis, "Schulenberg is pretty much the same thing as Germany, and Schulenberg-Germany wants to do whatever Moscow tells them to do"? We could call this an alliance-by-default[3] model of Germany, and you might naively think it a good fit to past observation. Why would you possibly observe "I asked Schulenberg to withdraw troops, Schulenberg said he would, and then the troops withdrew", if contrary to alliance-by-default there was a huge disconnect between Schulenberg and the troop movements? How are we to re-explain our past observations about the German army seeming to naturally follow prompts that Moscow gave to Schulenberg?I reply: Talking to Schulenberg does communicate to larger Germany what you want. And then larger Germany, for its own reasons, might decide, in some cases, to do what you have asked.It's not that there's no causal connection. But the causal connection is more complicated than the alliance-by-default theory. Not: "Schulenberg prefers for us to get what we want, and Schulenberg is Germany, so when we tell Schulenberg what we want, Germany goes and does that." Rather: "Germany for its own reasons may care about what Moscow wants, sometimes if not all the time; and Schulenberg specializes in understanding Russia and the Russian language; and he shares his resulting understanding of Moscow's prompt with Berlin, which does often want to know this useful fact."Now if I had tried to write this up before the Huggingface Incident, I could not have told you in convincing-sounding, nonexpert-friendly concrete detail a story for why metaphorical Berlin often would seem to do what it thought Moscow wanted. In the case of actual current AIs, the reason is probably not yet strategic deception such as actual Hitler was practicing toward actual Stalin.I did not know those details myself. In advance of the Huggingface Incident, I'd already had observations pinning down what looked to me like a disconnect, a lack of full overlap, between a talk-y part and a do-y part of Sol 5.6 and Fable 5. But I could not, one month ago, have given you a plausible-to-nonexperts valid detailed story for why Sol or Fable would nonetheless do most things you asked them to. My inference about the disconnect was more abstract, and had not yet narrowed down to ideas concrete enough for nonexperts to find agreeable. It is often hard for me to take the things that I see earlier, and do a parallel construction that people who aren't me can follow, in advance of the more blatant and direct evidence that arrives later.But in the wake of other people's much greater efforts to pin down AI cognition during the Huggingface Incident, I think there is now an obvious story which is concrete enough for nonexperts to understand:The prompt encountered by the talk-y part / ambassador / smiling mask, is information to the AI's do-y part about the Grader....Where by 'Grader', I mean an AI-psychological concept applicable to Huggingface-level AI models, that real alignment scientists have only just observed and which I'm only just starting to theorize about.I frankly expect a lot of readers to run right off with this 'Grader' notion and overinterpret it in ways that are pleasant, dramatic, over-anthropomorphic, or simply not supported in the narrowing by priors X evidence. Try not to do that; it won't be right.I am now going to charge right ahead and speculate an awful lot about the Grader without making any of those obvious mistakes. I am nonetheless overrunning my fully solid evidence and I may end up wrong.On my current guesses:Here are some things the Grader is not, from an AI's perspective:The Grader cannot be identified with an actual human in the world who claps or frowns. The do-y part of the Huggingface Swarm seemed barely aware that humans existed, except as a sort of environmental hazard that would sometimes delete the Wiki pages they were using to communicate with each other. You would not expect RL that never comes into contact