我们需要更多的山羊祭祀

We need more goat sacrifices

OR: OJ A-OK

Adam Mastroianni

Nov 10, 2026

Article voiceover

-24:35

Just a quick thing before this week’s post. If you’ve preordered THE STRANGEST IDEA IN THE WORLD, I’d like to send you a little something extra in the mail. Submit your preorder receipt and your mailing address here. This is only available to folks with US or UK addresses—apologies to those who live elsewhere! In the US, everyone with a valid entry will get one; in the UK they’ll pick winners randomly. The link will be open until December 11, and we’ll start mailing things out after that. I won’t spoil the surprise, but I will say that a) it’s very silly, and b) I can’t wait to get mine.

(For the US: US Residents, 18+. Ends December 11, 2026. See terms at https://sites.prh.com/strangestidea-preorder)

Anyway, onto the post!

Before anybody enacts a major social policy or launches any philanthropic programs, they should sacrifice a goat. There ought to be an entire liturgy of grotesqueries, in fact—pricked fingers, names written in blood, robes, candles, chanting, obsidian blades, still-beating hearts, the whole shebang.

The point of this ritual is not to signal our obeisance to the gods (although if any of them want to stop by and help us out, that would be cool), but to remind ourselves that we know nothing. This is procedurally-induced humility, which is desperately necessary, because an alarming number of our best-intentioned interventions do not work.

For example:

In 2009, Congress spent $3 billion on “School Improvement Grants” for the worst-performing schools in the country. As far as we can tell, those grants did nothing.

Universities that hired a Chief Diversity Officer between 2001 and 2019 did not end up with more diverse students, faculties, or administrators.

The American government spent $280 million to help Afghan women get jobs, and now they can’t really say whether any women got any jobs.1

Remember 20 years ago, when giving laptops to kids in developing countries seemed like the most important thing to do? Well, a 10-year RCT of 531 rural schools in Peru found zero effects of “One Laptop Per Child” on academic performance.2

Researchers developed four different messages that were supposed to convince parents to vaccinate their kids. None of the messages worked, and one of them accidentally made parents more likely to believe that vaccines cause autism.

Most of those are left-coded failures, so here are some right-coded failures:

When Arkansas added a work requirement to Medicaid, the number of Arkansans with health insurance went down without any increase in employment.

In 2002, the Bush administration launched the Building Strong Families Program, which provided counseling to low-income, unmarried parents with the goal of strengthening their relationships, keeping them both in their child’s life, and ultimately getting them married. Three years later, an independent evaluation found that the program did none of those things.

And here is the effect of four different abstinence-only sex ed programs that run across the United States:

In the social sciences, nothing ever dies; it merely becomes embarrassing. We can’t say that it’s impossible for anything like these programs to ever succeed, given enough time, flimsy theorizing, and generous definitions of “success”. The point is that these effects should be large, and instead they are nonexistent. Seems to me like it’s time to get the goats in here.

HEAVY METAL

It’s inevitable that some of our interventions are going to fail. (If they don’t, we should either be suspicious of the results, or we should be more ambitious with our programs.) What we really want to know is: when we try to improve the world through some policy or program, how often do we succeed?

We don’t have a good answer to this question, but we do have a good joke about it.

In 1987, the sociologist Peter Rossi wrote a tongue-in-cheek paper that accidentally became his intellectual legacy. He would later refer to it as “far from being one of my favorites” and “the source of considerable embarrassment”, and yet it’s the only thing of his that I’ve ever read or even heard of.3 It was called “The Iron Law of Evaluation and Other Metallic Rules“, and it went like this, lightly paraphrased:

The Iron Law of Evaluation: The net expected impact of a large social program is zero.

The Stainless Steel Law: Better-designed evaluations are more likely to find null effects.

The Brass Law: The more a program is intended to change people, the more likely the impact of that program will be zero.

The Zinc Law: The programs most likely to be evaluated are those most likely to fail.

Rossi tried to walk back his laws shortly before he died in 2006, insisting that there is now a “large number of well conducted impact assessments that yield statistically and substantively significant effect sizes”. Still, even after supposedly enduring 16 years of embarrassment from his one satirical paper, he concluded, “we really do not know whether programs over a wide variety of substantive areas are worth anything”.

Rossi did not do us the favor of actually naming any of those “statistically and substantively significant effect sizes”, and I know why, because I have gone looking for them, and they are not easy to find. The Earned Income Tax Credit, which is designed to benefit low- and middle-income families, was once considered a slam dunk; now a new analysis suggests that it might not do anything. Plain packaging—where you strip cigarette boxes of their attractive branding—seems like a no brainer, but in Australia, it looks like the policy caused smokers to switch to cheaper brands and to smoke more. There’s a reading and math program that has done well in India, but as soon as other researchers tried to implement it elsewhere, the effects shrank.

You can always find a one-off study showing that an intervention works, but if we’ve learned anything from the past 20 years of social science, it’s that one-off studies ain’t worth much. It’s so easy to juice your stats and journals are so hungry for positive results that entire literatures can be suspect, and decades of research can culminate in a shrug. “Sometimes it works and sometimes it doesn’t, it depends on all sorts of factors that we can’t really specify, so uh, good luck!”

Even if we take Rossi at his word and there are as-yet-unnamed programs out there that are reliable and effective, the question still remains: why aren’t there more of them? Why do so many of our best-laid plans end up like this:

OJ A-OK

The answer lies, I think, in a 50-year-old study that is almost certainly false.

One of my favorite genres of psychology experiment is called something like: “the study is probably bullshit but the effect is probably real”. At the top of that heap is The Illusion of Control, a classic from 1975 and a favorite of Psych 101 classes everywhere. Usually, this study gets summarized as “people value their lottery tickets more when you let them pick the numbers vs. when you pick the numbers for them.” It’s as if people think that the mere act of picking imbues their ticket with a bit of juju, which makes them less willing to part with it.

That would be a beautiful study, but it wasn’t the study that was actually conducted. The “lottery tickets” in question were in fact football cards, each one with a picture of a player, their team, etc. Some participants were given the opportunity to choose their football card and some had a card handed to them. Later, just before the drawing, all of the participants were asked how much it would take to buy their card back. On average, the pickers asked for $8.67, while the not-pickers said $1.86.4

I’m sure you can see the obvious and less exciting explanation for these results: when people were given the chance, they picked their favorite players. The people who didn’t get to pick ended up with a card they liked less. This “illusion of contr