Which persuasion science survived the replication crisis?

Sales playbooks lean on a dozen famous psychology experiments: the cookie jar that made scarce cookies taste better, the hotel towels reused because "most guests do", the jam stand where fewer flavors sold more jam. We checked each of them against the largest and most careful tests run since.

Of 12 persuasion effects, 4 held up, 6 gave mixed results, 1 failed and 1 has never been tested in a large study. Where a famous study and a large test could be compared, the large test found an effect a median 25% of the original size, and it was smaller for 6 of the 7 effects we could compare.

How we graded

We wrote down our predictions and grading rules before any search. We logged 836 queries across three literature databases, followed citations and included 132 sources. A second reviewer checked every number against its source.

Only strong tests could set a grade: replications run across many labs with at least 1,000 people, field experiments with at least 10,000 people or a public pre-registration, meta-analyses pooling at least 10 studies, and pre-registered replications with at least 500 people.

An effect is Replicated when every top-level test finds it in the original direction, with a confidence interval that excludes zero. It is Failed when they all find nothing, and Mixed when they disagree or when only meta-analyses without a publication-bias correction exist. Each effect is graded on the version a sales team would use. Effect sizes are in Cohen's d, a standard unit where 0.2 is small, 0.5 medium and 0.8 large.

What held up

Anchoring is the clear survivor. Across 36 samples in the Many Labs 1 replication, a high or low number planted before an estimate moved people's answers on all four test questions, by at least d = 1.17. That is larger than the classic study's 0.93, though the two used different designs. The useful version for sales is a plausible, relevant number such as a list price or a first offer. Numbers that only appear in passing, like a figure in a restaurant's name, did not move judgments in large tests. Anchoring is also the only effect we rate as transferring well to B2B buying.

Reciprocity held up in two strong tests. Swiss survey participants who got an unexpected voucher were 5.3 percentage points more likely to take part in the next survey round, across 17,332 people. A pre-registered rerun of the 1975 door-in-the-face study also worked. The pooled effect is d = 0.24, against 0.41 in the classic study.

Choice overload held up in direction but shrank to almost nothing. In the jam study, 30% of tasters at the small display bought jam, against 3% at the large one. In a test with 1.6 million shoppers on an online retail platform, recommending two products rather than four raised purchases from 4.29% to 5.13%. In the same experiment, one product sold worse than four (3.08%). The large-test effect is 7% of the jam study's.

Name personalization held up on one large email experiment: using the recipient's name raised sign-ups from about 12.5% to 15.6%. A 2023 replication of the best-known subject-line test found no effect (56.9% vs 58.5% opens). The gain may be fading now that every sender merges in first names.

Mixed results

Social proof is real but small. Of 12 large or pre-registered field tests of "most people do this" messages, 7 found an effect and 5 did not. The best convertible estimate, d = 0.12, is close to the hotel-towel study's 0.18, but both are very small. It is worth a few percent, not a transformation.

For scarcity and foot-in-the-door we found no large field experiment or multi-lab replication. Their meta-analyses find small effects (d = 0.28 and 0.44), but none corrects for publication bias, the tendency for null results to go unpublished. Scarcity's estimate is 25% of the cookie-jar study's.

Loss-aversion framing, telling people what they stand to lose rather than gain, barely moves anyone. Across three large tests the pooled effect is d = 0.02. The largest ran in 84 countries and found nothing reliable; one UK test found a small gain.

Priming and ego depletion, the replication crisis's best-known casualties, did badly. The two multi-lab priming replications found nothing: the best estimate is d = -0.04, against 1.04 in the classic elderly-walking study. For ego depletion, the idea that willpower runs out with use, the large tests averaged d = 0.08, 4% of the original 1.78. They stay Mixed rather than Failed on narrow grounds. For priming, two meta-analyses still find a small effect after correcting for publication bias; for ego depletion, one of three large replications found a small one.

What failed, and what was never tested

The decoy effect, adding an unattractive option to make another look better, did not survive its one large real-world test. On a flight-booking site with more than 140,000 search sessions, decoys did not shift bookings toward the option they were meant to promote (d = 0.00). It still shows up in hypothetical online choices.

The "because" effect, where any reason, even an empty one, is said to raise compliance, has never been tested in a large study as far as our search could find. The 1978 copy-machine study remains the main evidence.

How sure are we?

Four grades turn on a single judgment call. Anchoring would become Mixed if a set of pay-what-you-want field studies, public only as an abstract, counted as a null test. Choice overload would become Mixed if the one-versus-four comparison replaced two-versus-four. Loss framing's grade rests on its one positive test, and name personalization's on one email experiment. With every flagged call set against the effects, the count falls to 1 Replicated, 7 Mixed, 3 Failed and 1 untested.

The 25% median depends on the comparison studies. Using the 1981 risky-choice study as the classic for loss framing gives 16%, and a cruder formula for the cookie-jar study gives 42%. Either way, the large tests find well under half of the original effects.

Most of the evidence comes from consumers, students and survey respondents, not professional buyers. Some sources could be read only as abstracts, and most meta-analyses do not correct for publication bias, which inflates effects.

What this means for B2B sales teams

• Use anchors you can defend. A list price, a first offer or a reference deal. It is the one effect that held up large and transfers to B2B buying.
• Give something first, and expect a modest lift. Reciprocity works, but the effect is small.
• Use the buyer's name, but don't count on it. The one large email test found a gain; a newer replication found none.
• Social proof and fewer, clearer options may help a little. Think a few percent, not a new motion.
• Don't build a sales motion on priming, decoy pricing tiers or "decision fatigue". The strongest tests found little or nothing.