Anchoring effect

Evidence

Tversky and Kahneman (1974) had a wheel of fortune land on a number between 0 and 100 in the participants’ presence. Participants first said whether the quantity sought was above or below that number, then estimated it starting from there. For the share of African countries in the United Nations, the median estimate was 25 percent after a starting value of 10 and 45 percent after a starting value of 65 — although everyone could see the number came from a wheel of fortune. A payoff for accuracy did not change this.

This famous demonstration is thinner than its reputation suggests. The article is a review piece in Science; for the anchoring experiment it reports neither sample sizes nor measures of dispersion nor significance tests, only the two medians. A second demonstration in the same article works without any given number: high-school students had five seconds to estimate a product of eight factors. Computed downwards from 8 the median was 2,250, upwards from 1 it was 512. The correct answer is 40,320.

What was replicated, however, was a different procedure. Klein et al. (2014) tested four estimation questions from Jacowitz and Kahneman (1995): distance from San Francisco to New York, population of Chicago, height of Mount Everest, babies born per day. All four replicated in all 36 samples and markedly more strongly than in the original (d = 1.17 to 2.42 versus 0.93). The d measures how large an effect is — 0.8 already counts as large, and these values sit far above that.

The authors later qualified that strength themselves. Their implementation collapsed a two-step question into one step, and their own comparison test found that this version produces larger effects than the original. Comparing first and estimating second is already the 1974 procedure.

Limits

None of the four Many Labs questions involved a price; they concerned distances, populations, mountain heights and birth rates. Of the 13 effects tested, anchoring also varied most across samples. And in Many Labs 2 a related case failed: whether a product was called P97 or P17 no longer changed the sales estimates of 6,826 participants (d = 0.04, 95% CI −0.01 to 0.09; original 0.30). A nearby number is therefore not enough.

Fudenberg, Levine and Maniadis (2012) repeated the experiment of Ariely, Loewenstein and Prelec (2003) four times in the lab and found very little: for everyday goods one of six correlations was significant, for lotteries none. The effect was weakest precisely for willingness to pay — the quantity a price is about. Their own conclusion is nonetheless balanced: real, but quantitatively small. Field studies on the effect on pricing pages are missing.