Opinions are like belly-buttons, everybody’s got one. But not all opinions are treated equally. There are many experts and pundits and other consultants who are paid for their prognostications.

In Brill’s Content, they used to keep a scorecard on the predictions of the political pundits who regularly appeared on shows like This Week, the McNeil/Leher Report, or Meet the Press. From what I recall, most of them were wrong most of the time.

In a new book, a psychologist from Berkeley named Philip Tetlock studies over 82,000 predictions by 84 experts between 1983 and 2003, and concludes that people who appear on television, get quoted in newspaper articles, advise governments and businesses, and participate in punditry roundtables are generally no better than the average person.

Looking to Isaiah Berlin’s metaphor, Tetlock separates the experts into foxes and hedgehogs.

“Low scorers look like hedgehogs: thinkers who ‘know one big thing’ aggressively extend the explanatory reach of that one thing into new domains, display bristly impatience with those who ‘do not get it,’ and express considerable confidence that they are already pretty proficient forecasters, at least in the long term. High scorers look like foxes: thinkers who know many small things (tricks of their trade), are skeptical of grand schemes, see explanation and prediction not as deductive exercises but rather as exercises in flexible ‘ad hocery’ that require stitching together diverse sources of information, and are rather diffident about their own forecasting prowess.”

A hedgehog, explains Louis Menard, is a person who sees affairs as determined by a single bottom-line force or principle: balance of power considerations, or clash of civilizations, or globalization. The hedgehog might have a great man of theory history, according to which the Cold War doesn’t end without Ronald Reagan. Or an actor-dispensable thesis, according to which Communism was doomed. Whatever it is, the big idea, and that idea alone, dictates the probable outcome.

For the hedgehog, therefore, predictions that fail are only “off on timing” or “almost right” but derailed slightly by some unforeseeable event. There are always little curves in the road, but in the long run they are always proved right, eventually.

Foxes, on the other hand, tend to see the world “as a shifting mixture of self-fulfilling and self-negating prophecies: self-fulfilling ones in which success breeds success, and failure breeds failure, but only up to a point, and then self-negating prophecies as people recognize that things have gone too far.”

Tetlock believes that we are suffering from a primitive attraction to deterministic and overconfident hedgehogs, and argues that society would be better off if we forced the experts to state their policy beliefs in testable forms, monitored their performance, and held them accountable.

I personally don’t know whether I’m a fox or a hedgehog, but I do think it’s funny, and a little bit scary, that “experts” are so frequently wrong about things.

In their defense, though, I do see a difference between asking someone to be a fortune teller, on the one hand, and asking him to provide insight from experience, on the other.

The goal is not to find out, in advance, what is going to happen in the future, but to gain insight so that people can make better decisions about what to do next.

Ultimately, then, it’s not so important whether the expert is a fox or a hedgehog, but the person who is listening.


[See Louis Menard, “Everybody’s an Expert” The New Yorker, Dec. 5, 2005, p.98; reviewing, Philip Tetlock, Expert Political Judgment: How Good Is It? How Can We Know? (2005).]