Why pundits always get it wrong

Paul Raven @ 02-03-2009

Bill O'Reilly motivational poster spoofHave you ever wondered why the talking heads on television are so often utterly wrong in their predictions about the future?

According to a research psychologist from Stanford University, it’s because the people the media tends to hire as pundits are the sort of people who are psychologically predisposed to making predictions based more on their own beliefs than any rational assessment of the situation:

At first, Tetlock’s ongoing study of 82,361 predictions by 284 pundits (most but not all of them American) came up empty. He initially looked at whether accuracy was related to having a Ph.D., being an economist or political scientist rather than a blowhard journalist, having policy experience or access to classified information, or being a realist or neocon, liberal or conservative. The answers were no on all counts. The best predictor, in a backward sort of way, was fame: the more feted by the media, the worse a pundit’s accuracy. And therein lay Tetlock’s first clue. The media’s preferred pundits are forceful, confident and decisive, not tentative and balanced. They are, in short, hedgehogs, not foxes.

That bestiary comes from the political philosopher Isaiah Berlin, who in 1953 argued that hedgehogs “know one big thing.” They apply that one thing (for instance, that ethnicity and language are primal; ergo, any country that contains many ethnic groups will break up) everywhere, express supreme confidence in their forecasts, dismiss opposing views and are drawn to top-down arguments deduced from that Big Idea. Foxes, in contrast, “know many things,” as Berlin put it. They consider competing views, make bottom-up inductive arguments from an array of facts and doubt the power of Big Ideas.

Intriguing, no? But while foxes are arguably better than hedgehogs, here’s the sobering conclusion:

If there are three possibilities (say, that China will experience more, less or the same amount of civil unrest), throwing darts at targets representing each one produces a forecast more accurate than most pundits’. Simply extrapolating from recent data on, say, economic output does even better.

Unfortunately, extrapolated data doesn’t make for such good television… or blog posts, for that matter. Will the decentralisation of current affairs newscasting make this situation better or worse? [image by silas216]

I wonder if we could classify science fiction writers using the same dualism?

Be Sociable, Share!

One Response to “Why pundits always get it wrong”

  1. Tom James says:

    Chris Dillow comments on the strength of ego over rational empiricism on the subject of home schooling:

    One approach assumes that the resources of one’s own ego – prejudice and anecdote – are sufficient to settle a question. You know that home schooling is good (or bad), that there are no barriers to social mobility, that single parents are scroungers, that Israel (or Hamas) is right. And so on. So you just abuse the other side and wallow in your self-righteousness.

    The other approach, though, is to ask: what exactly is the evidence? What would count as good evidence?

    This latter approach, though, is a minority one. I suspect that we live not in an age of science and empiricism, but rather in an age of ego.

    As writing is a fairly egotistical pursuit anyway ( though I have no empirical evidence for that ! ;) ) I’d say SF writers can predict what they please: the problems emerge when you start taking hedgehog pundits seriously and ignore empirical evidence in making policy.