Book Review: Superforecasting

Superforecasting: The Art and Science of Prediction
Philip E. Tetlock and Dan Gardner
Cognitive science

I think there's about a 60-65% chance that I'd be a pretty good forecaster. I read widely, I'm comfortable with numbers (though not a mathematician), I'm good at thinking probabilistically, I try to be skeptical towards my preconceptions, my day job involves breaking big problems into tractable sub-problems, and I'm a fox rather than a hedgehog ("the fox knows many things, while the hedgehog knows one big thing"). On the other hand, I'm possibly a tiny bit intellectually arrogant--making me too slow to consult other people's views--and I'm probably not as good at self-criticism as I think I am.

That, in a nutshell, is the format that Philip Tetlock presents and defends in this book. Tetlock is known for having demonstrated that the views of experts and pundits are laregely worthless--indeed, the more certainty a pundit expresses, the less accurate his predictions are. In Superforecasting, he take the contrary tack. rather than focus on what makes bad forecasts, he analyzes on the truly astonishing ability of some people to make really good forecasts. And by "really good" here I mean "better than professionals, better than think tanks, and better than computer models."

This is fascinating stuff, and the book is very well written. As usual, the people who should read it are the people who are least likely to do so, or to believe it if they do. Also as usual, I reserve some degree of judgment on the basis of statistics; Tetlock addresses the notion that some degree of luck is involved, but without mathematical detail. That aside, I highly recommend this book. Whether you're interested in why so much decision-making in bad, or want practical ways to improve your own forecasting, Superforecasting is something of a revelation.

Update, February 13: Here's a good short article by Tetlock.

---

The bible of the growing literature in this area is Daniel Kahneman's Thinking, Fast and Slow. A similarly-titled book from 2008, Super Crunchers, deals with algorithmic prediction; it's less deep, but an interesting companion piece--particularly in light of recent research that suggests that human-computer partnerships do better than either does alone.