WFJ Book Club #11: The Signal and the Noise

Sometimes, when I am genuinely sick of the world and everything that those who should know better have to say about it, I have the presence of mind to thank God there's people like Nate Silver.

I should be clear, here.  I'm not one of the easily impressed who think that Silver is a genius, or that every prediction he makes is necessarily true.  As a follower of his blog fivethirtyeight.com from essentially the beginning, I'm generally aware of all his failed predictions.  But I'm also keenly aware of his successes, as well as his general orientation toward successful predictions.

I'm basically hopeless at mathematics, but Silver's approach to predicting Presidential elections is perfectly comprehensible to me in its broad outlines: he averages all the best polls together, and weights them according to their past accuracy.  His model then simulates the election hundreds of times, evaluating a candidate's odds of victory with those poll numbers, in light of a number of other factors like economic statistics and and public approval ratings.  It's a simple approach, and it works astonishingly well.  The fact that it is at all controversial (Silver has had many critics, particularly in advance of the last election) is a testament to how badly we need people like him in our public discourse.

The Signal and the Noise is not a book about politics, especially not in the noisome tradition of ideologically biased books by professional pundits.  It's a book about what makes prediction successful, whether it's applied to elections, baseball, or the path of a hurricane.  It's also about what we can reasonably expect from forecasts in inherently volatile fields, like poker, seismology, and the performance of the economy.  And it's an extremely valuable book, because it refuses to give us what we most want in those situations: certainty.

The problem is, humans are abysmally bad at evaluating trends.  The tricks our brains use to evaluate the world amidst an onslaught of data don't include a firm grasp of statistics or a proper understanding of what makes a trend.  We confuse correlation for causation, we place inordinate weight on the most recent data, and commit to a seemingly unending list of errors in trying to evaluate the future.  Simply put, we take an incredibly difficult task and try to do it in all the wrong ways.

And then along comes Silver and his book, with a few simple axioms that should be common sense, but sorrowfully aren't.  Don't confuse overwhelming data with careful analysis.  Don't assume all situations are equally predictable.  Don't mistake a confident forecast for an accurate one.  Don't assume that computers and other machines always perform infallibly.  Don't ignore data that doesn't confirm your preconceived notions.  Seriously, don't do these things.

Silver guides us through topics like sabermetrics, professional Texas hold 'em, artificial intelligence, and global climate models with equal measures of skepticism and appreciation for the genuine progress we have made at working out the future.  Half a century ago, efficient evacuations of particular regions in advance of a hurricane landing were almost impossible; thanks to better models, today they are feasible (though our lack of preparation can still bite us in the ass).  But an overconfident model can easily set itself up for failure when a statistically rare event occurs, as it inevitably must.

Simply put, there is an appropriate level of certainty for most kinds of predictions.  The best predictors work out their percentage of confidence before reaching their conclusions, and they don't inflate their certainty just to appear more convincing.  At a certain point, we have to accept what we don't know as much as what we do.

Like I said before, I'm basically hopeless at mathematics, and that includes statistics.  I'm not the one to explain all of this to you.  But The Signal and the Noise is just the book that people at large should be reading.  It isn't difficult, but it demands intellectual rigor.  It rejects blind ideology and takes the future seriously.  If you're open minded, it can change the way you think.

I've sung Silver's praises before, but I wanted to take the opportunity to do so more thoroughly today.  A book like The Signal and the Noise really is essential reading.  Taking it in won't make you a prophet, but it will change your conception of the future, and what we can say about that before it arrives.