Sunday, February 24, 2013

WFJ Book Club #11: The Signal and the Noise

Sometimes, when I am genuinely sick of the world and everything that those who should know better have to say about it, I have the presence of mind to thank God there's people like Nate Silver.

I should be clear, here.  I'm not one of the easily impressed who think that Silver is a genius, or that every prediction he makes is necessarily true.  As a follower of his blog from essentially the beginning, I'm generally aware of all his failed predictions.  But I'm also keenly aware of his successes, as well as his general orientation toward successful predictions.

I'm basically hopeless at mathematics, but Silver's approach to predicting Presidential elections is perfectly comprehensible to me in its broad outlines: he averages all the best polls together, and weights them according to their past accuracy.  His model then simulates the election hundreds of times, evaluating a candidate's odds of victory with those poll numbers, in light of a number of other factors like economic statistics and and public approval ratings.  It's a simple approach, and it works astonishingly well.  The fact that it is at all controversial (Silver has had many critics, particularly in advance of the last election) is a testament to how badly we need people like him in our public discourse.

The Signal and the Noise is not a book about politics, especially not in the noisome tradition of ideologically biased books by professional pundits.  It's a book about what makes prediction successful, whether it's applied to elections, baseball, or the path of a hurricane.  It's also about what we can reasonably expect from forecasts in inherently volatile fields, like poker, seismology, and the performance of the economy.  And it's an extremely valuable book, because it refuses to give us what we most want in those situations: certainty.

The problem is, humans are abysmally bad at evaluating trends.  The tricks our brains use to evaluate the world amidst an onslaught of data don't include a firm grasp of statistics or a proper understanding of what makes a trend.  We confuse correlation for causation, we place inordinate weight on the most recent data, and commit to a seemingly unending list of errors in trying to evaluate the future.  Simply put, we take an incredibly difficult task and try to do it in all the wrong ways.

And then along comes Silver and his book, with a few simple axioms that should be common sense, but sorrowfully aren't.  Don't confuse overwhelming data with careful analysis.  Don't assume all situations are equally predictable.  Don't mistake a confident forecast for an accurate one.  Don't assume that computers and other machines always perform infallibly.  Don't ignore data that doesn't confirm your preconceived notions.  Seriously, don't do these things.

Silver guides us through topics like sabermetrics, professional Texas hold 'em, artificial intelligence, and global climate models with equal measures of skepticism and appreciation for the genuine progress we have made at working out the future.  Half a century ago, efficient evacuations of particular regions in advance of a hurricane landing were almost impossible; thanks to better models, today they are feasible (though our lack of preparation can still bite us in the ass).  But an overconfident model can easily set itself up for failure when a statistically rare event occurs, as it inevitably must.

Simply put, there is an appropriate level of certainty for most kinds of predictions.  The best predictors work out their percentage of confidence before reaching their conclusions, and they don't inflate their certainty just to appear more convincing.  At a certain point, we have to accept what we don't know as much as what we do.

Like I said before, I'm basically hopeless at mathematics, and that includes statistics.  I'm not the one to explain all of this to you.  But The Signal and the Noise is just the book that people at large should be reading.  It isn't difficult, but it demands intellectual rigor.  It rejects blind ideology and takes the future seriously.  If you're open minded, it can change the way you think.

I've sung Silver's praises before, but I wanted to take the opportunity to do so more thoroughly today.  A book like The Signal and the Noise really is essential reading.  Taking it in won't make you a prophet, but it will change your conception of the future, and what we can say about that before it arrives.

1 comment:

  1. There is a school of thought that states that time is an illusion, albeit a convenient one as it keeps everything in our reality from occurring at once. But past, present, and future are claimed by believers in this philosophy to all coexist in the eternal now. I do not bring this up to belittle Silver's accomplishments but to place them within a different and larger context. If the future already exists, it seems to me at least, that all possible futures must already exist. This is in fact not just the wild imaginative dreaming of certain spiritual communities but a form of the many worlds hypothesis of contemporary physics as well. The question remains: what draws a single pattern of reality from the infinite number possible into our conveniently imagined present. One school of quantum physics claims that in the case of experimental procedures human measurement determines the outcome and measurement is itself determined by our minds. Perhaps the Universe, or rather the Multiverse as the physicists are now wont to refer to ultimate reality, is consciousness rather than matter. Wouldn't it be the ultimate shocker though to discover that we don't all live in the same present, that our lives actually just intersect at certain common points that we share, creating the illusion that we all live in one universe and will all share a common future, one that Silver will likely be better at predicting than most of us. Having not yet read his book I am not qualified to evaluate Silver's approach to predicting the future. I do though rather like the Three Laws of Prediction formulated by British author Arthur C. Clarke. They are:

    1. When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.

    2. The only way of discovering the limits of the possible is to venture a little way past them into the impossible.

    3. Any sufficiently advanced technology is indistinguishable from magic.

    Gotta love that Third Law.