The Signal And The Noise

The Signal And The Noise explains why so many predictions end up being wrong, and how statisticians, politicians and meteorologists fall prey to masses of data, when finding important signals is mostly a matter of being cautious, diligent and, most importantly, human.

Here are some of the people who make a living off predictions: sports commentators and broadcasters, stock analysts, the people in charge of the weather forecast, pollsters, poker players, economists, marketers and, of course, fortune tellers. Sadly, most of the people in the other categories have more in common with the last one, a typical fun fair scam, than we’d like them to.

Due to making so many errors, it’s hard to trust these people after a while. But why do they make so many of them in the first place?

Looking at economists as an example, they’ll usually say things like this: “We expect GDP (gross domestic product) to grow by 2.9% next year.”

In reality though, the result of their analysis had yielded something like this: “There’s a likelihood of 90% that GDP growth will lie somewhere between 2.1% and 3.7% next year.” This is a whole other story. Instead of just picking the middle and predicting an exact number, economists should admit that the best they can do is to give an interval.

Secondly, the accuracy of that interval is often greatly overestimated – forget those 90%. More like 50%. Since 1968, the actual GDP growth percentage has fallen completely outside of the given interval half of the time. They’re not only wrong, they’re confident about it too!

Where does this hubris and false sense of judgment come from? Mostly from turning off their common sense and relying solely on statistical data. Since the dawn of the internet, we have more information available to us than ever. Over 4,000,000 economic indicators are being tracked constantly, so it feels natural to rely on the hard facts and statistical data for making predictions. Given this extreme amount of data though, critical thinking and filtering based on your own reasoning has become all the more important. With so many correlating factors, some coincidences are bound to arise, and relying on them is certain to backfire after a while. For example, for 30 years, all the data pointed towards the stock market experiencing a surge in gains for the rest of the year, if the Super Bowl winner was a team from the NFL. If a team from the AFL won, that meant losses for the stock market. This hypothesis held up 28 out of 30 years between 1967 and 1997, leaving only a 1 in 4,700,000 chance that this is a coincidence. But guess what: it IS a…