The advertisements above do not necessarily reflect the views of this blog, its authors, or host.

Failing at Prediction, Succeeding at Bias

In Isaac Asimov’s Foundation series, the future can be predicted through a new method known as psychohistory. It’s a concept that turns up quite a lot — not only in science-fiction but from every so-called analyst, expert, advisor, talking head, and pundit across the spectrum of armchair philosophers and industry insiders.

We’re not talking about psychics here, but good old fashioned datahounds. Surely this suggests a more effective compass to predicting trends, right? Isn’t there a science to extrapolation that is more effective than reading the pattern of coffee grinds at the bottom of your Turkish coffee? Economics and technology and politics are not like trying to divine how many angels can stand on pinheads, after all.

Michael Shermer has a fine and sobering article today in the Huffington Post on how economists and Nostradamus aren’t so different with their respective track records:

 

At Skeptic magazine we routinely publish articles about the failed predictions of soothsayers, astrologers, tarot-card readers, palm-readers, and psychics of all stripes. But frankly scientists are not much better, especially in the social sciences where we depend on predictions of psychologists, sociologists, and most notably economists.

 

Shermer highlights the stealthy, insidious logical fallacy that is confirmation bias, as the gremlin in our would-be psychohistorical acumen:

One of the smartest and most deeply read historians of the 20th century, Arnold Toynbee, was spectacularly wrong in his blockbuster A Study of History, in which he thought he had identified a challenge-and-response cyclical pattern that all civilizations follow: birth, growth, expansion, empire, and disintegration. Starting with Greece and Rome, Toynbee dug through the historical record to find confirmatory evidence for his theory (culminating in his call for America to rise to the challenge of its alleged mid-century moral decline). You would think he would have taken heed from his inspiration, the German historian Oswald Spengler, who erroneously predicted the “decline of the west” in the 1920s. But that’s not how the mind works.

Why did Toynbee believe his own theory in the teeth of contradictory evidence presented by other historians? Because of the confirmation bias, which our brains employ to reinforce what we already believe while ignoring disconfirming data.

 

It isn’t merely the flashy discredited utopias that earlier ages expected — like flying cars or airships or lunar bases by 1999. The world is an astonishingly complex place, while our brains are constantly trying to simplify it into a distinct and candy-coated set of variables that will result in a conclusion ready-to-fit on a poster or soundbyte.

Writes Shermer:

Being deeply knowledgeable on one subject narrows one’s focus and increases confidence, but it also blurs dissenting views until they are no longer visible, thereby transforming data collection into bias confirmation and morphing self-deception into self-assurance. 

 

Not bad advice for the new year.

Leave a Reply