Experts Are the Worst Forecasters

It’s true — experts are the worst forecasters.  But don’t just take our word for it. In this blog we review a recent book by a Nobel Prize winner and explore what it says about uncertainty, prediction and strategic anticipation.  And also the oversize confidence we have in the prognostications of so-called experts, especially when the strategic horizon stretches out beyond the very near term. 

After decades of trying to help clients anticipate the widest range of plausible possible future events, the following quote rings quite true to us: “Those who know more forecast very slightly better than those who know less.  But those with the most knowledge are often least reliable.”

Daniel Kahneman, a winner of the 2002 Nobel Prize in Economic Sciences, published this conclusion in his book Thinking, Fast and Slow after extensive research including numerous statistical analyses.

One of his sources was work done by Philip Tetlock, a psychologist at the University of Pennsylvania’s Wharton School who gathered more than 80,000 predictions from experts on political and economic trends.  When Tetlock compared these predictions to what actually occurred he found that these experts had performed poorly.  In his words: “The results were devastating.  The experts performed worse than they would have if they had simply assigned equal probabilities to … the outcomes.

In other words, people who spend their time, and earn their living, studying a particular topic produce poorer predictions than dart-throwing monkeys who would have distributed their choices evenly…”

A look into why experts are the worst forecasters

Kahneman believes that forecast errors are inevitable because of the inherent unpredictability of the world.  Unfortunately, this conclusion flies in the face of the apparently natural assumption that a thorough, deep knowledge of the forces currently at work must confer a greater ability to understand how the future will evolve.  Major corporations and government agencies routinely base their strategies on an assumption that they know what’s coming.

Kahneman’s work provides an eloquent argument that this is folly. Expertise can be a seductive sort of tunnel vision, tempting us to stare through a telescope at one point in space, just when you really need to be scanning the entire sky with the naked eye.

Since the evidence shows conclusively that even the foremost experts (maybe even especially the foremost experts) cannot successfully predict the future, it stands to reason that strategies should be devised to manage the uncertainty rather than assume it away.  Rather than try to predict the single future that will occur, scenario planning presents a range of plausible futures that, as a set, present a plausible range of what could occur.  As a result, organizations that use scenario planning to create their strategies often report that they have a sense of déjà vu as the future unfolds– they have already rehearsed it.

Thinking, Fast and Slow systematically uncovers many other similar “errors of intuition.”  For example, we tend to revise what we originally believed once we know what has actually occured.  Experts go even further, rationalizing errors with phrases like “my prediction was only off on its timing” or it “would have been right except for an event that no one foresaw.”  Which contributes to the popular impression that experts are the worst forecasters.

But are they really the worst?

Experts can offer valuable insights about why and how things have occurred.  And we should all reach out to experts to understand the dynamics of their particular fields of expertise.  But in our opinion, scenario planning using multiple futures is a better way to anticipate what might happen than a single, detailed, expert forecast that just may be worse than a dart-throwing monkey. This is our proud professional claim to you as scenario consultants: “We aren’t experts…but we are just a bit better than a dart-throwing monkey!”