Here is an example of how the average person might make a prediction. Someone buys an old, used car. A friend says, "You just wasted your money. It’ll break down within the year (prediction). You would have done better buying a new one." We are not only good at making a lot of predictions we are also good at covering our tracks when the prediction doesn’t pan out. If the car is purring along after a year, the friend might say, "Of course, it lasted longer than a year. You took better care of it than most people." After two years, the friend might be saying, "Well, I guess you just got lucky and got a better-than-average care." Very few people would say, "I was flat wrong about my prediction because your car is still running."
Of course, we mortals act like this. That’s because we are not professional prognosticators. We all know that the pros do a much better job at making predictions. That is why they are interviewed for the knowledge they have to share.
The study also found that the more the experts knew the less reliable they were at guessing what would happen to the world in the future. Dr. Tetlock explained this by saying, "We reach the point of diminishing marginal predictive returns for knowledge disconcertingly quickly. . . . In this age of academic hyperspecialization, there is no reason for supposing that contributors to top journals—distinguished political scientists, area study specialists, economists, and so on—are any better than journalists or attentive readers of the New York Times in ‘reading’ emerging situations." This study also showed that forecasters who were well known would have exaggerated confidence in their forecasts. He concluded that "Experts in demand were more overconfident than their colleagues who eked out existences far from the limelight."
This is terrible. Why do our experts do so poorly? The best guess is based on human nature. People love to be right but hate being wrong. So, when we make a guess (forecast) we "fall in love" with our choice. No matter how it turns out we stick with the choice, justifying it right to the bitter end. This seems to be exactly what plagued the experts.
You would think that really smart people would learn from their mistakes. Even though experts are smarter than most of us, they are also human and subject to the thinking errors that affect all humans. One (of the many) thinking errors that might explain why smart people don’t learn from their failed predictions is called confirmation bias. This error describes how most people tend to dismiss new information that doesn’t fit with what they already believe. So if experts really believe their predictions actually come true most of the time, then they will automatically refuse to accept the failures.
This also happens to people who believe they have special powers to see into the future, like psychics. Years ago, a young woman contacted me to tell me that she had precognitive dreams. For example, she would dream about a terrible tragedy like an airplane crash prior to it actually happening. After she gave me some very impressive examples, I suggested she write down the details of her next tragedy dream immediately upon waking in the morning. After writing down the details, she was to seal the paper in an envelope and mail it to me immediately. She called me a few weeks later and excitedly told me about a dream she had about an accident that had killed many people. When she read about it a few days later all the details in her dream matched the newspaper account. I had her come to my office and we opened her letter together. She was surprised, shocked and disappointed with what she had written. The information was not even close to the event in the newspaper. Her mind had tricked her with something called retrograde memory. As she read about the accident in the newspaper the information went into that part of her brain that held long-term memory. As she tried to recall what she had dreamed, the dream details were replaced with the actually details.
Another smart person is more humble. Paul Krugman, the 2008 Nobel Prize winner in economics and a professor at Princeton University, says that "making predictions is hard … especially about the future, and sometimes about the recent past." Even that great philosopher about life, Yogi Berra, agreed with Dr. Krugman when he said, "It’s tough to make predictions, especially about the future."
If you are interested in the book from which much of this material was taken, it is called "Expert Political Judgment: How Good is it? How Can we Know?" (2005), Princeton University Press.