Last year, some colleagues and I invited people from around the world to compete in a tournament to predict the future. Over the course of seven months, more than 2,000 participants registered over 20,000 predictions. What would happen to the price of bitcoin? Which NHL team would win the Stanley Cup? When would Justice Stephen Breyer announce his retirement from the Supreme Court?
The results are in. The winner isn’t a specialist in economics, politics, or international affairs. He grew up in California, studied biology in college in Chicago, and now works as a business analyst at a medical group. His name is Greg Justice, and he’s only 24 years old.
The runners-up are an anonymous participant who works for an Australian government agency; Patrick Stanton, a mobile app development company founder in Washington; and Damodaran, an equity analyst in Mumbai.
We asked them how they did it, and here are their key principles and practices:
1. Lead with curiosity, not conviction. Great forecasters enjoy rethinking their views—they see it as an opportunity for discovery. In the tournament, most participants updated their forecasts once per question. Greg revised his predictions 9.5 times per question. His goal was to test his assumptions and beliefs, not to validate them. “I’m always cognizant of people’s temptation to hold unjustly bold opinions… I’m more comfortable with uncertainty,” Greg says. “I love seeing the different lenses that different people in different contexts develop for looking at the world.”
2. Look to the past for clues about the future. The patterns of the past don’t always apply to the future. But if an event had a high base rate of occurring before, it shouldn’t be discounted out of hand. As Patrick explains, “My process generally begins with collecting data and researching the historical precedent.” When he made Olympic predictions, instead of relying on his own knowledge, he examined which countries had a history of success in each sport.
3. When you make a forecast, make a list of conditions that would change your mind. Many people twist the facts to maintain their views. The best forecasters anticipate the kinds of new facts that should bend their beliefs. “I used to think changing one`s opinion is a sign of weakness and that used to make me wary of changing my stance as new data emerged,” Damodaran notes. Now “I usually ask myself under what circumstances will this prediction fail.” If you’re predicting the results of an election, the list of events that should prompt some rethinking might include the entrance of a newcomer into the race, a scandal affecting the incumbent, and the expansion versus restriction of opportunities to vote.
4. Don’t focus on being right; aim to be less wrong. When trying to anticipate the price of bitcoin, Greg realized that since it was impossible to get the precise answer, he was better off identifying the gaps in his knowledge. “It completely reoriented how I view good forecasting,” he notes. “I originally viewed forecasting as being about advancing your understanding of a question and outsmarting the crowd. Another forecaster showed me the value of devolving your model, properly assessing how much you don’t know and winning by being less wrong.”
5. Think again as the window of time for predictions to come true shrinks. Our Australian runner-up recommended “factoring in ‘time decay.’ If you make a prediction that some event will happen by the end of a month, and in half a month it still hasn't occurred, if no new information has come out your prediction about the likelihood of the event should drop—there is simply less time for it to have a chance of it occurring.”
6. Analyze your failures, not just your successes. By examining your forecasts, you can learn where your thinking is more and less reliable. “Analyzing your rationale after the question is over is helpful to understanding and improving your forecasting ability,” Patrick observes. “Honing my forecasting skill has taught me to question my own assumptions more often. Recently, I found myself having a thought and then trying to understand why I am thinking in that way. As a result, I now also seek out evidence that runs counter to my opinions more often.”
For more on the art and science of predicting the future, read Superforecasting. And to develop your skills, join a free open tournament on Good Judgment
