the first year of the tournament they made an open call for volunteers. After a simple screening, they invited thirty-two hundred to start forecasting. From those, they identified a small group of the foxiest forecasters—just bright people with wide-ranging interests and reading habits but no particular relevant background—and weighted team forecasts toward them. They destroyed the competition.
In year two, the Good Judgment Project randomly arranged the top “superforecasters” into online teams of twelve, so that they could share information and ideas. They beat the other university-run teams so badly that IARPA dropped those lesser competitors from the tournament. The volunteers drawn from the general public beat experienced intelligence analysts with access to classified data “by margins that remain classified,” according to Tetlock. (He has, though, referenced a Washington Post report indicating that the Good Judgment Project performed about 30 percent better than a collection of intelligence community analysts.)
Not only were the best forecasters foxy as individuals, they had qualities that made them particularly effective collaborators—partners in sharing information and discussing predictions. Every team member still had to make individual predictions, but the team was scored by collective performance. On average, forecasters on the small superteams became 50 percent more accurate in their individual predictions. Superteams beat the wisdom of much larger crowds—in which the predictions of a large group of people are averaged—and they also beat prediction markets, where forecasters “trade” the outcomes of future events like stocks, and the market price represents the crowd prediction.
It might seem like the complexity of predicting geopolitical and economic events would necessitate a group of narrow specialists, each bringing to the team extreme depth in one area. But it was actually the opposite. As with comic book creators and inventors patenting new technologies, in the face of uncertainty, individual breadth was critical. The foxiest forecasters were impressive alone, but together they exemplified the most lofty ideal of teams: they became more than the sum of their parts. A lot more.
* * *
• • •
A few of the qualities that make the best Good Judgment Project forecasters valuable teammates are obvious from talking to them. They are bright, but so were the hedgehog experts Tetlock started with. They toss around numbers easily, estimating this country’s poverty rate or that state’s proportion of farmland. And they have range.
Scott Eastman told me that he “never completely fit in one world.” He grew up in Oregon and competed in math and science contests, but in college he studied English literature and fine arts. He has been a bicycle mechanic, a housepainter, founder of a housepainting company, manager of a multimillion-dollar trust, a photographer, a photography teacher, a lecturer at a Romanian university—in subjects ranging from cultural anthropology to civil rights—and, most unusually, chief adviser to the mayor of Avrig, a small town in the middle of Romania. In that role, he did everything from helping integrate new technologies into the local economy to dealing with the press and participating in negotiations with Chinese business leaders.
Eastman narrates his life like a book of fables; each experience comes with a lesson. “I think that housepainting was probably one of the greatest helps,” he told me. It afforded him the chance to interact with a diverse palette of colleagues and clients, from refugees seeking asylum to Silicon Valley billionaires whom he would chat with if he had a long project working on their homes. He described it as fertile ground for collecting perspectives. But housepainting is probably not a singular education for geopolitical prediction. Eastman, like his teammates, is constantly collecting perspectives anywhere he can, always adding to his intellectual range, so any ground is fertile for him.
Eastman was uncannily accurate at predicting developments in Syria, and surprised to learn that Russia was his weak spot. He studied Russian and has a friend who was a former ambassador to Russia. “I should have every leg up there, but I saw over a large series of questions, it was one of my weakest areas,” he told me. He learned that specializing in a topic frequently did not bear fruit in the forecasts. “So if I know somebody [on the team] is a subject area expert, I am very, very happy to have access to them, in terms of asking questions and seeing what they dig up. But I’m not going to just say, ‘Okay, the biochemist said a certain drug is likely to come to market, so he must be right.’ Often if you’re too much of an insider, it’s hard