Range - David Epstein Page 0,44
like Kahneman’s curriculum-building team, which decided that thanks to its roster of experts it would certainly not encounter the same delays as did other groups. Flyvbjerg studied a project to build a tram system in Scotland, in which an outside consulting team actually went through an analogy process akin to what the private equity investors were instructed to do. They ignored specifics of the project at hand and focused on others with structural similarities. The consulting team saw that the project group had made a rigorous analysis using all of the details of the work to be done. And yet, using analogies to separate projects, the consulting team concluded that the cost projection of £320 million (more than $400 million) was probably a massive underestimate. When the tram opened three years late, it was headed toward £1 billion. After that, other UK infrastructure projects began implementing outside-view approaches, essentially forcing managers to make analogies to many outside projects of the past.
Following their private-equity-investor experiment, the outside-view researchers turned to the movie business, a notoriously uncertain realm with high risk, high reward, and a huge store of data on actual outcomes. They wondered if forcing analogical thinking on moviegoers could lead to accurate forecasts of film success. They started by giving hundreds of movie fans basic film information—lead actor names, the promotional poster, and a synopsis—for an upcoming release. At the time, those included Wedding Crashers, Fantastic Four, Deuce Bigalow: European Gigolo, and others. The moviegoers were also given a list of forty older movies, and asked to score how well each one probably served as an analogy to each upcoming release. The researchers used those similarity scores (and a little basic film information, like whether it was a sequel) to predict the eventual revenue of the upcoming releases. They pitted those predictions against a mathematical model stuffed with information about seventeen hundred past movies and each upcoming film, including genre, budget, star actors, release year, and whether it was a holiday release. Even without all that detailed information, the revenue predictions that used moviegoer analogy scores were vastly better. The moviegoer-analogies forecast performed better on fifteen of nineteen upcoming releases. Using the moviegoers’ analogies gave revenue projections that were less than 4 percent off for War of the Worlds, Bewitched, and Red Eye, and 1.7 percent off for Deuce Bigalow: European Gigolo.
Netflix came to a similar conclusion for improving its recommendation algorithm. Decoding movies’ traits to figure out what you like was very complex and less accurate than simply analogizing you to many other customers with similar viewing histories. Instead of predicting what you might like, they examine who you are like, and the complexity is captured therein.
Interestingly, if the researchers used only the single film that the movie fans ranked as most analogous to the new release, predictive power collapsed. What seemed like the single best analogy did not do well on its own. Using a full “reference class” of analogies—the pillar of the outside view—was immensely more accurate.
Think back to chapter 1, to the types of intuitive experts that Gary Klein studied in kind learning environments, like chess masters and firefighters. Rather than beginning by generating options, they leap to a decision based on pattern recognition of surface features. They may then evaluate it, if they have time, but often stick with it. This time will probably be like the last time, so extensive narrow experience works. Generating new ideas or facing novel problems with high uncertainty is nothing like that. Evaluating an array of options before letting intuition reign is a trick for the wicked world.
In another experiment, Lovallo and his collaborator Ferdinand Dubin asked 150 business students to generate strategies to help the fictitious Mickey Company, which was struggling with its computer mouse business in Australia and China. After business students learned about the company’s challenges, they were told to write down all the strategies they could think of to try to improve Mickey’s position.
Lovallo and Dubin gave some students one or more analogies in their instructions. (For example: “The profile of Nike Inc. and McDonald’s Corp. may be helpful to supplement your recommendations but should not limit them.”) Other students got none. The students prompted with one analogy came up with more strategies than those given no analogies, and students given multiple analogies came up with more strategies than those reminded only of one. And the more distant the analogy, the better it was for idea generation. Students who were pointed to Nike and McDonald’s