gas from reaching them in the first place. On the seventeen flights with no O-ring problems—akin to the seventeen Carter Racing races with no engine problems—the putty worked perfectly. Those flights provided no information whatsoever about how O-rings might fail, no matter the temperature, because the burning gas could not even get to the O-rings to cause a problem. Sometimes, however, small holes formed in the putty when the joints were assembled. On the seven flights that had O-ring issues, burning gas pushed through the holes in the protective putty and reached the O-rings. Only those seven data points were relevant to how the O-rings could be damaged or fail.
And on those seven shuttle flights—unlike gasket breaks in Carter Racing, which was the same problem every time—the O-ring issues came in two different varieties. The first: erosion. On five flights, burning gas that came shooting down the booster at ignition hit the O-rings and eroded the rubber surface. This was not a life-or-death condition. There was more than enough rubber for the O-ring to do its job. And erosion had nothing at all to do with temperature.
The second variety: blow-by. If the rubber ring did not expand instantly to fully seal the joint at ignition, burning gas “blew by” and could potentially shoot right through the booster wall. Blow-by was a life-or-death condition and, engineers would later learn, dramatically worsened when cool temperatures hardened the O-ring rubber. Two pre-Challenger flights had blow-by, but still returned home safely.
Thiokol engineers who opposed the launch on the emergency prelaunch conference call did not really have twenty-four relevant data points on O-ring failure to work with, as the Carter Racing study indicates. They did not even have seven, like the Harvard students. They had two.
Now what does the chart tell you?
Ironically, Allan McDonald, then director of the rocket-booster project at Morton Thiokol, told me, “Looking only at the relevant data points supported NASA’s [prelaunch] position that it was inconclusive.” There was no 99.4 percent certainty that was missed. The engineers were not poorly educated.
There was other important information the Thiokol engineers presented that could have helped NASA avert disaster. But it was not quantitative, so NASA managers did not accept it. The Carter Racing study teaches that the answer was available, if only engineers looked at the right numbers. In reality, the right numbers did not contain an answer at all. The Challenger decision was truly ambiguous. It was a wicked problem, rife with uncertainty, and outside of previous experience, where demanding more data actually became the problem itself.
* * *
• • •
The infamous emergency conference call convened thirty-four engineers —every manager was also an engineer—in three locations. Thiokol engineer Roger Boisjoly had personally inspected the joints after both flights with blow-by, and presented photographs from each. Following the 75-degree flight, he found a very thin streak of light gray soot beyond an O-ring in the joint, from a tiny amount of gas that had blown by before the O-ring sealed. It was nowhere close to a catastrophic problem. After the 53-degree flight, he found jet-black soot fanned out across a large swath of the joint. A lot of burning gas had blown past that time. In Boisjoly’s opinion, the reason the 53-degree launch looked so much worse was that cool conditions had hardened the O-rings and made them slow to expand and seal at ignition. He was right, but he did not have the data to prove it. “I was asked to quantify my concerns, and I said I couldn’t,” Boisjoly later testified. “I had no data to quantify it, but I did say I knew that it was away from goodness.”
Thanks to an extraordinarily strong technical culture, NASA had developed quantitatively rigorous “flight readiness reviews.” They were productively adversarial, like superforecasting team discussions. Managers grilled engineers and forced them to produce data to back up their assertions. The process had worked remarkably. The space shuttle was the most complex machine ever built, and all twenty-four flights had returned safely. But on the emergency conference call, that same quantitative culture led them astray.
On their engineers’ advice, McDonald and two Thiokol VPs on the call initially supported a no-launch decision. The Challenger had already been cleared, so this was an eleventh-hour reversal. When NASA officials asked Thiokol engineers exactly what temperature range was safe for flight, they recommended setting a limit at 53 degrees, the lower bound of previous experience.
NASA manager Larry Mulloy was flabbergasted. He thought the shuttle was supposed to be cleared