behind the news

Don’t treat worst-case scenarios as facts

How to avoid overhyping Ebola projections
October 7, 2014

On September 23, the Centers for Disease Control and Prevention released a statistical forecast of how far the Ebola virus could spread in Liberia and Sierra Leone in the next four months. In the best-case scenario, as many as 8,000 people in the two nations could be infected by September 30, a 72.5-percent increase in just a week. In the worst case, that number could be as high as 21,000. Unless the disease was effectively contained, as many as 1.4 million people could be infected by January.

Even though these computer-modeled projections were based on a count done in August using infection numbers that were likely underreported, and though they used an epidemiological model that didn’t take into account factors like the number of people that each Ebola patient infects, the idea that 1.4 million people could potentially be infected crackled through the news like a brush fire.

By the end of the day, headlines like, “Ebola Cases Could reach 1.4 million Within Four Months, CDC Estimates,” “Every Second Counts: New Ebola Report Predicts Huge Spike Without Action,” and “CDC: Ebola Could Infect 1.4 Million in Liberia and Sierra Leone by January,” were published respectively by The New York Times, NBC, and The Washington Post, whose story began with this alarming lede:

The Ebola epidemic in West Africa, already ghastly, could get worse by orders of magnitude, killing hundreds of thousands of people and embedding itself in the human population for years to come, according to two worst-case scenarios from scientists studying the historic outbreak.

Whether these hyperbolic headlines were about mobilizing concern, or just about getting more clicks is anyone’s guess. What’s certain is that they were framed around what could happen, even if it was an unlikely scenario: By September 28, as it turned out, the number of infections was nowhere near 21,000. If we go by a maxim laid out by George E.P. Wilcox, the British statistician who said that “all models are wrong, but some are useful,” the fact that this scenario did not play out was not surprising. But the media covered the story as though the worst-case outcome was quite probable. Framing a story around a potential situation that is based on unreliable projection is an exercise that doesn’t belong in a journalist’s toolbox.

But that didn’t stop The Wall Street Journal from releasing a video, “Stopping Ebola from Striking 1.4 Million People,” that all but assumes the worst-case estimates are on track to play out. “If the scenarios provided to us by the CDC come true, we’re talking about nothing less than the potential meltdown of this continent,” Jim Yong Kim, the World Bank’s president, says in the video.

Sign up for CJR's daily email

Some of the coverage did carry caveats. The New York Times piece was upfront about the model’s limitations, noting in its second paragraph that these scenarios were hypothetical at best. Then it quoted Thomas Frieden, the CDC director, who made a surprising confession: “My gut feeling is, the actions we’re taking now are going to make that worst-case scenario not come to pass. But it’s important to understand that it could happen.” Frieden’s quote–unscientific, a little bit paranoid–underlines how little we actually know about Ebola; in this particular instance, even the director of the CDC was relying on his gut.

Not everybody took the report at face value. On September 25, FiveThirtyEight published a piece titled, “We’ll Probably Never Know if The CDC’s Grim Forecast was Accurate,” in which Carl Bialik looked into the crystal ball and saw it for what it was: “The paper doesn’t include a measure of its own uncertainty, which Andreas Handel, an assistant professor in the department of epidemiology and biostatistics at the University of Georgia’s College of Public Health, would have liked to see.” Baliak continues: “Handel, whose research group models the spread of infectious diseases, likened the field to weather forecasting: ‘We trust weather forecasts up to a few days ahead, but would not give much weight to a forecast that’s months in the future,’ [Handel] said. ‘It’s similar for these kind of outbreak forecasts.'” Bialik ends the piece by saying such numbers are still useful if they get people to pay attention to Ebola.

September 3o came and went, and the number of infections rose to 7,178–a grisly, dispiriting number, but a far cry from 21,000, or even the lowball projection of 8,000 new cases. By then, though, the language of the CDC report was embedded in the discourse, like a stubborn semantic bur. On October 2, one day after a Liberian man in Dallas was diagnosed with Ebola, the word “pandemic” appeared in a headline, alongside the world “nuclear,” above an opening paragraph that described how the International Development Secretary had been shocked by the CDCs estimate that “1.4 million people may be infected by the virus if it continues to go unchecked.” Even though this article was something of an outlier, it was a good example of just how contagious a hypothetical can be.

“The outbreak is not going to become a pandemic,” Stephen Morse, a professor of epidemiology at the Columbia University Mailman School of Public Health, told me (although he added that he couldn’t say this with absolute certainty, and that scientists always have to hedge because it’s possible the moon will fall out of the sky tomorrow).

Morse did say that the virus is spreading remarkably quickly, considering how difficult it is to catch Ebola through direct contact. When asked what he thought of the media’s coverage of the CDC report, Morse said it ranged, “from the near-hysterical to the very balanced careful reporting, which is probably what you see in many other situations where there’s an evolving story.” He added, “It’s important to keep these issues in front of the public. Hopefully this will lead to a broader response and interest.”

Damaris Colhoun is CJR’s digital correspondent covering the media business. A reporter at large in New York, Colhoun has also written for The Believer, The New York Times, The Guardian, and Atlas Obscura. Find her on Twitter @damarisdeere.