Probability Problems

Clumsy numbers in coverage of MIT’s “Greenhouse Gamble” study

May 22, 2009
Clumsy numbers in coverage of MIT’s “Greenhouse Gamble” study

A study by the Massachusetts Institute of Technology’s Joint Program on the Science and Policy of Global Change, which found that end-of-the-century global warming could be twice as severe as previous estimates, drew a limited amount of press attention on Thursday. Few of the resulting articles, unfortunately, are totally satisfying.

One of the most conspicuous pieces was by USA Today’s Doyle Rice, who deserves credit for being one of the few reporters to seek outside, expert commentary on the study. But in the second paragraph, he flubs one of the study’s key statistics, writing:

The research, conducted by the Massachusetts Institute of Technology (MIT), predicts a 90% probability that worldwide surface temperatures will rise more than 9 degrees (F) by 2100, compared to a previous 2003 MIT study that forecast a rise of just over 4 degrees.

That’s inaccurate. According to the study, which was published this month in the American Meteorological Society’s Journal of Climate, that nine-degree (F) rise is actually the median probability of surface warming by 2100, meaning that there is an equal chance of the temperature increase falling above or below that value. There is a 90 percent probability that the increase will be between 6.3 and 13.3 degrees (F).

[Update – May 22, 3:00 p.m.: To understand why such minor gaffes can lead to major problems, read Judy Lowe’s explanation in The Christian Science Monitor of why “30 to 49 percent of people reading or hearing the news were going to be doubtful about some or all of it” in the first place. Or consider the recent Gallup survey, which found that many Americans think the media exaggerates global warming.]

One of the problems here is the word median. This is probably not a term with which most readers are familiar, and it really needs to be defined in every article where it appears. Take the second paragraph from Reuters article, for instance:

Sign up for CJR's daily email

Earth’s median surface temperature could rise 9.3 degrees F (5.2 degrees C) by 2100, the scientists at the Massachusetts Institute of Technology found, compared to a 2003 study that projected a median temperature increase of 4.3 degrees F (2.4 degrees C).

Unlike in USA Today, that sentence is technically correct; and the most important upshot, that things are perhaps twice as bad as they seemed, is clear. But these are the only numbers in the short article—so how many readers would be able to decipher the full import of median? Given that there are fifty/fifty odds of the actual temperature rise falling above or below that value, one could have written, “Earth’s surface temperature could likely rise….” But even that would be misleading because the construction is far too deterministic. The real likelihood is best described by the probabilistic statement: there is a ninety-percent chance that the increase will be between 6.3 and 13.3 degrees F.

Journalists have a real problem with odds, however, in most cases preferring more definitive, newsy sounding statements. Take, for example, the Telegraph’s lede:

The study, carried out in unprecedented detail, projected that without “rapid and massive action” temperatures worldwide will increase by as much as 7.4C (13.3F) by 2100, from levels seen in 2000.

Referring to that absolute high-end value is “catchy,” but totally betrays the purpose of doing a probabilistic study. At the very least, the reporter should have noted that such an extreme temperature rise is not the most likely outcome—according to the paper, there is fifteen-percent probability that the increase will be between six and seven degrees C. It’s not until the bottom of the story that the reporter gets around to the central conclusion:

The projections average out at a likely Earth temperature increase of 5.2C (9.4F) this century, and conclude there is a 90 per cent chance the temperature change will be between 3.5C and 7.4C (6.3F and 13.3F).

One begins to understand why the Telegraph led with the high-end forecast when he or she reads the article’s sensationalistic headline: “Global warming of 7C ‘could kill billions this century.’” Now, that last clause is in quotes, but it’s not clear whose they are. Down low in the story, the reporter quotes a climate campaigner from Friends of the Earth (a poor choice for comment at any rate; a climate scientist would have been better), who may be the culprit.

Whatever the case, the billions dead is not a wholly unrealistic statement if the planet does, indeed, suffer a 7 degree C temperature rise. But there is absolutely no mention of death in MIT’s study, and the Telegraph deserves a strong reprimand for making it seem as though this was its central point. It is truly shameless journalism.

The only journalist who really nailed this story is The Washington Post’s Andrew Freedman, who did an exemplary job. In fact, Freedman covered the study way back in February when MIT first released it—probably before it was even submitted for peer review and publication. He was one of the only reporters to do so, and got a well-deserved tip-of-the-hat from Climate Progress blogger Joe Romm, who chastised the rest of the media for being asleep at the wheel. This week, Freedman posted just a short recap of, and link to, his earlier piece.

As we’ve noted before, Freedman excels at delivering clear, concise, and accurate analyses of climate research. One of the most likeable aspects of his coverage of the MIT study is that he devotes almost equal attention on the research’s “policy scenario,” whereby global leaders get their act together and make some attempt to mitigate warming, as he does on its “no policy scenario.” Most of the coverage—evincing the if-it-bleeds-it-leads maxim of journalism—decided to address only the latter.

With the first viable (albeit watered-down) climate bill having just passed a historic vote in the House of Representatives on Thursday, that is a real shame. True, the policy scenario in the MIT study is a global policy scenario, and much different from the U.S.-only bill moving through Congress. Still, it is germane to note, as Freedman did, that “Under the [MIT] policy scenario, there is a 90 percent chance that climate change could be limited to below 3°C (5.4°F), compared to just a one percent chance of that occurring in the no policy case.” (Note the nice, probabilistic statement.)

Without that point, readers are still left with the important upshot that things are moving from bad to worst, climatologically speaking. But ignoring the point misses an even more important takeaway: that we still can, and should, do something.

Curtis Brainard writes on science and environment reporting. Follow him on Twitter @cbrainard.