Last September, CJR approached me to write an article for a special package on health and science journalism in the January/February issue. The subject: “one of the more significant trends in science journalism in the last 20 years — a shift in focus toward personal health news, especially diet and fitness.” And the premise: “the public craves simple answers and immediate solutions and the press indulges those unreasonable expectations.”
I responded that the assignment was interesting, particularly so “since I’m part of that trend and there are certainly journalists (and researchers) out there, including some old friends, who would say I’m pandering to the public’s desire to have easy answers.” Ultimately I turned it down, though, not for lack of interest but lack of time. Instead, I suggested they offer the story to my good friend David Freedman, who accepted.
The result was Dave’s cover article, “Survival of the Wrongest,” which oddly enough singled out my work, along with that of The New York Times’s Tara Parker Pope, as examples of health journalism gone awry. Both articles were on obesity and the implication was that Parker Pope and I might be, at least in these cases, sacrificing the tenets of good journalism for the benefits of promoting a favored theory that bucks conventional wisdom.
Two journalistic questions were raised by my friend Dave’s article that are worth addressing, the second admittedly more critical than the first:
1) Should we add to the multitude of reasons why a freelancer should never turn down an assignment, the possibility that the friend whom you recommend instead (or the writer the editors choose independently) will publicly disparage your own work in the subsequent article?
2) Considering that most of what is published in the medical journals is not nearly as meaningful or even correct as we might like to believe—a problem that Dave does such a good job of addressing—how should those of us who cover these fields report on these issues? This question embraces both reporting on deadline for tomorrow morning’s paper and the kind of long interpretive/investigative articles that Dave was criticizing.
I wrote about this problem 14 years ago (“Telling time by the second hand,” in Technology Review) and it was not particularly new then. At the time, I quoted the philosopher of science John Ziman describing the front-line of research— where those of us covering science and medicine make our living—as a place where reliable knowledge is hard to come by and “controversy, conjecture, contradiction, and confusion are rife.” Using physics as an example, Ziman estimated that 90 percent of the science in the journals was probably wrong, compared to only ten percent in the undergraduate textbooks. (The job of science, he suggested, was to separate the wheat of the textbooks from the chaff in the journals.) Dave uses the recent research of John Ioannidis of Stanford to estimate that two-thirds of the papers in the medical journals are unreliable, although that’s only the “top medical journals” and we can assume the lesser journals, of which there are multitudes, have a batting average considerably worse.
With these numbers in mind, as Dave says, “a reporter who accurately reports findings is probably transmitting wrong findings.” This issue is the subject of the bulk of Dave’s article and it should engender the most consensus, at least in theory if not necessarily in practice: If we have to cover these articles at all, the liberal use of caveats and conditions, and context (the proportion of articles on the subject that came to different conclusions in the past) would be an excellent idea. As I noted in my 1998 essay, the very first reports on any new finding—the ones most likely to get media attention—were also the ones most likely to be wrong. This is the nature of science. It is fundamentally at odds with what we do in journalism, whether personal health or not, and we have to always keep it in mind.