More than 10,000 stories came out of the annual meeting of the American Heart Association (AHA), which took place in Los Angeles last week, but it was the media’s ticker that was beating too fast and too hard, according to the media criticism website HealthNewsReview.org.
The cardiac confab is always a big draw for journalists, and as with other scientific conferences, reporters have a tendency to hype the tentative research that is presented there. Health News Review, which uses a standard satisfactory/unsatisfactory rating system, based on ten criteria, to evaluate news coverage of medical treatments and procedures, gave low marks to many of the stories flowing out the AHA meeting.
The website was particularly disappointed by coverage of the biggest story to emerge from the conference, about a new class of cholesterol-reducing drugs called PCSK9. They’re designed to replace or augment the use of statins, which is the current course of action for treating high cholesterol, but pharmaceutical manufacturers have a long way to go before PCSK9 drugs enter the clinical market—a fact that was underemphasized in many articles, according to Health News Review.
Stories from Reuters and HealthDay both received only three stars on the site’s five-star scale, meaning they earned satisfactory grades on only 41 to 60 percent of the evaluation criteria, which include things like adequately assessing the benefits, harms, and costs of a treatment or procedure. Both outlets exaggerated the potential advantages of PCSK9 drugs, Health News Review concluded. In its review of HealthDay’s article, the team remarked:
Is it true that an experimental treatment to lower LDL cholesterol “may help” patients who can’t tolerate standard statin treatment? That’s what this story reports. But the fact is that no one knows, because this trial and others other similar tests have measured only how the drug affects blood test results in the short term, not whether there are any real health benefits in the long run… News reports about experimental cholesterol treatments should always point out the distinction between lab test results and meaningful health outcomes.
In addition, the Health News Review’s publisher, Gary Schwitzer, criticized a New York Times article by Andrew Pollack that didn’t receive a formal review. The article included a number of “caveats” about PCSK9 drugs, such as the fact that while they lower cholesterol, “it has not yet been shown that they actually reduce the risk of heart attacks, strokes or other cardiovascular problems.” But those caveats didn’t come until after Pollack quoted one doctor saying that together with statins, the drugs can help “virtually everyone” with high cholesterol. Schwitzer called that claim “totally unsubstantiated” and accused the Times of playing “editorial ping-pong” in the structure of its story.
The media did better with other stories that came out of the AHA meeting. The Associated Press and USA Today won praise for skeptical stories about a study that involved using chelation, a controversial therapy used to treat heavy-metal poisoning, to treat heart disease. Even in applauding the USA Today story, however, Health News Review’s team criticized the lackluster coverage of more common, but still problematic therapies:
While we applaud the aggressiveness with which some stories like this one punched holes in this study, we pause to point out that many of the complaints being made here—marginal benefit, lots of dropouts, possibly biased investigators, etc., etc. — can also be made about studies of more “mainstream” treatments. We will continue to push journalists to call out those studies as aggressively as they do complementary or alternative health research.
Ironically, the perils of making too much of the provisional research that dominates scientific meetings came up in the AHA’s own journal, Circulation, just as its gathering was coming to a close. In a paper titled, “The Conversion of Cardiovascular Conference Abstracts to Publications,” researchers concluded that less than a third of the abstracts presented at the annual meetings of three major cardiology groups between 2006 and 2008 (including the AHA) were published within two years of the conferences, and only about half were published within five years.
“So while some journalists are glued to coverage of these meetings, publishing daily for general news audiences that may not understand the nuances of the limitations of drawing conclusions from talks presented at scientific meetings, most of this stuff isn’t even published in the medical literature for at least 5 years – if ever!” Schwitzer wrote on his blog at Health News Review. “Why does this matter? Such news coverage creates a rose-colored view of progress in research. It may not be inaccurate but it most certainly misleads and lacks important context if it doesn’t present caveats about the limitations.”
Schwitzer also reminded his readers of a 2002 paper published in The Journal of the American Medical Association titled, “Media Coverage of Scientific Meetings—Too Much, Too Soon?” which concluded:
Results are frequently presented to the public as scientifically sound evidence rather than as preliminary findings with still uncertain validity. With some effort on the part of meeting organizers, journalists, and scientists, it will be possible to better serve the public.
Along the same lines, The Knight Science Journalism Tracker’s Paul Raeburn suggested that Health News Review’s guidelines should inform more coverage in general. “I’m beginning to think that Schwitzer’s criteria for judging stories ought to be printed on wallet cards for reporters, like Miranda warnings, to remind them what questions to ask,” he wrote. “I could use one of those myself.”
It’s a great idea, but unfortunately one that may never come to pass. According to Schwitzer, Health News Review took a 30-percent budget cut this year from the Informed Medical Decisions Foundation, which has been the website’s sole source of support for the last seven years, and he worries that funding may dry up entirely by July.
Given Health News Review’s substantial contributions to the field of media criticism, its closing would be a tragedy—and one way to ensure that coverage of scientific meetings, like the AHA’s conference, never gets any better.
Curtis Brainard writes on science and environment reporting. Follow him on Twitter @cbrainard.