A review of nearly 1,500 health-medical articles over the last five years has found that while journalists are nailing a few key categories of quality reporting, they’ve been falling down on the most important ones, like the costs, harms, and benefits of care.
HealthNewsReview.org—a website that reviews news stories about specific treatments, tests, products, or procedures—released the “scorecard” on the occasion of its fifth anniversary this week. The site uses a standardized, satisfactory/unsatisfactory rating system based on ten criteria to evaluate articles from a roster of the country’s top news agencies.
It used to review the top twenty-five newspapers by circulation, plus Time, Newsweek, and U.S. News & World Report, the Associated Press, and ABC, CBS, and NBC news stories. In December 2009, it cut back to the top papers by circulation (excluding the New York Post and Daily News in the name of geographic diversity), dropped the TV networks, and added CNN, NPR, MSNBC, HealthDay, Reader’s Digest, Reuters Health, and WebMD. It does not follow hard copies or broadcasts, and instead relies on the websites of all the outlets.
Tabulating the results of 1,488 reviews, HealthNewsReview found that more than two-thirds of the articles received a satisfactory score in areas like explaining the availability of a treatment or procedure, explaining the novelty of a treatment or procedure, avoiding disease- or fear mongering, and avoiding overreliance on a news release. Yet only forty-one percent appropriately covered alternatives to a treatment or procedure, and just over half did an acceptable job quoting independent sources and letting the public know about potential conflicts of interest. And journalists fared worst in the most vital categories, with only about three out of ten articles adequately describing the costs, benefits, harms, and evidence underlying a treatment or procedure.
“We think all ten criteria are important, but clearly some are more important than others and they’re the ones that, sadly, do the worst,” said Gary Schwitzer, the founder and publisher of HealthNewsReview.
Asked why that might be so, Schwitzer added that the most crucial criteria are undoubtedly the toughest to get right. In an effort to help reporters overcome that challenge, HealthNewsReview just released a “first stab” at an online list of resources to help journalists explore the costs of treatments and procedures. The list is the latest addition to a fourteen-part toolkit designed to help journalists improve their health-medical coverage, which also includes primers on medical devices, phases of drug trials, animal and lab studies, and absolute versus relative risk, among other subjects.
Schwitzer stressed that his site’s mission is not to criticize, but rather improve the accuracy of health and medical reporting (in addition to helping consumers evaluate the evidence for and against screening tests, drug therapies, surgeries, nutritional advice, et cetera). In an interview, he mentioned a 2006 Associated Press article about the launch of HealthNewsReview, which quoted Cristine Russell, a veteran health reporter and CJR contributing editor, saying she hoped the project would not “end up being another media-bashing exercise.”
“I worked really hard to overcome that idea and frame this [website] as exactly what it is, an attempt at constructive outreach,” Schwitzer said. “I wouldn’t have given up a tenured position—I wouldn’t have been doing this every day for the last six years—if I didn’t think that it was helping people,” he added, referring to the job he left last year teaching health journalism at the University of Minnesota.
For her part, Russell agrees that the site has not indulged in any gratuitous media bashing—far from it. “HealthNewsReview has played a constructive role in setting the bar for what good health journalism should be,” she wrote in an e-mail. “It has set tough standards that may be hard for some journalists and organizations to meet, particularly at a time of limited staffs and added pressure to feed the blogs and websites. But at least they know where to aim. And HNR plays an important watchdog role when it calls out bad practices and ethical lapses.”
The site has rapid growth in traffic in the last year and a half, according to Schwitzer, climbing from around 3,000 unique visitors a day in January 2010 to about 5,000 a day in the last couple months. He relies on a team of reviewers comprising half a dozen freelance journalists and a mix of more than two dozen doctors and health services professionals (many in the latter group work for the Foundation for Informed Medical Decision Making, a nonprofit organization that helps people evaluate healthcare options and provides the sole financial support for HealthNewsReview). In almost all cases, three reviewers evaluate each article.
Its accomplishments notwithstanding, the site has seen its share of frustration. It stopped reviewing television network news stories in 2009 in order to reallocate resources from a time-consuming process that did seem to be helping.
“After 3.5 years and 228 network TV health segments reviewed, we can make the data-driven statement that many of the stories are bad and they’re not getting much better,” Schwitzer wrote in a September 2009 post on his HealthNewsReview blog headlined, “It doesn’t make sense for us to review health news anymore.”
TV news articles were receiving an average of about two stars on the zero to five-star scale that HealthNewsReview uses, meaning that the article satisfied only 21-40 percent of the site’s ten evaluation criteria (five star stories satisfy 81-100 percent).
In an August 2009 post explaining why “network TV morning health news segments may be harmful to your health,” Schwitzer wrote that HealthNewsReview had discerned a clear pattern at ABC, CBS, and NBC, in which the networks:
• Unquestioningly promote new drugs and new technologies
• Feed the “worried well” by raising unrealistic expectations of unproven technologies that may produce more harm than good
• Fail to ask tough questions
• Make any discussion of health care reform that much more difficult
“We urge TV health news decision-makers to realize how often they’re doing more harm than good with so many of their non-evidence-based, cheerleading promotions of treatments for wrinkles, weight loss, baldness, toenail fungus, etc.- and breathless enthusiasm for new devices and other ‘new stuff’ in health care,” Schwitzer wrote in the September 2009 post. In an interview, he emphasized that HealthNewsReview does not ignore the networks, however.
“I’ve continued to comment on television news on my blog and, in fact, in so doing probably touch on more TV news now than I ever did,” he said. “It’s just in a more subjective blog style than it is with the more objective standardized criteria driven evaluations of HealthNewsReview.”
Indeed, Schwitzer’s blog, which used to be located at a separate website, was incorporated into HealthNewsReview during the 2009 overhaul and now accounts for roughly 50 to 60 percent of the site’s traffic depending on the day.
Despite network news’s perennially poor performance and the mixed grades revealed in the five-year scorecard, not all is lost in the world of health and medical reporting. When I spoke to Schwitzer, he had just returned from the annual meeting of the Association of Healthcare Journalists, which took place April 11-14 in Philadelphia. He said he found plenty of reassuring signs. To begin with, the meeting had unprecedented attendance, with more than 625 journalists and health experts taking part.
“To have a record number of people show up is, at some level, an acknowledgement that people know they need to improve and know where they need help, and are somehow finding the resources or fellowship support to come out and learn,” Schwitzer said. (HealthNewsReview’s toolkit includes a link to “Covering Medical Research: A Guide For Reporting on Studies,” that the association of published last September, which Schwitzer authored with help from Reuters Health executive editor Ivan Oransky, an MD.)
Asked whether the thinks health-medical coverage is, on the whole, getting better or worse, Schwitzer said it was “impossible to answer” because of the question was so broad.
“But I think you are able to identify the ends of the spectrum that stand out,” he added. “As Don Barlett, of the Barlett and Steele [investigative reporting] team, told the convention when it met in Chapel Hill in 2005, we’ve never had better special-project, investigative healthcare journalism in this country. It’s the daily drumbeat of news stories that is a disgrace. The peaks of excellence still happen too rarely and the valleys in between seem to wider and deeper.”
Over two dozen examples of those peaks can be found in the Association of Healthcare Journalists’ awards for outstanding coverage in 2010. It is also worth checking the latest issue of TVWeek’s NewsPro magazine, which focuses on health journalism and the association’s meeting in Philadelphia. It includes an article titled, “Weathering the Storm—Health Journalism Is Alive and Well,” which quotes Schwitzer and association president Charles Ornstein citing new outlets that have produced noteworthy coverage. Among them are ProPublica (where Ornstein is a senior reporter), Kaiser Health News, California Watch, The Texas Tribune, Health News Florida, Georgia Health News, the Connecticut Health I-Team, and the Kansas Health Institute News Service.
“But even in the mainstream press there are signs that health care coverage is starting to rebound, Ornstein said, citing The Boston Globe, The Philadelphia Inquirer, and NPR as organizations that are seeking to beef up coverage.”
Likewise, in an interview, Schwitzer emphasized “the power of individual reporters” making a big difference in the mainstream media. He commended Blythe Bernhard and Jeremy Kohler, the medical and investigative projects reporters (respectively) at the St. Louis Post-Dispatch; Karen Brown, the health care and social welfare reporter at WFCR public radio in Amherst, Massachusetts; and Jeff Ballion, an investigative reporter at the Fox News affiliate in Minneapolis-St. Paul, Minnesota. Schwitzer also applauded the solid health-medical reporting at wire services.
“We should be so thankful that AP and Reuters Health have the kind of journalists that do skeptical, evidence-based medical reporting on their teams,” he said. “If they didn’t, and if they weren’t doing what they’re doing, we’d be in a lot worse shape, because so many [other outlets] subscribe to them.”
As for the next five years, Schwitzer has a number of ideas in mind. One is to “get ahead of the curve.” Information about upcoming papers or announcements related to new treatments and procedures is available through conferences, news releases, and other channels. HealthNewsReview could provide advance tip sheets to reporters outlining important points to consider and relevant resources for emerging stories.
“This is something that, understandably, journalists have been asking us about for quite some time,” Schwitzer said. “We could be proactive in that manner and see if it makes a difference.”
Another promising avenue is collaboration with a growing number of websites around the world that use basically the same standardized evaluation criteria that HealthNewsReview does. “In a sense, there is a network of six of us now,” Schwitzer said.
He modeled HealthNewsReview after Media Doctor Australia, which launched in 2003 and developed the evaluation criteria, and gave Schwitzer permission to adapt it for his site. The Australian original also inspired Media Doctor Canada, Media Doctor Hong Kong, Media Doctor Germany, and Media Doctor Japan. People in Italy and Sweden are considering starting similar sites as well, according to Schwitzer.
“I’ve been thinking for years now that we could collaborate with these international friends, pool findings—since we all do the same thing—and maybe make this message [about the importance of improving health-medical coverage] that much stronger,” he said.
Indeed, a database comprising thousands of article reviews from around the globe could be invaluable in terms of figuring out where journalists need support resource-wise, or where journalism schools and on-the-job training should be focusing their attention. And it would be enlightening to compare different countries’ annual scorecards. Who knows, a little competition might even encourage U.S. journalists to bring their grades up.
Curtis Brainard writes on science and environment reporting. Follow him on Twitter @cbrainard.