JR: They don’t like the metrics for readmission rates. There’s concern that safety net hospitals that see poor patients can end up having higher readmission rates. Those patients have a harder time paying for medicine and following discharge instructions, and often don’t have the social support structures to improve.

TL: What are the big teaching hospitals objecting to?

JR: They don’t want to be compared to community hospitals, and the other thing is they don’t think the risk-adjustment that CMS does use really distinguishes between sick and extremely sick patients.

TL: Are they correct about that?

JR: As we discussed, these measures are coarse. But to lose a lot of money, hospitals will have to do badly in multiple domains. If they’re just a laggard in one area, such as patient safety, but above average in outcomes or patient surveys, it will balance out.

TL: Have consumers been using any of these measures—patient safety measures, satisfaction scores, and the so-called process measures like making sure a patient gets an antibiotic one hour before surgery?

JR: Most evaluations show that consumers don’t use these data in selecting hospitals.

TL: Why don’t they use them?

JR: A lot of things that put you in the hospital are immediate problems that don’t lend themselves to comparison shopping, and consumers are directed to hospitals based on their doctors’ preferences, their insurance coverage, geographic convenience or their general sense of a hospital’s reputation.

TL: Are there any measures being used by consumers?

JR: That’s a great question. I don’t think a majority of consumers use any of them.

TL: That brings us around to our fellow journalists. How should they use these measures, if at all?

JR: Journalists can use them, but they should do so carefully. No one measure captures the overall hospital quality, and you should be careful about your comparisons. With the right context, I don’t think there’s anything wrong with using the patient safety indicators in stories.

TL: Can you give me an example where a news organization used them properly?

JR: The Dallas Morning News is a prime example of how to responsibly use the measures its coverage of Parkland Memorial Hospital. Texas has good discharge data, and the paper did its own analysis of patient safety measures, comparing Parkland to other large Texas hospitals. Patient safety measures were just one piece of the coverage. They used lawsuits and government inspection reports. So, bottom line: Patient safety measures can be a good piece of a larger mix, but they themselves are not solely definitive.

TL: What should journalists not do?

JR: They should not use them without talking to the hospitals, and they should be very cautious in comparing teaching hospitals to community hospitals. They should also make clear that they should not imply the safety ratings cover all the patients in a hospital. They cover only certain types of cases and accidents. They should also not assume that a hospital rated better than average by Medicare is in fact superior or trouble-free—those could be underreporting problems.

TL: So should they construct any rankings of hospitals based on the measures?

JR: It depends on the context. I wouldn’t do the ten worst or the ten best hospitals.

TL: How should reporters use the mortality measures, considering that the latest research reported in Health Affairs shows they haven’t reduced mortality?

JR: Just because publication of data hasn’t led to improvements doesn’t mean the data isn’t accurate. The mortality data—rates of people dying within 30 days of discharge—is pretty good.

TL: Do you have any other advice for journalists?

JR: As complex as these measures are, they are great ways to get conversations started with a hospital executive you are interviewing. Also, when hospital executives say Medicare data is not accurate, press them to produce their own internal data to prove their data are more accurate than Medicare’s.

TL: Think for a moment about reporters just starting to cover hospital metrics. What should they do?

Trudy Lieberman is a fellow at the Center for Advancing Health and a longtime contributing editor to the Columbia Journalism Review. She is the lead writer for The Second Opinion, CJR’s healthcare desk, which is part of our United States Project on the coverage of politics and policy. Follow her on Twitter @Trudy_Lieberman.