In a highly touted effort to improve the quality of hospital care, the federal government has started disclosing data that ostensibly reveals which hospitals are best (and worst) at keeping their patients safe. But a few weeks ago, Kaiser Health News presented some not entirely unexpected news that turned conventional wisdom about patient safety data into, well, not-so-conventional wisdom. A piece by Jordan Rau raised serious questions about the efficacy of the federal government’s efforts to turn patients into savvy shoppers. The data, it seems, may not be ready for prime time.

I rang up Rau, a veteran health journalist and an expert in these matters, for a Q and A to help all of us who may be tempted to use the data in ways we probably shouldn’t. Here’s what he had to say:

Trudy Lieberman: What’s the problem with the patient safety measures?

Jordan Rau: These metrics, which measure such things as serious blood clots and accidental cuts and tears, were created for a different purpose. The original aim was to help hospitals look at and track internal problems. They were not set up to compare one hospital with another.

TL: Isn’t that what the hospitals are squealing about?

JR: Yes. The concern is that when you compare hospitals that are very different—for example, a teaching hospital that does a lot of complex surgery with a community hospital where fewer patients are going under the knife—you get a distorted result.

TL: Have hospitals themselves caused some of the problem?

JR: Yes. Some have become so meticulous and adept in their coding and billing Medicare that more mistakes are showing up, indicating they have higher error rates. So was something really a problem, or an exaggeration due to exhaustive coding? It’s hard to know.

TL: What else is wrong with these measures?

JR: Hospital quality measures are still in the teenage years of their development. All these measures are imprecise, and that includes measuring readmission rates, infections, and patient experiences.

TL: So is the government saying, “Let’s get this stuff out, even if it’s not so great”?

JR: Implicitly, yes. But in their defense, you can’t make the perfect be the enemy of the good. You can make a case that you can’t get hospitals on board to be measured publicly if you don’t start somewhere.

TL: Which measures are they starting with?

JR: This fall they are going to adjust payments to hospitals based on their performance on the patient experience surveys and process measures like “Did the patient get an antibiotic before surgery, or a beta blocker after a heart attack?”

TL: Most survey research—involving all kinds of services, not just hospitals—shows most people are generally satisfied with whatever is being measured. How will the Centers for Medicare and Medicaid Services (CMS) differentiate hospitals?

JR: Some patients are more satisfied than others. So CMS will determine some point in a scale, and those hospitals above will get a little bit more money and those below will get a bit less. It’s like the race to the top for schools. And CMS gives credit for improvement, so underperformers can still get bonuses if their scores are getting better at a faster clip than are other hospitals.

TL: What kind of bonus will they get?

JR: The bonus is a withhold of one percent of their aggregate Medicare reimbursement. Medicare will hold back this one percent and then dole it out in the form of bonuses. So it’s sort of a penalty and bonus at the same time. The amount withheld grows to two percent in the fall of 2016.

TL: Have the hospitals been squawking about that?

JR: For the most part they’ve given up complaining, because it’s part of the law now. Instead they are focusing on trying to influence CMS about the choice of measures and the weight they give each measure in setting payment.

TL: Which measures don’t they like?

JR: They don’t like the measure requiring them to report their rates of hospital-acquired infections, especially because there’s a separate penalty for hospitals that have higher rates. They consider that double jeopardy.

TL: Are there other metrics hospitals don’t like?

JR: They don’t like the metrics for readmission rates. There’s concern that safety net hospitals that see poor patients can end up having higher readmission rates. Those patients have a harder time paying for medicine and following discharge instructions, and often don’t have the social support structures to improve.

TL: What are the big teaching hospitals objecting to?

JR: They don’t want to be compared to community hospitals, and the other thing is they don’t think the risk-adjustment that CMS does use really distinguishes between sick and extremely sick patients.

TL: Are they correct about that?

Trudy Lieberman is a fellow at the Center for Advancing Health and a longtime contributing editor to the Columbia Journalism Review. She is the lead writer for The Second Opinion, CJR’s healthcare desk, which is part of our United States Project on the coverage of politics and policy. Follow her on Twitter @Trudy_Lieberman.