Sen. Barbara Boxer was defiant. As Campaign Desk reported recently, the gentle lady from California said in no uncertain terms that if the Congressional Budget Office did not give the Senate the numbers it was looking for in estimating savings from preventive care, she, for one, would not follow the CBO’s advice: “We’re going to look at OMB and CBO and make our own decision as to who is right.” She said members of Congress had just heard from the CEO of Safeway, “who said his insurance costs went steadily down since they instigated incentives for prevention.”

It so happens that CEO Steven Burd also made his claims on In an op-ed, Burd explained that, in 2005, his company designed a plan that has made continuous improvements each year. The plan, which relies on rewarding healthy behavior, bases premiums for the insured nonunion workforce on health behaviors that influence weight, blood pressure, cholesterol, and smoking. Employees with lower numbers are charged lower premiums. Burd asserts that the program has held Safeway’s per person health care costs flat at a time when “most American companies’ costs have increased thirty-eight percent over the same four years.” He said results “have been remarkable.”

Burd’s anecdote is just that—an anecdote, based on what researchers call an observational study, not on hard-core evidence from, say, a randomized controlled trial that controls for other things that might account for the results. Any good health and medical reporter knows that. Even a quick read of Burd’s op-ed demands skepticism—from journalists who might be tempted to forget about the hierarchy of evidence and especially from senators who are in the policymaking business.

Health policy researchers Louise Russell and Jonathan Oberlander, experts on cost savings whom we have featured in our Excluded Voices series, dug into the Safeway claim a little more. “Policymakers need to be cautious about over-interpreting this anecdotal report as evidence of savings from prevention,” they said. In effect, they were doing the job more journalists should be doing when they hear CEOs of major corporations make such claims.

Russell and Oberlander did find a good story in the San Francisco Chronicle that helped guide their digging. The plan Burd seems to refer to is called Healthy Measures, and the Chronicle reported that it “just went into effect on Jan.1” of this year. It appears to be the same program described in the Wall Street Journal, offering premium reductions for staying within certain limits for the risk factors of smoking, obesity, blood pressure, and cholesterol. Russell and Oberlander say: “If this is true, the rebates simply cannot be responsible for the fact that Safeway’s health care costs have been flat since 2005.”

It also turns out that Safeway began a new health insurance program in 2006, called Market-Based Health Care, that establishes health reimbursement accounts for workers. Each year, Safeway pays the first $1000 of medical costs, and the employee pays the next $1000; after that, the company pays 80 percent of the costs. As Russell and Oberlander point out: Studies tell us that when people have to pay more of the costs for their care themselves, they tend to use less of it—and of course, spend less. That change by Safeway would be enough by itself to explain Safeway’s lower health care costs.

There are a number of other questions reporters and senators should be asking to keep from being fooled by the next company that comes along and touts savings from its wellness programs. Here are a few to consider:

• Has the health plan changed in other ways that might contribute to controlling costs?

• Does the plan cover the same services over the time period measured?

• Have reimbursement rates stayed the same?

• Have there been changes in the plan’s system of reviewing and approving services for payment?

• Are there changes in provider networks that could be lowering costs?

• Have copayments, deductibles, and other cost-sharing features changed?

• Are managers hiring healthier employees to avoid claims for expensive illnesses?

• What are the other comparison groups? In Safeway’s case, Burd calls them “most American companies.” But just which ones—other national food chains, car makers, drug companies, or toothpaste sellers?

• Are there data that show employees are actually using fewer medical services?

• Are there data that show employees have actually changed their bad behaviors if indeed the cost reduction is due to such a change?

And, while the diggers are at it, they should remember that companies touting their anecdotal evidence may have other agendas. Burd says that “we are constrained by current laws from increasing these incentives.” By that, he means the payments a company can give employees who, say, have stopped smoking are too small. “Reform legislation needs to raise the federal legal limits so that incentives can better match the true incremental benefit of not engaging in these unhealthy behaviors,” says Burd. Some emerging bills do contain such incentives. Are these good investments for taxpayers? Such op-eds, which are really another form of lobbying, don’t often answer that question.

Trudy Lieberman is a fellow at the Center for Advancing Health and a longtime contributing editor to the Columbia Journalism Review. She is the lead writer for The Second Opinion, CJR’s healthcare desk, which is part of our United States Project on the coverage of politics and policy. Follow her on Twitter @Trudy_Lieberman.