“One of my first jobs was to license all of this information provided by places like Mayo and the Cleveland Clinic [also world-renowned non-profit research hospital],” he said. “So our content was built around the core of this very credible, dispassionate stuff, but it tended to be framed more in response to commercial concerns … it’s very hard to live in that environment, where you have advertisers who want to appear directly across from a specific piece of content.”

The landscape is rapidly changing, however. Stoltz was surprised to see the Pfizer ad on the Mayo Clinic’s fibromyalgia page, for example. Ginger Plumbo, a spokesperson for Mayo, says that the site has taken medical ads since in launched in 1995, but there have been other changes. In 2009, Mayo signed a deal with the Everyday Health Network to expand its advertising to include general consumer products like cars and cosmetics (however, a MedCity News article about the partnership reported that ad revenue is a secondary breadwinner for the site, which relies mostly on creating personalized health portals for private organizations).

There also used to be two separate Mayo sites—MayoClinic.com, which focused on providing health information, and MayoClinic.org, which focused on the clinic and its services. They merged into a single site in January, Plumbo said, and there is now a higher level of integration between general health information and information about treatments and services offered by the clinic. Other non-profits are pursuing similar strategies. The Cleveland Clinic’s main website does not accept advertising, but it unveiled a sister site in July 2009, called Cleveland Clinic Health, which does. One health reporter contacted for this review thinks these developments at Mayo and Cleveland Clinic are an indication that nonprofits are not any more “dispassionate” about pushing services than for-profits.

In light of such changes and the many subtle factors that affect a site’s credibility, Stoltz agreed it would be very useful to have some sort of regularly updated database that people could use to evaluate different sites. The closest anyone has come to building such a thing was HealthRatings.org, a creation of Consumer Reports’s WebWatch project that launched in 2005 and evaluated the top twenty health information sites. WebWatch shutdown in 2009 when its grant-derived funding ran out, however, and the ratings are no longer available online.

Thomas Goetz, the executive editor of Wired magazine, who has written extensively about personal health and information technology, said that “it’s easy to scapegoat the pharmaceutical industry” when evaluating sites, arguing that drugs often do help people. He stressed, however, that all information providers should be sensitive to the general public’s propensity for “cyberchondria.” Outlined in a survey performed by Microsoft in 2008, the term describes people’s tendency to gravitate toward the worst-case—and most unlikely—explanations for symptoms (a brain tumor, rather than caffeine withdrawal, in relation to a headache, for example).

Goetz doesn’t think there is an inherent difference between for-profit sites like WebMD and non-profits like Mayo Clinic when it comes to “poking the fear button.” Like former journalist turned PR rep and blogger Brian Reid, Goetz noted that the real fear-mongering comes from places like anti-vaccine sites, which push egregious pseudoscience. The existence of those sites does not negate the need to evaluate more serious and professional health sites, however.

Stoltz pointed out that general information sites like WebMD and Mayo Clinic are often just starting points for health searches, and that sites devoted to specific conditions, such as ACOR.org (the Association of Cancer Online Resources), can be even more helpful. And Goetz, like a number of other journalists contacted for this review, recommended government sites like the National Institutes of Health’s MedlinePlus or the Center for Disease Control’s website, which he said often go overlooked.

“As taxpayers, we’ve spent billions of dollars on the research that informs those sites,” he said, “and it’s unfortunate that not many people got to government sites right off the bat.”

Curtis Brainard is the editor of The Observatory, CJR's online critique of science and environment reporting. Follow him on Twitter @cbrainard.