The incessant coverage of nutritional studies that make tenuous claims about the harms or benefits of consuming various foods and beverages has come under heavy fire from critics in recent months.
On Thursday, science writer Gary Taubes launched the latest broadside against credulous reporting of flimsy epidemiological research. “The last couple of weeks have witnessed a slightly-greater-than-usual outbreak of extremely newsworthy nutrition stories that could be described as bad journalism feasting on bad science,” he wrote in a guest post for Discover’s blog, The Crux.
A flood of stories about papers linking red meat to a higher chance of death and chocolate to lower body weight sparked Taubes’s ire.
The first study, from a team at Harvard School of Public Health and published in Archives of Internal Medicine, reviewed over two decades’ worth of data from more than 121,000 men and women who participated in either the Health Professionals Follow-up Study or the Nurses’ Health Study, two long-term observational research efforts. It claimed that one serving per day of unprocessed red meat was associated with a 13 percent increased risk of mortality after controlling for factors like smoking, exercise, and body weight, and that one serving per day of processed red meat (a hot dog or bacon, for example) was associated with a 20 percent increased risk.
The other study, from researchers at the University of California, San Diego (UCSD), was also published in Archives of Internal Medicine, and collected new data from over 1,000 men and women who participated in an observational study at the university. It found that eating chocolate was associated with a lower body mass index and claimed that eating it on a regular basis could lead to weight loss.
“The problem with observational studies like the ones from Harvard and UCSD,” Taubes explained, is that they identify associations rather than causal relationships or definitive links. In other words, they generate hypotheses, which, he added, more rigorously controlled experiments usually fail to support.
The difference between observational epidemiology and clinical trials—the “gold standard” in such matters—was a subject that Taubes discussed at length in a 2007 cover story for The New York Times Magazine, “Do We Really Know What Makes Us Healthy?” which CJR praised at the time as a great backgrounder in medical science for reporters. His blog post reminded readers that he “first wrote about the questionable nature of observational epidemiology in Science back in 1995,” in a piece titled “Epidemiology Faces Its Limits.”
Taubes has very particular views about diet and nutrition, however. His books—Why We Get Fat: And What to Do About It in 2010 and Good Calories, Bad Calories: Fats, Carbs, and the Controversial Science of Diet and Health in 2008—and other articles for the Times Magazine—“Is Sugar Toxic?” in 2011 and “What if It’s All Been a Big Fat Lie?” in 2002—have promoted the benefits of a high-fat, high protein, low-carbohydrate, zero-sugar diet. Critics such as Scientific American’s John Horgan and the Knight Science Journalism Tracker’s Paul Raeburn have questioned the evidence for Taubes’s formula and argued that his work is too intent on persuading readers to accept it.
At times, Taubes is similarly heavy-handed in his criticism of observational research, asserting in his blog post for Discover that it’s “closer to a pseudoscience than a real science.” And the no-to-meat, yes-to-chocolate advice in recent coverage clearly ran counter to his deeply held beliefs about what is healthy.
His cautions about the pitfalls of covering epidemiology are on the mark, however, and he’s not the only one making them.
In a post at the Knight Science Journalism Tracker titled “Red (Meat) Scare,” Pulitzer-Prize winning science writer and University of Wisconsin journalism professor Deborah Blum criticized “the alarmist tone of some coverage.” She faulted headlines such as “Eating All Red Meat Increases Death and More Reasons to Never Eat Meat” from The Daily Beast and “Scientists Warn Red Meat Can Be Lethal” from Sky News in the United Kingdom, while praising more balanced work elsewhere.
It is all well and good to point out the obvious: there are many (perhaps the vast majority) of articles on diet and nutrition that either largely wrong or fact free. Nothing will change until the cost of disseminating such is increased, either by exposing academic ineptitude more forcefully or driving poor journalists from their profession. At present neither the academic or journalistic institutions seem willing to deep dive to determine who is failing the system.
The most common defense of the plethora of terrible diet/ nutrition studies is the difficulty in performing a scientifically valid study. If physics took the same approach, we would still be stuck with alchemistry. Scientifically understanding the role of diet on human health has to be much easier than observing neutrinos and anti-particles.
#1 Posted by Robert Bramel, CJR on Wed 11 Apr 2012 at 12:08 PM
"Scientifically understanding the role of diet on human health has to be much easier than observing neutrinos and anti-particles."
Not necessarily. The human body isn't exactly a standardized or clean lab to work in, unlike the environments physicists can manufacture for themselves.
#2 Posted by Thimbles, CJR on Wed 11 Apr 2012 at 03:37 PM
Thimbles is right: I think it's actually a LOT harder to understand the role of diet on human health than it is to observe neutrinos and anti-particles. There are many variables at play and ethical limitations that limit what we can and cannot control for. For example, the claim has been made that it's impossible for someone to become obese without consuming carbohydrates, but we can't test that in a meaningful way because overfeeding experiments are no longer considered ethical and would have a hard time getting approved by human subjects committees. Individuals perform these types of experiments on themselves (see the Twinkie diet, for example), but N=1 experiments don't tell us much because we know that response to diet and exercise varies across individuals.
Personally I don't think observational epidemiology is "bad science," I think it just gets misinterpreted, and much of that misinterpretation is the media's fault. Researchers can insert all the caveats they want about correlation vs. causation, but journalists (or their editors) will continue to ignore them in the quest for an eye-catching story. Observational epidemiology can reveal apparent associations that help focus additional research.
#3 Posted by brad, CJR on Wed 18 Apr 2012 at 11:10 AM