How much chocolate does it take to fool a journalist? Turns out not much. Earlier this month, Peter Onneken and Diana Löbl, a pair of documentary filmmakers from Germany, and John Bohannon, a biologist and science journalist based at Harvard, revealed that they had tricked millions of people—including their peers at The Daily Star, Cosmopolitan’s German site, and the German and Indian sites of The Huffington Post—into believing chocolate could help them lose weight.
The hoax exposed how easy it can be to turn shoddy research into headline-making news. They pulled it off with little more than a mock clinical trial, some cooked statistics, and one fake website—plus a small army of journalists who were either too stressed or too lazy to check the facts, and a diet industry that profits off concerns about obesity, obsessions with slimming down, and scientific illiteracy. Their study, “Chocolate with high cocoa content as a weight-loss accelerator,” was quickly accepted by The International Archives of Medicine, a journal that claims to do rigorous peer reviews. When it was published two weeks later, it read exactly the same.
There are pineapple diets. There are beer diets. It wasn’t very easy to make a hoax with chocolate because there are so many things that people already think chocolate will do for them.
Bohannon has done this sort of thing before. Last year he ran a “sting operation” for Science on fee-charging for open-access journals, an increasingly lucrative sector of academic publishing. This time, Bohannon was working with Onneken and Löbl, who timed the hoax to coincide with the release of their latest film, a German documentary about how lousy science helps the nutrition industry sell diets—a likely racket in which they felt the media plays a part. The filmmakers’ idea to implicate the media by spoofing it belongs to a long and storied tradition of media pranks that goes all the way back to Benjamin Franklin. CJR sat down with Onneken to discuss how the hoax came to be, the ethics of gonzo journalism, and why chocolate proved to be the perfect vehicle.
This hoax has an interesting backstory that begins with the concept for your film. How did things get started? Diana and I got a call in September 2013 to do a film for Arte, a public TV station that does productions between Germany and France. It belongs to a longer history of initiatives that began in the 1950s to overcome all the battles between the two countries—not just World War II, but going back centuries.
So the chocolate hoax began as an effort in German-French diplomacy? To be fair, our first idea was to do a hoax study about a “beer and schnapps diet,” which is about as scientific as a chocolate diet. We thought a beer and schnapps diet would be perfect for that time of year in Germany we call “the summer lull,” when there’s nothing on the news except some Loch Ness Monster stories. Unfortunately, France has different laws about how you can show people drinking on the telly. So the commissioning editor asked us to go for another diet. At one point it was Coke. Dr. Gunter Frank, a critic of home diet interventions and one of our experts in the film, is the one who proposed dark chocolate—which, to be fair, is not that easy to pull off.
Why is that? The whole problem with telling someone to eat food and constructing a diet that doesn’t exist is that market is so big, not every diet holds up. There are pineapple diets. There are beer diets. It wasn’t very easy to make a hoax with chocolate because there are so many things that people already think chocolate will do for them.
So the challenge was figuring out how to cut through the noise? Indeed. There are already a lot of strange chocolate studies out there, so I wasn’t sure until it was actually published if this would work out. I guess a lot of editors didn’t read our story—not because they didn’t think it was serious, but because they thought, “Well, we know chocolate does this and that.”
So what was your angle? It turned out that I underestimated people’s love for Easter and Easter eggs. We had the right timing for that, so it got picked up by the papers.
The swindle is part of the game. The way Diana and I put on our white coats and measured their waists, I would say 98 percent of them (there were only 16 people in the study) knew that this wasn’t a proper study.
Your movie is about the connection between the diet industry, the umbrella organizations of medical experts, and how lousy science helps them sell diets. What did you discover in the course of investigating this world? We looked at six diet studies proposed by the S3 guideline, a European evidence-based guideline. The interesting part was if you look at these diets and then look at the medical professionals proposing them, you start seeing connections to these industries. One of them was on the advisory board of Weight Watchers, so they proposed the Weight Watchers diet. One of them was working for OPTIFAST, which is owned by Nestlé. There were so many connections between the industry and these professionals on these advisory boards that we decided to talk to them. One of them admitted that yes, it’s true, our whole scientific approach isn’t that accurate, and yes, we have to be defined by the industry because no one else is giving us money. They didn’t call their whole science bogus, but they didn’t really convince us that it isn’t.
You went to lengths to make your study look as real as possible. You had subjects and doctors. You ran a supposedly “clinical” trial. Wasn’t the film enough? The main issue, if you talk to some German experts in the food industry, is that most or all of these studies don’t actually fulfill the criteria of proper, high-quality studies. Their sample sizes are small. They only look for effects of a diet for a year or two. So we thought, why not do it ourselves? Of course, we’re not saying that our study meets the same standards as theirs. Theirs might not be that good, but ours was even shittier.
Some people have questioned the ethics of your hoax, saying you’ve misled people. What do you say to your critics? Most important for us was that we didn’t harm them. We knew that eating chocolate wouldn’t be too big an issue. Dr. Frank did a proper examination of the subjects to make sure there were no anorexic people or anything. But the swindle is part of the game. The way Diana and I put on our white coats and measured their waists, I would say 98 percent of them (there were only 16 people in the study) knew that this wasn’t a proper study. I mean, we didn’t tell them, but all of them had academic backgrounds. They weren’t idiots. Of course, we informed them before we went public.
It wasn’t very new, it wasn’t very original, and we fed it to them through our “medical nonprofit think tank that wanted to cure obesity in the developed world,” which just sounds like absolute rubbish.
Were you surprised by anything your hoax revealed about journalism? We work in the media, we know our colleagues. To be fair, if they don’t check studies, it’s not the biggest surprise. It’s too easy to say, “ask the experts.” Yes, that’s the right way to do it, but there are deadlines. The bigger issue is that this story had no need to get published. If we had found the stone of wisdom, or promised the cure of cancer, it would have been different. But there was no need to tell this story. It wasn’t very new, it wasn’t very original, and we fed it to them through our “medical nonprofit think tank that wanted to cure obesity in the developed world,” which just sounds like absolute rubbish.
Turns out to be good publicity for your film, though, right? Yes, we did this for the documentary. But we also wanted to show that the media plays a very important role in selling this stuff and telling us what to look like. If scientists do junk science and nobody ever hears about it, then nobody is harmed. It’s one thing when a British tabloid tells us about research saying there are dinosaurs on Mars. But there are so many people who want to lose weight and who are actually in desperate need of diet interventions, which lots of studies show don’t help them at all.
So there’s an advocacy element to your hoax? We found one study by the University of Copenhagen that had 16 subjects and fed some of them normal chocolate and some of them dark chocolate, and then they feed them pizza after that, hoping to measure some impact on how dark chocolate helps keep your appetite down. (There were originally 17 subjects, but one was excluded because he consumed a large amount of alcohol on the evening before the second test day.) If a proper university is selling that kind of science, I understand how you end up giving your readers junk science. The PR strategy of universities, which are funded by governments and by society, bear some responsibility in this. But as journalists, we have to check stories. That’s part of our job.
NOTABLE MEDIA PRANKS THAT EXPOSED SOMETHING ABOUT THE PRESS
Infamous prankster Joey Skaggs pulled one over on The Washington Post and GMA with his Fat Squad prank (in which he claimed that for $300 a day, a NY-based company would send out a team of tough guys to follow people around and keep them on their meal plans, using force if needed). Skaggs showed how easy it was to fool a reporter into thinking you’re someone you’re not.
Notable prankster Alan Abel sent up American morality—often fueled by the press—with pranks like a campaign against breastfeeding in public, and the Society for Indecency to Naked Animals. He also posed as Deep Throat and faked his own death, via a New York Times obit.
The Yes Men created fake editions, or “special editions,” of The New York Post and The New York Times, featuring headlines they’d like to see.
Benjamin Franklin may be America’s original media prankster. He was particularly good at information warfare (a distant precursor to ISIS and Putin’s Media Trolls?). Like the time he sent grisly reports to a British paper detailing how British soldiers had hired Native Americans to murder and scalp Americans who had fought in the Revolutionary War (the reports were false, but he hoped they would arouse the sympathies of British citizens in time for peace negotiations). Or the time he wrote an essay about a Muslim warrior who aimed to enslave all Christians; the fictional account turned out to be a send-up of an American congressman whose views on slavery were no less enlightened.
Ryan Holiday, known as “the media manipulator,” figured he could get bloggers to write about anything. So he used HARO—a free source that puts reporters in touch with sources—to become the go-to expert on a variety of topics for a slew of publications, including The New York Times, ABC, CBS, and others. Topics included insomnia, Generation Yikes, and winterizing your boat. He now contributes regularly to Forbes.
Steve Colbert coined the word “truthiness” in the pilot episode of his show. It mocked how “gut-feeling” had become a rhetorical device in contemporary political discourse, especially during the Bush years. Although a stunt word, it was named Word of the Year for 2005 by the American Dialect Society and for 2006 by Merriam-Webster.