A much-hyped study about a diet that lets you eat chocolate daily was an elaborate hoax
They were reporting the results of a study published in the impressive-sounding International Archives of Medicine, an online open-access journal. The study's authors claimed eating chocolate along with a low-carb diet had helped people lose weight and increase their general well-being, compared to people who just ate a low-carb diet.
The problem is, the whole study was an elaborate hoax.
Journalist John Bohannon and others designed, ran, published, and publicized a diet study of purposely dubious scientific merit to make a point about how claims with little to no scientific backing make headlines and drive diet fads, Bohannon reveals in io9.
How they did it
The study was designed to be part of a documentary about the junk-science diet industry, and the filmmakers had a doctor (to run the trial) and a statistician (to play with the numbers) all lined up. Bohannon, who has a PhD in the molecular biology of bacteria and a known history of successful scientific pranks, was brought on to help get the study published and publicized.
They recruited 15 participants, who were paid €150, screened for any health problems, and told they were going to be in a documentary film about dieting - which was true.
B HU/Flickr
Lo and behold, the group that ate chocolate plus a low-carb diet lost weight 10% faster than the group on the low-carb diet without chocolate. The difference was statistically significant, and Bohannon had a hook to sell.
It wasn't hard for Bohannon to get the study published - journals have widely varying standards for peer review, and a lot of them accepted the study. Once the study was accepted, Bohannon wrote a press release, created a fake site for his made-up research center, and - sure enough - reporters started writing about the sham study.
It seemed they'd pulled it off.
What it means
In Bohannon's story on io9, he lists major German daily newspaper "Bild," the German "Cosmopolitan" website, and the German and Indian "Huffington Post" websites among the news outlets that fell for the ruse. He also includes screenshots of coverage from the "Prevention" website, the "Daily Mail," and the June issue of "Shape" magazine.
Bohannon seems pretty pleased that these publications with large audiences picked up his study, "fooling millions," the io9 headline claimed. But as journalist Daniel Engber pointed out on Twitter, maybe it wasn't such a feat after all:
Wait, so no reputable outlets whatsoever (and few overall) covered this fake study. Did it really "fool millions"? http://t.co/fMmRNkvQB4
- Daniel Engber (@danengber) May 28, 2015
Still, it's worth taking the warning to be a little more skeptical about reports that a certain diet or food has major effects on your health - especially when those reports show up in a tabloid.
This study wasn't so different from many other diet studies that also use small sample sizes. It's unfortunately not common for scientists to tweak their data by running an experiment multiple times or excluding outliers to get statistically significant results, mostly innocently.
A good study typically has a hypothesis and then tests it. This study had no real hypothesis and tested 18 different things - not one thing - to see if any statistically significant associations popped up. In poorly designed diet studies, that's not actually so unusual. If enough things are measured in such a small group, that approach is bound to yield... something. Bohannon explained it this way:
Think of the measurements as lottery tickets. Each one has a small chance of paying off in the form of a "significant" result that we can spin a story around and sell to the media. The more tickets you buy, the more likely you are to win. We didn't know exactly what would pan out-the headline could have been that chocolate improves sleep or lowers blood pressure-but we knew our chances of getting at least one "statistically significant" result were pretty good.
- Colon cancer rates are rising in young people. If you have two symptoms you should get a colonoscopy, a GI oncologist says.
- I spent $2,000 for 7 nights in a 179-square-foot room on one of the world's largest cruise ships. Take a look inside my cabin.
- An Ambani disruption in OTT: At just ₹1 per day, you can now enjoy ad-free content on JioCinema
- SC rejects pleas seeking cross-verification of votes cast using EVMs with VVPAT
- Ultraviolette F77 Mach 2 electric sports bike launched in India starting at ₹2.99 lakh
- Deloitte projects India's FY25 GDP growth at 6.6%
- Italian PM Meloni invites PM Modi to G7 Summit Outreach Session in June
- Markets rally for 6th day running on firm Asian peers; Tech Mahindra jumps over 12%