Friday 29 May 2015
A hoax paper claiming dark chocolate can help you lose weight makes a valuable point about the state of science publishing today. But the real problem comes when such papers chime with political agendas.
News stories about the latest AMAZING diet secret are commonplace - and a standing joke. The media has a steady stream of reports that adding this or avoiding that will soon translate into disappearing inches from our waistlines. Most of this stuff is quickly forgotten. Today’s astonishing research finding is tomorrow’s fish-and-chip paper.
So it was no wonder that a team in Germany, working with documentary makers producing a film about the junk science of the diet industry, was able to catapult a story about a wonder food into the mainstream media. The scam closely modeled common practices in the field: the team picked a likely candidate for a wonder food, dark chocolate; they conducted an actual clinical trial that was far too small and poorly controlled to give meaningful results; and tortured the resulting data until a statistically significant result popped up. With an (almost) straight face, they were able to claim that people who ate a low-carb diet but also ate a small bar of dark chocolate each day lost more weight than those who ate a low-carb diet alone.
Then, to burnish the credentials of their ‘study’, they got it published in an open access, pay-to-get-published journal called International Archives of Medicine. These veritable vanity publishers are supposed to peer review the studies they publish. But it is pretty clear from the experience of the German team that no meaningful peer review took place; the paper when from submission to publication in just two weeks.
Then the paper was widely press-released in the UK and Germany. Lo and behold, the story was widely picked up, especially by the kind of outlets that stuff their inside pages with ‘churnalism’. Most didn’t bother to read beyond the press release. A few did get in touch with the authors, but mostly to check trivial details rather than to question the quality of the study.
The story is a nice piece of spoofing to make a point: be very careful about believing this kind of story. Such dubious research only makes it harder to understand the real causes of obesity or disease, creating a storm of conflicting advice about what we should or should not eat to lose the flab or enjoy a healthy diet. Nutrition research - indeed, any research on the causes of ‘non-communicable disease’ - is fraught with difficulty. The results need to be very strong and clear, arising from well-designed research, before they should be taken seriously. Beyond the well-tested observation that active cigarette smoking causes lung cancer and other respiratory diseases, vanishingly few such studies produce reliable conclusions for how we should live.
But this is not just a problem of attention-seeking researchers, vanity science publishing, journalists who are lazy or just ill-equipped to assess such claims, and a readership eager to lap up new, apparently scientific findings. Such tabloid stories pass through our consciousness quicker than All-Bran through our guts. (Which is why the headline of the story about the scam, I Fooled Millions Into Thinking Chocolate Helps Weight Loss. Here’s How.
, is as dubious as the spoof study.) The real problem is when such junk science feeds into wider agendas about public policy. So claims that, for example, plain packs cut smoking rates are errant nonsense, but still get dredged up to prove the efficacy of introducing bans on cigarette branding. The claim that the sugar consumption of a country is directly related to the incidence of type-2 diabetes is barely more sophisticated than a back-of-an-envelope calculation.
Yet these dubious studies ultimately have a real impact on our ability to live our lives as we would choose to and are frequently the product of activist researchers trying to provide the ‘evidence’ to back up their desire policy prescriptions, whether it is smoking bans or sugar taxes. When science gets into bed with politics, both science and public policy suffer.
I'm a writer and author on a wide range of issues. I'm the former deputy editor of the online magazine spiked and I currently work at the Institute of Ideas. I'm the author of Panic on a Plate: How Society Developed an Eating Disorder (Imprint Academic, 2011).