WILTW: Bad Science Reporting
If you are trying to eat better, one giant obstacle is sorting through all of the “information” that’s about. Low fat? Low salt? High fat, low carb? High carb, low protein? Protein, no carbs? Red wine and dark chocolate are a winning combo? Meat protein over plant protein? Vegan? And it goes on and on and on…
How can this be? Thousands and thousands of scientific studies floating about–conflicting left, right and sideways. How can you separate the good studies from the bad one? How can you tell which ones are really significant?
If you have any faith in news media to help straighten it all out…nope.
What I learned this (last) week: there’s a good chance that the journalists working on the science section/segment of whatever news media you consume are regurgitating press releases and study summaries rather than actually investigating and reporting.
John Bohannon purposely did a flawed study, submitted the paper to a scientific journal and newspapers across the world just repeated his “findings” despite the red flags.
It was, in fact, a fairly typical study for the field of diet research. Which is to say: It was terrible science. The results are meaningless, and the health claims that the media blasted out to millions of people around the world are utterly unfounded.
You can read his whole confession: I Fooled Millions Into Thinking Chocolate Helps Weight Loss. Here’s How.
Once the major news media reports on a health study, smaller outlets, bloggers, etc pick it up–everyone remixing or rewriting the same thing–with few people casting a critical eye on the actual study itself.
Fortunately, there is an organization/website dedicated to actually reviewing science news stories and science news press releases. HealthNewsReviews reviews the big news stories of the week and scores them based on 10 reasonable criteria, including whether or not the article is exaggerating a study’s conclusions. Even when a story passes on a criteria level, HNR still points out if readers should be weary.
For example, here’s the HNR review of an NPR story on headaches and migraines.
HNR doesn’t cover every news story. What it means is that readers of science news, especially in regards to diet/nutrition/medicine/health, have to take even articles from reputable news outlets with a grain of salt. Or, if you prefer, a pinch of chocolate.