Skip to Content

The Sad State of Human Dietary Research

I wanted to highlight this post at Five Thirty Eight, because it makes an important point about something that chemists, biologists, and MDs get to hear a lot about: nutritional science. Headlines have been produced for decades about how you should eat this superfood, and avoid that toxic one, and how this berry will protect you from Alzheimer’s and that vegetable from colon cancer, and how eating X is associated with Bad Disease Y, and so on and so on.

And you know what? The scientific rationales behind most of these are pitiful. Have a look. And have a look at how the numbers are generated – food diaries, attempts at recalling what you’ve had to eat over the past three months, that sort of thing. No one locks up five hundred people in a warehouse and feeds them precisely measured portions of People Food Mix, and without that, the numbers are always going to be fuzzy. Really, really fuzzy, to the point that the great majority of all these eat-this stories are noise, sheer noise.

Although concerns about self-reported dietary intakes have been around for decades, the debate has come to a head in recent years, said David Allison, director of the University of Alabama’s Nutrition Obesity Research Center in Birmingham. Allison was an author of a 2014 expert report from the Energy Balance Measurement Working Group that called it “unacceptable” to use “decidedly inaccurate” methods of measurement to set health care policies, research and clinical practice. “In this case,” the researchers wrote, “the adage ‘something is better than nothing’ must be changed to ‘something is worse than nothing.’”

Indeed. Bad data are worse than no data, and the situation is made far worse by the headline-ready nature of the material, not to mention its store-shelf-ready nature. People, understandably, would like to know if there’s some food that they’re eating that is substantially raising their risk of disease, or substantially lowering it. Journalists know that people want to read about this stuff, and marketers know that there are vast sums to be made by catering to hopes and fears: “Just eat this!” You couldn’t ask for a setup that could lead to more hype and muddle, and that’s what we’ve got.

My strong impression is that if we could put error bars on the results of observational nutrition studies, that they’d be appalling. That goes both for the amounts and types of food that people are putting down as having eaten (or avoided) and for the conclusions about human disease. Just a look at the constant weathervaning over the last thirty years should be enough evidence: coffee/wine/butter/what-have-you are good, bad, good again, actually sort of bad, good for some people, bad compared to X, good compared to Y, were good all along, were thought to be good but now are bad. . .it’s never-ending. And that’s because human nutrition is extremely complex, varies from person to person (and population to population), and because the numbers are crap.

Short of arsenic berries, should those exist, it’s not easy to pin down One Specific Food that has a gigantic, inarguable effect on long-term human health. Yeah, I think that a twenty-year diet of bacon-wrapped Twinkies would probably be inadvisable. But so would a twenty-year diet of nothing but brown rice and zucchini. We should all try to stay in between those two, and stop reading food headlines.

43 comments on “The Sad State of Human Dietary Research”

  1. Vader

    “Yeah, I think that a twenty-year diet of bacon-wrapped Twinkies would probably be inadvisable. But so would a twenty-year diet of nothing but brown rice and zucchini.”

    Yeah. But at least the first option includes a complete protein.

    And has the advantage that you’ll enjoy your meals before scurvy sets in.

    1. Oliver in reply to Vader

      <3 that comment makes me happy.

  2. John Thacker

    These saddest one I’ve seen is this study from the University of Maryland about a specific brand of chocolate milk and concussions.

    1. Allan in reply to John Thacker

      Appalling! Nearly criminal. What University in its right mind would put this out as a press release?? Especially given the last sentence “The aforementioned study results are preliminary and have not been subjected to the peer review scientific process.”

  3. Rule (of 5) Breaker

    Ironically, as I am reading the comments here, there is an ad on the side for “66 Powerful Superfoods packed into a 1 oz shot.” I mean…wow. Anyway, I have always considered that being a scientist comes with the pain of having to endure just how scientifically unaware the general population is. This is just another example of the burden we must carry.

    1. Bubba Zinetti in reply to Rule (of 5) Breaker

      This, so much this. I sometimes feel that if I hear one more person say something about detoxing, or boosting their immune system I will go crazy.

      1. cancer_man in reply to Bubba Zinetti

        Just as economists have to listen to scientists go on about some economic theory they came up with over lunch with their friend.

  4. Kevin Parker

    Or you could get a ‘genetic’ meal plan from DNAFit and ilk. It’s based on your genes* after all…

    1. Sam Adams the Dog in reply to Kevin Parker

      Well, it could be based on your microbiome.

    2. Brendan Gray in reply to Kevin Parker

      There was a diet book out about 10 years ago or so that suggested we eat diets based on our blood types. I thought it was interesting, except for all the foods that I liked that it told me to stay away from.

  5. agesilaus

    I came to that conclusion years ago after investigating the history behind sodium restrictions. It turns out that the origin of this bit of nonsense happened in 1939 when Dr. Walter Kempner hospitalized patients with malignant hypertension and put them on a 2000 cal rice and fruit diet. This was very low sodium and some of the patients improved. There was no other treatment available at the time and survival time after diagnosis was six months.

    And that is the basis for the years of sodium restriction that followed.

    I had a couple of arguments with my physician over this but gave up arguing with him and just ignored his sodium recommendations. And now I’ve been proven right, I haven’t checked with him to see if he changed his mind tho.

  6. johnnyboy

    Urg, “something is better than nothing” – my second least favorite phrase in research, right after “well that’s the way we’ve always done it”

  7. Gene

    This post made me think of this report on CNN:
    http://www.cnn.com/2010/HEALTH/11/08/twinkie.diet.professor/

    Especially the last paragraph.

  8. Pat

    Another thing about this junk science is that it makes the public more skeptical of science in general. If we keep changing our minds about something as apparently simple as whether coffee is good or bad for people, why should anybody believe we know what we’re talking about with vaccinations or global warming? And when it’s five thirty-eight that calls out this stuff, rather than any of the supposedly rigorous scientific societies, why should the public trust our procedures of replication and peer review?

    1. Kyle Serikawa in reply to Pat

      Actually, I think the deeper issue is not that this makes the public more skeptical of science, it’s that the public has a poor understanding of how science works in general. Science communication is definitely an issue here–both on the part of PR departments who oversell every incremental finding, and on the part of the press which, while having some very thoughtful, nuanced and excellent communicators, also has some people who report poorly and give the impression that things are clearer than they are. To flip your thought around, I think if the public had a better understanding of how science works, they would expect changes in knowledge and that would make them more confident in peer review and replication. And they would be forgiving when our understanding of and recommendations for health-related actions change. Although that perhaps is wishful thinking, since our resistance to change and desire for certainty seems baked in…

  9. Pharma Newbie

    In case you have not seen this one before: https://xkcd.com/882/

    Am I alone in thinking we need a big repository of negative results?

  10. z

    Well, rice is a (comparably) decent source of arsenic should you desire that.

    Hmm, their statement that our bodies are “pretty good at telling us what we need” makes me wonder if there’s been much research on cravings and how they are generated (and possibly manually tested/augmented for those that can’t “hear” their bodies that well).

  11. cthulhu

    Gary Taubes’ “Good Calories, Bad Calories” and Nina Teicholtz’s “The Big Fat Surprise” do a good job of documenting the incredible amount of sloppy and downright fraudulent science that has gone into the nutrition field. Taubes and Dr. Peter Attia, a mechanical engineer + MD, founded the Nutrition Science Institute in 2012 with the mission to do the rigorous experiments that are needed to truly start unraveling the unknowns in the science of human nutrition. I hope they succeed. (I am not affiliated with anybody in this field, but as somebody with a hard science background who is struggling with some metabolic issues, I despair of finding reliable information – and the government dietary guidelines are among the worst offenders.)

  12. Ash (Wavefunction)

    One could say something similar about a lot of psychology and economics research.

    1. Jeff in reply to Ash (Wavefunction)

      Hey! Don’t go knocking Economics. Sure, we come up with some tenuous conclusions but our issue is massive volumes of data some of which is GREAT and some of which is TERRIBLE and significant difficulty in differentiating the two. Contrast that to the nutrition studies that are pretty much universally horrible and it’s night and day.

  13. Anon

    And yet “the woman who only ever eats Monster Munch crisps/chips seems perfectly healthy:

    http://www.mirror.co.uk/news/uk-news/the-woman-who-only-ever-eats-232644

  14. Emjeff

    People lie on diaries?? I’m, shocked, shocked. Well, no, I’m not. Ever look up patient adherence to medications? You’ll see what liars and cheats patients are about how they take their medicines – I’d expect the same with regard to what they eat. Surprising it’s taken this long to come to that conclusion…

  15. These studies aren’t easy to do. One such has been done in the past year, and it shows the folly of general dietary guidelines (even for diabetics who really need them).

    Look at how much work was actually done

    [ Cell vol. 163 pp. 1079 – 1094 ’15 ] People who eat identical meals display different blood glucose levels afterwards, thanks in part to differences in their gut microbes . Large spikes in blood glucose after eating increase the risk of type 2 diabetes, so dietary guidelines rank foods based on their glycemic index — an indicator of their effects on blood glucose.

    Each individual was connected to a continuous glucose monitor which measured interstitial fluid glucose every 5 minutes for 7 full days, using subcutaneous sensors which are as accurate as blood measurements.

    Eran Elinav and Eran Segal of the Weizmann Institute of Science in Rehovot, Israel, and their colleagues continuously monitored the diets and lifestyles of 800 people over a week (52,005 meals), and found that meals with the same glycaemic index caused widely different glucose levels in participants (e.g. it was useless for prediction in a given individual). By analysing data on the participants’ gut microbiomes, physical activity and other clinical factors, the team created personalized diets for 26 people and found that these resulted in lower glucose levels after meals than did non-personalized diets. They also created an algorithm to predict glycemic indices in a given individual (based on blood parameters, dietary habits, anthropometrics physical activity and gut microbiota) and tested it in an independent 100 person cohort and showed that it accurately predicted the postprandial glycemic index.

    The study could partly explain the limited efficacy of universal dietary guidelines.

    [ Nature vol. 528 pp. 484 – 486 ’15 (editorial on the Cell paper) ] During the week some 5,107 standardized meals were given to the 800 (but otherwise they ate their regular diet during the week). There were 1.5 million glucose measurements. A banana could have a bigger glycemic index than a cookie in one person with the reverse in another.

    As expected, increased carbohydrate consumption was closely tied to a raised PostPrandial Glucose Response (PPGR). Fiber increased the PPGR shortly after eating but decreased over the next day.

    Akkermansia muciniphilia bacteria degrade the glycoprotein mucins lining the gut. Their presence is associated with higher PPGRs.

  16. Ricky Morse

    Hi! Did you mean ” attempts at recalling what you’ve had to heat over the past three months” to have the word “heat” in it? Or should that be “eat”?

  17. Fred the Fourth

    The law blogger Ann Althouse is very fond (as am I) of the expression “Better than nothing is a high bar.”

  18. HFM

    I agree with Pat – IMO, the diet-by-epidemiology efforts over the past few decades have done more damage to the credibility of medical science than a busload of Shrekli clones could dream of. It’s clickbait-bait of a very high order, and most of it is indeed terribly sloppy work, so the BS headlines reverse themselves with every new study. It makes us look like idiots.

    I am encouraged by some recent efforts to do it right, especially with NuSI coming online. But my goodness, what a mess.

    (I’m on a medical ketogenic diet – as in, I get a veto-proof majority of my calories from saturated fat and cholesterol, because my nervous system gets cranky otherwise. By conventional wisdom I should be heading straight for an octuple bypass…)

    1. Chiara in reply to HFM

      And perhaps you will.

  19. gippgig

    There is an underlying problem that getting accurate results requires solving a system of nonlinear equations in darn near infinite variables. This could require a study size larger than the human population. Reminds me of the Heisenberg Uncertainty Principle.

  20. There’s only one diet!
    Pasta+Wine+Espresso followed by some dark chocolate.

    1. Me in reply to jbosch

      Do you take that as a concentrated shot in a power shake?

    2. dearieme in reply to jbosch

      We call that the Friday Evening Diet.

  21. Anon

    The amount of complete BS out there is scary. I once read a report that chewing willow bark could alleviate headaches. Unbelievable, and so dangerous to make such nonsnse claims!

    1. Chiara in reply to Anon

      🙂

  22. david

    One basis of the problem is the old adage “you are what you eat”, which has been the basis of lots of fad diets. “Don’t eat fat and you won’t be fat”. “Eating things with Cholesterol causes high cholesterol.” Another is assuming that if a large group of people “tend” to eat certain foods then that must be why their health is different from other groups. Why doesn’t the FDA actually base recommendations based on real data and real outcomes? Until they do, I think their recommendations are completely bogus.

  23. flem

    I’m waiting for an infomercial selling newly formulated AIR for weight-loss: based on the scientific fact that “We are what we Breath” 65% of our mass is oxygen.. “Order now and get twice the AIR(c) (only pay for shipping and handling)”

  24. Anon

    If “we are what we eat”, then why aren’t we either cows or cannibals?

  25. Jada Loris

    “The amount of complete BS out there is scary. I once read a report that chewing willow bark could alleviate headaches. Unbelievable, and so dangerous to make such nonsnse [sic] claims!”

    Yeah, well, this:
    http://www.drugs.com/npp/willow-bark.html

  26. newnickname

    The USDA used to maintain the list of “Oxygen Radical Absorbance Capacity (ORAC) of Selected Foods” that companies would use to justify making health claims about their beans or berries. In 2010, the UDSA the ORAC list with the following note:
    http://www.ars.usda.gov/services/docs.htm?docid=15866

    “Recently [2010] the USDA’s Nutrient Data Laboratory (NDL) removed the USDA ORAC Database for Selected Foods from the NDL website due to mounting evidence that the values indicating antioxidant capacity have no relevance to the effects of specific bioactive compounds, including polyphenols on human health.

    There are a number of bioactive compounds which are theorized to have a role in preventing or ameliorating various chronic diseases such as cancer, coronary vascular disease, Alzheimer’s, and diabetes. However, the associated metabolic pathways are not completely understood and non-antioxidant mechanisms, still undefined, may be responsible. ORAC values are routinely misused by food and dietary supplement manufacturing companies to promote their products and by consumers to guide their food and dietary supplement choices. […]”

  27. newnickname

    Ooops. Typos. Corrected: “In 2010, the USDA removed the ORAC database with the following note.”

  28. Crimso

    I’m reminded of an interview some years ago on NPR with a physician regarding the Atkins diet. The interviewer gave a quick description of the diet, and then asserted (in the form of a question to the MD) that such a diet didn’t seem to make any sense (something like “it seems like you can eat as much fat and protein as you want and still lose weight”). The MD responded “Well it does seem to violate the laws of thermodynamics, doesn’t it?” I was screaming at the radio “Material and energy balances!!!!! I had a whole class in it!!!!!! THINK ABOUT IT!!!!!!!!!” They are assuming that everything you put in your mouth=calories in your body. Not so. As most of you understand, it ain’t in your body yet.

    So this got me to thinking about how much caloric “intake” is actually absorbed by your body, and how much passes on through. The Gibbs free energy of feces is not 0. So I did some cursory literature searches, and this is where things get hazy for me. I could have sworn I read work published by Burkitt wherein he actually did measure caloric values of food intake and (presumably) caloric content of feces to see how many calories were actually being absorbed. IIRC, he did this work in Africa, and compared his results for British soldiers stationed in the area (who had their own particular diet) with the local population. I recall that anywhere from 40% to 60% of the calories were actually absorbed, and it differed between the two populations (not surprising). My problem is that I’m having a really difficult time tracking down exactly where I read that. Anybody got that reference, or did I just dream that I read it?

  29. Mark

    Red meat was in the headlines again in the UK a few weeks ago – the cancer scare stories associated with it never seem to go away. David Colquhoun did an excellent article on the subject recently:

    http://www.dcscience.net/2013/04/13/another-update-red-meat-doesnt-kill-you-but-the-spin-is-fascinating/

    The take-home message is, as studies investigating the link between red meat and cancer get bigger in size, effect size goes down. To quote: “This reduction in size of the effect as samples get bigger is exactly what’s expected for spurious correlations… The estimate of the harm done by red meat has vanished entirely in the latest study.” (I think this latest study even included the infamous cured red meat, complete with evil” nitrites and all).

    Point is, the same is probably true of most studies in the field of nutrition: they are seriously underpowered. But they do make good headlines (and possibly grant applications), so I doubt they’re going away any time soon.

  30. Anonymous Researcher snaw

    Interesting study finds that individuals respond very differently to food.

    http://www.bbc.com/news/magazine-35193414

Leave a Reply

Your email address will not be published. Required fields are marked *

Solve the math problem. *