This article is a collaboration between 鶹ý and:
A vast trove of dietary data used in the formulation of nutrition guidelines for millions of America is unreliable and should not be used, claims a new review.
The National Health and Nutrition Survey (NHANES) started in the 1960s but has been done every year since 1999, and it relies on respondents recalling what they ate. But most men and women don't correctly remember what they ate and report inaccurate values, says , at the University of Alabama Birmingham School of Public Health, the lead author of the study.
In fact, research shows that 67.3% of women and 58.7% of men gave an estimated intake that wasn't "physiologically plausible," meaning they reported either so many or so few calories that they "could not have survived" on the values they presented, according to Archer. He and his colleagues published their review in on June 9.
"The essence of science is the ability to discern fact from fiction," said Archer in an interview with 鶹ý. "But in nutrition and obesity research there is no accountability -- the health of our population is at risk." He added that relying on NHANES for accurate dietary data is "pure pseudoscience."
These findings aren't new. Archer wrote about them in a study in 2013, and questions have been raised about the reliability of memory-based dietary assessment methods for more than a decade. But Archer said that the findings were largely ignored by the dietary community when his 2013 paper came out. And when they were addressed, researchers conceded that the data were limited.
But Archer said that the inaccuracies aren't just a limitation of the data; they're a refutation. "On the population level, the NHANES being representative of energy and nutrient intake is patently false," he said. But with this latest paper, Archer is again finding a skeptical dietary community.
The Study
Archer and colleagues wrote the study to show that "the explanatory and predictive failure of epidemiologic nutrition research is explained by its reliance" on memory-based recall methods, according to the study. "The biggest public health problem we have isn't ignorance -- it's the illusion of knowledge," said Archer in an interview.
In the study, the authors first gave evidence that recall methods bear little relation to actual energy intake. This was based mostly on Archer's earlier study. The authors then reviewed research of the human memory, attempting to show that to assume that the brain provides literal or precise reproductions of past events is naive. In addition, they argued that the data from recall methods aren't subject to independent observation and are "pseudoscientific and inadmissible."
Archer's 2013 study used different methods to determine the validity and plausibility of NHANES data. In an editorial in in 2013, , concluded that the reported energy intake measures inferred from the questionnaire are "incompatible with life."
There are statistical problems with using recall methods, added Archer. Many people under-reported how many calories they consumed, but some also over-reported. If one person under-reported 500 calories, and another over-reported that same amount, it would look as if the survey was accurate, said Archer.
"But if you put your head in the fridge, and your feet in the fridge, it doesn't mean that you're experiencing average temperatures," he said.
The timing of the article coincides with the formation of new guidelines from the 2015 , which used mostly data from the NHANES to make recommendations. The guidelines -- a draft of which was published online earlier this year -- have faced from various sources. A final version is expected later this year. But Archer and colleagues are arguing that the heavy reliance of the data on recall is a fundamental shortcoming of the committee.
"Attempting to develop recommendations to improve health is a complex enterprise because of the interactive nature of genetics, environmental factors, and individual behavior," wrote two authors of an , , and , both at the Virginia Polytechnic Institute and State University.
The authors suggested that several tools can be used to help increase the accuracy of dietary recall, including the doubly labeled water technique, which tracks energy intake.
In that editorial, the authors referred to seeking to test the validity of NHANES data by using the doubly labeled water technique. But Archer said that even then, a significant number of the obese, female patients in the survey of 524 volunteers under-reported by 500 calories. "That's the equivalent of a body thermometer that measures human body temperature at 80 degrees," he said.
, who works in human nutrition at the United States Department of Agriculture, has also studied the doubly bonded method before.
"Dr. Archer's approach was to take existing data from NHANES, use only a single 24-hour recall of the two collected, apply his own mathematical formula, and conclude that the total calorie estimate was not physiologically plausible," wrote Klurfeld in an email to 鶹ý. "It has been well known for decades that because of the day-to-day variability in intake, more than a single 24-hour recall is required."
Klurfeld added that in the study he and his colleagues did, they found that normal-weight people underestimated by only 3%. Obese people under-reported by 18%.
Another flaw of Archer's research, Klurfeld added, is that he "extrapolated from a study designed to provide a cross-sectional snapshot of the population average intake to a single individual." He added that "NHANES is often used incorrectly for such purposes by academic researchers."
But Archer said that the study by Klurfeld isn't looking at the same population of NHANES. And under-reporting by 18% is significant -- that can be 500 calories, which is physiologically implausible, according to Archer.
Conflicts of Interest, and a Generational Theory of Obesity
At least two prominent obesity researchers questioned the ties to Coca-Cola that Archer and his colleague reported.
"This is a fairly highly funded global campaign for them to show diet data is not good and only physical activity data are, and that reduced physical activity is the cause of all obesity," said , at the University of North Carolina, in an email to 鶹ý.
And in an interview, he added that he thinks there are times and places to use diet data. "[Archer's] sweeping conclusions are rather unscientific based on one set of crude methods that others have questioned," he added.
One of the three authors was , of the Ochsner Health System in Louisiana. Lavie reported also receiving fees from Coca-Cola, and he has written a book The Obesity Paradox, which looks at why skinnier people are sometimes unhealthier than overweight or obese people.
, at the T.H. Chan School of Public Health at Harvard, wrote in an email that the latest review was an attempt by Archer and his colleagues to discount the role of diet in obesity. They are "paid by big soda to try to discredit anything that might threaten their product."
But Archer said he found these accusations offensive. "Before I came back to science I was playing polo on the beach in Florida," he said. "The honoraria from Coke wouldn't have paid my bar tab." The accusations have also prompted Archer to accuse his detractors of -- a tactic named after a politically successful biologist in the Soviet Union who manipulated data to reach a pre-determined conclusion that was in line with Soviet ideology.
"When they can't attack the science, they attack the scientist," Archer said.
One researcher, Yoni Freedhoff, MD, an obesity doctor in Ottawa, wrote in a that he found the review on NHANES data compelling. But he said that much of the review is focused on a generational theory of obesity, which Freedhoff thinks lacks evidence. That theory argues that obesity is the result of nongenetic evolutionary forces like social or cultural evolution, which have led fat cells to dominate.
"What struck me from the piece was the fact that despite his thesis that self-reported data was unreliable and consequently conclusions on its basis couldn't be made, he seemed very comfortable making the very broad, sweeping, and entirely non-evidence-based conclusion that the standard American diet is not only not harmful, but is in fact responsible for improvements to morbidity and mortality," wrote Freedhoff in an email to 鶹ý.
But Archer said he thinks there is data to back him up. He recently on his hypothesis, but the causes of obesity are still not very well understood.
Whatever the case, there's little debate that the NHANES data are in some way flawed. In cited by Archer and colleagues, researchers admitted that, "We measure energy so poorly" and that the 24-hour recall used by NHANES "may be particularly problematic in the obese." Archer said that's an important admittance, and that these weaknesses are significant enough to be extremely skeptical of the dietary data.
"The greatest problem that we have in obesity research is not being able to differentiate fact from fiction," he said.
Disclosures
The research was supported by a grant from the National Institute of Diabetes and Digestive Kidney Diseases of the National Institutes of Health.
Archer has received honoraria from the International Life Sciences Institute and the Coca-Cola Company.
Lavie has received consulting and speaking fees from the Coca-Cola Company and has written a book on obesity.
Primary Source
Mayo Clinic Proceedings
Archer E, et al "The inadmissibility of what we eat in America and NHANES dietary data in nutrition and obesity research and the scientific formulation of national dietary guidelines" Mayo Clin Proc 2015; DOI: 10.1016/j.mayocp.2015.04.009.