Can you recall everything you ate yesterday, including those little snacks you snuck, with reasonably accurate estimates of amounts? How about two days ago? Most people, it turns out, have a hard time recalling with good fidelity their recent food intake, or they have a hard time reporting it accurately not only because of the fallibility of memory but because of biases and denial.

Despite this, a great deal of nutrition research is based upon subjects recalling and accurately reporting what they ate. A recent article in Mayo Clinic Proceedings by Edward Archer et al challenges the legitimacy of this research. They write:

The reliance on M-BMs to inform dietary policy continues despite decades of unequivocal evidence that M-BM (memory-based dietary assessment methods ) data bear little relation to actual energy and nutrient consumption.

This is pretty damming. They charge that the current dietary guidelines are based upon fatally flawed data. They specifically state that such data is based upon fallible memory, uses data collecting techniques known to promote false recall, cannot be independently verified, and often does not contain objective data on physical activity. They go as far as to call such research an, “unscientific and major misuse of research resources.”

The nutrition research community has had a variety of responses, from agreement to rejection. Some researchers have noted that this type of data has led to important discoveries, such as the connection between fat intake and vascular disease.

This debate is embedded in a deeper discussion over how science itself is conducted. I have been a strong advocate of using metascience – the science of science itself – to improve the institutions of science at every level, in order to make science more efficient, transparent, and reliable. The way I see it, over historical time using scientific methods has been fabulously successful at building a model of how the universe works. Scientific theories have proven incredibly useful, as measured by practical applications, predicting future observations and experimental outcomes, and building a coherent explanatory model of nature.

Further, science progresses first by picking the low hanging fruit, asking and answering basic questions. It then builds on this basic knowledge by asking ever more subtle, deep, and fundamental questions. There are numerous analogies to demonstrate this. One of my favorites was put forward by Isaac Asimov. Early scientists (really philosophers just making basic observations) figured out that the world was a sphere. Later more accurate observations refined this model, discovering that the earth bulges at the equator, so it’s an oblate spheroid. Satellite data which is highly accurate further discovered that the southern hemisphere is a bit larger than the norther hemisphere, so the earth is slightly pear-shaped as well. The idea that the earth is a sphere is still basically correct, but the details have been progressively refined.

In addition these observations about the earth can be combined with other theories and data concerning gravity, the shape of other planetoids, observations and theories about how solar systems form, the influence of the moon, data concerning the internal composition of the earth, thermodynamics, and other disciplines. It all ties together into one coherent description of reality.

Nutrition science is no different. Scientists previously worked out the basics of nutrition, including what fats, proteins, and carbohydrates are, how they are used for energy, their structural roles in the body, the existence of vitamins and minerals and what they do, and the various effects of malnutrition. They then built upon this basic knowledge asking deeper and deeper questions that are more subtle and detailed. There are different types of fat, and carbohydrates have different glycemic indexes, for example.

As the questions get more subtle and detailed, the research techniques need to become more rigorous. In this case, the memory-based diet research methods may have been adequate for picking out the big signals in the data, but are likely not adequate to see higher resolution details. It seems like a good time to question whether or not we need to abandon current methods of research in favor of higher rigor. In this specific case we have also learned more about the fallibility of memory which informs these research methods.

But let me also play devil’s advocate a bit here. In applied research context is very important. Certainly for basic nutrition research, where we are asking fundamental questions, we need research of the highest rigor feasible. However, for applied research the techniques used will depend greatly on the questions being asked. We are not asking how the world works, but rather how we can best apply that knowledge in real life.

Specifically, we are asking what people should eat to optimize health, maintain a desired weight, and compensate for specific disease states, such as diabetes. One could argue that we only need to detect big signals in this data, because anything more subtle would be too complex for the average person to apply in their daily lives anyway.

Many have also argued essentially that if we look for dietary effects that are too subtle we quickly get down into the noise, and this only serves to generate a great deal of confusion. A study, for example, found that 72% of common cookbook ingredients had studies showing they were associated with either an increase or decrease in cancer risk. A recent commentary by Thinking Nutrition highlights this problem further, showing that you can find evidence to suggest that broccoli is toxic. Everything is either toxic or a panacea, depending on which evidence you want to cherry pick.

The ultimate point is that all of this nutritional and health data is down in the noise and is best completely ignored by the average person.  It may not be worth trying to tweak your diet to this level. This follows the principle of diminishing returns. Getting the broad brushstrokes correct is certainly a good idea and is not difficult to understand and apply. The more detailed you try to get the more effort and perhaps expense that is required for smaller and smaller health returns. In fact, there may be a net health negative as big benefits are compromised in the pursuit of tiny and even illusory benefits.

Conclusion

The recent commentary by Archer does raise a very valid point – relying upon memory for dietary information is old technology and has perhaps surpassed its usefulness in nutrition science. As nutrition science asks more and more subtle questions, more rigorous research techniques are required. This higher level of detail does have practical applications, to the food industry, for example, and for crafting optimal supplements and diets for individuals with specific health conditions. Advancing our basic understanding of nutrition science also has generic benefits that go beyond any immediate application (as is true of basic science in general).

At the same time, applying nutrition science to everyday life is tricky. The media sensationally reports preliminary data, much of which cannot be replicated, that probably causes more confusion than useful advice for the average person. This data is useful to researchers, but there is a huge need for more and better communication on nutrition science to the public (not to diminish the sources that are out there).