This is a great idea, and in fact is long overdue. The NIH is awarding various grants to establish educational materials and centers to teach principles of scientific rigor to researchers. This may seem redundant, but it absolutely isn’t.

At present principles of research are taught in basic form during scientific courses, but advanced principles are largely left to individual mentorship. This creates a great deal of variability in how well researchers really understand the principles of scientific rigor. As a result, a lot of research falls short of scientific ideals. This creates a great deal of waste in the system. NIH, as a funding institution, has a great deal of incentive to reduce this waste.

The primary mechanism will be to create teaching modules that then can be made freely available to educational and research institutions. These modules would cover:

biases in research; logical fallacies around causality; how to develop hypotheses; designing literature searches; identifying experimental variables; and reducing confounding variables in research.

Sounds like a good start. The “biases in research” is a broad category, so I’m not sure how thorough coverage will be. I would explicitly include as an area of education – how to avoid p-hacking. Perhaps this could be part of a broader category on how to properly use statistic in research, the limits of the p-value, and the importance of using other statistical methods like effect sizes and Bayesian analysis.

Prior research has shown that when asked about their research behavior, about a third of researchers admit (anonymously) to bad behavior that amounts to p-hacking. This is likely mostly innocent and naive. I lecture about this topic all the time myself, and I find that many researchers are unfamiliar with the more nuanced aspects of scientific rigor.

This is good news and bad news, in that this is clearly a problem, but it is also one with an obvious fix. That is where the NIH initiative comes in. The idea is to make education about scientific rigor more uniform. The target population are already highly educated, used to continuing education, and already exist within an educational infrastructure. This is a true knowledge deficit problem that is easily fixed.

In fact, I would go as far (perhaps this will be phase 2) as requiring certification in the principles of scientific rigor in order to be eligible for NIH grants. Once the educational modules are created and in use, this would be a trivial step to add – just make an online test providing certification. Academics and clinicians already do this for many purposes, adding one more would not be a burden.

And of course, once the NIH requires certification, this will almost certainly make it uniform within academia, at least on the biomedical side. Then we need other research granting institutions to replicate this, also requiring certification. It basically should become impossible to have a career as a researcher in any field without some basic certification in the principles of research rigor.

This is also completely fair. If you are going to request public funding for your research, you should be required to document minimal competency in how not to waste those funds. To me this is a no-brainer, and the infrastructure already exists to add this requirement. Creating the educational program is really the only piece that is missing.

This also represents a broader trend in certain professions, such as medicine – moving away from an individual mentor model to a standardized training and certification model. The individual mentor will not, and really cannot, be eliminated from the training process. The point is not relying on individual training to assure basic competency. This has to be standardized in some way.

The NIH initiate is also designed to be transparent and bidirectional, meaning that researchers can provide feedback to make the material and the process better. They will be full participants, and not just have this system imposed upon them. That is typical in academia, and is part of the strength of the system.

I look forward to seeing what this initiative produces, and hopefully participate in it as well.