David Gorski and I have just published a paper in Trends in Molecular Medicine titled: Clinical trials of integrative medicine: testing whether magic works?

While we have published literally thousands of online articles discussing these issues here, at Science-Based Medicine, and other venues, it’s great to get an article in the peer-reviewed literature, which hopefully will spark more of a discussion in academic circles.

The full article is available online at the link above, but here’s a quick summary of the main points:

The question is – should we devote limited research resources to investigating CAM (complementary and alternative medicine) methods? Those resources include not only money, but researcher time, available patients, and space for reporting and discussion at conferences and in the published literature. CAM is actually a false category, in my opinion, used really as a marketing strategy and not a meaningful designation. It makes it difficult to answer this question, because we first have to operationally define CAM. (As an aside, “integrative” medicine is essentially the same as CAM, just a different marketing term.)

The real question is – how far down the scale of plausibility should we go in allocating research resources? Should it matter?

David and I argue that it does matter. The primary reason is that the probability of scoring a hit (finding a treatment that works) becomes lower and lower as the plausibility of the treatment decreases. Of course we now have to define “plausibility.” Plausibility essentially refers to our best estimate of how likely a treatment is to work based upon all existing scientific evidence and our current scientific understanding of how the universe works.

If a treatment requires a dramatic change in our understanding of a disease, and that understanding has a solid basis in scientific evidence, then the treatment can be deemed relatively implausible. As an example, the liberation procedure for multiple sclerosis is based upon they hypothesis that MS is caused by blockages in the veins that drain blood from the brain. However, decades of research point to MS being primarily an autoimmune disorder.

This does not imply that our current understanding is always correct. We are talking about probability. Are we making a sound investment with this research, or playing the lottery.

Even more implausible would be treatments that violate basic laws of physics. In order for homeopathy to be true not only does our understanding of health and disease need to be wrong, but major areas of chemistry and physics would also need to be wrong. This gives homeopathy a plausibility that approaches zero.

Researching unlikely treatments is problematic for reasons other than just a low probability of finding an effective treatment. Statistical analysis and reviews of the published medical literature (specifically the work of John Ioannidis) find that the less plausible a hypothesis is the more likely it is to generate false positive outcomes.

Preliminary research itself has a false positive bias, and when such exploratory studies are positive they provoke follow up research which is larger, more rigorous, and therefore consumes more research resources. It may take 10-20 years of pre-clinical and clinical research to properly sort out if a treatment hypothesis is valid. This problem is exacerbated and more wasteful the less plausible the original hypothesis.

Therefore, from a purely scientific and statistical perspective, research into highly implausible treatments is very likely to waste vast amounts of research resources without finding effective treatments. In fact, this has been the exact experience of the National Center for Complementary and Alternative Medicine (NCCAM). Through them Americans have invested billions of dollars in researching implausible CAM therapies, with nothing to show for it.

There are also less tangible social reasons to be concerned about researching implausible therapies. Proponents of such therapies (which usually means people with a vested interest in selling them) often point to the very fact that a treatment is being studied to argue that it is therefore legitimate (why else would they study it?). Combine with this the tendency of preliminary studies to be false positives, and a completely bogus therapy may enjoy 20 years of false legitimacy before definitive trials show that it is completely useless.

Even worse, CAM does not occur in a vacuum. It is part of a culture that “uses science as a drunk uses a lamp post, for support rather than illumination.” They have a history of distorting the process of science to suit their needs, engaging in bait-and-switch tactics, and attempting to change the rules of science in order to make negative evidence seem favorable. CAM often goes hand-in-hand with an anti-scientific, unscientific, or pseudoscientific world view. It is difficult to quantify the overall negative effect this has on our society.

Conclusion

Researching highly implausible medical therapies is very unlikely to result in the addition of new treatments to our medical options. It is unlikely to actually help reduce human disease and suffering.

It is very likely, however, to waste research and medical resources, to exploit desperate patients, create false hope, and distort the process of science and medicine. It is feeding into a culture that wishes to reduce or eliminate the standard of care in medicine, and to empower anyone who wishes to hang a shingle to sell any nonsense they choose with hyped health claims. The only real net effect of CAM research is to offer a patina of legitimacy to nonsense in order to better exploit the public with snake oil.