The interpretation of clinical trials is very complex, even for practicing physicians who have to base medical decisions on such evidence. For the public, who rely heavily on media-filtered information about clinical research, the task is almost impossible. The result is general confusion.<\/p>\n
The story of Echinacea, and whether or not it is useful for the treatment of the common cold, is a good example of the complexities involved \u2013 especially when dealing with an unregulated product marketed directly to the consumer. Echinacea has been marketed for decades as a cold remedy. Early clinical trials showed mixed results, but proponents cherry picked the positive studies to promote Echinacea to the public. Then a series of well designed trials all showed no effect, and it seemed that Echinacea would be relegated forever to the fringes of CAM.<\/p>\n
Now, a recent meta-analysis<\/a> of 14 studies involving the use of Echinacea for the treatment of the common cold\u00a0 found that \u201cPublished evidence supports echinacea’s benefit in decreasing the incidence and duration of the common cold.\u201d The press is promoting this as a new \u201cstudy\u201d that shows that Echinacea works.<\/p>\n Confusion ensues.<\/p>\n The Echinacea story reflects the basic misrepresentation of clinical evidence to the public, and even to professionals. Here are some basic principles to keep in mind when evaluating such evidence.<\/p>\n Confusion of Basic and Clinical Research<\/strong><\/p>\n Medical research is divided into basic science ( test tube and animal studies), and clinical research (studies on people). Often the purveyors of drugs, supplements, and all variety of medical devices and products will make clinical claims based upon basic science evidence. This is disingenuous and misleading.<\/p>\n For example, with Echinacea, there are studies showing that certain chemicals in certain preparations of certain parts of the Echinacea plant have affects on certain parts of the immune system, like T-cells and phagocytes. Echinacea promoters then extrapolate from these test-tube effects to the clinical claim that Echinacea \u201cboosts immune system function\u201d and then further from that to the claim that it helps the common cold.<\/p>\n But serious medical researchers know this to be utter folly. It is difficult to impossible to extrapolate from chemical or cellular reactions in a test tube to a net medical effect on an entire organism. What we think should happen based upon basic science information rarely predicts what exactly happens in people. Basic science should be used to guide clinical research, but it cannot replace and never should trump clinical research.<\/p>\n So rule #1 is to beware clinical claims based upon basic science alone.<\/p>\n Reliance on individual studies<\/strong><\/p>\n The medical research is a complex evolving beast, and to truly understand if a treatment works one must look not only at plausibility but all the clinical research to see what the overall pattern is. Individual studies are quirky, often biased or flawed, and never tell the whole story. It takes time \u2013 years \u2013 for a research question to work itself out to a reasonable degree of reliability.<\/p>\n Rule #2 \u2013 beware preliminary research, or conclusions based upon 1 or a few studies.<\/p>\n Confusing different types of \u201cstudies\u201d<\/strong><\/p>\n Often, the media will report anything published as a \u201cstudy,\u201d as they did in this case with the recent meta-analysis of Echinacea, as if it is new data or data that bears directly on efficacy. But a study could be a survey or it could be just a reanalysis of old data \u2013 as in this case. The meta-analysis did not present any new evidence for Echinacea. Calling it a \u201cstudy\u201d is misleading.<\/p>\n Rule #3 \u2013 Know what kind of study is being referred to.<\/p>\n Meta-analysis<\/strong><\/p>\n A meta-analysis takes several previous studies and then analyzes the data lumped together. It is a very problematic and complex analysis to do, and is often misused. The greatest weakness of a meta-analysis is that it lumps together different studies with different methods and outcome measures. There is also great risk of selection bias in which studies to analyze.<\/p>\n A meta-analysis misses what I think is a critical aspect of an evolving research literature on a question \u2013 how does the quality of the study relate to the size of an effect, if any? Also, do the various results present a consistent picture, or do they contradict each other.<\/p>\n For a large and complex literature, a systematic review is a much better tool for analyzing the research. Systematic reviews look at all studies on a question and look to see what the overall pattern is \u2013 what do the best studies show, how consistent are they, what was the effect on outcome of fixing criticisms of the early research, etc.<\/p>\n Rule #4 \u2013 Beware the meta-analysis<\/p>\n Anecdotes and Testimonies<\/strong><\/p>\n Do I even need to go into this again?<\/p>\n Rule #5 \u2013 Beware anecdotes and testimonies<\/p>\n Back to Echinacea<\/strong><\/p>\n With all this in mind, let\u2019s take another look at the Echinacea question. The plausibility of Echinacea is low but not zero. Unlike homeopathy, herbs contain actual chemicals that can have a pharmacological effect in the body. Herbs are drugs. But treating the common cold has a low probability. The cold is caused by a large number of viruses that are constantly evolving. There is no silver bullet that makes the immune system function better than it evolved to function against viruses, at least not that anyone has credibly found so far.<\/p>\n The basic science findings of Echinacea are not very compelling. They are the types of non-specific changes that can be found with hundreds of compounds, and basically just mean that the immune system is reacting to a foreign agent.<\/p>\n So the basic science is unconvincing and the prior probability is low but not zero. What about the clinical evidence?<\/p>\n I think this Cochrane Review <\/a>is a much better assessment of the evidence than any meta-analysis.\u00a0 It concludes that the data is inconsistent and there is insufficient evidence to conclude that Echinacea works for the common cold.<\/p>\n I am also compelled by the fact that the best studies<\/a> in terms of methods and controls were all negative.<\/a><\/p>\n I do not think that this current meta-analysis adds anything to the body of evidence on Echinacea, but it will serve to confuse the public on the issue. The herbalist community has happily accepted the results and will contribute to the confusion.<\/p>\n