Dana Ullman, a notorious homeopathy apologist, actually has a regular blog over at HuffPo. For those of use who follow such things, the start of his blog there marked the point of no return for the Huffington Post – clearly the editors had decided to go the path of Saruman and “abandon reason for madness.” They gave up any pretense of caring about scientific integrity and became a rag of pseudoscience.

Ullman’s recent blog post is typical of his style – it is the braggadocio of homeopathy. I am sure others will skeptically dissect his piece so I won’t go into every point here. I want to focus on Ullman’s claim that the clinical and basic science research supports homeopathy. Here is the paragraph on which I want to focus:

Most clinical research conducted on homeopathic medicines that has been published in peer-review journals have shown positive clinical results,(3, 4) especially in the treatment of respiratory allergies (5, 6), influenza, (7) fibromyalgia, (8, 9) rheumatoid arthritis, (10) childhood diarrhea, (11) post-surgical abdominal surgery recovery, (12) attention deficit disorder, (13) and reduction in the side effects of conventional cancer treatments. (14) In addition to clinical trials, several hundred basic science studies have confirmed the biological activity of homeopathic medicines. One type of basic science trials, called in vitro studies, found 67 experiments (1/3 of them replications) and nearly 3/4 of all replications were positive. (15, 16)

Those numbers are references that allegedly support his claims – 14 papers (they are not all studies, some are reviews) that allegedly make the case that homeopathy works. Most reader do not independently check references to see if they say what the author claims. Some may foolishly assume that the editors at the HuffPo have done that already.

First, Ullman is a notorious cherry picker. Any large and complex body of research will have enough noise that you could support just about any claim you wish regarding the research in you cherry pick only results that support your conclusions. The way to get to the essence of a body of research is through a systematic review – a review that looks at all the research and examines each piece for quality. You need to examine the relationship between quality of research and outcome. Ullman, rather, prefers to simply count studies.

I don’t know if his claim that most homeopathy studies show positive results is true, but I am willing to concede that this is probably true – because this is true of most research areas, even into therapies that we now know do not work. We know from the work of John Ioannidis that most published studies, in retrospect, are wrong. This is because there is a large amount of preliminary and poorly controlled research leading up to the large definitive trials that finally answer questions. Preliminary research is unreliable and biased – most of it is wrong. But we can still get to reliable answers in the end. Meanwhile, there is also researcher bias, publication bias, and the various placebo effects that conspire to make medical research look positive, even when there is no effect.

Ullman’s first reference to support his claim, however, is this meta-analysis: Are the clinical effects of homeopathy placebo effects? A meta-analysis of placebo-controlled trials. This study does not show that most published homeopathy studies are positive – that’s not what a meta-analysis is for. Here is what they concluded:

The results of our meta-analysis are not compatible with the hypothesis that the clinical effects of homeopathy are completely due to placebo. However, we found insufficient evidence from these studies that homeopathy is clearly efficacious for any single clinical condition. Further research on homeopathy is warranted provided it is rigorous and systematic.

This is hardly a ringing endorsement of homeopathy – and that’s the best reference Ullman could come up with. I disagree with the authors that the evidence is not compatible with placebo – I think it is, even by their own data. The authors should read John Ioaniddis. These results are perfectly compatible with just the expected noise of clinical research. But they were on the money with their second sentence – you cannot conclude from the evidence that homeopathy actually works for any specific indication. This is a good clue in itself that we are dealing with noise – when you focus on any one indication, the evidence is not there. Yet Ullman includes this as a reference in a paragraph in which he claims the opposite.

His second reference to back this point does not even address his point. It was a re-analysis of the Shang study that showed that homeopathic treatments are placebos, and the analysis concluded that “The conclusions on the effectiveness of homeopathy highly depend on the set of analyzed trials.” Well, of course they do. That’s why systematic reviews are better than meta-analysis. What do the systematic reviews of homeopathy show? Edzard Ernst did a systematic review of systematic reviews of homeopathy – that’s about as thorough as you can get. He found:

The findings of currently available Cochrane reviews of studies of homeopathy do not show that homeopathic medicines have effects beyond placebo.

Ullman then goes on to claim that there is evidence for homeopathy for specific conditions – despite the conclusions of his own references that he neglected to mention. I have to note at this point that Ullman takes on the skeptics in his article, writing:

It is remarkable enough that many skeptics of homeopathy actually say that there is “no research” that has shows that homeopathic medicines work. Such statements are clearly false, and yet, such assertions are common on the Internet and even in some peer-review articles.

This is a typical Ullman strawman. Skeptics don’t say there is “no research” – what we say is that there is “no good research” – meaning large, blinded, placebo controlled trials that show a replicably positive effect. What we do see is the positively-biased noise of placebo vs placebo research. The better controlled the study, the smaller the effect and greater the chance of no effect. Systematic reviews reveal this pattern – Ullman’s cherry picking does not.

Ullman references one study and his own review for the next claim dealing with rhinitis. But an independent review, which Ullman did not reference, found:

Some positive results were described with homeopathy in good-quality trials in rhinitis, but a number of negative studies were also found. Therefore it is not possible to provide evidence-based recommendations for homeopathy in the treatment of allergic rhinitis, and further trials are needed.

Next up is homeopathy for influenza. He chose a Cochrane review of Oscillococcinum for influenza. There are two big problems with this reference. The first is the conclusion:

Though promising, the data were not strong enough to make a general recommendation to use Oscillococcinum for first-line treatment of influenza and influenza-like syndromes. Further research is warranted but the required sample sizes are large. Current evidence does not support a preventative effect of Oscillococcinum-like homeopathic medicines in influenza and influenza-like syndromes.

It seems Ullman saw the word “promising” and his eyes glazed over with such joy that he could not read the rest of the conclusion. He also missed the other major problem with this reference – it has been WITHDRAWN (in big capital letters at the beginning of the title). So even the wishy-washy support for this treatment was thought to be not up to the Cochrane’s usual standards (which, in my opinion, have slipped recently). If you want to see how silly Oscillococcinum is (beyond the generic homeopathic silliness) read this article by Mark Crislip.

I could go on in detail, but it will get tedious. Suffice it to say that the rest of Ullman’s references show the same pattern – they are to small or unblinded studies with weak evidence, or reviews of the same. Most are unblinded, which in this context means they are worthless. The one blinded study he directly references is for homeopathy in ADHD – which was a small study with barely significant results. Again – I prefer systematic reviews, like this one, which concludes:

There is currently little evidence for the efficacy of homeopathy for the treatment of ADHD. Development of optimal treatment protocols is recommended prior to further randomised controlled trials being undertaken.

Conclusion

Again – there is no good evidence for homeopathy, but there is low-grade evidence that apologists like Ullman can cherry pick and misrepresent. Ullman’s references do not support the claims he is making – sometimes directly contradicting them. But most readers will just see lots of reference numbers after Ullman’s claims and be impressed – and that’s probably what he is counting on.

The goal of the apologist is to provide cover, not to make a fair and scholarly assessment of the evidence. I think we can see what Ullman is doing here.