Mar 26 2013

Debating Homeopathy Part II

Yesterday I discussed a recent debate in which I participated at UCONN, focusing on the plausibility of homeopathy. Today I will discuss the clinical evidence, and address some of the strategies employed by my opponent in the debate, Andre Saine.

Does Homeopathy Work?

Yesterday I made the case that homeopathy is highly implausible in many ways, and after two hundred years of scientific advance this extreme implausibility has only become greater. Two centuries has apparently not been enough time for homeopaths to make their case and convince the mainstream scientific community. The only reasonable explanation for this is that homeopathy is simply not valid.

I also took the position that overall scientific plausibility must be considered when looking at any new claim – how well does it comport with existing scientific evidence? In medicine this means, when considering clinical evidence for a treatment, that evidence needs to be put into the context of the scientific plausibility of the treatment.

Given homeopathy’s extreme implausibility, it would take a great deal of iron-clad scientific evidence to convince most scientists that there was a real effect.

However, the clinical evidence for homeopathy is so clearly negative, that even if we set aside plausibility arguments entirely we could still confidently conclude that homeopathy does not work. In other words, the clinical evidence regarding homeopathy is negative using any reasonable threshold, and not adjusting for the extreme implausibility.

Systematic Reviews

Individual studies are not a good way to understand the evidence base for any medical claim. Differences in methodology, rigor, and statistical fluctuations will ensure a random scatter of results. Meanwhile, researcher bias and publication bias will tend to bias that random scatter toward the positive end. For this reason, if you cherry pick individual studies on any topic that has been widely studied you can support any position you choose to take. Only by systematically looking at all the evidence for the overall pattern can you see what the science is really saying.

There are several ways to do this: systematic reviews are just that – they look at all published studies on a given question or topic and then come to a conclusion about what that evidence says.  One pattern to look for is the relationship between the quality of the study and the outcome. You can assess heterogeneity – do high quality studies tend to agree, or are the results all over the place? A systematic review may include a funnel-plot analysis – a measurement of publication bias.

Another way to look at multiple studies is a meta-analysis, which combines data from multiple studies into one new statistical analysis. This can be combined with a systematic review.

Finally there is a best-evidence synthesis, which is similar to a systematic review but emphasizes a detailed analysis of the best studies, including effect sizes, and the relationship between rigor and outcome. (It sounds very similar to the four criteria of compelling evidence I outlined in yesterday’s post.)

So – what do systematic reviews tell us about homeopathy? The most comprehensive such review was conducted by Edzard Ernst in 2010, a systematic review of systematic reviews, and he concluded:

The findings of currently available Cochrane reviews of studies of homeopathy do not show that homeopathic medicines have effects beyond placebo.

A 2000 review came to a similar conclusion:

There is some evidence that homeopathic treatments are more effective than placebo; however, the strength of this evidence is low because of the low methodological quality of the trials. Studies of high methodological quality were more likely to be negative than the lower quality studies. Further high quality studies are needed to confirm these results.

While the author tried to put a positive spin on the result, the penultimate line is where the money is – high quality studies were more likely to be negative than lower quality studies.

The evidence is clear and overwhelming, after years and hundred of trials, studies of homeopathy have been unable to generate convincing evidence that homeopathy works for anything. Different reviewers can spin the results in different ways depending on their biases, but the evidence is the same – no consistent, replicable, clinically significant effect seen in the highest quality studies.

Saine Responds

Saine echoed the standard response from the homeopathic community. First he played the “no true Scotsman” card, claiming that most studies of homeopathy are not really studies of homeopathy, but of “homeotherapeutics.” Only individualized treatment by a homeopath is true homeopathy, not the standardized treatment in most clinical trials.

The results of individualized homeopathy, however, are similar to that of standardized homeopathy. A recent review concludes:

The results of the available randomized trials suggest that individualized homeopathy has an effect over placebo. The evidence, however, is not convincing because of methodological shortcomings and inconsistencies. Future research should focus on replication of existing promising studies. New randomized studies should be preceded by pilot studies.

By now you can recognize the positive spin of a proponent and look past it – the bottom line is that the evidence is “not convincing.”

Saine also tried to dismiss the Ernst review by making an ad hominem attack. First he stated the straw man premise that I rely heavily on Ernst for my opinions (not true – I am capable of assessing the literature on my own, thank you), and then he character assassinated Ernst, saying he is not a “real” homeopath and therefore doesn’t know what he is doing. The fact that Ernst was the first professor of complementary and alternative medicine and has over 1000 published studies to his credit apparently did not impress Saine.

In any case – the evidence is what it is. Ernst is simply reviewing the evidence. His findings are also no different than anyone else, only the language used in the conclusion is different based on author bias.

At one point in the debate I was able to ask Saine, putting aside all his special pleading excuses,  if he thinks there is any indication for which homeopathy has been adequately proven to be effective. He cited the Jacobs 2003 review of three clinical studies of homeopathy for diarrhea. Jacobs concluded:

The results from these studies confirm that individualized homeopathic treatment decreases the duration of acute childhood diarrhea and suggest that larger sample sizes be used in future homeopathic research to ensure adequate statistical power. Homeopathy should be considered for use as an adjunct to oral rehydration for this illness.

However – other reviewers (Altunç U, Pittler MH, Ernst E.) looked at the same studies and concluded:

The evidence for attention-deficit/hyperactivity disorder and acute childhood diarrhea is mixed, showing both positive and negative results for their respective main outcome measures.

Further, a 2006 study by Jacobs of homeopathic combination therapy was entirely negative.

So – Saine referred to the 2003 review that was positive, despite the fact that other reviews of the same data were negative, and a high quality follow up study by Jacobs was negative. This was his best evidence for homeopathy, which supports my point – homeopathy has not been adequately demonstrated to work for anything.

Saine also spent a great deal of time quoting from epidemiological evidence, historical cases of homeopathy used to treat epidemics (mostly from the 19th century) and similar data. This is all, of course, uncontrolled and unblinded data.

The pattern is clear – when the rigorous, blinded, controlled data is negative, turn to the unblinded and uncontrolled data. That is the pattern of pseudoscience, used to support a claim that is not true.

Conclusion

There is a great deal more detail that I could go into, and Saine and I might engage in a written exchange to follow up our live debate. This would be a great opportunity to delve deeply into the research of homeopathy.

On every point proponents fail to make the case for homeopathy. It remains extremely implausible, and even setting aside that implausibility, the clinical evidence is negative. The pattern of results that we see are consistent with the null hypothesis – a scatter of results, a positive bias in the preliminary studies, but as methodological rigor increases studies tend to become negative. The evidence is a giant arrow pointing at the null hypothesis – homeopathy does not work.

The basic science and clinical evidence for homeopathy has not come anywhere close to the four criteria I outlined for compelling scientific evidence – statistically significant outcomes with adequate signal to noise ratios in high quality studies that survive replication.

What we have instead are excuses, special pleading, appeals to low grade evidence, some conspiracy mongering and bashing of mainstream medicine, ad hominem attacks, and other logical fallacies.

And yet Saine marvels at how skeptics can remain skeptical.

 

Share

19 responses so far

19 Responses to “Debating Homeopathy Part II”

  1. DavidCTon 26 Mar 2013 at 9:21 am

    How did this most reasonable set of arguments play to the house of true believers? Keeping the faith in the face of evidence seems to be the very human response. Sometimes you do start sowing he seeds of doubt. Thank you for making the effort.

    I am personally still waiting for some evidence before ordering my homeopathic starter kit from Dr. OZ. That makes me closed minded in the world of woo.

    A posting of the written exchange should be an interesting study in logical fallacies.

  2. champenoiseon 26 Mar 2013 at 9:37 am

    Kind of painful though, that after society has spent trillions of dollars in research over many decades and millions of people graduate from University (of Connecticut), this needs to be debated. Science is for science people and the rest is into magic.

  3. Heptronon 26 Mar 2013 at 12:20 pm

    So when it was all said and done, did you get laughed at again or did the crowd give you any sort of positive response (applause that wasn’t forced)?
    Also, did anyone come up to you after the show and ask you questions or seem like they had been ‘converted’?

  4. Lswanon 26 Mar 2013 at 5:35 pm

    If I understand Homeopathy, then sperm diluted in water should be the perfect birth control. I would ask the true believers to test that for about 12 months and bring in their results…lol

  5. HarryBlackon 26 Mar 2013 at 7:33 pm

    Back when I was a believer in Homeopathy I was totally unaware of any controversy. It just wasnt mentioned. In fact I was told many doctors were using it.
    Although the proposed mechanism seemed strange (And yes, now I can see how utterly ludicrous it is) I had faith in people close to me who said it worked and when I read about it I figured firstly – ‘Well if it absolutely didnt work then they wouldnt be allowed sell it right?’
    And secondly – ‘ No one could be brazen and dishonest enough to sell something at this level which they KNOW cant work right?’
    So I tried it for a small non specific problem I was having and hey presto the problem went away!
    I was totally sold. And I have a good friend who had the same problem, received the same treatment and also never suffered the problem again.
    She is more scientifically literate than I was but cant see past the anecdote.
    The sad thing is that the friend who introduced me to it is now, with the help of his true believer parents, taking a homeopathic alternative to the medication he should be on for his schizophrenia. Needless to say its not going well and Im utterly disgusted at the people who would take what cant be more than 100 euros to destroy the life of a promising young man.
    Although I never had a those thoughts about homeopathy specifically there were other pseudo scientific ideas that I couldnt believe were not accepted by science and mainstream medicine. I had ‘seen’ them work. My friends were peddlers and ‘saw’ them work everyday.
    In fact the only reason I was able to break away was that I have always been a little intellectually insecure and started learning about critical thinking to try and do something about that. My lack of investment in my own perceptions meant I was ready to accept that I may have interpreted something the wrong way and made an error of judgement. Because many people feel inadequate in one way or another, the only thing some of us feel pretty confident about is our memories and interpretation of events. I have found that questioning this is seen as questioning their very selves which most arent willing to do.
    I think until every adult leaves school with a knowledge of what studies are, why we do them, what good standards are and how to check them this issue will go on with the fence sitters leaning toward the wishful side of these debates and the true believers shaking their heads because they dont actually know the definition of good evidence.

    Steve, what have you generally found to be the after effects of such debates? do you often get anyone waking up and getting in touch to tell you so? My assumption would be that if someone is invested enough to attend a debate they have looked at the evidence and ignored it once already.

  6. Davdoodleson 26 Mar 2013 at 10:50 pm

    Again, this makes me wonder: EVEN the most sympathetic reading of Jacobs, itself, the best evidence Homeopaths can provide, says that homeopathy (ie water) might possibly shorten the duration of diarrhea somewhat better than just oral rehydration (ie other water) alone.

    It might, just possibly (they think) be better than nothing.

    On the other hand, Imodium already works really well.

    So, why do homeopaths get so damned thrilled about homeopathy?
    .

  7. Davdoodleson 26 Mar 2013 at 11:20 pm

    “If I understand Homeopathy, then sperm diluted in water should be the perfect birth control.”

    A conundrum.

    The Lore says that “(highly diluted) like cures like”. Thus, a molecule of arsenic in an ocean of water, when drunk, will cure arsenic poisoning. Or somesuch nonsense.

    But “pregnancy” is not a problem, unless you don’t want to be pregnant. And vice-versa, sort of.

    So, presumably, sperm diluted in water is actually a cure for infertility.

    While the the perfect birth control would be highly diluted foetus.

    All perfectly sensible. Really…
    .

  8. jt512on 27 Mar 2013 at 1:21 am

    Steve wrote:

    The evidence is a giant arrow pointing at the null hypothesis.

    What a great sentence. I shall have to steal it.

  9. PharmD28on 27 Mar 2013 at 11:46 am

    HarryBlack….

    Excellent points IMO

    I part time work at a CVS pharmacy…most people simply believe that the homoepathic products that are mingled all in with the other OTC treatments are simply just another treatment that must be legit because, well, how else could they sell it?

    Also your comments reminded me of a conversation with a patient about a diet pill…she pointed out to me that a radio personality endorsed the product as highly effective and that it was “clinically proven”….in her mind, it seemed like this had to carry some weight otherwise it would be misleading and possibly illegal….

    But we know, it is not…

  10. RickKon 27 Mar 2013 at 3:34 pm

    HarryBlack – excellent post. Thank you for sharing. If you don’t mind, I’m going to copy and quote your comment in future debates as a good example of how an open mind actually seeks truth.

    PharmD28 – I challenged the pharmacist at our local CVS with a $15 box of Oscillococcinum and asked him how he could sell something he knew didn’t work. He just smiled, pointed toward the supplements aisle, and said “we sell all kinds of stuff that doesn’t work, because people buy it”.

  11. Badly Shaved Monkeyon 27 Mar 2013 at 4:57 pm

    Should we find it curious that Saine did not repeat his claim that homeopathy reliably cures rabies and was left rather pathetically promoting that limp and useless diarrhoea work?

    Was it fear that even with a sympathetic audience such a clam would be met with angry astonishment?

  12. ccbowerson 27 Mar 2013 at 11:43 pm

    RickK-
    I hope you found that answer satisfying enough because that is the correct answer. I assume you know that that person is an employee that has no say in what is sold at the store, and at best can only comment truthfully when asked (whether he is a shruggy or not is a different question). This is a problem that extends beyond this, and applies to many situations.

    If the SGU became a widely syndicated show, it is almost certain that a bunch of commercials for a number of crappy supplements would be paired with the show, and there would be nothing they could do about it (except to criticize the very same products on their show, perhaps). Its an unfortunate reality that explains why challanging the individual at the bottom will accomplish little if the surrounding structure remains the same

  13. Murmuron 28 Mar 2013 at 7:00 am

    @HarryBlack,

    That is similar to my experience, and thank you for your post.

    I think growing up we come in contact with a lot of information and we trust those around us quite often without question. It is also very funny that since I have deviated from belief in quackery, inane spirituality that my friends and family cling to that I am now the one being called closed minded and intellectually stunted, despite being the only one who has actively hunted for ghosts (ran a casual ghost hunting company for 6 months before finances and boredom overtook us) and the only one of two who actually got a CAM qualification (Reiki for my shame).

    Discovering the SGU two years ago has been the final straw for me, it gave me the freedom to have my own thinking process. It only took me 33 years to find it, but two years in I feel so much better not having the yoke of “belief” to drag me down… I am free to analyse and nothing… nothing at all is taboo for me anymore. When I was in a band we had a saying when it came to our music “Murder your darlings”, meaning those riffs and lines in songs we were so attached to we analysed the closest and if they didn’t improve the song then they went first. Very often we found that those little darling moments we loved were what were holding the overall music back, if they stayed, they were the best bits of music we ever wrote.

    I do this now in my life. I look at the ideas, thoughts and beliefs I cling to most and really put them through the ringer. It is a very humbling thing and I know I am a better person for it and I am much better off for seeing I was wrong and admitting it.

  14. SkeptiDCon 29 Mar 2013 at 10:28 am

    Here is the link to the video of the debate if anyone is interested: http://mediasite.uchc.edu/mediasite41/Play/f45177db9279460797ffe70714a3f5611d

    Steve, I’m always amazed at how calm you remain in these situations. Saine came across as quite frazzled by raising his voice, theatrical gestures, and stumbling throughout. One of the highlights was when he forgot the third country in the diarrhea study and you interjected with “it was Mexico.” Classic. The guy is accusing you of not studying homeopathy but doesn’t even know the details of the studies that he quotes.

  15. BillyJoe7on 29 Mar 2013 at 6:05 pm

    SDC,

    Am I right in thinking that English is not Saine’s native language (either that or he has a rather idiosyncratic way of pronuncing many of the words he uses). If so, his stumbling can be forgiven. But he probably also made the mistake of trying to fit too much into his presentation as evidenced by him continually going overtime and having to be docked for each segment because of it and then still running overtime. Also he didn’t accuse Dr. Novella of not having studied homoeopathy, he “accused” him of not reading the homoeopathic literature (actually, he asked him if he had read it, probably knowing that he hadn’t, so it came across as an accusation to some extent). Finally, forgetting the third country was really an irrelevant detail (in other words, that was the least of his problems).

  16. tmac57on 29 Mar 2013 at 7:36 pm

    Thanks SkeptiDC for the video link (I wish YouTube videos streamed as seamlessly ;) ).
    I thought Steve did a great job,but them I am biased toward the rationalist position I guess.
    I thought that Saine was trying to get in a big ‘gotcha’ with the stacking of homeopathic literature,and maybe that influenced some,but to me,that would be like stacking up a bunch of witchcraft tomes,and saying if you haven’t waded through all of this nonsense,then you can’t criticize the art of casting spells. And I would also be careful if I were in Saine’s shoes,because he repeatedly cast aspersions on ‘allopathic’ medicine, and I would bet that he has not read even a thousandth of the available literature that a med student has to learn.

  17. ChrisHon 29 Mar 2013 at 10:34 pm

    What is interesting is to see the slides roll by as Saine was speaking, compared to them being almost static during Dr. Novella’s portion. It is also amusing that Saine brought up over century year old incidences.

    I am about halfway through.

  18. ChrisHon 29 Mar 2013 at 11:10 pm

    My ears hurt listening to Saine shout at you, Dr. Novella. I hope you did not lose any hearing.

  19. SkeptiDCon 04 Apr 2013 at 3:39 pm

    Billyjoe -

    Saine’s first language does not appear to be English. He came across as crazy regardless.

    He did in fact ask Steve at the very end if he had made every effort to study homeopathy to determine whether it was valid or not (I’m paraphrasing here). Really much of Saine’s argument had a strong theme running throughout that Steve had not properly examined homeopathy (or even “real” homeopathy as he kept clarifying.) But I do think it’s quite telling that he forgot the third country. While it’s a small detail, to me it shows that he doesn’t know the literature well, and in fact Steve knows it better.

Trackback URI | Comments RSS

Leave a Reply

You must be logged in to post a comment.