Mar 11 2010

Acupuncture Does Not Work for IVF

Acupuncture is a so-called complementary and alternative medicine (CAM) modality I frequently tackle because it often provides excellent  teaching points on the relationship between science and the practice of medicine. My reading of the literature is that acupuncture is highly implausible and the evidence does not support its efficacy for any indication.

And yet it is one of the more popular CAM modalities (although still a small phenomenon – only 6% of Americans have ever used it), especially in its penetration of hospitals and academia. There is a great deal of misinformation out there about acupuncture, and this seems to garner the most attention from naive physicians.

In vitro fertilization (IVF) is one of the applications of acupuncture that has been most touted by proponents. The evidence for any positive effect from acupuncture for IVF, however, has been consistent with no effect. By this I mean that there are poor quality studies with mixed results, but trending positive (as poor quality studies tend to do), especially in China and other nations culturally predisposed to acupuncture, but the better designed studies tend to be negative.

There have been a number of systematic reviews of acupuncture trials for IVF with mixed opinions. The most recent review was conducted by the British Fertility Society, and they conclude:

The guidelines found that there is currently no evidence that having acupuncture or Chinese herbal medicine treatment around the time of assisted conception increases the likelihood of subsequent pregnancy.

They reviewed 14 randomized controlled trials and performed meta-analyses on the various outcomes and timing of treatment, and in all cases found no difference between acupuncture and control groups. Other systematic reviews have also come to the same conclusion, for example this 2008 review concluded:

Currently available literature does not provide sufficient evidence that adjuvant acupuncture improves IVF clinical pregnancy rate.

Yet other reviews came to different conclusions. This BMJ review from 2008 concluded:

Current preliminary evidence suggests that acupuncture given with embryo transfer improves rates of pregnancy and live birth among women undergoing in vitro fertilisation.

Why the discrepancy? Partly it is due to a judgment call – the difference between “preliminary evidence is positive” and “there is insufficient evidence” – which is likely a result of the biases of the reviewers. How much weight do you put on preliminary evidence. Those reviews that emphasized the best studies (which I think is the appropriate approach) showed no effect from acupuncture.

One of the best studies of acupuncture in IVF was this one published in 2009. They found that placebo acupuncture was better in some measures than true acupuncture, although no better in others. The authors should have concluded that acupuncture simply does not work, and that these differences found were likely due to chance. However, they concluded that:

Placebo acupuncture was associated with a significantly higher overall pregnancy rate when compared with real acupuncture. Placebo acupuncture may not be inert.

This is just bizarre, but not uncommon in the CAM world. It is just a form of special pleading to argue after a negative trial that placebos work also. In this study the authors are left trying to justify the conclusion that placing the needles in the “correct” locations had a disadvantage over placing them in random locations, which would also mean that expertise in acupuncture is a disadvantage.

Proponents of acupuncture, following the BFS announcement, were quick to insulate themselves from the evidence using special pleading.

One high-profile practitioner, Dr Xiao-Ping Zhai, of The Zhai Fertility Treatment Clinic, said there were indeed problems with the way in which these trials were designed and that different analysis would show the benefits.

“Certainly for those with unexplained fertility problems in particular, we know acupuncture and traditional Chinese medicine can be beneficial. What matters is both the expertise and experience of the practitioner, but most of all the treatment of the patient as an individual. It is the tailored treatment which is key.

My primary question for Dr. Xiao-Ping Zhai is – how does he know that acupuncture works to improve IVF success? He seems to be taking the classic pseudoscientific approach of using science to validate beliefs, rather than test them. When controlled trials are negative, proponents then blame the trials – they must be wrong, because they “know” that acupuncture works.

One of their tactics is to claim that acupuncture needs to be “individualized” – using a standard treatment does not work. But they are unable to provide a coherent explanation for why individualization should matter, at least in physiological terms. Invoking “life energy” and the flow of “chi” is nothing but superstition.

But even if we grant for the sake of argument that individualization of treatments makes a difference, that does not mean the existing trials are wrong and acupuncture works. At best it means we still do not know if acupuncture works for IVF. It is now on the proponents of acupuncture to design and execute what they claim are better trials that allow for individualization.

The problem with this historically is that such trials sacrifice some of the blinding in order to individualize. They have to be especially careful in their design to accomplish both ends – individualized and properly blinded. This has been done, in acupuncture for back pain, for example, with no difference between standardized and individualized treatment groups. So far the “individualization” special pleading has not been supported by the evidence.

“Expertise and experience” also does not seem to matter. This is just another claim without evidence. It may seem intuitive that experience should matter, but actually it matter far less than experts (including physicians) would like to think. To clarify this point – this is in the context of applying treatments. Generally speaking, following evidence based guidelines yields better results than shooting from the hip based upon personal experience.

I would also argue that expertise in a pseudoscientific discipline should be considered pseudoexpertise. Knowing where to place acupuncture needles according to complex Traditional Chinese Medicine principles would only matter if the TCM principles were valid. However, the totality of acupuncture research is consistent with the interpretation that it does not matter where you place the needles – so all that expertise appears to be worthless. It is the equivalent of being able to provide an expert astrology reading – expertise in astrology matters not, because astrology is 100% nonsense.

At the very least, if proponents are going to argue that existing evidence against the efficacy of acupuncture for IVF should be dismissed because the treatments were insufficiently individualized or that they were given by insufficiently experienced practitioners – they should provide evidence that either of these things matter. And even worse, they are committing the logical fallacy of concluding not only were the trials not definitive, but that acupuncture works (rather than it is still unknown).

What about plausibility? Proponents argue thusly:

A statement from the British Acupuncture Council noted: “Fertility focused acupuncture treatment has been found to help increase blood flow to the reproductive organs, balance hormone levels, regulate the menstrual cycle and help improve the lining of the uterus and quality of eggs released.”

This is a common ploy by proponents of dubious treatments – they cite research which essentially amount to anomaly hunting – looking for physiological changes that correlate with the treatment. First it must be pointed out that basic science evidence, like physiological changes, should not be used to extrapolate clinical claims – the clinical evidence for efficacy must also be there, and it isn’t.

But the more subtle point is that medical researchers are familiar with the fact that if you do something to an organism or a clump of cells and then look for changes, you will find them. Biological systems respond to stimulation. Stick a needle in someone and there will be a local inflammatory reaction, causing a host of changes.Are any of these changes actual markers or causes of a specific clinical effect? That is a different question.

Further, biological systems are highly variable, so looking for changes is often rewarded. Knowing what those changes actually mean is another matter.

When anomaly hunting for physiological changes is rewarded, proponents also often make another unjustified leap of logic, assuming that “changed” equals “improved.” So changes in hormone levels becomes “balance hormone levels” – without any evidence that the non-specific changes represent a functional improvement.

The take-away message is that finding changes in physiological markers in correlation with a treatment is helpful in designing future research and both testing and generating hypothesis – but they are no substitute for clinical research. Finding physiological changes is also not as impressive as is often presented – it is actually an extremely poor predictor of clinical efficacy. Anyone doing translational research is keenly aware of this – most treatments that are promising based upon pre-clinical evidence do not pan out clinically.

Conclusion

Acupuncture remains an implausible treatment, and lacks sufficient evidence to conclude that it works for any specific indication. The history of acupuncture research is following a familiar pattern. Initial research is mixed but trends positive. Basic science research shows non-specific anomalies, but no consistent pattern that accords with scientific theory of mechanism for acupuncture. For each indication, as better and more rigorous studies are designed, the effects shrink until the best studies are negative.

Proponents then engage in special pleading to dismiss the evidence, while simultaneously citing poor quality or pre-clinical evidence, secure in their faith that acupuncture works.

We see this pattern with homeopathy research, the ESP literature, UFO investigations, and all manner of fringe science. Recognizing this pattern is essential to interpreting any large body of such research.

20 responses so far