SGU listener, Brandon, recently sent in this question:
Hello all, just want to say I really enjoy the show. I had a quick question/comment on the homeopathy discussion during the most recent podcast. While criticizing the dubious homeopathic studies, Dr. Novella stated that the Cochrane Collaboration was putting in charge of these studies ideological advocates. While I completely agree with Dr. Novella, I have a question about this statement. Isn’t the argument that the proponents of an ideology will only propagate their own agenda the reasoning used by creationist and the like? They use this argument to state that “Darwinist” aren’t following logic and will not review or consider Intelligent Design. So isn’t Dr. Novella’s argument in this instance weak? Well, I just thought I would raise this question/comment, if I am mistaking please let me know. Again, just let me say how much I appreciate the show and have learned so much from it. Thank you all for your time and input.
Thanks for the question, Brandon. These types of questions come up frequently – by which I mean a false analogy stemming from generalizing from a principle that was only applied in the specific. For example, we have argued that grand conspiracies are inherently improbable because of their size, scope, and complexity. Yet we frequently receive feedback arguing that conspiracies do exist, often accompanied by an example.
However, we never argued that conspiracies do not exist – only that they become more unlikely as they grow in size. Also, if one is going to argue for a conspiracy as an explanation in a particular case, then that creates the burden of providing evidence for the conspiracy itself. Also, the accusation of a conspiracy is often used as a panacea to explain away all contradictory evidence and the absence of confirming evidence for a claim.
This is a similar situation. First I must point out that during the SGU episode to which Brandon refers I never argued that we can conclude the Cochrane review of homeopathy for cancer treatment side effects was poor or misleading because the lead author is biased. Rather, I made a completely separate case for the low quality of the review – there were very few studies (8), that addressed three different clinical questions, only a few of the studies were of decent quality, and at least one of the high quality positive studies was actually not a study of homeopathy at all but of an undiluted drug (calendula) already known to be effective.
I further argued that the positive spin the review placed on the data – saying it suggested a positive effect and the need for further study – was at odds with the more conservative style typical for Cochrane reviews. I then concluded that the poor quality and positive spin of the review was likely due to the bias of the lead author, who is a practicing homeopath.
With regard to the argument put forward by Intelligent Design (ID) proponents, they are saying that the widespread acceptance of Darwinian evolution by the scientific community is due, not to the compelling nature of the evidence, but to a pervasive atheistic and materialist bias. This is the central theme of the pro-ID movie, Expelled.
This is a grand conspiracy of scientific bias. It is used by ID proponents and creationists to re-frame the debate and to whitewash the actual scientific arguments. Here we are dealing with a clear-cut ideological agenda with continuity over the last near century. They have utterly lost the scientific debate and are simply using the accusation of systematic bias as a attempt to win politically what they have lost scientifically. And just as the 9/11 truth movement has failed to provide any compelling evidence for a widespread government conspiracy, the creationist/ID movement has failed to provide evidence for a pervasive pro-evolution ideological bias in science.
But just as the former does not imply that conspiracies do not exist, the latter does not imply that bias does not exist in science. It certainly does – at the individual, group, institutional, and even professional level. We are all humans, and humans are emotional creatures burdened with many sources of intellectual bias.
There are two general strategies, however, for mitigating the effects of bias in science. The first is the scientific method itself, which is specifically designed to minimize the effects of bias. This is precisely why studies must be blinded, and outcomes must be as objective as possible.
The second strategy is the scientific community itself. Bias is most intense at the individual level. The more people who are involved in assessing a specific scientific question, however, the more likely it is that individual biases will vary and will therefore counteract each other to some extent. It is always easier to spot bias in someone else, and scientists generally keep each other honest. The process of peer review and publication serves this purpose. It is also why science must be transparent to work.
It is also invaluable that older scientists eventually retire or die, and there is always a new crop of young scientists looking to make a name for themselves by overturning established notions.
The result is a messy and imperfect process, but slowly biases are hammered out of science.
The theory of Darwinian evolution has now been around for 150 years. It has survived numerous challenges, and while it has had to be modified significantly, it has also survived the development of new sciences such as genetics, developmental biology, radio-isotope dating methods, and a dramatic expansion of the fossil record.
It has also withstood a century and a half of direct challenges from those who were and are extremely ideologically biased against it. Creationist have been trying all this time to knock down evolution, and they have not been able to.
The claim that the scientific consensus that common descent and evolution through variation and natural selection is the result, not of science and evidence, but of bias is absurd and historically ignorant. It is a desperate political strategy and nothing else.
By contrast, homeopathy (which is about 200 years old) has not weathered scientific advance well. The more we learned about chemistry, physics, and biology the less sense homeopathy made. It is now demonstrably false – a quaint and bizarre belief that stands in direct opposition to two centuries of scientific advance. And yet it still has its proponents who, for whatever reason, cling to this pre-scientific philosophy of health.
While still doing well in Europe, although recently on the wane even there, homeopathy is also clearly past it prime. It has failed to gain mainstream scientific acceptance and remains on the fringe, criticized sharpy whenever scientists feel it is worthy of their time to do so. Lack of general acceptance has made proponents of homeopathy somewhat desperate for legitimacy.
This does not prove that any individual homeopath is biased in any particular claim or assessment that they make, but it certainly raises the concern of bias. An organization such as the Cochrane Collaboration, which is premised on the very promise of unbiased and evidence-based reviews of medical claims, should take this potential for bias very seriously. I think it is fair to say that proponents of dubious and fringe scientific claims should not be the only ones to review evidence for such claims.
And, in this particular case, for the reasons I outlined above, I think the result was a poor quality biased review.
As an aside, I think there is often a conflict between expert review and independent review. If only proponents are deemed experts by institutions, which is increasingly the case with regard to CAM (complementary and alternative medicine), then only proponents will perform expert reviews or provide expert opinions for regulatory bodies or academic and professional organizations. But then we have no independent review – review from skeptics or those with a different opinion or bias. Independent review is essential to science, and yet CAM proponents have been successful in setting themselves up as the only experts, thereby weeding out any skepticism of their claims. (A notable exception is Edzard Ernst, who started out as a proponent but was beaten into a CAM skeptic by the evidence.)
To summarize – bias certainly exists in science, but there are mechanisms for the mitigation of such bias, and good scientists are always on the look out for sources of bias. There are situations in which those mechanisms of bias reduction fail, or are excluded from the process ( for example, whenever proponents or ideologues can control the institutions of science). This is why journals or institutions that are dedicated to a specific belief system should always be suspect.
This is also why transparency, as well as substantive and vigorous debate, are so essential to science.