We all do it. In fact, we are generally very good at it. Smart and educated people are better at it.

Rationalizing is a daily practice, part of the “default mode” of human thinking. We make up reasons to justify believing what we want to believe. Often we are only dimly aware of why we want to believe something, the calculus largely occurring in the subconscious depths of our brains.

We defend beliefs because they are pleasing to our egos, because they minimize cognitive dissonance, and just because they are our beliefs. They resonate with our world-view, our internal model of reality.

We have at our disposal a long list of logical fallacies that we can marshal to the defense of our beliefs. Notions that are based on solid evidence and logic do not require such vigorous defense. Those beliefs that cannot be defended by logic and evidence require that bad logic and bad data be invoked to defend them. Luckily we have no problem distorting and cherry picking facts and twisting logic into pretzels.

One very common bit of bad logic is called special pleading. I think it is common because it is so insidious – it creeps up on us unaware. Special pleading is the process of inventing a special reason to explain away inconvenient evidence or the lack of predicted evidence.

Take, for example, ESP research. I wrote previously about a series of experiments by Daryl Bem in which he claims to have found evidence for precognition. The results of the research are not impressive, mainly due to small effect sizes that are within the range of noise that plagues such research. Given the high implausibility of the phenomenon Bem claims to have documented, at the very least we would like replication before taking his results seriously.

Wiseman, Ritchie, and French replicated some of Bem’s research, with negative results. They are having a problem, however, getting their replications published. This is a flagrant display of publication bias at work. So Wiseman has created a registry for anyone attempting to replicate any of Bem’s studies – so that all the data can be captured. This is a great idea.

Meanwhile, they continue to try to get their paper published. The latest attempt also met with failure, but with an interesting wrinkle. The paper was sent to two reviewers, one of whom recommended publication and the other (a psi believer) recommended rejection. His reason, as recounted by Stuart Ritchie, is an excellent example of special pleading.

This latter review didn’t find any problems in our methodology or writeup itself, but suggested that, since the three of us (Richard Wiseman, Chris French and I) are all skeptical of ESP, we might have unconsciously influenced the results using our own psychic powers.

So – skeptics of psychic powers have psychic powers of their own that they use (whether consciously or subconsciously) to precisely counteract psi effects and make experiments come out negative. This idea, sadly, is not new. Psi proponents have been making this claim for years – that skeptics mess up their psychic mojo.

This is special pleading not because it has to be wrong, but because there is no particular a priori reason to think that it is right. It is a special excuse invented out of whole cloth to explain away negative studies.

At best such special pleading objections are new hypotheses that themselves need to be investigated before they can be invoked as actual explanations. It’s OK to say that maybe skeptics have special anti-ESP vibes. This cannot legitimately be invoked as a reason to reject negative evidence, however, until it is independently demonstrated as a real phenomenon.

This, of course, has never been done. The closest that any research has come to demonstrating a “skeptic effect” has been demonstrating that the same ESP experimental protocol tends to have different outcomes when a believer is running the study than when a skeptic is running the study. There is no reason to assume, however, that this is a “skeptic effect”. It could be a believer effect, or more simply – it’s an experimenter effect, also known as experimenter bias, something which is well known.

Fine – there is experimenter bias. That is well established, and just brings us back to the need for replications. Real effects should hold up under replication, because they are not entirely the result of experimenter bias. Those results that are the result of bias will tend not to hold up under different bias conditions – they will fail to replicate.

But there is also another layer to this. The best way to deal with experimenter bias is through increased rigor of study design and execution. A sufficiently rigorous study should minimize experimenter bias, hopefully to insignificance.

The reviewer who recommended rejection of the Wiseman et al replication made this justification (again, according to Ritchie):

The story behind this is that Richard has co-authored two papers where he and a believer in psi both did the same experiment, and the believer found positive results but he didn’t. However, the most recent time they did this – which was the best-controlled and largest size – neither found results.

Here is more special pleading – the reviewer invoked the fact that previous research of Wiseman’s demonstrated an experimenter bias effect. But this bias was already further investigated by doing larger and more rigorous studies, which were negative. This means that the bias was really on the part of the believer, not the skeptic.

If anything the reviewer made a point for publishing Wiseman’s negative replication, and for the probability (or at least possibility) that Bem’s results are due to experimenter bias. Certainly before Bem’s results are taken seriously experimenter bias needs to be ruled out – by replication at least, and with more rigorous studies that minimize experimenter bias.

A sad note on this episode is that the editor of the journal accepted the second reviewer’s special pleading and rejected Wisemen et al’s paper for publication.

The history of PSI research is that whenever this process works itself out – replication and better study design – apparent initial effects vanish. This has been dubbed the decline effect (a general tendency for the magnitude of research results to decline over time), which I feel is mostly a combination of publication bias and researcher bias. The question is – does the effect size shrink but remain positive, or does it shrink to zero. For PSI research effects always shrink to zero. The most parsimonious explanation of this fact is that PSI is not real.

But never underestimate the power of rationalization. Special pleading and other fallacies can always be invoked to avoid the parsimonious explanation when it is not pleasing to our world view.