Dec 08 2011

ESP Special Pleading

We all do it. In fact, we are generally very good at it. Smart and educated people are better at it.

Rationalizing is a daily practice, part of the “default mode” of human thinking. We make up reasons to justify believing what we want to believe. Often we are only dimly aware of why we want to believe something, the calculus largely occurring in the subconscious depths of our brains.

We defend beliefs because they are pleasing to our egos, because they minimize cognitive dissonance, and just because they are our beliefs. They resonate with our world-view, our internal model of reality.

We have at our disposal a long list of logical fallacies that we can marshal to the defense of our beliefs. Notions that are based on solid evidence and logic do not require such vigorous defense. Those beliefs that cannot be defended by logic and evidence require that bad logic and bad data be invoked to defend them. Luckily we have no problem distorting and cherry picking facts and twisting logic into pretzels.

One very common bit of bad logic is called special pleading. I think it is common because it is so insidious – it creeps up on us unaware. Special pleading is the process of inventing a special reason to explain away inconvenient evidence or the lack of predicted evidence.

Take, for example, ESP research. I wrote previously about a series of experiments by Daryl Bem in which he claims to have found evidence for precognition. The results of the research are not impressive, mainly due to small effect sizes that are within the range of noise that plagues such research. Given the high implausibility of the phenomenon Bem claims to have documented, at the very least we would like replication before taking his results seriously.

Wiseman, Ritchie, and French replicated some of Bem’s research, with negative results. They are having a problem, however, getting their replications published. This is a flagrant display of publication bias at work. So Wiseman has created a registry for anyone attempting to replicate any of Bem’s studies – so that all the data can be captured. This is a great idea.

Meanwhile, they continue to try to get their paper published. The latest attempt also met with failure, but with an interesting wrinkle. The paper was sent to two reviewers, one of whom recommended publication and the other (a psi believer) recommended rejection. His reason, as recounted by Stuart Ritchie, is an excellent example of special pleading.

This latter review didn’t find any problems in our methodology or writeup itself, but suggested that, since the three of us (Richard Wiseman, Chris French and I) are all skeptical of ESP, we might have unconsciously influenced the results using our own psychic powers.

So – skeptics of psychic powers have psychic powers of their own that they use (whether consciously or subconsciously) to precisely counteract psi effects and make experiments come out negative. This idea, sadly, is not new. Psi proponents have been making this claim for years – that skeptics mess up their psychic mojo.

This is special pleading not because it has to be wrong, but because there is no particular a priori reason to think that it is right. It is a special excuse invented out of whole cloth to explain away negative studies.

At best such special pleading objections are new hypotheses that themselves need to be investigated before they can be invoked as actual explanations. It’s OK to say that maybe skeptics have special anti-ESP vibes. This cannot legitimately be invoked as a reason to reject negative evidence, however, until it is independently demonstrated as a real phenomenon.

This, of course, has never been done. The closest that any research has come to demonstrating a “skeptic effect” has been demonstrating that the same ESP experimental protocol tends to have different outcomes when a believer is running the study than when a skeptic is running the study. There is no reason to assume, however, that this is a “skeptic effect”. It could be a believer effect, or more simply – it’s an experimenter effect, also known as experimenter bias, something which is well known.

Fine – there is experimenter bias. That is well established, and just brings us back to the need for replications. Real effects should hold up under replication, because they are not entirely the result of experimenter bias. Those results that are the result of bias will tend not to hold up under different bias conditions – they will fail to replicate.

But there is also another layer to this. The best way to deal with experimenter bias is through increased rigor of study design and execution. A sufficiently rigorous study should minimize experimenter bias, hopefully to insignificance.

The reviewer who recommended rejection of the Wiseman et al replication made this justification (again, according to Ritchie):

The story behind this is that Richard has co-authored two papers where he and a believer in psi both did the same experiment, and the believer found positive results but he didn’t. However, the most recent time they did this – which was the best-controlled and largest size – neither found results.

Here is more special pleading – the reviewer invoked the fact that previous research of Wiseman’s demonstrated an experimenter bias effect. But this bias was already further investigated by doing larger and more rigorous studies, which were negative. This means that the bias was really on the part of the believer, not the skeptic.

If anything the reviewer made a point for publishing Wiseman’s negative replication, and for the probability (or at least possibility) that Bem’s results are due to experimenter bias. Certainly before Bem’s results are taken seriously experimenter bias needs to be ruled out – by replication at least, and with more rigorous studies that minimize experimenter bias.

A sad note on this episode is that the editor of the journal accepted the second reviewer’s special pleading and rejected Wisemen et al’s paper for publication.

The history of PSI research is that whenever this process works itself out – replication and better study design – apparent initial effects vanish. This has been dubbed the decline effect (a general tendency for the magnitude of research results to decline over time), which I feel is mostly a combination of publication bias and researcher bias. The question is – does the effect size shrink but remain positive, or does it shrink to zero. For PSI research effects always shrink to zero. The most parsimonious explanation of this fact is that PSI is not real.

But never underestimate the power of rationalization. Special pleading and other fallacies can always be invoked to avoid the parsimonious explanation when it is not pleasing to our world view.

11 responses so far

11 thoughts on “ESP Special Pleading”

  1. Skeptico says:

    This does suggest areas for Negative Energy Research. I’m available.

  2. ccbowers says:

    It is special pleading that begs the question

  3. HHC says:

    Daryl Bem posited that we infer our attitudes from our behavior. This would imply that humans run on instinct and reasoning and emotion follow literally as an afterthought. So ESP would be a type of instinctual precognition (id) which requires a special pleading (ego-based) to be sensible. Freud and Bem can’t create a statistical basis for research in 2011. Psychologists were the true philosophers of the Twentieth Century. And Freud was dominant in the Nineteenth Century. A more coherent social psychological theory is required 😉

  4. As long as we’re going to be special pleading and making up negative psychic effects due to disbelief, what happens if you are both strongly skeptical, and also strongly want something to be true despite your skepticism?

    What would the net result of the Houdini effect be in a Bem reproduction experiment?

    Obviously the way to find out would be to get some Houdini type skeptics to run the experiment. If they confirm Bem’s results, the effect is obviously either nill or positive, but if they fail to confirm Bem’s results, the effect must be net negative.

    Special pleading is fun! 🙂

  5. ConspicuousCarl says:

    “WAAAAAAAHHHH! It’s not working because you scared away my invisible friends!”

    How does that kind of thinking end up on the wrong side of an observation window? You would think they could stop people from going through the wrong door, what with all the pass cards and badges they have these days.

  6. SimonW says:

    “This means that the bias was really on the part of the believer, not the skeptic.”

    No, it means “If the original result was due to bias, it was bias on the part of the believer.”. Wiseman’s collaborator’s work could have been positive due to other factors such as random statistical variation.

    It could be the later experiments failed because of the increased interest from skeptics with all that “negative power”, in which case PSI researchers should give up since they’ll never be any reproducible results once they’ve gone public with a positive result with all the skeptics around these days.

    You are of course right that none of these are reasons not to publish, indeed not publishing would make it harder for someone to later prove there is a “skeptic effect”, such as the reviewer suggested.

  7. tmac57 says:

    “This latter review didn’t find any problems in our methodology or writeup itself, but suggested that, since the three of us (Richard Wiseman, Chris French and I) are all skeptical of ESP, we might have unconsciously influenced the results using our own psychic powers.”


    On Peter Pan’s ability to fly:

    “In the play and novel, he teaches the Darling children to fly using a combination of “lovely wonderful thoughts” (which became “happy thoughts” in Disney’s film) and fairy dust;”

    Maybe the researchers should try ESP with a touch of fairy dust,just to be sure the are maximizing the effect.

  8. SARA says:

    I continue to be boggled at this research. If ESP existed, it would be obvious and clear to the population at large.
    Even if it existed in only a small percentage of the population, it existence would be well known and their ability would exploited to their own benefit and to the benefit of governments and corporations.
    If they have to spend so much time searching for minuscule evidence, evidence that is highly debatable, it doesn’t exist on any level that is worthwhile – either for research or use.
    So I wish they stop trying to find it and use their research skills on something worth researching.

  9. tmac57 says:

    Even if it existed in only a small percentage of the population, it existence would be well known and their ability would exploited to their own benefit and to the benefit of governments and corporations.

    SARA,you do realize that there is a large percentage of the population that already believes those things to be true,right?

    That’s why these researchers just won’t let this ‘rubber ducky’ sink.

  10. Khym Chanur says:

    Psi proponents have been making this claim for years – that skeptics mess up their psychic mojo.

    Seems like this would be easy enough to test. Set up the experiment so that it can be observed from a special room through a one-way mirror. Unknown to the participants or the believer-experimenter (until after the fact), the room will at random be filled (or not) with skeptics disbelieving as hard as they can. Then they can look to see if there’s any correlation between lower scores on the psi-tests and the presences of skeptics.

  11. Murmur says:

    I can understand why believers would cling to the idea behind why skeptics don’t get results, and as mentioned in a few other comments it is easily tested.

    What this brings me to though is if PSI exists, and is being throttled by the government or whatever, what kind of a “skill” is it? I am in muddy waters here myself, but it got me thinking how belief seems to be something we really put a lot of … well… belief in. For me, PSI, if it existed would be a talent, more like musical talent, as opposed to a skill. An example:

    X people are seen in the real world to catch a ball coming at them hard and at a strange angle. Stick half in a room with someone saying they can’t do it again, and half in a room saying they have a natural talent and see if they can do it again… I would expect the half in the room being told they can’t would fair worse (Or maybe not), but there would be a decay in the skill none the less i am sure.

    So, then take x people who can play a particularly hard piece of music, do the same as above… I am pretty certain your success rate would be much higher than the ball catching. Belief would have little to no difference here… yet this involves much higher brain function (I would assume, feel free to shoot me down here).

    Anyway… it just got me thinking about where is the boundary at which belief in something actually has an effect?

Leave a Reply