Randi’s Million Dollar Paranormal Challenge is an icon of the skeptical movement. The challenge basically offers $1 million to anyone who can, “show, under proper observing conditions, evidence of any paranormal, supernatural, or occult power or event.” So far no applicant has passed even the preliminary test for the million dollars.

It should not be surprising that the challenge is a thorn in the side of all proponents of the paranormal and charlatans whose living depends on belief in supernatural powers. The challenge is therefore under frequent attack by such proponents – always, in my experience, using unfair and often factually incorrect charges.

For full disclosure, even though this information is already on my author page, I am a senior fellow at the JREF (the James Randi Education Foundation, who offers the challenge), and I have participated in several preliminary tests. I have actually run three preliminary tests, and have participated in the development of protocols for others. The three tests I ran were designed and executed independently by me, with no input from Randi or the JREF, but following the rules they lay out and approved by the JREF before being executed.

The purpose of the challenge is not to design and run scientific experiments, and it is not to scientifically prove or disprove the existence of the paranormal or any particular supernatural phenomenon. Randi and the JREF have always been crystal clear about this. Rather, the point of the challenge is to be a public demonstration. There are many people who claim dramatic paranormal abilities. If their claims were anything close to the truth, it should be easy to demonstrate their abilities. As Randi says – he is only asking them to do what they claim to do on a daily basis, just under conditions that make it impossible (or at least very difficult) to cheat. That’s it.

A recent blog post by Steve Volk once again attacks the million dollar challenge. The attack amounts to one giant straw man, typical of such criticisms. I will also point out that this post, like all criticisms I have seen, focuses on Randi the man. I suspect this is probably because of his position, but also because they feel he is an easy target. Volk writes of him:

Randi, an amateur magician who found fame as an opponent of paranormal claims, has long served as the cranky elf of the skeptical movement.

“Cranky elf” is more charitable than most opponents. This article also typically ignores the fact that Randi is not even involved in the million dollar challenge anymore. The challenge is now run by Banachek, who is also a professional magician. Banachek’s name does not even appear in Volk’s article.

The core of his criticism is this:

The Challenge has muddled the very boundaries of science, allowing Randi-ites to say paranormal claims don’t hold up to scientific scrutiny while conceding, when pressed, that the Challenge isn’t science.

The link that apparently justifies this statement goes to a JREF announcement in which president D.J. Grothe is quoted as saying, “We have a longstanding commitment to investigate paranormal claims in a fair and scientific way…” Saying that investigations are done “in a scientific way” is not the same as saying that they are scientific experiments. The challenge develops protocols that use proper blinding, multiple trials, and statistical analysis. These are scientific methods, but even still no one claims that these are rigorous scientific experiments worthy of publication in peer-reviewed journals.

Volk goes on to complain that the thresholds for beating the challenge are too high – far above what would be required for scientific significance:

The result is that an applicant can—and did—achieve statistically significant positive results, yet was deemed to “fail” the challenge.

This is an absurd criticism, however, and does not even make consistent sense. The threshold for statistical significance is often set at 0.05, which essentially means that, by random chance alone, 1 experiment in 20 will reach statistical significance. If Randi were to set the threshold at a P-value of 0.05 then he would be giving $1 million dollars to every 20th applicant. It should be obvious why this is not done. Even a level of 1 in 100 would be ruinous. It is perfectly reasonable to set the threshold at 1 in a million, where it has traditionally been set (or the lower threshold of 1 in 100,000, which they are considering doing).

The link Volk provides above as evidence of his position is to a challenge, actually run by Richard Wiseman, not Randi, in which the threshold was set at 50 to 1 and the applicant failed, but barely.

Volk’s criticism is that the challenge is rigged, so people with real psychic ability cannot win. But the very article he links to also says:

“Professional statisticians and scientists work with applicants to develop tests that minimize the possibility of charlatans winning by mere chance, while also minimizing the possibility of any actual psychic losing by “bad luck.””

What other option is there but to set the threshold at a point that is a reasonable balance between minimizing false positives and false negatives. This is just good science, even without $1 million dollars on the line.

Volk complains that small psychic effects would require hundreds of hours – years – of testing to reach the threshold of the million dollar challenge. This is true, but irrelevant. The million dollar challenge is not designed to scientifically test subtle or tiny effects, but rather to test the dramatic claims of people who are publicly proclaiming they have genuine paranormal abilities.

The most recent public test, for example, was held at TAM 2012 in July. This test involved Andrew Needles, who contacted the JREF with the claim that he invented a version of the “power bracelet” that really worked. A completely fair test was designed and executed on stage and before cameras. Needles agreed to the protocol ahead of time as completely fair. He expected to succeed. He completely failed, with results essentially at chance level.

The key to the challenge is not the ability to scientifically test subtle effects, but to demonstrate that dramatic claimed abilities utterly vanish under proper observing conditions, as was the case with Needles. Often the thresholds for success are set far below what the applicant claims to be able to do. They are given the best chance to succeed, and their every requirement is met – as long as this does not equate to the ability to cheat (whether consciously or unconsciously).

The attacks on the million dollar challenge are likely to continue. This is a sign, in my opinion, of the success of the challenge. Con artists know they cannot beat the challenge, and so they have no choice but to try to discredit it. Those who truly believe they have abilities but fail the challenge almost universally make up post hoc excuses for their failure.

The fact remains – the million dollar challenge has seen a long series of applicants with dramatic paranormal claims yet who cannot demonstrate their abilities when even the most basic controls are set into place. This does not prove that there are no paranormal abilities anywhere ever.  It does demonstrate, however, that the applicants are likely to have been self-deluded (or conscious frauds). And this says something very important – that people can delude themselves to that degree.

At the very least, the challenge teaches us, basic controls should be put into place before any paranormal claim is taken seriously. So far no claims has crossed the threshold to be taken seriously.