I am often asked if skeptics and skeptical organizations should undertake first-hand investigations. Of course, it depends upon what your goals are. But I think the question can be re-phrased to mean – is there any value or benefit to first hand investigation, and to this the answer is a definite “yes.”

But this is not to denigrate the value of skeptical review from the comfort of your computer chair. This kind of activity has sometimes been referred to as “armchair skepticism” – meant to be derogatory. While I see the value in going out into the field, armchair skepticism has a valuable and complementary role to play.

In fact, these two activities mirror what real scientists do, and are roughly analogous to peer-review vs experimental replication.

Armchair Skepticism

The community of scientists keep each other honest, and keep the process of science grinding forward, in various ways – only one of which is going into the lab to replicate a study or do follow up research. When a colleague publishes a paper, or presents a paper at a meeting, his colleagues provide analysis and criticism. Ideas are examined for logic, internal consistency, and plausibility. Other options, perhaps neglected by the researcher, are explored. And existing research, perhaps not taken into account by the researcher, is brought up and discussed.

This feedback is provided without ever doing any actual investigation. When skeptics perform the exact same service to paranormal or fringe claims, this should not be denigrated at all, but seen as providing in our area of expertise the same kind of analysis that scientists provide in theirs.

This “peer-review” takes several forms. First, the term “peer-review” is often used to refer to the formal process of reviewing a paper that has been submitted for publication. I am not referring to this formal peer-review (which I do not think has any analogy in skeptical activity), but rather to the informal peer-review that collectively refers to all the efforts of the scientific community to hammer errors and flaws out of scientific thinking.

Informal peer-review has various manifestations, all of which have analogies in skeptical activism. For example, scientists will often dissect a specific published paper, analyzing it for weaknesses of methodology, strength of the outcome, how well the authors interpret their own findings, and putting it into the context of plausibility and other published research. This analysis may be published as a letter or commentary in the same journal as the original study, or incorporated into a talk at a meeting. Skeptics will often do the exact same thing, but these days published as a blog or article in the skeptical literature.

Scientists will also publish systematic reviews or topic reviews, reviewing all the published evidence and arriving at a bottom-line assessment of the state of the science. Skeptics do this as well. This activity does involve “research,” but not laboratory or field research – rather it involves researching the literature. It is getting easier and more viable to perform this activity sitting in front of a computer with internet access, as there are online libraries of published research, and many journals and news outlets have online versions.

Depending upon the topic and the depth of one’s investigation, it may be necessary to venture into a physical library, but this is getting less and less necessary. This is the level of research one would do when writing a book, but not for a daily blog post.

Armchair skeptics therefore provide a valuable service, similar to the activity of working scientists. We can analyze specific claims, topics, or published research for quality, plausibility, and historical and scientific context. We can then tailor our writing to communicate with our colleagues, the public, or both simultaneously. We can also provide reference material (like The Skeptic’s Dictionary), which fills the role of a science textbook or reference website.

Skeptical Investigation

While I am a strong advocate of armchair skepticism, if you have the opportunity to go into the field and do first-hand investigation, you will likely find the experience very illuminating. Theory and book-learning does only go so far, and there are aspects to paranormal investigations that you would simply not imagine until you are there to see it first hand.

I and my colleagues have performed a number of investigations – mostly haunted houses, but also EVP, channelers, exorcism, and psychics. Most of these were with other (less-than-skeptical) groups. What was always very striking was how unimpressive the paranormal investigators or claimants were. We always gave them more credit than they deserved, and were surprised at how easy it was to analyze their evidence.

For example, when we investigated a channeler in Connecticut who claimed to channel the 600 year old spirit of a man from Nepal, I was prepared to have the Nepalese analyzed to see if it was modern or appropriate to the claimed time period. However, the channeler did not speak any dialect spoken in Nepal, just English with a cheesy regional accent (it sounded Indian to me).

We also went on an EVP (electronic voice phenomenon) recording session with some local ghost-hunters. EVP is basically an exercise in audio pareidolia – listen to static long enough and you will make out words. We sat in the attic of an allegedly haunted restaurant. Even though it was relatively quiet in the location, I was struck by how much background noise there was. When you listen carefully, you can hear voices in the street, the rumblings of any building, and blowing of the fan, and other noises of unclear source. It was far more noisy than I had anticipated, providing a rich source of raw material for later imaginative listening.

I also actually sat through hours of recorded exorcisms. This is not an activity I am eager to repeat, but it certainly gave me a more thorough perspective on what goes on in such exorcisms – basically nothing. They are incredibly boring non-events.

I will also relay the experience of Susan Blackmore, a former ESP researcher who eventually gave up ESP research as fruitless. She noted that there is only so much you can infer about the quality of another researcher’s methodology from the published report. When you actually go into their lab and examine the methods first hand, you get a much better idea of the quality, and may identify flaws that were not apparent from the published description.

And of course, Joe Nickell, who does investigations full time, has many tales of paranormal claims and stories that could not be definitively resolved without on-the-site investigation.

Conclusion

Both armchair skepticism and first-hand investigation are important to skeptical activism. They are complementary, each filling different needs. Both also are analogous to activity that working scientists perform in the process of peer-review, new investigations, and replicating previous research.

In the last decade or so I feel that the skeptical community has honed its ability to perform meaningful peer-review of paranormal claims, and communicate the results of that review to the public. It is also my sense that our overall activism would benefit from increased efforts to perform more first hand investigations.

I would even go beyond replicating the typical haunted house investigation, and do some real hypothesis testing. This of course takes time and resources, but would be well worth it.