There is an historical pattern with which skeptics and scientists should be very familiar – the dubious phenomenon that vanishes under double-blind testing. We saw this with N-rays, which now stands as a famous cautionary tale. The ephemeral rays could only be seen by those who knew what to look for, mostly French scientists. The introduction of a blinded test, however, quickly proved N-rays to be all illusion and wishful thinking. We have seen this with mesmerization, homeopathy, and a long list of other useless medical interventions, with electromagnetic sensitivity, and recently with the Power Bands. In countless cases over hundreds of years many people were utterly convinced of the reality of a phenomenon, and they could even apparently demonstrate or detect it (when they knew what they were supposed to see), but under blinded conditions the phenomenon evaporated.

It is for this reason that scientists do not (and should not) accept the reality of a new phenomenon until it has been demonstrated in such a way that bias and illusion have been ruled out. There is a general tendency to underestimate the degree to which bias and suggestibility can influence perception, no matter how many times it is experienced.

Perception is especially vulnerable to suggestion, and the influence of other senses. For example, Morrot, Brochet, and Dubourdieu performed a study in which they colored white wine red, and then had 54 tasters describe the wine. They used red metaphors to describe the wine, as would typically be used to describe a red, rather than white, wine.

In another study researchers exposed wine tasters to positive and negative statements about the wine; they did this both before and after they tasted the wine, but before they reported their impression of the wine. The comments affected the tasters’ evaluations of the wine when they occurred before, but not after, tasting the wine. This suggests that the tasters were not just following what they were told about the wine despite their experience, but rather than the positive and negative statements actually affected their experience of the taste of the wine.

In yet another study the price tag associated with wine affected the experience of “pleasantness” of the wine. There seems to be a consistent pattern in the research – subjective experiences can be modulated by suggestion, expectation, and other sensory cues.

With wine the question that many people have is this – can even experts discern the quality difference alleged to be present in really expensive wines? This is actually a more generic question that can be applied to many areas.

One such question that I confronted recently was the quality of instruments. My older daughter has been playing the flute for several years and is dedicated enough that we thought it was time to get her a finer quality flute than the old student version she was using.

There is no question that there is a significant difference in quality of construction and the materials used between the low end “student” flutes and the professional flutes. Flutes start at about $200 at the low end, and student models go to the upper hundreds. Professional flutes start at 1-2 thousand and go up from there.

This much is what I expected, about an order of magnitude difference between low end an high end flutes. But when investigating what was available we found that professional flutes had a wide range of prices, from 2 thousand to 40 thousand dollars (leaving out rare or historical instruments).

This made me seriously wonder if there was a real difference in quality between a $2000 flute and $40,000 flute. I have still not answered that question to my satisfaction, but I suspect that for about 4-5k you can get a solid silver gold-plated flute that has as good construction as any flute out there. Beyond that I suspect you are paying for prestige, not craftsmanship. (I wonder what professional flutists think of this, if any read this blog.)

I was reminded of this again with a recent article in which a researcher had 17 professional violinists try to tell the difference among six violins – two Stradivarius, one Guarneri, and three modern violins. They were literally blinded to which violin they were playing (they were blind-folded). Seven stated they could not tell which one(s) were a Stradivarius, seven guessed incorrectly, and three guessed correctly. This is consistent with random guessing.

This was a small study, but appears to have been properly blinded. The subjects were all professional violinists. If these results are reliable, that implies that a well-constructed modern violin can be just as good as the iconic Stradivarius. I am more interested, however, in the fact that this supports a larger body of research showing that people will often perceive what they expect, and so any evidence based upon unblinded perception is dubious.

Blinding is one of the most basic aspects of scientific methodology. It is well established that without blinding results can be entirely fictitious, and it is likely that people will observe exactly what they expect to. And yet it is still common practice to rely upon unblinded observation in everyday life, even for controversial conclusions.

That is one skeptical lesson I would truly like the public in general to learn – don’t trust unblinded subjective experience. It is best to assume that there is no practical limit to the extent to which we can deceive ourselves.