Masaru Emoto thought that emotions can affect inanimate objects. If you are nice to water and then freeze it, it will make pretty happy crystals. If you are mean to water and then freeze it, it will make ugly unhappy crystals. He wrote:

The result was that we always observed beautiful crystals after giving good words, playing good music, and showing, playing, or offering pure prayer to water. On the other hand, we observed disfigured crystals in the opposite situation.

I know – this is ridiculous. Why even bother? Scientific skeptics study pseudoscience for several reasons. What is interesting about Emoto’s research is not the research question he was pursuing (which has close to zero plausibility) but how he had managed to convince himself that his research supported his fantastical notion. Further, he had managed to convince (or at least intrigue) a large segment of the public that he was onto something, and so this presents an opportunity to teach the public about how science works and how we can distinguish it from pseudoscience. Finally, even serious science can fall prey to error and self-deception. Blatant pseudoscience is an excellent opportunity to see pathological science in the extreme, which helps us understand it better phenomenologically, and hopefully then avoid more subtle manifestations elsewhere.

I do want to emphasize that I have no problem with Emoto researching this question (as long as he was not wasting limited public research funds). Exploring seemingly wacky ideas may bear unforeseen fruit. The probability is low, but that is the nature of exploratory research. Fringe research is a good way to keep us on our toes, keep us from getting too complacent or narrow in our view. And very occasionally, we may just get surprised. Even if the hypothesis itself turns out to be hopelessly wrong, we may find something unexpected along the way.

What I do have a problem with, however, is doing bad or shoddy research in order to confirm a silly idea, and then claiming that the silly idea is scientific. That, in my opinion, is what Emoto was doing.

In case anyone is holding out that there is some mysterious plausibility to Emoto’s claims – there isn’t. He claimed that simply writing negative words on a container of water is enough to change the physical properties of that water. Such a claim would require a fundamental change in our understanding of the basic forces of nature and how nature works. Simply appealing to the limits of human knowledge is not enough to rescue such a hypothesis.

While we don’t and can’t know everything, and must always leave the door open to new evidence and new ideas, we have accumulated a body of knowledge that we can use to evaluate new hypotheses. Essentially we can ask – is this alleged phenomenon compatible with what has already been established and does it require new physics? These are two distinct criteria. Simply requiring new phenomena in order for a hypothesis to be viable does make it less likely to be true, simply because we are venturing into the unknown. Most new ideas in science turn out to be wrong.

Far more damning, however, is when a new hypothesis would overturn a mountain of existing research. The larger the body of verified research that would need to be wrong in order for a hypothesis to be true, the greater the burden of proof for the new idea. So – I would not say that you could never convince me (that is not a scientific approach), but rather that the evidence for the new hypothesis has to be of a quality and quantity greater than the evidence that suggests the hypothesis must be wrong.

This is one of the primary ways that pseudoscientists go wrong – they put up an ant-hill and claim it disproves a mountain. When the scientific community is not impressed, they accuse them of being closed-minded or engaged in a conspiracy.

In order to maintain their claims they generally overestimate the magnitude of their own evidence, and are either ignorant or dismissive of the established evidence. Part of overestimating the value of their own research is ignorance of methodology and grossly underestimating the role of self-deception and bias in research.

For example, Emoto’s research is fairly universally criticized for poor methodology. He used sample sizes that are too small, outcomes that are subjective, and methods that are insufficiently blinded. For this reason, his results do not stand up to replication when proper methods are used. He was engaged in the classic pseudoscientific process of starting with a conclusion and then seeking to prove that conclusion, rather than genuinely trying to prove his own hypothesis wrong.

To illustrate this, Emoto had expanded his research from water to rice. He filled jars with rice and added water for the rice to absorb. He then left them for days to see how much mold and fungus they grew. Some jars of rice were exposed to positive emotions, some to negative emotions, and some were ignored. Unsurprisingly, he found that the rice with positive emotions grew less fungus than the rice exposed to negative emotions.

And again – his methodology is not convincing. He used a small sample size, his outcomes were open-ended, and his observations were not blinded. How long did he observe the rice? What did he consider a positive or negative outcome? How many times did he do the experiment before he liked the results?

Unsurprisingly, his experiment does not hold up to even basic replication. Attempts to repeat the experiment with a larger sample size were entirely negative.

The lessons here are basic to understanding scientific methodology. In order to avoid p-hacking (getting the results you want by tweaking the experiment) you need to use rigorous methods. Sample sizes need to be large enough to have statistical power. You need to determine before the experiment what the outcome measure will be, how long the observations will be, how may subjects there will be, and what kind of analysis you will make. You cannot make these decisions after you start to collect data, because then you can easily subconsciously p-hack the results.

Emoto gives us an extreme example of this. It shows that there is no hypothesis so ridiculous that you cannot p-hack your way to an apparently positive result. This is an important lesson for serious scientific researchers, to remain vigilant against more subtle manifestations of p-hacking.

All scientists and science enthusiasts should be students of pseudoscience.