Oct 01 2012

See What You Feel

One of the main themes of scientific skepticism, at least one of my favorite themes, is that we cannot take the accuracy of our own perceptions for granted. We cannot trust what we remember about what we think we experienced – a principle I call neuropsychological humility. Human brains process information in a complex way, making assumptions and adjustments that are useful most of the time, but introduce multiple opportunities for misperceptions. This is partly why we need objective evidence as a check on our perceptions.

Neuroscientists continue to document the many ways in which our perceptions can be fooled. One category of such phenomena are so-called cross-modal interactions – one sensory modality influencing another. The basic concept here is that our brains are receiving multiple streams of information simultaneously and they weave those streams into one seamless experience of reality. Therefore what we see influences what we hear, and what we hear influences what we see, which influences what we feel, etc.

By exploiting these cross-modal interactions researchers can trick the brain into a false experience – by bending or breaking the rules of these interactions. They do this somewhat like magicians, creating scenarios for which evolution would likely not have prepared us.

A recently published study in the journal Perception demonstrates one type of cross-modal interaction that most people are probably not aware of – that what we see affects what we feel. The article is Visual influence on haptic torque perception. “Haptic” refers to exploring the world immediately around you through touch. The researchers tested the ability of subjects to sense in which direction a stick they were holding upright in one hand is weighted to one side or the other. A weight hanging on the left side of the stick would tend to pull or torque the stick to the left, which we would feel as twisting our wrist counter clockwise.

The researchers found, unsurprisingly, that subjects could feel the direction of torque even without seeing the weight on the stick. But then they used mirrors to flip the apparent side of the weight, so it appeared to be on the right even when it was on the left. This caused some subjects to feel the weight on the side their visual input told them the weight was, even though it was really on the other side. This effect was fairly robust, although was stronger for lighter weights – so there was a threshold beneath which visual input had a greater influence than haptic input.

This is a haptic illusion based upon a cross-modal effect. Interestingly, the illusion works even when subjects were aware of exactly what was going on. Even when they knew the weight was really on the left and mirrors were being used to make it appear as if it were on the right, they still felt the torque to the right.

This means that the effect is involuntary and not affected by awareness, which reflects an important underlying concept of neuroscience. Much of our brains’ processing of information, sensory input and otherwise, occurs below the level of awareness. The processing is automatic, and the results are just “presented” to our surface consciousness. Sometimes we can consciously influence this processing, and sometimes we cannot – we see the illusion so matter what we do.

I find these type of phenomena fascinating – peeking behind the curtain of our own brains and seeing how the kluge works. These narrow specific perceptual phenomena can be extrapolated to the working of our brains in general – our thoughts, feelings, and memories (in addition to our perceptions) are all the result of complex evolved algorithms that contain multiple assumptions, estimations, and calculated trade-offs, and they mostly occur beneath our awareness. It takes a great deal of introspection and metacognition to wrest a small measure of cognitive control from these subconscious processes.

Or perhaps this is just another layer of illusion of control – but at least it’s one level up from just going with the flow of our default processing.

9 responses so far

9 thoughts on “See What You Feel”

  1. d3dude says:

    This reminds me of the experiment with glasses that flipped the image upside down. The test subjects adapted to function normally and eventually became unaware that everything was flipped.

    I’m curious though, were any of the test subjects completely immune to the “haptic torque perception” tests? If they were, do you think this means their brains are abnormal and would other brain “issues” be evident as well?

    I really enjoy reading your blog posts. You have the knack of making complex ideas relatively easy to understand. Or possibly you are very adept at tricking my brain into thinking that.

  2. Quine says:

    I think this subject is very important, not only from the neuroscience standpoint, but also from the standpoint of trying to understand why we have such a strong impression of Cartesian Duality. It reminds me of a lecture in which Thomas Metzinger asked what Natural Selection would favor: “There is an iconic representation of a wolf associated with this point in my visual cortex” v. “There’s a wolf.”

  3. SimonW says:

    My hypothesis is that the “illusion” or “idea” of Cartesian duality is in part wrought from the effective working of the blood brain barrier. That is your body can take quite a lot of damage and abuse, and your mind can still function surprisingly well (as long as this abuse isn’t in the form of chemicals easily or actively transported across the barrier).

    e.g. The feeling we are in our heads is partly that is where most of the sense organs are, and partly that we have evolved a lot of separation for the brain from the other more disposable bits (like finger, toes, or the occasional arm, leg, kidney, or even eye that goes missing); both chemical and physical separation.

    If you have a medical condition where the brain does start to suffer significant defects with the status of the medical condition, I think it can change one’s “normal” perspective, and one realizes one is properly a unity of ones parts – even if some bits are better insulated from the outside world than others. I guess like in this experiment you get to peak behind the curtain and see the machine for what it is, rather than the apparently magical outputs it produces so effortlessly under better conditions.

  4. ConspicuousCarl says:

    If you have a medical condition where the brain does start to suffer significant defects with the status of the medical condition, I think it can change one’s “normal” perspective,

    This little chunk of brain appears to be one such place:

    If it isn’t working right, your brain appears to lose track of its own location in relation to your body.

    Also very strange is that the right damage to the Angular Gyrus can cause you to lose the ability to distinguish amongst your different fingers.

  5. daedalus2u says:

    This is interesting. I would argue that haptic sensation has to be mediated this way, in that cross-calibration from the visual system is necessary for eye-hand coordination to occur.

    I don’t think that anyone can be “immune” to this effect because this is how motor function (and visual pattern recognition) is learned. Infants are unable to control their motions, the neuroanatomy that controls motion is not developed at birth. Similarly, the cells that detect light in patterns on the retina are not connected into neural networks with high fidelity to do pattern recognition on the nerve impulses from those light patterns. As those nerves grow and wire together, they require feedback to maintain consilience between the visual world and the haptic world “views”.

  6. elmer mccurdy says:

    I imagine if you kept the mirrors there long enough, one would adapt.

  7. elmer mccurdy says:

    At least your vision would, no?

  8. elmer mccurdy says:

    Also, what makes this noteworthy is that it’s so freakish, and so unusual. The vast majority of our subjective perceptions are accurate, else we wouldn’t be able to function.

  9. Lenoxus says:

    Shouldn’t this post’s title be “Feel What You See”? 😉

Leave a Reply