May 21 2021

The Neuroscience of Robotic Augmentation

Imagine having an extra arm, or an extra thumb on one hand, or even a tail, and imagine that it felt like a natural part of your body and you could control it easily and dexterously. How plausible is this type of robotic augmentation? Could “Doc Oc” really exist?

I have been following the science of brain-machine interface (BMI) for years, and the research consistently shows that such augmentation is neurologically possible. There is still a lot of research to be done, and the ultimate limits of this technology will be discovered when real-world use becomes common. But the early signs are very good. Brain plasticity seems to be sufficient to allow for incorporation of robotic feedback and motor control into our existing neural networks. But there are still questions about how complete this incorporation can be, and what other neurological effects might result.

A new study further explores some of these questions. They studied 20 participants who them fitted with a “third thumb” opposite their natural thumb on one hand. Each thumb was customized and 3D printed, and could be controlled with pressure sensors under their toes. The subjects quickly learned how to control the thumb, and could soon do complex tasks, even while distracted or blindfolded. They further reports that over time the thumb felt increasingly like part of their body. They used the thumb, even at home, for 2-6 hours each day over five days. (Ten control subjects wore an inactive thumb.)

They used fMRI to scan the subjects at the beginning and end of their training. What they found was that subjects changed the way they used the muscles of their hand with the third thumb, in order to accommodate the extra digit. There were also two effects on the motor cortex representing the hand with the extra thumb. At baseline each finger moved independently would have a distinct pattern of motor cortex activation. After training, these patterns became less distinct. The researchers refer to this as a, “mild collapse of the augmented hand’s motor representation.” Second, there was a decrease in what is called “kinematic synergy.”

This is an interesting phenomenon in itself. One general principle of brain evolution and function is that efficiency takes a high priority. The brain is complex and energy hungry, and tasks require a lot of computational power. So evolution found a lot of shortcuts to reduce the processing load on the brain. One of those shortcuts is to automate frequently repeated tasks. As you train with any motor task, like the iconic example of shooting baskets, you have to consciously think about the details of your movement less and less over time. It becomes “automatic” and subconscious. You develop “muscle memory” which is really brain memory for specific coordinated muscle movements, but below the level of conscious awareness. While I am typing these words, I am not thinking at all about where each key is or which finger I have to move.  The words I am thinking just appear on the screen. I have been typing for so many years it is now an automatic process.

Part of this “muscle memory” is kinematic synergy, because different muscles need to work together in order to produce a specific motor task. Activating multiple muscles becomes a single action (neurologically), rather than a group of actions. It’s like a macro for frequently executed functions. It also comes at the expense of a decrease in degrees of freedom. Muscle movements are not as open-ended and flexible, because certain activations patterns will tend to occur together.

Getting back to the study – they found that over the five days of examination the subjects had a decrease not only in the distinctness of their motor cortex activation patterns relating to the augmented hand, but a decrease in their kinematic synergy. Essentially the brains followed Yoda’s advice – “You must unlearn what you have learned.” This makes sense. If the brain is going to maintain a certain level of flexibility and plasticity, it would need to undo previous learned patterns in order to have the potential for new functions. First you must regain degrees of freedom before a new action can be learned. A week later, however, these fMRI changes reverted back to baseline. This makes sense too. Years of use will not be undone by 5 days of trying something new. Muscle memory is enduring, like the old saying about riding a bike. But it remains to be seen what far longer experience would do.

What this and other research suggests is that at least there is the potential for sufficient neurological flexibility to functionally incorporate robotic enhancement. Even extra digits or limbs could feel natural and be effectively controlled. Imagine if the extra thumb were controlled not with a foot pedal but with your mind, and imagine that it provided sensory feedback when you moved it, in order to close the loop. Evidence suggests these factors would greatly enhance neurological incorporation.

This goes beyond robotic replacement of lost or non-functioning body parts. It’s possible that in the future robotic augmentation will be common. Many jobs might require it. Developing facility with a particular augmentation over years might become a critical job skill, an asset not easily replaced. It will also be interesting to see the effects of augmentation at a very young age when plasticity is much greater. That introduces some interesting ethical questions. It is very likely that if you fit an infant with an augmented robotic part (like a tail, for instance) their brains would develop to incorporate the extra part much more thoroughly than any adult can. But would and should parents have the right to do such a thing. It is also possible that we might figure out a way to return an adult brain to a state more like an infant, with greater plasticity. I suspect that sufficient genetic tweaking will do it.

This all points to our cyborg future. The implications will likely be greater than we currently imagine.

No responses yet