Jun 03 2008

Monkey Controls Robotic Arm With Its Mind

Published by under Uncategorized
Comments: 14

This is the next step in the progress toward brain-computer interfaces. Previously researchers had enabled human subjects to control the cursor on a computer screen with an EEG-computer interface. After a significant training period, through thoughts alone the subject could move the cursor on the screen. This seems like a small accomplishment but it was a significant proof of concept. Also, while modest, such an interface could enable quadraplegics to communicate or control external devices, like a wheelchair.

Researchers at Duke University led by project leader Miguel Nicolelis have previously published research in which they trained monkeys to manipulate a cursor onscreen using only a brain interface, without having to move any part of their body.

Last week in Nature online was published the results of similar research from from the University of Pittsburgh led by Dr. Andrew Schwartz. He took the next step by connecting the interface to a robotic arm. First they mapped the activity in the frontal and parietal lobes – the parts of the brain responsible for movement of the arm – while the monkeys moved their limb normally. Then they connected these signals (produced from a few hundred neurons) to a computer controlling a robotic arm. The monkeys were then able to move the robotic arm by moving their own arm. Since the signals had been mapped the researchers were able to match the robotic arm movements to the monkeys own arm. Eventually the monkeys learned how to move the robotic arm without moving their own arm. They were then able to control the arm well enough to feed themselves with it, while they were otherwise restrained and unable to move their own limbs.

This is an exciting development in this line of technology – the interface of brain and computer. This shows how quickly the brain can adapt to controlling an external device, and even separate the movements from the movements of the body. From this point better and more exquisite control is only a matter of more precise mapping and measurement of neuronal firing. This is like coming up with the basic concept of a printed circuit – from there continued progress of computing power was almost assured.


The monkeys used their own visual feedback to learn how to control the arm. Some kind of sensory feedback is critical for the brain to learn how to control the device. Control of the arm, as well as the naturalness of the feel of such an external device, would likely be significantly improved if the subject could somehow feel the arm. There are multiple types of sensory feedback that our brains receive from our limbs. Not only does the skin have tactile sensation, limbs have proprioception which is the sense of where the limb is in 3-dimensional space. This is why you can precisely control your limbs even with your eyes closed. Also, the muscles provide sensory feedback on the degree to which they are stretched. All of this sensory information is important for the brain to control the limb and for you to feel as if you own the limb, that it is part of you.

It seems that the human brain is plastic enough that it can remap both sensory input and motor output. This happens often after nerve injuries, when the nerves have to regrow. If they do not grow back to the exact location they previously terminated, then the brain has to remap to the new layout of the nerves. This can be done with practice and visual feedback. The same principle would likely apply to learning how to interface with a robotic arm. If it had the same motor input and sensory output as a real limb eventually it could feel as natural as the limb you were born with.

Adapting to new prosthetic limbs of this type would be much easier and more successful the younger the recipient. However, it is also likely that in the future through stem cell therapy or other approaches that the plasticity of the brain (the ability to rewire itself to adapt to new uses and inputs) could be increased even in older adults.

The next step in this technology, starting with motor output, then progressing to sensory input, would be cognitive input. This would involve a computer interface where the computer or prosthetic would not just give raw sensory information to the brain but complex information. This is Matrix territory – downloading languages or having access to vast amounts of information stored outside the biological brain. One computer-brain interfacing is a reality, this seems to be the inevitable arc of this technology.

The applications of such technology are too numerous to discuss fully. The most obvious uses in the short term would be prosthetics for amputees and those suffering from paralysis. Like the Bionic Man – limbs could be replaced with robotic limbs. For those with intact but paralyzed limbs, robotic exoskeletons could be used.

In addition to limb replacement and augmentation, there is direct control of external devices. This could enable a paralyzed person to control their entire home without having to lift a finger. Computers could be used for communication.

Beyond helping the sick and injured, this technology could also be used to augment the healthy. For example, direct interfaces could be used to provide exquisite real-time control of robots and machinery, either as an exoskeleton (like the Iron Man suit) or independent robotics. Or how about just controlling software – thinking what you want to write and having the words appear onscreen.

Some, like Ray Kurzweil, believe that this technology will inevitably lead to computer-brain symbiosis – we will become biological computer hybrids, “cyborgs.” I can think of no reason why such technology cannot work, and the rapidity with which it is currently developing speaks to the feasibility of such concepts. Exactly how long such technology will take to develop and the precise forms it will take are impossible to predict with any confidence – but I think we can paint the broad brushstrokes. We are seeing the beginning steps to the age of computer-brain interface technology. This is a transformative technology, and even the most enthusiastic futurists probably underestimate the degree to which it will affect our lives.

Share

14 responses so far

14 Responses to “Monkey Controls Robotic Arm With Its Mind”

  1. ellazimmon 03 Jun 2008 at 9:38 am

    I think this is fantastic! I look forward to the day when bomb squads can use a much more human like remote device so as not to put themselves in so much danger. I suppose remote controlled soldiers are being thought of already as well.

    I suppose once I learn to control an artificial limb I can plug in different “ends”. Say I’m trying to repair a system on the shuttle with a “robot”; I can swap my “hand” for a cutter or a screwdriver or a glue gun.

    I wonder how long it will take for someone to try and develop an application for the porn industry.

  2. DevilsAdvocateon 03 Jun 2008 at 12:11 pm

    Thank you, Dr. Novella, for these mental images I cannot shake of Planet Earth ruled by the Monkey Borg.

    [[[shudder]]]

  3. Steve Pageon 03 Jun 2008 at 3:08 pm

    We’re on the edge of something quite wonderful here. I can’t see a future of borg-like humans as the norm, but for those who’ve lost limbs, or even those without limbs (for example, those with a developmental deficit that prevented the limbs from forming), this could be amazing. Kudos to those at Duke Uni.

  4. superdaveon 03 Jun 2008 at 3:25 pm

    Just anecdotal, I actually have conversed with this scientist in email and he was extremely nice, you should see about getting him on the podcast.

  5. cuervoon 03 Jun 2008 at 3:42 pm

    Hey Schteve (can’t remember where that cultural reference is from…Fireball XL5?)

    I’m interested in the whole ‘futurist/transhumanist’ schtick. Some if it seems a bit flaky, with the life-extention – quasi-religious parallels, but perhaps with the benefit of being possible. The idea of the ‘singularity’ is intriguing i.e. Moore’s Law gallantly holding up, brain augmentation and AI…is this a step in that direction?[answer:yes]

    Perhaps an interesting SGU interview/discussion idea?

    Personally, I think we should be investing billions of ‘pounds’ into this.

  6. Roy Nileson 03 Jun 2008 at 4:14 pm

    Who will be in charge, in this brave new world to come, of regulating the power that the cyborgs will necessarily have to effectively enslave the lesser augmented masses?

  7. DevilsAdvocateon 03 Jun 2008 at 4:41 pm

    I just asked my health insurer if they’d cover surgery to implant my TV remote in my forearm. Now that’s augmentation.

  8. [...] Steven Novella placed an interesting blog post on Monkey Controls Robotic Arm With Its MindHere’s a brief overviewHowever, it is also likely that in the future through stem cell therapy or other approaches that the plasticity of the brain (the ability to rewire itself to adapt to new uses and inputs) could be increased even in older adults. … [...]

  9. geoff394on 04 Jun 2008 at 12:43 am

    The date on that article says 13 October 2003. It must be a mistake or something is out of date.

  10. Steven Novellaon 04 Jun 2008 at 11:46 am

    Geoff,

    Thanks for pointing that out. I had the wrong link – one to an older study. I have updated the entry with the correct descriptions and links.

    To clarify (as is now correct in the entry) Duke researchers in 2003 were able to get monkeys to move a cursor onscreen with only their thoughts.

    Now, Pittsburgh researchers have done essentially the exact same thing but hooked the output up to a robotic arm and showed that they could use the arm to perform a useful task, like feeding themselves.

  11. DevilsAdvocateon 04 Jun 2008 at 4:22 pm

    All kidding aside, and assuming that’s even a possibility for me, this thing has huge implications and virtually boundless potential. Then, when you factor in its combination with other imminent breakthroughs in bio-tech….. it honestly takes my breath away to think of what might soon be possible.

  12. ADR150on 19 Jun 2008 at 8:06 am

    Steve,

    not sure if you’ve hear the latest Science Talk podcast, but it deals with the singularity. Mersky interviews and editor and a journalist from IEEE Spectrum magazine. I found it interesting, although they raised some questions that you seem to think differently on. For example they say that the Singularitarians are mainly techies and they don’t understand the brain because “nobody really understands the brain”. They also say the problem of consiousness raises all kinds of philosophical questions and our subjective experience is “profoundly mysterious”. However, they are explicit in their assertion that no mystical or supernatural explanation is needed. There are some other points that I thought you might take issue with. All that said, these guys are journalists for a tech magazine, not scientists and it would be good to hear your take on the episode. Thanks

    Andrew

  13. SOG kniveson 17 Jul 2008 at 3:33 pm

    SOG knives…

    Interesting ideas… I wonder how the Hollywood media would portray this?…

  14. [...] Honda announced recently that they have built a device that reads the electrical impulses and blood flow of a person’s brainenabling them to control a robot with their thoughts. Sounds cool – but actually I did not find this to be a significant breakthrough. Other research centers have already achieved similar goals – remember the monkey that learned to control a robot with his mind? [...]

Trackback URI | Comments RSS

Leave a Reply

You must be logged in to post a comment.