Mar 12 2019

Robots Learning to Walk

Researchers at the USC Viterbi School of Engineering have developed a robotic limb with artificially intelligent control that learns how to walk by trying to walk. This may seem like a small thing, but it represents a fascinating trend in AI and robotics – shifting more and more to a bottom up rather than top down approach to programming.

This recent advance is very incremental, but worth pointing out. The researchers tried to designs a limb based on biological principles. Rather than programming the limb with the processes necessary to walk, including dealing with difficult terrain and recovering from a trip, they developed an algorithm that will learn how to walk and adapt by trying to do it. This type of learning algorithm from scratch is nothing new, but the researchers claim this is the first time it was applied to this particular task.

The results were impressive – the robot was able to learn how to walk within minutes. Because the learning is mostly trial and error, different iterations of this algorithm will hit upon different solutions, so different robots might have distinctive gaits.

The first thing I thought of when I read this news item is – what about Boston Dynamic’s Big Dog? This is a four-legged robot about the size of a large dog developed as a pack mule for the military, and capable of handling rough terrain. Watch the video – it’s impressive. I tried to find out how much of the Big Dog walking algorithm is learned vs programmed, but what I found is that “it’s proprietary.” But the consensus of opinion seems to be that it is partly both, a lot of developed walking algorithms but maybe incorporating some learning AI. If true the USC robotic limb would be the first fully self-learning walking robot algorithm, as they claim.

What I find most interesting is the general trend I have seen (being a technophile) over the last few decades when it comes to AI. This was partly my own learning curve, and partly how AI was portrayed in science fiction, but also reflecting the actual technology. My 1980s concept of AI was mostly top-down, meaning that I naively thought we would develop general AI in order to accomplish specific tasks, including walking. Further I assumed that much of the abilities (not necessarily the knowledge of the world) of an AI controlled robot would be “hard wired” and pre-programmed. Both of these assumptions on my part were wrong.

Over the last 30 years the actual technology has taken increasingly a different course. First, task specific AI turned out to be much more powerful than I think anyone thought, virtually eliminating the need for truly general AI in order to accomplish what we need. In other words – we do not need to create a self-aware AI with human level general intelligence in order for it to be able to play chess better than any human, drive cars, identify faces, or even write poetry or make art. Task-specific AI is getting better and better at such things, without being on a path to anything resembling self-awareness or general intelligence.

Throughout this entire process many people felt that, sure, AI can play chess, but it will never beat a grand master. Then when it did they said, sure, it can beat a grand master at chess, but it will never do X (insert any complex task). Then when it accomplished X, they moved the goalpost to Y, etc. But now I think we have seen this process play out long enough that the consensus is moving to the opinion that – maybe there really isn’t any inherent limit to what a task-specific AI can do.

The increasing power of task-specific AI, in turn, is partly due to a shift from top-down to bottom-up programming in another way – using learning algorithms rather than pre-programming. (Keep in mind, I am observing this as an interested outsider. Actual AI experts will be able to put this in more technical and specific terms.) Learning algorithms mean that we don’t need to write hundreds or thousands of rules for playing a game like go, or model playing behavior on master players. Rather, we just let the AI play the game a few million times and learn what works. This can apply to any task.

We are just scratching the surface of the potential of this technology. Computers are relentlessly getting more powerful, and increasingly being paired with big data and learning AI. The results are staggering, and only going to get more powerful.

Another layer to all this is how these trends in AI and robotics have mirrored biological systems. It seems we had something to learn from hundreds of millions of years of evolution. As one researcher of the new robotic limb put it:

“I envision muscle-driven robots, capable of mastering what an animal takes months to learn, in just a few minutes,” said Urbina-Melendez, a doctoral candidate in biomedical engineering who believes in the capacity for robotics to take bold inspiration from life. “Our work combining engineering, AI, anatomy and neuroscience is a strong indication that this is possible.”

Over this same time period as I was seeing the changes in AI, I also went to medical school and became a neuroscientist. Seeing the interplay between neuroscience and computer science has also been fascinating. When you walk, for example, you are not using your top-level cortical function to do so. You are not thinking about every tiny muscle movement, consciously sensing the direction of gravity or the feedback from the tension on your tendons. This is all happening at a subconscious level, and most of it at a subcortical level, in the most primitive parts of the brain.

Subconscious processing is more efficient. Our brains evolved to shift processing of tasks from conscious to unconscious as we practice and learn the tasks better. Our brains have a learning algorithm that uses task-specific rather than general biological intelligence whenever possible. It really is an exact analogy to the direction of AI.

Our brains also learn by doing. You learn to walk by trying to walk. Sure, there are circuits in the brain that are prepared to learn the task, and optimized for certain tasks, but you still have to develop those circuits by doing. Even the very basic subcortical pathways will not develop without the proper stimulation. Grow up in zero G and you will not develop the anti-gravity systems in the brainstem that allow you to maintain an upright posture. Grow up without language, and the language centers in your brain will not develop. Patch one eye of a child and they will never develop binocular vision. Use it or lose it.

It should not be a surprise that we found AI functions most powerfully when it follows the same kind of path that biological intelligence evolved – more task-specific programming and more learning algorithms rather than pre-programming.

In fact I have come to the belief that we may never have to develop general AI in order to accomplish whatever it is we want with our AI and robots. We probably still will, if for no other reason than just to do it. Also, we are using computers to model brain function and modeling computers after brain function – these two sciences are working in tandem. So we will probably (this is my guess) develop general AI as part of our research into modeling human brain function. But I don’t think we will develop general AI to drive our cars, operate our robots, or perform any task we might want them to do.

We’ll see. Of course my thinking is very different now than it was 30 years ago, and may be different in another 20-30 years.

No responses yet