It has been fascinating, perhaps especially so as a neuroscientist, to watch the progress being made in artificial intelligence (AI), robotics, and brain-machine interface. Our understanding of biological intelligence is progressing in tandem with our attempts to replicate some of the functioning of that intelligence, as well as interface with it. Neuroscience and AI/robotics inform each other.

Here is another example of that – adding sound to the perception of an AI/robotic system to help it distinguish different objects. Before now AI object recognition has mostly been purely visual. (I always qualify these statements because I am not aware of every lab in the world that might be working on such things.) Visual object recognition is a good place to start, and this is what we probably think of when we imagine identifying an object ourselves – we look at it and compare it to our mental database of known objects. That is how our visual processing works.

But that is also not the whole story. We tend to underestimate, or simply not be aware of, the extent to which our brain are simultaneously processing multiple different sensory modalities to make sense of the world. When we listen to someone talk, for example, our brains also process the movement of their lips in order to make sense of the sounds as language (this is called the McGurk effect). When we identify someone else’s probable gender, we are strongly influenced by their voice.

Continue Reading »

For adults who had vision but then lose it due to eye disease or damage to the optic nerve, their visual cortex is still intact. It is deprived if input, but is theoretically capable of functioning normally to create images. The ultimate technological expression of this potential would be something like the visor of Geordi La Forge – a device that can see (even in frequencies and particles humans cannot normally see) and transfer that information to the visual cortex. Obviously we are a long way away from any such technology, but we have taken the first baby steps in that direction, including a recent study which makes one tiny advance (but more on that shortly).

Early research into this approach involved animals and simply tried to determine if the monkeys could “see” the stimulation. Often simple behaviors, like moving their eyes, were used to see if the stimulation was having any effect. Some of the research also comes from trying to map the visual cortex, not necessary allow the blind to see. This research has been encouraging, because it shows that the primary visual cortex is arranged in a way to reflect the images it sees (so-called retinal mapping). The neurons, in short, are like a bitmap of an image. So if you stimulated a circle of neurons in the primary visual cortex, subjects would see a circle.

Of course there is more to vision than the primary visual cortex. A lot of processing occurs in the nerves  and pathways carrying information to the cortex. After the basic image is formed it is then sent to higher visual cortical areas for further processing, so a two-dimensional image is given shape, shading, movement, distance, three-dimensionality, and ultimately meaning. But hopefully we wouldn’t need to worry about all that higher levels of processing because once the image is presented to the primary visual cortex, the rest should take care of itself.

Continue Reading »

A new study offers a small advance in our understanding of the body ownership illusion, but it is a good opportunity to review this cool and important neurological phenomenon. It also has practical implications. The body ownership illusion is the subjective sense that you occupy or own your body and its parts. There is also a separate phenomenon that gives you the subjective sense that you control your body parts.

What I find extremely interesting about these phenomena is that we initially didn’t know they needed to exist. People don’t generally wonder what creates the sense that they own and control their own bodies. It’s not even a question people think to ask (unless they are somehow involved in neuroscience). At first it may seem obvious – we are our bodies, and we do control our bodies, so why shouldn’t we have that sense? But like everything you experience and feel, this is not a passive or automatic sensation. It is an active construction of your brain. There are dedicated circuits in the brain whose specific function is to create these sensations.

As with most neurological phenomena we first suspected their existence by encountering patients who have suffered brain damage (like from a stroke) and therefore have a lack of some subset of these functions. For example, there is alien-hand syndrome. This is the sense that a body part is acting “on it’s own” without your conscious control. There are also cases that demonstrate the separation of body parts from the sense that we own those body parts. Phantom limbs, for example, occur when a limb is removed but the brain circuitry that creates the sense of ownership of the limb is still there. There can even be supernumerary phantom limbs – an illusion of an extra limb that was never there. This can happen when a stroke paralyzes a limb but spares the circuitry that creates the sense of ownership. There are also cases of the apparent opposite – when a limb is present and functioning, but the person has the uncomfortable sense that it does not belong to them. We also have evidence from certain drug-induced and other states of feeling entirely separated from our bodies.

We have a partial understanding of how our brains create these sensations. They all involve circuits that compare two or more inputs. When they are synchronous we have the subjective sense of ownership or control. For the sense of control the circuits compare our intention to move and our actual movements. When they match, we feel we have control. For ownership the primary circuit involves “visual-motor synchronization” or visual-sensory synchronization. If what we see of ourselves matches what we feel, that synchronization produces the sense of ownership. Obviously, we don’t immediately have an out-of-body experience when we close our eyes. There are other sensory inputs that act redundantly – proprioception, vestibular function, tactile sensation, and muscle feedback. It’s a robust system, which is why the illusion rarely breaks in day-to-day life. You have to disrupt the main parts of the brain producing this sensation with drugs, oxygen deprivation, or something else.

Continue Reading »

This is another entry in my informal series on interfacing machines and the human brain. Yesterday I wrote about Neuralink, which is a project to develop electrodes to interface with the brain itself. Today I write about another incremental advance – in the July 17th issue of Science Robotics, researchers published, “A neuro-inspired artificial peripheral nervous system for scalable electronic skins.”

This seems to be a serious advance in the architecture of artificial skin.

“We demonstrate prototype arrays of up to 240 artificial mechanoreceptors that transmitted events asynchronously at a constant latency of 1 ms while maintaining an ultra-high temporal precision of <60 ns, thus resolving fine spatiotemporal features necessary for rapid tactile perception. Our platform requires only a single electrical conductor for signal propagation, realizing sensor arrays that are dynamically reconfigurable and robust to damage.”

This configuration is more scalable than the current designs, which “are currently interfaced via time-divisional multiple access (TDMA), where individual sensors are sampled sequentially and periodically to reconstruct a two-dimensional (2D) map of pressure distribution.” As the number of sensors increases, with this design, the delay in signal processing gets greater. The new design does not suffer that limitation.

Biological skin evolved to have a host of desirable features. It is soft, has a variable number of sensors as needed, and yet is robust, still operates with minor damage, and is self-repairing. Artificial skin would be optimal if it shared all these features. This new eskin gets us closer to this ideal.

Continue Reading »

I’m still trying to figure out if Elon Musk is a mad genius or a supervillain. Perhaps that’s a false dichotomy. Seriously, I do like his approach – he has billions of dollars laying around, so he decides that we need some specific technology in order to build the future, and he builds a company dedicated to developing that technology. Wherever he sees holes, he tries to fill them.

SpaceX has been, in my opinion, his most dramatic success. He has pioneered the technology of reusable rockets, and anyone who has seen one of his falcon rockets landing vertically has to be impressed. Tesla cars are impressive as well, but from what I understand he still has to make the company profitable. I’m still skeptical about the hyperloop, but at least he’s trying. It all depends on how cheap he can make tunneling, and the real innovation may be in his Boring company.

Not all of his companies involve travel. He also wants to change humans, in order to ultimately keep up with the AI he thinks we will inevitably create. In 2017 he tweeted, “If you’re not concerned about AI safety, you should be. Vastly more risk than North Korea,” along with a picture declaring, “In the end, the machines will win.” The existential threat of AI is a separate question.

Now for most people, if you are worried about AI you talk about it with your friends and colleagues. Perhaps you have a blog where you can share your concerns with the world. But if you are Elon Musk you can start a 100 million dollar company designed to thwart the perceived threat. So that’s what he did.

Continue Reading »

We are on the cusp of several technologies that promise to transform medicine – genetic manipulation, brain-machine interface, and stem cell therapies among them. One of the hopes for stem cells is that we will be able to grow from them entire replacement organs. Imagine that you have kidney failure, and face years of dialysis with the hope of finding a kidney donor, so you can exchange that dialysis for a regimen of anti-rejection drugs. You may wait years on the list because we simply don’t have enough organs to go around.

Now imagine that even before your kidneys completely fail doctor take a skin or mucous membrane sample from you, and then over the next few month they grow a new kidney from your own cells. The kidney is your own tissue, and so there is no rejection at all. Eventually the new kidney is surgically implanted and you are good to go.

It doesn’t take much imagination to see how awesome this could be. If we could grow new hearts, lungs, livers, pancreases or kidneys from one’s own tissue that would transform medicine. The threshold for doing transplants could go way down, because we no longer need donors, we can grow them. The risk would go down because there is no longer the possibility of rejection and therefore the need for powerful anti-rejection drugs, so we could do transplants in more situations. We wouldn’t have to wait for organs to completely fail. Also, if you have cancer we don’t have to try to preserve as much of the organ as possible – just take the whole thing out, making sure you get all the cancer, and replace it with a new one.

This would be a truly transformative medical advance that would bring us into a new age of medicine. I don’t think we are close (meaning <10 years) to such applications, but we are close enough to say that researchers are working on it, and to see a path to get there. It’s enticing.

Continue Reading »

As our electronic and computer technology advances, the technology of implantable medical devices is opening up. Things like pacemakers are already old established tech, but ambitious researchers are looking at much more than just pacing the heart. There is potential for brain-machine interfaces, spinal cord stimulators, cochlear implants, and even replacement organs. One major technological limitation, however, is how to power these devices.

Right now the state-of-the art is small batteries. A pacemaker, for example, can last 5-10 years on one battery, which then would have to be replaced. Replacing a battery requires another surgery, which is fairly low risk but not negligible. Researchers are working on essentially two options to get around this limitation. The first is recharging batteries from the outside using a coil. You generate an electromagnetic field on the outside which induces a current in an implanted coil which then recharges the battery. This works, but is limited by the fact that tissue tends to block the field, and also the coils are bulky which limits the biological spaces in which they can be placed.

Still, external recharging can work for some applications. For example, a pacemaker can have a battery pack just under the skin that a coil can be placed directly over to recharge.

Perhaps a better option would be if the device could harvest energy continuously from the body itself. The body generates a lot of energy in multiple forms, and if even a small amount of that could be harvested that could keep something like a pacemaker going indefinitely, without any further recharging or surgery. Further, a totally self-charging device could be placed deep within tissue, without the need to have a coil visibly just under the skin.

Continue Reading »

I have been following the development of brain-machine interfaces and their application to artificial (prosthetic) limbs. A recent paper documents another incremental but significant advance – combining a new surgical technique for amputation with a prosthetic designed to take advantage of it.

The goal is to close the loop between voluntary muscle control and sensory feedback, which is critical to that control. The way our brains normally work is to constantly monitor various sensory streams in order to create the subconscious and conscious sensation that we occupy, own, and control the various parts of our body.

We may take these phenomena for granted, but they are an active construction of the brain that are critical to proper function. Without the sensation that we occupy our body, we would have “out-of-body” sensations, like we were floating in space, and this would make it difficult to interface with the physical world.

Without the sense of ownership, we don’t feel like a limb is part of us, and it is therefore not incorporated into our internal model of our own body. This is the current state-of-the art for normal prosthetics – they feel like they are attached to the user, but not part of them. It therefore takes more conscious effort to use them, and control is not as good as a normal limb.

Continue Reading »

It’s interesting to follow truly cutting edge research that has the potential to significantly change our world. I include in this category research into brain-machine interfaces, regeneration through stem cells, genetic engineering, and fusion energy. I would also add research into creating synthetic life.

Synthetic life research views living organisms like a technology. It is, in a way, the original nanotechnology, using complex tiny machines to manufacture chemicals, collect and store energy, degrade toxins, and other functions. Scientists have been very successful in tweaking existing organisms to harness them as tiny factories. Many modern drugs are now made in this way, making drugs like insulin widely available.

Some researchers, however, want to go beyond tweaking existing organisms. What if we could create synthetic organisms, even just single cells, entirely from scratch? That is Craig Venter’s dream – to strip down cells to their bare essentials, and then use that as a template to create a completely artificial minimal generic cell. That basic artificial cell, which we understand well because we built it from the ground up, can then be modified to perform endless functions – designer cells.

There is more to this vision, however. Once we free ourselves from the constraints of existing organisms we can explore novel properties that did not happen to evolve. Evolution is powerful, and it has had several billion years to experiment with life, but evolution is also constrained by its own history. For example, all life on earth uses the same genetic code, based upon two pairs of bases in the DNA – cytosine bonds with guanine and thymine bonds with adenine. This produces a 4-letter alphabet for the genetic code (CTAG), and also gives DNA the double-stranded structure and its ability to make copies of itself. The code consists of 64 3-letter words.

Continue Reading »

Futurists have a love-hate relationship with artificial intelligence (AI). Elon Musk represents the fear side of this. In two recent articles we see two sides of this fear of AI. In a Vanity Fair piece we learn:

He told Bloomberg’s Ashlee Vance, the author of the biography Elon Musk, that he was afraid that his friend Larry Page, a co-founder of Google and now the C.E.O. of its parent company, Alphabet, could have perfectly good intentions but still “produce something evil by accident”—including, possibly, “a fleet of artificial intelligence-enhanced robots capable of destroying mankind.”

We also learn from The Verge:

SpaceX and Tesla CEO Elon Musk is backing a brain-computer interface venture called Neuralink, according to The Wall Street Journal. The company, which is still in the earliest stages of existence and has no public presence whatsoever, is centered on creating devices that can be implanted in the human brain, with the eventual purpose of helping human beings merge with software and keep pace with advancements in artificial intelligence. These enhancements could improve memory or allow for more direct interfacing with computing devices.

So Musk thinks we need to enhance our own intelligence digitally in order to compete with the AI that we are also creating, so they don’t destroy us. Musk is joined by Bill Gates and Stephen Hawking raising the alarm bells about the dangers of AI.

On the other end of the spectrum are Ray Kurzweil, Mark Zuckerberg and Larry Page. They think AI will bring about the next revolution for humanity, and we have nothing to worry about.

So who is right?

Continue Reading »