Dec 16 2008

Mind-Reading Software

Published by under Uncategorized
Comments: 18

Headlines declare, “Mind-Reading software could record your dreams.” Of course, those in the business of writing headlines for news articles are notorious pathological liars.

The technology being reported on is indeed very intriguing, but it actually has nothing directly to do with recording dreams. This is just one speculative future application when and if the technology significantly matures. But that’s what the headline writer pulled out.

This is similar to the reporting of advances in so-called metamaterials that have a negative refractice index. These materials have many possible and plausible applications. Perhaps the least plausible and most highly speculative application would be the creation of an invisibility cloak. So of course, the headlines read that scientists make breakthrough in creating an invisibility cloak.

This technology is potentially very cool, even without sensational exaggeration. Jack Gallant and colleagues at the University of California, Berkeley have demonstrated that they can predict which image out of a set of candidates a subject is viewing just by looking at their fMRI scan – at the pattern of activity in their visual cortex.

Even better, Yukiyasu Kamitani at ATR Computational Neuroscience Laboratories in Kyoto, Japan has demonstrated that he can predict what a subject is viewing even without having candidate images to choose from.

What they are doing is this – they show subjects test patterns while recording their brain activity using fMRI. They then use software to correlate the brain patterns to the viewed images. Kamitani specifically used this technique to create a grid of black and white pixels representing visual cortical activity. They then show the subject a new image and the software interprets the fMRI to see which pixels are active, and it reconstructs what the subject is looking at. Kamitani was able to use this technique to read the word “neuron” that a subject was viewing.

This technology is still very crude – in it’s infancy, if you will. The images produces are fuzzy and low-resolution. Subjects need to be “calibrated” before they can be read. And only black and white, static, crude images can be generated.

What this is, however, is a proof of concept. It means that patterns of brain activity are reproducible enough to be interpreted by software using the current resolution of fMRI scanning. With higher resolution fMRI, better software, and more elaborate calibration it may be possible to view much more detailed images.

Researchers plan to add color. It may be possible to add motion – create video. It remains to be seen if the technology can image what a person is thinking, as opposed to what they are currently viewing, but previous research suggests that the pattern of brain activity should be similar. And then other aspects of brain function, such as language, can be investigated. There is a great deal of work to be done before this kind of technology becomes more than a laboratory curiosity.

My primary question is what are the ultimate limitations of this kind of technology? Brain activity is pretty chaotic, even though there are stable patterns. Might that chaos of activity always blur the patterns, limiting resolution and fidelity?

Brains slowly evolve over time as we learn, experience, grow, and age. How quickly will the calibration become obsolete?

The resolution of fMRI is inherently limited. It images blood flow and infers brain activity from that. Likely we will need a more direct imaging of neuronal firing to produce significantly higher resolution “mind reading.” What will that require? Can it be done remotely, or will an invasive “brain jack” be required.

I guess we’ll figure this out as we go along. It is an intriguing concept – and perhaps will be looked back upon as the first tentative steps into true computer-brain interfacing, along with parallel research looking into software control through brain electrodes (like the monkey’s who can control robotic arms).

I don’t think we’ll be recording dreams anytime soon, however. Even optimistically, that application will likely take decades.

18 responses so far