Dec 16 2008
Mind-Reading Software
Headlines declare, “Mind-Reading software could record your dreams.” Of course, those in the business of writing headlines for news articles are notorious pathological liars.
The technology being reported on is indeed very intriguing, but it actually has nothing directly to do with recording dreams. This is just one speculative future application when and if the technology significantly matures. But that’s what the headline writer pulled out.
This is similar to the reporting of advances in so-called metamaterials that have a negative refractice index. These materials have many possible and plausible applications. Perhaps the least plausible and most highly speculative application would be the creation of an invisibility cloak. So of course, the headlines read that scientists make breakthrough in creating an invisibility cloak.
This technology is potentially very cool, even without sensational exaggeration. Jack Gallant and colleagues at the University of California, Berkeley have demonstrated that they can predict which image out of a set of candidates a subject is viewing just by looking at their fMRI scan – at the pattern of activity in their visual cortex.
Even better, Yukiyasu Kamitani at ATR Computational Neuroscience Laboratories in Kyoto, Japan has demonstrated that he can predict what a subject is viewing even without having candidate images to choose from.
What they are doing is this – they show subjects test patterns while recording their brain activity using fMRI. They then use software to correlate the brain patterns to the viewed images. Kamitani specifically used this technique to create a grid of black and white pixels representing visual cortical activity. They then show the subject a new image and the software interprets the fMRI to see which pixels are active, and it reconstructs what the subject is looking at. Kamitani was able to use this technique to read the word “neuron” that a subject was viewing.
This technology is still very crude – in it’s infancy, if you will. The images produces are fuzzy and low-resolution. Subjects need to be “calibrated” before they can be read. And only black and white, static, crude images can be generated.
What this is, however, is a proof of concept. It means that patterns of brain activity are reproducible enough to be interpreted by software using the current resolution of fMRI scanning. With higher resolution fMRI, better software, and more elaborate calibration it may be possible to view much more detailed images.
Researchers plan to add color. It may be possible to add motion – create video. It remains to be seen if the technology can image what a person is thinking, as opposed to what they are currently viewing, but previous research suggests that the pattern of brain activity should be similar. And then other aspects of brain function, such as language, can be investigated. There is a great deal of work to be done before this kind of technology becomes more than a laboratory curiosity.
My primary question is what are the ultimate limitations of this kind of technology? Brain activity is pretty chaotic, even though there are stable patterns. Might that chaos of activity always blur the patterns, limiting resolution and fidelity?
Brains slowly evolve over time as we learn, experience, grow, and age. How quickly will the calibration become obsolete?
The resolution of fMRI is inherently limited. It images blood flow and infers brain activity from that. Likely we will need a more direct imaging of neuronal firing to produce significantly higher resolution “mind reading.” What will that require? Can it be done remotely, or will an invasive “brain jack” be required.
I guess we’ll figure this out as we go along. It is an intriguing concept – and perhaps will be looked back upon as the first tentative steps into true computer-brain interfacing, along with parallel research looking into software control through brain electrodes (like the monkey’s who can control robotic arms).
I don’t think we’ll be recording dreams anytime soon, however. Even optimistically, that application will likely take decades.
18 Responses to “Mind-Reading Software”
Leave a Reply
You must be logged in to post a comment.
I seem to recall B. Alan Wallace taking a swipe at fMRI in the SGU interview with him. In making his case for our ignorance of how the brain works, I remember him balking about how it just measures blood flow, implying that insights made using this technology were necessarily crude. And yet, correlating such a crude observation such as blood flow in the brain can produce such amazing and clearly testable insights.
Cool story, Steve.
Headline writers- I agree with you there.
If we can image what the person is viewing, it is probably possible to view what he is ‘picturing’ in his mind’s eye.
This is not the same as ‘what he is thinking’, but would be a cool step in between.
This would be an interesting test.
Hubbub, that Wallace interview was by far the most painful to listen to. That guy sounds like even if he was on the skeptics side, he would still be an arrogant mean spirited jerk.
” – they can predict which image out of a set of candidates a subject is viewing just by looking at their fMRI scan – ”
Not to quibble, but deducing what they “is” doing is not exactly a prediction. The magical inference just isn’t the same.
“Of course, those in the business of writing headlines for news articles are notorious pathological liars.”
Oh please, that’s just silly! Not to mention seemingly a bit naive. Do you understand the function of a headline in the business of news? It’s to grab attention in the midst of competing headlines (of course, your comment about headlines could itself be viewed primarily as a sensationalist attention grabbing device too). Its whole job is to be as sensational as possible so people will buy/consume the article attached to it (wherein lies the accurate or inaccurate information). Quite often, a headline isn’t written by the same person who wrote the article. A headline is not news, it’s an ad for the news…a teaser…a lure…a headline is a girl dancing on a table in a short skirt promising to take off her top…. metaphorically speaking, of course since topless girls dancing on tables isn’t really news or a headline unless it’s Hillary Clinton or perhaps someone whose breasts most of us would like to oggle.
Apparently many people who write think others are interested in a technology that may one day be able to “read” dreams – it’s certainly a sci-fi cliché – and think it makes a good attention grabber!
How can you be sure Hillary isn’t at bottom oggleworthy?
[[[shudder]]]
“And then other aspects of brain function, such as language, can be investigated.”
I can see the headlines now…
Scientists make breakthrough in Universal Translator, holodeck “just around the corner” says Dr. N.
Very Interesting.
Look forward when they add the colour and motion, and see how accurate they can interpret.
What the fMRI BOLD technique (functional Magnetic Resonance Imaging Blood Oxygen Level Dependant) measures is prompt changes in blood flow caused by neurogenic release of NO. The fundamental thing being measured can be thought of as the regions of the brain where the local NO level is high enough to activate sGC and cause vasodilation (a few nM/L, the exact levels are not well known and are likely variable).
Presumably it is not the vasodilation that is actually causing the neuronal activity because the neuronal activity slightly precedes the vasodilation; the vasodilation is merely correlated with what is actually causing the neuronal activity (that would be the local NO level being above some threshold close to where sGC is activated).
If it were increases in the local blood flow that were causing the neuronal activity, then increases in blood flow alone would be enough to cause neuronal activity. The increase in flow can either increase the concentration of a metabolite carried by the blood, or decrease the concentration of a metabolite carried away by the blood. If the former, the increase in metabolite concentration can only be very small, on the order of the change in blood flow. If the latter, then a source (constant or variable) of the metabolite being removed is needed.
It turns out that blood is the sink for NO, so the neurogenic NO release causes acute local vasodilation which increases the local concentration of hemoglobin which removes the NO as nitrate/nitrite (at kinetics limited by diffusion to hemoglobin).
Fifi – Are you saying that it’s okay to lie, so long as you’re just doing it to make money?
Febo – I’m saying that expecting an apple to be a sheep is not reality based thinking. Newspaper headlines serve the same function now as they always have. Newspaper headlines have ALWAYS been sensationalist, they have always been intended to grab eyes and sell papers, and newspapers have always been commercial concerns influenced by their publishers. Now, you can go on about how morally wrong you happen to feel this is but it’s a bit silly and not dealing with how the real world functions. (I think commercial medicine raises many ethical issues that are far more important than the fact that headlines are used to sell newspapers – however I also accept we live in the real world so compromises and imperfection are part of life, even as we try to live and create our ideals.) I’m also pointing out that the blog does exactly what it decries in the opening – it sensationalizes something (partly by waving around a bit of a red herring), presents it in a way that’s not particularly reality based for dramatic impact and so on. I don’t take any issue with trying to make something interesting to read – or even creating a bit of drama for entertainment purposes – but it’s a bit silly to do it while simultaneously decrying others for doing it!
Fifi,
I am very aware of the purpose of headlines. I recently wrote about it here: http://www.theskepticsguide.org/sgublog/?p=393 – where I make all the same points you do.
My purpose in writing about it is to point out the phenomenon to my readers. For science new stories, headlines resort to cliche’s and greatly distort the content of the story. Unfortunately, many people just read headlines and maybe the opening paragraph to get a sense of the news. This will leave them with a very false impression of the actual science.
I simply want to raise awareness that science news headlines can be misleading and are not to be trusted.
Also – I obviously have nothing against writing in an interesting and provocative manner. I am not advocating that popular news stories should be dry and technical. But you can do that without misleading, which is what I try to do, and therefore I think your analogy is a false one.
Dr Novella – Sorry but I found your opening gambit misleading and to be doing exactly what you decried (and I’ve explained why so I guess we’ll just disagree on that one – I guess you’re more attuned to misrepresentations of science, while I’m more attuned to misrepresentations of journalism, an entirely natural bias on both our parts!
)
I agree that people often just read headlines and don’t bother reading the actual article. However people also do all other kinds of foolish things and make grand assumptions based on things other than headlines. Plus, the headline may even have been pulled directly from the press release sent out by the lab/university/corporation behind the research. A great deal of pseudoscience/fictionalizing of science to make it “fun” and “attention grabbing” – like these kinds of headlines – are a being promoted and sold to the media by those doing the science. Now, I’m not trying to negate the journalist, editors and publications’ responsibilities (to their audience, journalists, publishers and the paper’s bottom line) – I’m just pointing out that much of the cause of this kind of thing actually comes out of academic and commercial science. Puffing on dandelions (getting upset about headlines) isn’t nearly as effective as digging up the roots (or the whole network of roots). Seems to me that putting some pressure on the the science end would be more effective than complaining about headlines not being something other than what they’ve already been because you want them to serve your purpose not their historical one!
@superdave
That interview? Annoying? I thought you were supposed to be a skeptic. I had lunch with a prominent physicist and he said it was amazingly informative and entertaining.
Also, if you takes you longer than 10 milliseconds to read this comment, then dualism must be true.
Fifi – I grant you that science press releases are also part of the equation. They are often written by the PR department of Universities or other institutions – not by the researchers. I have complained before about such press releases, and the passing along of such releases unfiltered by the press.
I think you are getting too defensive about a side point I made in this blog entry. This was not meant to be a thorough discussion of science news reporting. I was just expressing irritation at the tendency to sensationalize headlines for science news stories in this particular fashion – highlighting the most speculative and unlikely application of a genuinely interesting development.
You also misrepresent my position. I already acknowledged that headlines are meant to be attention grabbers. I am not complaining that they are not serving “my” purposes. Rather, I think such headline writing is just lazy. It’s grabbing the easy angle – what sticks out the most.
It takes more effort, but it’s possible, to write a headline that grabs attention without grossly misleading.
Dr Novella – Please feel free to think I’m being too defensive, it’s an opinion. In my opinion, you were being hyperbolic and falsely laying all responsibility for “false news reporting” at the door of journalists and editors – if the “lies” and misrepresentations are actually reflections of the press releases sent out by institutions to promote the work of various scientists (which they often are), blaming editors and journalists is not only misleading but it’s not even addressing the root cause of the problem. It takes more effort to address the rot in our institutions – and is more personally dangerous – than it does to lay the blame at the door of others. There are valid critiques to be made of science journalism – simply blaming journalists for not questioning press releases that emmanate from exactly the institutions they turn to for expert advice pretty much amounts to slapping journalists around with a red herring – not to mention actually obscuring the real source of so much disinfo about science and the true source of the hyperbole!
Your Mind-Reading Software story reminds me of an artist who always labels his paintings after he draws them. He sold a painting once which had three mounds of colors. The title he chose was “Three Nuns”. Everyone at the gallery was trying to see the characteristics of the title in the picture.