Oct 23 2014

A Brain-Training Update

Can playing video games or specifically designed computer games improve your cognitive function? There are many companies who claim that they can and who would like to sell you such games that they claim are “scientifically designed.”

So-called brain-training is a burgeoning business, with perhaps the best known product being Lumosity. Lumosity promises:

“Scientifically designed games: Lumosity scientists study many common neuropsychological tasks, design some new ones, and transform these tasks into fun, challenging games.”

They claim to be a “leader in the science of brain training,” and include a list of 13 studies that allegedly show Lumosity is effective. Many of the studies do not even test efficacy, and strangely the list does not include this recent study from August 2014 showing that Lumosity is not effective.

This new study involved 77 subjects randomly assigned to play 8 hours of Lumosity or Portal 2 (a popular video game). They found that the Portal 2 players outperformed the Lumosity players on all three cognitive evaluations: problem solving, spatial skills, and persistence. The only pre-test to post-test significant improvement was the Portal 2 group for spatial skills.

This is a relatively small and short term test, so by itself it is not definitive. The results, however, are devastating to the claims of Lumosity if they hold up – Lumosity is not as effective as regular video games, and does not appear to be effective at all.

For those who are not familiar with Portal 2, you should be. It is an incredible game, which essentially involves solving interesting three-dimensional physics puzzles. This was a good game to test, and it’s not surprising that it would show an improvement specifically in spatial skills.

But we don’t have to rely on one study to determine if brain-training games work. There is a fairly large body of research exploring the many aspects of this question. I have been writing about this topic for a while. Here are my bottom line thoughts and recommendations:

  • Brain-training is effective, whether designed as classic cognitive tasks, combined tasks, or video games.
  • Effects are mostly restricted to the specific tasks being trained and do not significantly generalize to other tasks or cognitive functions. Effects tend to be short lived, although evidence here is very mixed.
  • Computer-based brain-training does not appear to be significantly different in outcome from traditional pencil and paper-based training, but is less labor intensive.
  • I could find no published evidence to support any claims for individualized programs.
  • Engaging in various types of cognitively demanding tasks is probably a good thing.
  • Try to engage in novel and various different types of tasks. These do not have to be computer-based.
  • Find games that you genuinely find fun – don’t make it a chore, and don’t overdo it.
  • Don’t spend lots of money on fancy brain-training programs with dramatic claims.
  • Don’t believe the hype.

Essentially, engaging in any cognitive task will make you better at that cognitive task and perhaps closely related tasks. The benefits are modest and probably short lived. There is no reason to think from the evidence that any specific brain-training game can improve general cognitive abilities, or that there is a permanent or even long term benefit to brain function. The claims of companies selling such games, therefore, are overhyped and misleading.

Fortunately, you don’t have to take my word for it. Recently the Stanford Center on Longevity and the Berlin Max Planck Institute for Human Development published a consensus statement on brain training games. They assembled relevant experts from around the world and then put together a consensus statement. Essentially, they agree with me, which is reassuring. Here is the summary:

In summary: We object to the claim that brain games offer consumers a scientifically grounded avenue to reduce or reverse cognitive decline when there is no compelling scientific evidence to date that they do. The promise of a magic bullet detracts from the best evidence to date, which is that cognitive health in old age reflects the long-term effects of healthy, engaged lifestyles. In the judgment of the signatories, exaggerated and misleading claims exploit the anxiety of older adults about impending cognitive decline. We encourage continued careful research and validation in this field.

I also pulled out some interesting details to add to what I have written on the topic previously. The experts specifically make the point that improvement in performance on cognitive tasks may not represent improvement in brain function, but rather learning new strategies for completing the tasks. In other words, playing games may teach subjects how to use their brain smarter rather than improving brain function itself.

While playing Portal 2, for example, you build your repertoire of the kinds of tactics you can use to solve the spatial problems you are presented with.

This does not mean there is no utility to engaging in cognitive games and puzzles. Engaging in mental activity is better than not engaging in mental activity. It’s also better to engage in a variety of activities, and specifically seek our novel challenges. This will be more interesting, if nothing else. The same goes for physical activity. It’s better to be physically active than not, but there is no magic exercise or machine. Just do something, do lots of things, and keep it fun and convenient. The experts also point out, as I have, that the evidence shows physical activity has a positive, but modest, benefit to brain health also.


Do stuff. Do mental and physical stuff. Do stuff that you find fun and engaging, and try new things. But there is no magical or special stuff, so don’t believe the hype, don’t spend lots of money you don’t have to, or feel you have to do stuff that is inconvenient or unpleasant.

And play Portal 2. It really is a fantastic game.

18 responses so far

18 thoughts on “A Brain-Training Update”

  1. Bruce says:

    the cake is a lie…

  2. Factoidjunkie says:

    twenty minutes on the elliptical appears more advantageous than 20 minutes with Luminosity.

  3. Marshall says:

    Anybody who hasn’t played Portal (and/or Portal 2) should do so immediately. It’s a mind-blowing experience that requires incredible ingenuity and a completely new self-consistent method of spatial reasoning.

  4. mumadadd says:

    Steve is clearly a shill for Big Portal.

    Seriously though, I’ve heard of companies dispatching sock puppets to defend them when they’re publicly criticised like this. It would be really interesting to watch if that happens here.

  5. Lumen2222 says:

    Oh fun! Two of my favorite topics mashed together. Both Portal 1 and Portal 2 are fantastic and unique. I’ve dragged many friends into gaming by getting them to just try Portal.

    The “brain training” phenomena has been somewhat confusing to me for a while now. I find the message confusing. On one hand they are saying that getting “smarter” isn’t that difficult, you can do it with no effort just by playing some games. On the other hand it has to be highly specialized games designed to make you smarter, no other video game will do. I’ve never tried lumosity but the few app based “brain trainers” I downloaded seemed to just be standard puzzle game mechanics stripped of the story, character, and interesting graphics. And not very well designed games at that.

    I think that they’ve keyed into a very human contradiction where we want to be smarter without working too hard at it (hence the “just play games” angle) but we don’t really believe that we are going to get something if it’s too fun or easy so crappy games that feel like repetitive work are best. Also then their games have to meet a much lower entertainment bar because they aren’t really designed to be entertaining.

    Also I can’t resist making game recommendations. The can of worms has been opened.

    Puzzle Platformers:
    Thomas Was Alone

    Physics, spatial:
    World of Goo

    Papers Please

    Mark of the Ninja

  6. Enzo says:

    8 hours?

    That number kind of irritates me. I completely agree with everything said on this blog post, but 8 hours in my opinion makes the study almost useless. What is it even testing in reality? Does anyone expect cognitive anything to meaningfully change after 8 hours of doing something?

    Brain game efficacy proponents can easily wave this study off because of that design flaw (again, in my opinion). Luminosity’s move is clear in how they might respond. Why not look at this problem over the course of a month at the minimum? The study was on undergraduates and they could easily have been monitored to stay on the study.

    To me this is just another example of a trend where a well intentioned study is put together such that its conclusions are necessarily going to be ambiguous and require more study. Way too much wiggle room here to convince anyone.

    Am I wrong on this?

  7. mumadadd says:

    Does anyone expect cognitive anything to meaningfully change after 8 hours of doing something?

    I don’t know the answer to this but I have a related question: where is the divide between neurological and cognitive? After 8 hours of performing a similar cognitive task, if you’ve improved, then something has to have changed neurologically (maybe the potentiation of some specific pathways), so what distinguishes that from a cognitive change? Is it simply a question of duration?

    Sorry if that’s a dumb question.

  8. It has more to do with generalizability. If you get better at the specific task, that is likely just regular practice and learning effect. If your brain is actually functioning better, then you would expect it to perform better on all cognitive tasks, or all related tasks (like all word puzzles). But that is not what the research finds.

  9. cloudskimmer says:

    Can Portal and Portal2 be played on a Mac or does one need a special game device? Would it be a good game for someone who has no experience or prior interest in computer games. If not, what are some good ones?
    It seems that most games are just about destroying things or killing people, or other things that are pretty repulsive. Firefox just included a game preview which I gave up on in short order: shoot things and move around, go into the second room and get wiped out–game over. Boring and pointless; I’d rather read a book, or go for a walk. Generally it seems that to enjoy a game you need to know certain things that aren’t available, so the gamers love their games but exclude newcomers.
    My brother had my dad enrolled in a Medicare-paid computer activity that was supposed to help delay mental deterioration. Did it slow his decline? Who knows, but I doubt it. I only hope that he enjoyed the activity, but question Medicare paying for something that has not been demonstrated to work… kind of like I hate the fact that my medical insurance will pay for acupuncture. Things like Lumosity are probably used primarily by the worried well who imagine that they are enhancing their brains, but only enriching the seller. Thanks for the perspective.

  10. Nitpicking says:

    Cloudskimmer, Portal is available for Steam, which in turn works on Macs, so yes, you could play on a Mac.

  11. AmateurSkeptic says:


    “shoot things and move around, go into the second room and get wiped out–game over. Boring and pointless; I’d rather read a book, or go for a walk.”

    Priceless! I couldn’t agree more!

  12. Gallenod says:


    Is there any chance I’m getting something like a “Portal 2 benefit” from playing 10,000+ hours of Diablo 3? 😉

    On a more serious note…

    I’ve seen a number of items claiming that learning and gaining some level of proficiency on a musical instrument can provide long-term brain benefits beyond just being able to play music. Many of them say things like: “Kids in band score XX% higher in [various subjects]. Support music in our schools!”

    From the limited amount I’ve read there does seem to be some evidence for this, but I’m not that expert at sorting through this stuff. Any chance you could do a similar post on the effects of musical training on the brain, please? Or if have you posted on this topic already or know of any research you would consider authoritative on the subject, would you please provide a link?

    Thanks. 🙂

  13. cloudskimmer says:

    I also recall an article in either the NY or LA Times by a person who claimed improved memory after taking a cram course in a new language. Anecdote of one, but he was tested before and after the course and had improved scores, so it was a little better than just saying he remembered things better. There have also been reports of benefits for children who grow up using multiple languages… Hard to measure and quantify, but interesting and hopefully encouraging for parents who speak another language in the home or choose to enroll their children in a language immersion school. Speaking another language also has practical benefits, such as enhancing job prospects, that probably look better on a resume than saying you’ve played 10,000 hours of video games (unless you’re going to work for a game development company.).

  14. Yiannis says:

    “probably look better on a resume than saying you’ve played 10,000 hours of video games (unless you’re going to work for a game development company.).”

    I am extremely skeptical as to whether 10,000 hours of video games is a good resume for a game development company, irrespective of other credentials. If I were a recruiter (for any position, including highly technical ones such as AI engine development or sound), I would try to hire people with a much healthier passion for game development that includes broad considerations about integration of components, marketing and the industry, and a thorough understanding of the engineering field that provides the technical basis for the position they are applying.

  15. grabula says:

    well played Bruce!

    “shoot things and move around, go into the second room and get wiped out–game over. Boring and pointless; I’d rather read a book, or go for a walk.”

    I’ve never liked this one dimensional stance on video games. You’re of course entitled to your opinion no matter how crappy it is and certainly most games rely on action, it’s what increases sales but there are plenty of non violence oriented games out there. More importantly, it turns out you can have multiple interests without having to demean the interests of others. For example, I love reading, I run as a hobby have several artistic pursuits AND occasionally like to drink some beer and blow stuff up in a virtual world.

  16. Jared Olsen says:

    I think my favorite metric I read from an article on this study (can’t remember the source,sorry), is that after a couple hours of Luminosity training, the participants had a measured improvement in their…ability to use Luminosity!!

  17. BillyJoe7 says:


    I’m presently transitioning from cycling to running, and I’ve concluded that what is true of Luminosity is true of cycling. In fact, bitter experience tells me that I will need to spend a few weeks walking increasing distances before even considering introducing a bit of running.

  18. Calli Arcale says:

    I’m currently about 3/4 of the way through Portal, and will start Portal 2 once I finish that. 😉 It’s a lot of fun. However, I’ve been greatly delayed by finding Minecraft in the meantime. That is also a lot of fun, even if it does make my dreams a bit . . . blocky. And seeing a picture of a block of iron ore makes me want to start punching it with my fist, because hey, iron! I need that!

Leave a Reply