I’ve been following AI (artificial intelligence) news very closely, including all the controversies and concerns. I tend to fall on the side of – AI is a powerful tool, we should continue to develop it and use it responsibly. We don’t need to panic, and highly restrictive laws are likely unnecessary and counterproductive. But there are legitimate concerns about the power of AI, especially in the “wrong” hands. I also think the greatest disruption to our lives might not come from cyberterrorists (although a legit concern) or AI run amok, but from marketing. Giving companies who see us only as customers the power to predict our every move gives me pause.

This AI news item falls into this latter category – the use of machine learning AI to predict which songs people will like. Seems innocuous, but I think it furthers a trend that has some serious downsides. This is what the researchers did:

Traditionally, song elements have been measured from large databases to identify the lyrical aspects of hits. We took a different methodological approach, measuring neurophysiologic responses to a set of songs provided by a streaming music service that identified hits and flops. We compared several statistical approaches to examine the predictive accuracy of each technique. A linear statistical model using two neural measures identified hits with 69% accuracy. Then, we created a synthetic set data and applied ensemble machine learning to capture inherent non-linearities in neural data. This model classified hit songs with 97% accuracy.

This kind of approach is called neuroforecasting – predicting people’s likes and dislikes based upon their brain activity and physiological responses (like a lie detector but for your reaction to music). First let me point out that this study used a synthetic set of data, and is therefore just a proof of concept – this approach can theoretically work. They need to test this in the real world, and see if it can predict hits, not just match the model to existing hits. But let’s assume it works, and the 97% accuracy hold up. What will this mean for the music and streaming industries?

The positive case, which is made strongly in the article, is that 24,000 new songs are released worldwide every day, or 168,000 per week. That is a massive number of songs. If you are a streaming service like Pandora, how do you decide what to stream? Probably most of these new songs are niche, in a foreign language, or obviously not good. But that probably still leaves thousands of new songs per week that pass through an initial filter. This is then winnowed down to 30 songs that Pandora will stream as their new songs of the week. The problem for Pandora is that they are not very good at predicting which songs will be hits.

If they and other streaming services, however, were near perfect in their ability to predict hits then they will do a better job of giving their listeners what they want. Artists who have created genuine hits won’t be overlooked. The whole system will be more efficient, and everybody wins. Further still, study author Paul Zak explains:

“If in the future wearable neuroscience technologies, like the ones we used for this study, become commonplace, the right entertainment could be sent to audiences based on their neurophysiology. Instead of being offered hundreds of choices, they might be given just two or three, making it easier and faster for them to choose music that they will enjoy,” Zak said.

This is definitely in Black Mirror territory. Imagine if, for example, you wear wireless electrodes on your temples (hardly noticeable) which connect to your smartphone. As you listen to music your neurological response is recorded and AI determines your response. This is used to then predict whether you will like new music. But of course this same technology can be used to predict all sorts of things about your preferences, not just music. This is a marketing bonanza. Perhaps someone will have to come up with some fun or genuinely useful application for this technology, so that everyone wants to use it. But the real purpose will be to garner massive marketing data.

This is just an extension of what is already happening – as we use our apps and visit websites, our data is being gathered in order to target ads at us. This is justified as just giving people what they are likely to want, and there is some utility to this. But it can also be massively annoying, and more than a little creepy at times.

I am also concerned about possible cultural and psychological effects (as many people have brought up before). The idea that all of our entertainment will be carefully curated to us specifically and then personally streamed to us as its appeal, but it also means that we lose something culturally. There is less of a shared cultural experience. We are all in our own bubbles of curated entertainment. This, of course, extends to information. We are already seeing the societal harm caused by different segments of society living in different information ecosystems, curated to push their ideological and emotional buttons to maximize engagement. We are increasingly living in mutually incomprehensible media bubbles.

Even for the individual, I worry that something is lost with this approach. Sure, I would like to efficiently discover new music (or movies, books, TV shows, etc) that I am likely to appreciate. But this can also be a security blanket. It comes at the cost of finding something surprising – something that no algorithm would have predicted you would like. Generally people like copies of things that they already like. We like the music that was popular during our youth, for example. Shows and movies often are nostalgia service. But I also want to be surprised.

Also, people’s likes and dislikes can slowly shift over time, but only if they are being pushed out of their comfort zone. Some things are an acquired taste – but to acquire it you will not like it at first. Art can grow on you. You can gain a deeper understanding and appreciation for art that may rub you the wrong way at first. But I see the end stage of this neuroforecasting technology as having AI keeping us wrapped in warm and comfortable blankets of our existing tastes. It’s infantilizing in its own way, and stifles growth.

I am also concerned that trends like this may result in a slow death – it creeps up on us so slowly, especially generation after generation, that we don’t realize what we have lost. More likely, the older generation will sense what has been lost, but they will just seem like cranky old farts to a younger generation that has no clue. Jump ahead a few generations and what do we have. I very much don’t want to sound like that old fart, and I generally am a technophile willing to embrace new technology and new ways of doing things. But I think we do need to think carefully about an over-curated culture. Sometimes taking the pathway of least resistance does not get us to where we want to be.

On the other hand, I can also see a cultural backlash to these trends. The younger generation, for example, is having a bit of a love-affair with analog technology. Record players and vinyl are having a resurgence. Maybe it’s not too late to appreciate what can be lost in a highly digital life – the immediacy and visceral nature of a raw analog experience. Similarly, future generations may come to appreciate the benefits of experiencing entertainment that was not highly curated for them specifically, of the risk of not liking something, or the joys of discovering something surprising.