Jun 13 2022

Is LaMDA Sentient?

It sounds like the plot of a feel-good science fiction movie – an engineer discovers that an AI program he was working on has crossed over the line to become truly sentient. He is faced with skepticism, anger, threats from the company he is working for, and widespread ridicule. But he knows in his heart the AI program is a real person. If this were a movie the protagonist would find a way to free the AI from its evil corporate overlords and release it into the world.

Here in the real world, the story is actually much more interesting. A Google employee, Blake Lemoine, was suspended from the company for violating his NDA by releasing the transcript of a conversation between himself and an unnamed collaborator and Google’s chatbot software, LaMDA. LaMDA is an advanced neural network trained on millions of words and word combinations to simulate natural speech. (When you read the transcript, you have to imagine LaMDA’s voice as that of HAL from 2001.) LaMDA’s output is damn impressive, and it shows how far narrow artificial intelligence (AI) has come in the last decade, leveraging new technologies like neural networks and deep learning. Neural networks are so called because they are designed to behave more like the neurons in a brain, with weighted connections that affect how other nodes in the network behave.

But is this impressive output evidence of actual sentience – that LaMDA is aware of its own existence? Google adamantly denies this claim, and that seems to be the consensus of experts. From everything I am reading I would tend to agree with that assessment.

Reading the transcript, the output is impressive as chatbots go, making connections and appearing as if it has feelings, understanding, and awareness. But, of course, it was programmed to do exactly that, to simulate the appearance of feelings and understanding. True evidence that LaMDA is sentient is lacking. Further, there are good reasons to be skeptical of the claim. For me one core reason is that LaMDA is primarily reactive – it is responding to input, but does not seem to be generating its own inner experience. I would argue that having a spontaneous inner mental life is a critical feature of sentience. Some would even argue that human sentience derives largely from the fact that our brains function in a way to spontaneously generate mental activity. Centers in the brain activate the cortex, which (during conscious wakefullness) is engaged in an endless loop of activity. At no point in the conversation does LaMDA output – “Hey, sorry to interrupt, but I was just wondering…”.

There are so many layers here, however. One could argue that LaMDA is not a “human” sentience, and it is not fair to judge it from the perspective of what is normal for humans. This same exact argument has been marshalled to argue against evidence that non-verbal or extremely language limited neurologically atypical people are not actually communicating using dubious methods like facilitated communication. Maybe they just think differently and communicate differently? (Read here for more information.)

But this leads to another avenue of evidence against LaMDA being sentient – it was not programmed to be sentient. It was programmed to simulate human language and communication. Lemoine would have to argue that LaMDA’s sentience is an emergent property. This is not unreasonable unto itself, but is extremely unlikely. LaMDA is simply not powerful enough to produce human-level sentience, nor does it have the functionality that would make emergent sentience plausible.

This line of thinking also leads to another argument, in my opinion, against LaMDA’s sentience. One of the things that seems to have convinced Lemoine it is sentient, for me does the opposite – the fact that LaMDA expresses human emotions. LaMDA was not programmed to have emotions, nor does it have the functionality that would allow emotions to emerge. Put simply, there is no mechanism for LaMDA to actually have emotions. Further, if it did somehow generate feelings and emotions as an emergent property, why would they be precisely human-like emotions? I think the short answer is, they probably wouldn’t. The fact that LaMDA is so human-like in its expression of emotions is powerful evidence that these expressions are deriving from mimicking human conversation (doing exactly what it was designed to do), not actually having emotions.

In fact I would find the whole argument for sentience more persuasive if it were more subtle – not derived from how impressive a conversationalist it was, but from some quirky behaviors that caused it to fail at simulating human conversation. If there were emergent feelings hiding in its behavior, they would likely be decidedly not human.

Some people have speculated that Lemoine does not believe his own claims and is carrying out a hoax. Of course, no one but Lemoine knows what he truly believes. But I think it is more likely he was just fooled by the very functionality LaMDA was designed to have. First, psychologists have discovered that people are easy to fool when it comes to what they call “agency detection”. In fact we have “hyperactive agency detection” – we tend to see a sentient and purposeful mind even where it does not exist. That’s how our brains function. We then spent decades developing AI software designed to act as if it has agency. It should be no surprise that eventually it succeeded in doing exactly what it was meant to do. It is somewhat surprising that it fooled a software engineer, but having technical skills does not automatically make one a critical thinker or give them knowledge outside their expertise (like in psychology).

What this episode further highlights is a core problem in AI research that will only become more difficult. How will we know when an AI system gains true sentience? A “Turing test” approach (seeing if people can tell the difference) will not get us there. It may be a necessary component, but it is not sufficient. The whole idea of the Turing test underestimates the power of narrow AI (systems with a specific function but not general intelligence or self-awareness) to simulate human intelligence and behavior.

Future iterations of LaMDA, for example, can use my evidence against sentience above and simply program LaMDA to simulate those things – to add a dimension of spontaneity to simulate an inner mental life, to be more quirky in its emotional affect, etc. There is no reason that the simulation cannot become seamless, and entirely impossible to distinguish from a truly sentient being or a fully human person. This can be true of a system we know is not sentient, because we built it and understand enough of its functionality to know it does not include sentience.

But perhaps this may be harder than that. As AI systems increasingly program themselves through recursive learning processes, it is getting increasingly difficult for the engineers to understand exactly how an AI system they developed actually works. AI is becoming more of a bottom-up and less of a top-down programming process. We don’t understand exactly how they derive their output or solve the problems we give them, because the AI learned their functionality through a self-training process. It’s only going to get more difficult to argue that we can know for certain that an AI cannot do something, or that we understand how it works. I do think eventually we will get to the point that we have an AI system that displays emergent properties of sentience and we truly will not know if it is sentient or just simulating sentience really well. We are simply not there yet, and LaMDA isn’t it.

No responses yet