May 25 2020

Can We See Personality?

Is someone’s basic personality type written on their face? This is an interesting question, that research has not definitively answered. A new study uses AI to add one more piece of information, suggesting that the answer is – maybe, sort of.

Let’s start with a technical definition of personality:

“Personality refers to individual differences in characteristic patterns of thinking, feeling and behaving.”

It is uncontroversial that different people have different personality traits, although there are different schemes for how to divide up all the different recognizable personality traits people might display. One of the more accepted schemes is OCEAN (the big five) – Openness, Conscientiousness, Extroversion, Agreeableness, and Neuroticism. This does not capture every aspect of one’s personality, nor the rich background of experience and culture that helps mold our behavior, but it does seem to capture something fundamental about how humans vary.

Far more controversial is whether or not there are different personality types, meaning a suite of personality traits that tend to go together. There are many tests based on the assumption that people can be sorted into a small number of different personality types, but none of them have established validity. The best evidence we have so far, in my opinion, does not support the notion of personality types in any meaningful way.

For personality traits the question remains – to what degree are they genetic vs environmental. It is clear that they are both, some probably having a stronger genetic component than others, with different contributions in different individuals, and with genetic predisposition interacting with development and the environment. It is easy to see how this tangled mess would be difficult to untangle into one clean simple scheme.

If, however, personality traits have a significant genetic contribution (which seems clear) then we can ask – to what extent do personality traits go along with physical characteristics? This is plausible because sometimes a single gene will code for a protein or enzyme that has multiple functions, and therefore could (for example) affect the development of both the brain and the face. Also, genes that are next to each other have a statistically higher chance of sorting together – so if you have one you are more likely to have the other. This means that it is theoretically possible that if you have one particular personality trait you may be more likely to also have some facial characteristic.

The arrow of cause and effect may also go the other way. People who look a certain way may have a particular life experience that contributes to certain personality traits. A baseline aloof expression may, for example, contribute to introversion. It may also be possible that personality can affect our resting facial expression (not emotional expression, but just how we carry our face at baseline). It also makes sense, being an extremely social species with a lot of brain tissue dedicated to seeing and interpreting human faces, that we evolved to detect not only emotion but personality in the faces of others.

Most of this is speculation, but if there is any real connection (whether directly causal or incidental) between personality traits and how faces look, the ultimate test is that we should be able to guess at higher than chance how someone will perform on a measure of personality based on a static picture of their face, showing no emotional expression. That is the premise of the current study.

The study was done in a sample of 12 thousand volunteers who completed a self-report questionnaire measuring personality traits based on the “Big Five” model and uploaded a total of 31 thousand ‘selfies’.

That is a pretty large data set. Of interest, they had to eliminate all pictures showing emotional expression and of cats. Seriously – they specifically mention they had to eliminate pictures of cats.

Next, an image classification neural network was trained to decompose each image into 128 invariant features, followed by a multi-layer perceptron that used image invariants to predict personality traits.

What they found was that the trained neural network AI was able to predict the relative personalty trait of two randomly chosen individuals based on photographs with a 58% accuracy, where chance guessing would result in 50%. So in other words, the AI had pictures of two individuals and had to guess which one was more agreeable, or more neurotic. These results are spectacularly unimpressive. The AI was essentially given the simplest version of estimating personality trait (a binary choice), and still only did slightly better than chance. Of note the AI did a little better with female than male faces, and also performed best on the conscientiousness personality type. These may be random flukes of the data, however, and would need to be replicated before taken seriously.

So what does this mean? Even though the results are slim, they are statistically robust. Also, there was some internal consistency in that different photos of the same person yielded the same results. If this study holds up under replication, this could mean that there is a real, but slight, correlation between physical faces and personality. The limited success may reflect underlying reality – perhaps there is only a 58% or so actual correlation between faces and personality. Or, the actual correlation may be higher but this AI system was not perfect. Again, future research might enlighten this question.

From my perspective, this does fit what we might predict, that there is some real correlation, but it’s weak.

The other question is – can this technology be put to some practical use? The press release reports:

There are a vast number of potential applications to be explored. The recognition of personality from real-life photos can complement the traditional approaches to personality assessment in situations where high speed and low cost are more important than high accuracy. Artificial intelligence can be used to propose products that are the best fit for the customer’s personality or to select the possible ‘best matches’ for individuals in dyadic interactions, such as customer service, dating or online tutoring.

Um, no. I don’t buy it at all. The correlation is too slight to be of any practical use as speculated, even if the accuracy can be pushed a little higher. Doing slightly better than a coin flip is of little practical value. Imagine, for example, a dating app using this approach. You have lots of data on prospective dates, their education, likes and dislikes, a self-description which is likely to be revealing in certain ways, their career, the photos they choose to show of themselves, etc. Added to this the app will tell you – that this person is 58% likely to be more agreeable than this one other person. There is no info about their absolute agreeableness, just that they might be more agreeable than one other person. But they might also be slightly more likely to be less open.

But to be fair, such an app could make thousands of such comparisons. It could then rank potential dates (or hires, or prospective tutors, etc) based on multiple comparisons in each of the five traits, perhaps giving each person a percentile for that trait based on how many other individuals they were better than. This kind of approach could be useful – if the accuracy were substantially greater than 58%. My fear is this will give the illusion of information, but that information will be highly unreliable. What we would need to see is the results of an app that use this approach, and then see if the results have any actual validity, meaning that they predict something useful. So we are a few steps away from using this technique in the real world, and I am not optimistic given the 58%. But I also predict that some company will use it, because the idea is very marketable, even if it doesn’t really work.


No responses yet