A recent study asked subjects to give their overall impression of other people based entirely on a photograph of their face. In one group the political ideology of the person in the photograph was disclosed (and was sometimes true and sometime not true), and in another group the political ideology was not disclosed. The question the researchers were asking is whether thinking you know the political ideology of someone in a photo affects your subjective impression of them. Unsurprisingly, it did. Photos that were labeled with the same political ideology (conservative vs liberal) were rated more likable, and this effect was stronger for subjects who have a higher sense of threat from those of the other political ideology.

This question is part of a broader question about the relationship between facial characteristics and personality and our perception of them. We all experience first impressions – we meet someone new and form an overall impression of them. Are they nice, mean, threatening? But if you get to actually know the person you may find that your initial impression had no bearing on reality. The underlying question is interesting. Are there actual facial differences that correlate with any aspect of personality? First, what’s the plausibility of this notion and possible causes, if any?

The most straightforward assumption is that there is a genetic predisposition for some basic behavior, like aggression, and that these same genes (or very nearby genes that are likely to sort together) also determine facial development. This notion is based on a certain amount of biological determinism, which itself is not a popular idea among biologists. The idea is not impossible. There are genetic syndromes that include both personality types and facial features, but these are extreme outliers. For most people the signal to noise ratio is likely too small to be significant.  The research bears this out – attempts at linking facial features with personality or criminality have largely failed, despite their popularity in the late 19th and early 20th centuries.

Is it plausible that environmental effects or one’s personality and behavior affect their face? We can change our facial expression with mood, and if someone has a persistent mood that will definitely affect how other people perceive them. What about our “resting face” – the default expression on our face when we are not experiencing any particular strong mood? This is a trickier question. There are clearly some cultural influences here, as anyone who lives in the Eastern US and has traveled along the North-South axis can attest. Really good actors can change their appearance through facial expressions and how they carry and move their face to create the impression of completely different personalities.

Many of the studies on facial characteristics and personality try to account for this by having subjects photographed with a neutral expression. This way permanent facial features can be isolated from transient mood-based facial expression.

There are several websites that allow you to test your own criminal detecting ability. You look at a bunch of pictures and try to determine who is a criminal and who isn’t. It’s basically random guessing. Of course, once you know who the criminal is, you can see it in their eyes. Is this, therefore, completely subjective?

There definitely appears to be a strong subjective component. In addition to the current study, other studies show that just by labeling a photograph, people change their subjective impressions of a photo. In this study, telling people the picture is of a scientist correlated with characterizing his eyes as kind and penetrating. But if the same picture is labeled as that of a criminal, those same yes become impudent and inhuman. This then contributes to the false sense that we can tell criminality in the face – a form of subjective validation combined with confirmation bias.

Evolutionarily speaking, why would this be the case? I can only speculate, but a few thoughts come to mind. First, it does make sense that there would be evolutionary pressure to be able to detect potential threats simply by looking at people. But likewise, there would be evolutionary pressure not to appear threatening. The end result is a wash, and in fact the pressure not to appear evil, unsympathetic, and threatening may be far stronger. Also, there likely is simply no strong correlation between the genes that control facial structure and personality traits. There features are all genetically complex in any case, and behavior is strongly influenced by environment as well. There therefore would likely not be an easily detectable facial feature that predicts criminality.

The science of “physiognomy” – trying to discern criminality or personality from facial features, was essentially a failure and was largely abandoned a century ago. However it is making somewhat of a comeback, partly fueled by facial recognition and AI technology. Facial recognition allows for more subtle and complex measurements of facial features, providing lots of data to train AIs to see if they can come up with some correlation. So far this has not produced any convincing evidence for physiognomy, but it has produced algorithms that seem to perpetuate stereotypes. This is a problem more generally for some AI algorithms which can simply carry forward biases and stereotypes fed to them in the data.

There is a lot of complexity here I don’t have time to go into in this post. The bottom line of a century of research, however, is that there is no reality to physiognomy and that facial perception is highly subjective. What we do seem good at is seeing patterns and then weaving those patterns into a narrative. Once we are told someone is menacing, we then associate their face with menacing characteristics. This makes it seem like we evolved more to remember than to predict.