Feb 23 2016

Identifying Real or Fake Images

CGI-fakeFor anyone active on social media it is almost a daily occurrence that a photo being passed around as if it were real is revealed as a fake. In fact, if you don’t want to look silly, it’s a great idea to Google before you share. A basic search is often all that is necessary, and if the photo is fake it is very likely that Snopes has you covered.

For the more intrepid, you can also use reverse-photo search websites. These will find matches to the photo you select, which can often reveal the original photo that was “photoshopped” in order to create that iconic representation of whatever ideology is being promoted.

Some people have a better eye for photo manipulation than others. Sometimes context is all you need – if the photo seems too perfect to be true, it probably is.

The task of sniffing out fake photos, however, (at least from a technical perspective) is getting more difficult. There are two basic ways to make a fake photo. The most common is to take a real photo and manipulate it. Just replace the words on that protest sign to say whatever dumb thing you want to mock.

It is also possible to make an entirely computer generated (CG) image, or even video animation. As CG technology improves, detecting these fakes with the naked untrained eye is getting more difficult. Researchers at Dartmouth have been tracking just that, and they have recently released their most recent findings.

The researchers are focusing on CG images of human faces. This is the most challenging to create because of the uncanny valley. The human brain has a tremendous capacity to detect subtleties of the human face, probably because of our need to be very sensitive to the facial expressions of others, and to recognize individuals under a variety of conditions. In neurological terms, there is a large part of the cortex dedicated to processing visual information about human faces.

What CG animators discovered is that when artificial human images get closer and closer to realistic people tend to like them and relate to them more. However, when they get very close but not quite realistic, that affinity sharply drops (the uncanny valley) because the images start to look creepy. That effect goes away only when the images become almost indistinguishable from real faces.

This is why CG animated movies either focus on non-humans (robots, toys, aliens and insects) or they avoid the uncanny valley with cartoonish characters. The Polar Express is one notable exception – they went for realistic, and landed right in the middle of the uncanny valley.

As CG technology advances, however, it is slowly pushing through the uncanny valley, leading to realistic images and videos that are not always easy to tell apart from the real thing. Take a look at the photo at the top of this article – is it real or CG? Decide that before reading further.

The image is CG, but it’s pretty good. I actually think this was not too hard to tell, there is something wrong with the eyes (it’s always the eyes). It’s especially easy if you see the real and CG photos side-by-side (see bottom of post). It also might be more difficult if the photo is not of someone famous with which you are already familiar.

So how did the subjects do?

Observers correctly classified photographic images 92 percent of the time, but correctly classified computer-generated images only 60 percent of the time.

That means 40% of the time they mistook CG images for real images. That is almost a coin flip. At 50% subjects essentially cannot tell the difference when looking at a CGI image. It is interesting that when you are looking at a real image, you know it (at least 92% of the time).

The researchers then gave the 250 subjects some basic training in detecting CG images:

In a follow-up experiment, the researchers found that when a second set of observers was provided some training prior to the experiment, their accuracy on classifying photographic images fell slightly to 85 percent but their accuracy on computer-generated images jumped to 76 percent.

This was reported as an “increase in accuracy” but I don’t completely agree. It seems more like a shift toward calling everything CGI – an increase in true positives and false positives. They called more real photos CG and more fake photos CG.

This is actually not uncommon – when people have beginner training in detection, they tend to go through a phase where they are biased toward positive detecting. With more training they then tend to weed out the false positives better.

Perhaps more important than how the subject did on this study was how they compared to the same study from 5 years ago. The researchers report that subject did much better previously, and their ability to separate real from CG images is getting worse as CG technology improves (as one might expect).

The researchers discuss the real world implications of this fact, specifically as it relates to the regulation of child pornography in the US. In 1996 Congress passed a law making it illegal to own any explicit sexual representation of a minor. In 2002 the Supreme Court upheld the law, with the exception of completely CG images. They argued that since no real child was exploited, CG child pornography is protected free speech. In 2003 Congress responded by passing a new law making CG child pornography “obscene,” however in practice this does not have the same force as the law against real images.

So – it is now a matter of important legal concern whether or not an image or video depicting child pornography is CG or real.

I think it is a perfectly reasonable extrapolation of current trends, supported by this current research, to conclude that in the not-too-distant future it will become almost impossible for a human to tell the difference between a CG image and a real image. It will likely still be possible with computer analysis for quite some time, and it is interesting to speculate whether it will eventually become impossible even for computer analysis to tell the difference.

A defendant charged with possessing child pornography could always claim that they had good reason to believe the images were CG.

There are many more implications of this technology as well. The most obvious is – will studios forgo paying live actors once they can replace them with realistic CG characters? Of course this will happen, the question is, to what extent?

There are myriad legal and scientific implications as well. A photograph as evidence might eventually become worthless.




21 responses so far

21 thoughts on “Identifying Real or Fake Images”

  1. carbonUnit says:

    Imagine trying to unravel crimes and deal with conspiracies once a lot of pictures and video can’t be treated as evidence. “I was framed!” How reliable will news reporting be if the images can’t be trusted?

  2. mumadadd says:

    I recently found that the native photo viewer on my phone has the ability to splice faces from one photograph onto another. It will automatically recognize faces, and realign, resize and crop the spliced face to fit the new image, all in about 1.5 seconds. At first it was obvious that images had been doctored (though still piss funny), but then I realised it was mismatched lighting and focus that was behind this — if you photograph two people together, in the same light, then splice, it’s impossible to spot the fakery.

    Now I’ve found an app that can apply new hair and make up, and one that can splice faces in live video recordings. You can actual watch yourself while you are filming in selfie mode, with somebody else’s face perfectly superimposed over yours.

    Friggin insane. I had no idea this sort of tech was so widely and cheaply available.

  3. mumadadd says:

    The cg image looks real, but I didn’t spot that it was Russell Crowe, I just dismissed it as somebody who happened to look a bit like him.

  4. mumadadd says:

    …whereas the real image of Russell Crowe is more obviously Russell Crowe.

  5. mumadadd – I agree. I think it is easier to tell that the picture is not quite Russell Crowe than to tell that it is CG (although I think I can tell this one is CG). It seems it would be harder if the image is of someone unknown to you and not famous.

  6. petrossa says:

    so now that eyewitness accounts of perpetrators are to say the least ‘unreliable’ even photographic evidence is just as wonky. Coupled with the understanding that who to blame for a deed, the ‘consciousness’ which only interprets data extremely manipulated by diverse neural networks and gets to answer for it, or the real perpetrator.

  7. Fair Persuasion says:

    As of 2003, the CG pornography of children is classified as obscenity. So, if the family downloads the images, and pays for their computer service, the children of the purchasing family are at risk. The possession of material remains available to the entire household. This is not in the best interests of the family’s children. Most likely the viewers have a combination of real and CG downloads. The real child pornography uses the missing children from all over the world to supply the human acts. You may be free to receive obscene materials, but do you think about the wrongs that you commit by possession of it?

  8. FP – My sense is the distinction the Supreme Court made is this: Actual child pornography is produced at the expense of great harm to a minor, who is incapable of consenting.

    Freedom of speech is limited mainly when it causes demonstrable harm (inciting violence, defamation, fraud, death threats,etc.). Freedom of speech is not limited by offensiveness.

    Parents are also generally given great deference in such areas. Parents might choose to be nudists, or think that shielding children from sex or profanity is misguided, etc.

    Congress passing a law making CG child pornography obscenity was essentially a political move. There may not be any tangible legal ramifications, and if there are and it is challenged I would not be surprised if the SC knocked it down.

    Otherwise you are essentially making a moral, not legal, argument.

  9. The Sparrow says:

    I wonder how many people would recognize the CG images as such if the subjects weren’t informed beforehand that some of the images were CG, and were just asked to look for general oddities in the pictures.

  10. BillyJoe7 says:

    I thought the CG image was real – Russell Crowe from “Master and Commander” – until I saw the real image.

  11. mumadadd says:

    I think the cg image looks more like Paul Hollywood.


  12. Fair Persuasion says:

    Steven Novella, If there exist moral arguments, then prosecutorial legal arguments follow immediately. The U.S. laws are based on the Judeo-Christian ethics of our times. Currently, states can and do prosecute the felon for the possession of child pornography and possession of obscene materials, real or character-generated.

  13. RAnt says:

    Avatar remains the only animated film that I had trouble picking out the CG from real photography. I knew the scene the CG at the top was trying for, but I knew it wasn’t the actor it was trying to be. As you say, it was the eyes that were wrong as well as some of the features in the cheeks.

  14. slm says:

    Two points: I thought the multiple sources of light on the CG image were a giveaway. Both the right and left jaw have a faint shine, which might be because of movie lighting, but seemed too much.

    Should possession of CG child porn be illegal? Dr. Novella makes the point that a great concern is the harm to children done in making child porn, which does not happen in CG. I worked for years in the Child Protective field as counsel for the services. One thing that was advanced against child porn was not that the existence of it in the house was itself a danger, but that people don’t download or keep caches of porn if the porn does not arouse them. I thought of pedophilia as like an orientation. If someone in the home has a strong preference for pedophilia, then the children in that house are at risk.

    I’m not doing that work right now, so have not followed studies that investigate if pedophilia is an orientation, or if possession of the porn collates to acting out. It seemed to in many cases, but often those cases were in no way borderline. The possessor of the porn frequently also had a criminal record for sexual offenses. In the abstract, should a single person who has no history of acting out be criminalized for having CG porn? Right now, in law, yes. I’m completely on the fence on this one, except if the possession of the porn strongly links to acting out. Then we are back into actual harm to actual children, with the porn being like a red flag. To me, the biggest problem in not criminalizing the possession is that the acting out would likely create significant harm to the victim. How far does the prevention style of argument take us?

  15. BillyJoe7 says:


    I agree. CG paedophilic porn is preferable to the real version because at least children are not directly harmed by it. However, the effect could be cancelled out if this led to increased availability of paedophilic porn which, in turn, led to increased acting out. I’ve no idea how this would play out, but I guess eventually it will be played out.

  16. So you’re saying that free speech should be limited if it can be demonstrated that it encourages acts which harm others (especially children)? That seems valid, but it also seems that you recognize how deep into the gray zone we are.

  17. tmac57 says:

    I thought that the top image was probably real but touched up in the way that images of famous people tend to be unless they are caught candidly. Most professional images of celebs or promotional images (like for a movie) found in the media are touched up to show them in their best light. So you could have a sort of hybrid cg/real image that is real enough, but still ‘fake’ as well.
    What other sort of criminal activity could be faked with CG that law enforcement might decide to prosecute? Probably only things with sexual content, I would guess. Using existing movie and gaming content as a guide, simulated murder (even of children) is OK, complete destruction of humanity OK, most forms of violence, every crime imaginable OK except those concerning a few sexual acts. Not saying it shouldn’t be that way, but interesting none the less.

  18. etatro says:

    Dan Savage discusses these things a lot on his podcast. So taking out the charged and immoral child porn aspect and generalize it. Say one is in a closed relationship and one partner has a fantasy involving acts that the other partner isn’t willing to engage in, or there are impossible things (like the fantasy of being eaten), they discuss alternative of engaging the imagination, fulfilling sexual desires through reading, writing, drawing, fantasizing, etc. So no one is hurt and the energy or desire is diverted to solo acts. I think there needs to be some study on behavior and fantasy or porn consumption. There’s probably a wide range of responses, varying by personality and predisposition, but the conventional wisdom that I know of is that more porn consumption correlates with fewer sexual encounters.

  19. etatro says:

    I thought the top image was CG, based on the texture of the clothes. It’s something I’ve paid attention to from gaming, and have kept an eye on over the years in CG movies, the way the light reflects seemed like the surface was too smooth and there weren’t the random imperfections that we see in images of real clothes. Like tmac, I think we’ve seen so many touched up images of people in the media, that issues like overly-smooth skin, and too-sharp cheekbones, or too-well-formed eyebrows, we chalk it up to retouching and assume that it was based on a actual photo.

  20. tmac57 says:

    etatro- You are right that more (and better) studies need to be done on the behavior effects of porn consumption as well as pseudo-porn consumption. There have actually been a lot done previously such as the widely challenged Meese Report done in the 1980’s and others, but the results are all over the map, just like gun violence, video games, movie and TV violence, etc.
    One study that I read about expected to see increased rates of sexual abuse and violence against women as the explosion of porn emerged, but found rates actually going down. The author felt like there must have been (and looked for ) other factors such as increased law enforcement and harsher laws along with educating the public on these issues. Sounds a bit like moving the goal posts or motivated reasoning, but who knows, since this type of issue is difficult to study properly.
    It would be nice to know the true correlation if any, so that we could address it appropriately with actual data as a society, rather than shoot from the hip (gut), or from our moral outrage, with an eye to punish that with which we find personally distasteful, rather than that which causes true harm.

  21. slm says:

    Dr. Novella,

    Oh yeah, way into the gray area here. Freedom of speech is not protected the same way, necessarily in other jurisdictions (Canada, England, etc) and you come across anti-hate laws restricting speech relating to incitement to acts of hatred. I believe US law draws the line with respect to speech that can cause harm directly to individuals, the classic being shouting fire for no reason in a crowded theater. The harm is the reason for the limit.

    In straight fantasy, there are few limits legally, except relating to depictions of sexual acts and again hate speech. Where that line is drawn seems to come from community standards, and is a moving target.

    The toughie as I see it is the child porn, given the theoretical risk to defenseless victims. It feels as if it should be treated differently, but getting the data is very hard. A major Canadian decision from 2001 (R v Sharpe, [2001] 1 S.C.R. 45, 2001 SCC 2, or there is a wiki page.) discussed where the line should be drawn and decided that there is an exception for self-created images of oneself and with the consent of anyone else in the image (so ability to consent comes in and would not be presumed to be possible for a young child to give), also for self-created writing or drawing kept exclusively for oneself and which were meant to be private. That was it. Outside those exceptions, possession of images of child porn were illegal. The court seems to have accepted that child porn of itself can lead to bad actions. I wish there was more data on that.

Leave a Reply