Attack of the Uncanny Valley

Roboticist Masahiro Mori conceived of the “uncanny valley” in his 1970 essay of the same name. Mori was originally concerned with the development of human-like robots, but his theory has since been expanded to explain the creepiness of everything from wax sculptures to computer generated imagery – anything that looks human, but not quite.

For the past five years, Stephanie Lay has been exploring the uncanny valley as part of her PhD research for Open University – recording her findings on her blog Almost Too Human and Lifelike For Comfort. Some consider Mori’s uncanny valley to be a kind of vague pseudoscience rather than a proven scientific theory. Even Lay is not quite sure what to make of it.

“In the early part of my studies,” says Lay. “I spent a while thinking about whether I should be using the term ‘uncanny valley theory’ or ‘uncanny valley effect’ or should it be ‘idea?’ ‘Hypothesis?’ ‘Concept?'”

In spite of her uncertainty, Lay has yet to come across anyone who is immune to the effect, and when it come to uncanny faces people can detect minute flaws. In an entry on her blog, Lay discussed an “uncanny” image from the video game Assassin’s Creed: “One respondent to the original post noticed that part of the wrongness of the image came from the fact that her eyes were actually mirrored, giving unrealistic reflective points in her eyes.”

Artists regard these flaws as challenges to overcome, and researchers like Lay may play a role in helping them navigate beyond the uncanny valley. But is this territory we should be crossing?

During the 2000 Super Bowl, a company called Nuveen Investments aired a TV ad featuring the famous actor and quadriplegic Christopher Reeve. In the commercial, we see Reeve rise from his wheelchair and walk to the center of an amphitheatre, seemingly cured of his paralysis. The commercial was obviously computer-generated; there was something not quite right about the way Reeve moved and the look of his head during that close-up. And yet, after it aired, the National Spinal Cord Injury Association was inundated with calls from viewers who wanted to know how Reeve was cured. In spite of the “Uncanny Valley” effect of the computer animation, the television audience, fooled by the technology – or perhaps seeing what they wanted to see – believed Reeve had been cured.

Professor Philip Brey, Ph.D., is chair of the philosophy department at the University of Twente in the Netherlands. Brey is concerned with questions regarding the ethical implications of computing and information technology. And he has some concerns about how truly photorealistic CGI will be used in the future, especially in interactive entertainment.

“A potentially troubling development would be the use of photorealistic CGI characters that depict real persons without their consent,” says Phillip Brey.

There are laws that are supposed to prevent the unauthorized use of a person’s likeness, but, sometimes, determining ownership can be a muddy affair. And with the arrival of new technologies like photo-realistic CGI, the lines are even less clear.

Consider the current class-action lawsuit against the NCAA and EA Sports for the unauthorized use of player likenesses in video games. Students sign over the rights to their likeness to the NCAA while playing college-level sports, but, long after they graduate, the NCAA continues to use their likenesses in lucrative merchandising deals like video game licensing. This wasn’t a problem twenty-years ago when video game characters were barely-indistinguishable clumps of pixels on a screen, but today, companies like EA Sports pride themselves on being able to push the boundaries of video game photorealism. In doing so, they are also pushing the boundaries of ownership and legality.

“A case could be made that such uses should not be restricted because games are fictional works and the depiction of real persons in games is simply a matter of free speech,” says Brey. “On the other hand, such depictions can be very defamatory and invasive of the privacy of persons. Imagine that a photorealistic copy of you is found performing disgusting or embarrassing actions. I think you would feel violated and would worry that your character is being assassinated.”

One doesn’t have to simply imagine such things. There have already been instances of celebrity faces (sometimes convincingly) inserted into pornographic images by mischievous computer nerds and the media is frequently caught “adjusting” images to meet editorial objectives.

For a story on ABC News journalist Elizabeth Vargas, Marie Claire magazine featured a photo of Vargas breastfeeding in-studio at her newsdesk. It was a composite image meant to illustrate the balance between work and parenting. But the photo was convincing enough; one could be forgiven for wondering if Vargas really takes her baby to work.

When tennis player Andy Roddick was featured on the cover of Men’s Fitness, his biceps were digitally enlarged. When Roddick saw the magazine at a newsstand he did a double take. “I’m not as fit as the Men’s Fitness cover suggests,” said Roddick. “… little did I know I have 22 inch guns and a disappearing birth mark on my right arm.” A spokesperson for the magazine replied, “I don’t see what the big issue is here.”

And in 2006, an Ohio Republican Party newsletter featured a picture of Al Franken dressed like a bunny in adult diapers while clutching an oversized teddy bear. In reality, the photo had been doctored by transplanting Franken’s face into a scene in which he never participated. The deception was so convincing, years later (July 2009), well after Franken had been seated in the United States Senate, the image surfaced in the Cincinnati Enquirer. In a caption next to the forged image, columnist Peter Bronson asked: “Is this who you want making decisions about your health care?”

The advent of trick photography and digital photo-editing has changed the veracity of photographs and forced people to become more skeptical of images. But photo-editing technology can only manipulate still images that have already been captured from real life.

No such limitations exist with CGI. Images can be conjured-up out of thin air.

As CGI becomes increasingly realistic, the distinction between fantasy and reality will only blur further. Video games like The Sims already allow players to create their own virtual worlds populated with simulated friends, family and even pets.

“Increasing realism could expand that,” says Melanie Beisswenger, assistant professor of digital animation at Nanyang Technological University. “[CGI] technology could blur the line between real humans and virtual humans … real life and virtual life.”

“There is only a finite amount of time and resources that people can invest into their surroundings,” says Brey. “As people invest more of their lives in virtual worlds, they have [fewer] investments to make into the real world. Virtual worlds are often more attractive, more exciting and more controllable than the real world. This may cause people to lose themselves in them, to such an extent that they start neglecting their ‘real’ life. They may even emotionally invest in affectionate relationships with CGI characters at the expense of such relationships with real people.”

It’s not hard to imagine such a world when there are already people emotionally invested in inanimate objects. The New York Times recently ran a story about a Japanese sub-culture in which men use the 2-D images of anime characters as surrogates for flesh and blood human beings. “As long as you train your imagination,” says Toru Honda, a 2-D lover featured in the Times piece, “a 2-D relationship is much more passionate than a 3-D one.”

Long after Nuveen’s Christopher Reeve Super Bowl commercial aired, pundits and columnists admonished the use of photorealistic CGI in depicting fantasy as reality. It seemed almost cruel that that Nuveen had gone through so much trouble to convincingly depict something that doesn’t exist when so many paraplegics are waiting for a very real cure. If the voice-over hadn’t established the hypothetical premise of the ad, many more people would have been fooled.

Then again, isn’t fooling people the point? Don’t we want to be fooled? After all, achieving computer-generated photorealism appears to be the driving force behind the computer graphics industry. According to Tim Sweeney, founder of Epic Games, photorealistic CGI is no more than 10-15 years away. That may be true for producing still images, but creating interactive, photorealistic CGI characters is a different matter.

“Some CGI characters are already quite realistic,” says Brey. “However, in spite of the impressive advances in realism over the past thirty years, I believe it will still take a long time before CGI characters will be truly indistinguishable from real humans. This would not only require further improvements in computer graphics and physical modeling, but also major advances in artificial intelligence that would enable the modeling of highly realistic behavior by CGI characters.”

Depictions of real people may present even bigger challenges.

“A perfect posthumous computer generated version of a famous actor would probably be eerier than the exact match to someone living but unknown,” says Stephanie Lay. “Our knowledge and expectations of what the famous figure should look like, how they move and what they sound like, would drive whether or not we’re able to accept the facsimile.”

For now, Lay’s research is focused on finding that “something” that makes uncanny faces so unsettling; the missing link that creates the disconnect between fantasy and reality. But are we ready to cross the uncanny valley?

Christopher Reeve couldn’t really walk. Al Franken didn’t really pose as a diaper-wearing bunny. These were lies. So, naturally, we were unsettled by their existence. But who is to say whether or not a person’s love for an anime character is real?

And that’s where the trouble of photorealism begins. If that Super Bowl ad had offered nothing more than the sight of Christopher Reeve walking; or if that Al Franken photo had been a perfect forgery, all we’d have to discern fantasy from reality is our feelings. And when we’re at the sole mercy of our feelings, we’re liable to believe in anything.

Haasim Mahanaim is a freelance writer based in Ottawa, Canada. He also keeps this blog.