August 18, 2011
Computer Reads Facial Expressions

Computers will get more and more able to read us. Meanwhile, we won't be able to see what their electrons are up to.

Researchers have developed new computational tools that help computers determine whether faces fall into categories like attractive or threatening, according to a recent paper published in the journal PLoS ONE. Mario Rojas and other researchers at the Computer Vision Center in the Autonomous University of Barcelona in Spain, in cooperation with researchers from the Department of Psychology of Princeton University, developed software that is able to predict those traits in some cases with accuracies beyond 90%.

Imagine blind people (and even autistics) using a camera tied to a computer to let them know in real time (perhaps thru an ear jack) what facial expressions people around them are presenting. Or imagine cameras in airports, banks, and (especially) post offices scanning for someone about to go on some sort of rampage.

You can read the full paper.

State of the art machine learning methods are applied to a) derive a facial trait judgment model from training data and b) predict a facial trait value for any face. Furthermore, we address the issue of whether there are specific structural relations among facial points that predict perception of facial traits. Experimental results over a set of labeled data (9 different trait evaluations) and classification rules (4 rules) suggest that a) prediction of perception of facial traits is learnable by both holistic and structural approaches; b) the most reliable prediction of facial trait judgments is obtained by certain type of holistic descriptions of the face appearance; and c) for some traits such as attractiveness and extroversion, there are relationships between specific structural features and social perceptions.

I wonder whether any secret projects are underway to attempt to read facial expressions, eye dilation, pulse, voice intonation, and other indicators to detect lies. Governments would love that capability. Ditto companies. But it works the other way too. Is the boss sincere about a promotion if you work long hours for months and make some massive deadline? What's really going on?

Share |      Randall Parker, 2011 August 18 08:17 AM  AI Understanding Humans

Lou Pagnucco said at August 18, 2011 9:52 AM:

Well, the TSA and other police agencies would love this invasive technology, but would politicians and newscasters like to have lies outed by a computer that reliably analyzes their facial expressions, body movements and voice modulation? (I'd buy a TV with that built-in feature.)

This could also become a cell phone app, or maybe even accomplished with an inconspicuous mini-microphone/camera.
Perhaps it could combined with the facial identity recognition app, and also ultra-sensitive scent detector to identify drug use or disease. I believe all of these are all possible today - maybe already in use by governments?

The following paper is also relevant -
"Multimodal emotion recognition from expressive faces, body gestures and speech"

PacRim Jim said at August 18, 2011 4:39 PM:

Notice that it did not say "Computer Correctly Reads Facial Expressions.
I predict an avalanche of lawsuits.
Achtung, class action attorneys.

Randall Parker said at August 19, 2011 11:19 AM:


Yes, I saw it. But it doesn't fit into a credential system. We need ways to earn credentials more than we need more ways to watch a class. Make the credential system where one can just take tests and then the videos and classes to prepare for the credential test will pop up.

Post a comment
Name (not anon or anonymous):
Email Address:
Remember info?

Go Read More Posts On FuturePundit
Site Traffic Info
The contents of this site are copyright