Facial Expressions, Computational Perspectives

Facial Expressions: Background

With the publication of The Expression of Emotions in Man and Animals in 1872, Charles Darwin initiated the systematic, scientific study of facial expressions of emotion—on the encoding side. His work indicated that some emotions might have a universal facial expression and that animals, as well as humans, have a complex repertoire of emotional expressions. To understand decoding mechanisms, one should first consider to what extent expressions are a reliable, automatic manifestation of the internal state of the individual, which is still a debated issue. Facial expressions are often regarded as a veridical reflection of the signaler's emotional state, an approach followed, among others, by Paul Ekman. In his work, Ekman has identified a set of basic expressions that he considers to be universal across cultures and to reflect physiological changes in the signaler, identifiable with emotions. If this position is accepted, then relatively fixed, perhaps even genetically specified, decoding mechanisms would largely suffice. An alternative view includes facial expressions in the wider discussion about the honesty of signals in animal communication. Within this approach, expressions are regarded as strongly modulated by the social context in which they are produced, and they function to manipulate the behavioral responses and the emotional state of the receiver. Therefore, they do not necessarily transmit honest information: Sometimes they can be [Page 350]largely veridical, while at other times they can be quite deceptive. Much more flexible decoding mechanisms are then required, to be honed by individual experience through learning and memory. Whatever system is used to interpret fine details in the visual input, based on a meaning-driven representation of repertoires of facial expressions, this system has to be continuously updated.

Are Facial Emotions and Facial Identity Processed Separately?

The functional model proposed by Vicki Bruce and Andy Young in 1986, one of the most popular “box models” from that period, had a strong influence on the study of face recognition, in particular with respect to the localization of perceptual mechanisms in the brain. The authors put forward the notion that facial expressions and facial identity are processed along distinct pathways, which diverge early on in the visual stream. The system devoted to expression analysis receives inputs from an early stage where the fine structural properties of the visual stimulus are already extracted, whereas a separate route is followed to abstract an expression-invariant description of the face, which will lead to the attribution of identity and other semantic information. This seemingly reasonable functional distinction between processing facial identity and emotion has since been assigned an anatomical basis, with the identification of cortical regions that are differentially involved in the visual analysis of distinct aspects of faces. Static (or perhaps virtually invariant) features of faces, including identity, seem to be especially encoded by the lateral fusiform gyrus, whereas the superior temporal sulcus seems to be involved in the representation of dynamic, changeable properties of faces, which include emotions, eye gaze, and lip movements.

...