Face Recognition in Humans and Computers

Image-Based Representations

Face recognition for both humans and machines begins with an image projected onto the retina or sampled by a camera. Early approaches to machine recognition operated by extracting and quantifying the discrete features of faces (e.g., eyes, nose). A fundamental shift in this strategy occurred in the early 1990s with the use of principal component analysis (PCA) applied to face images. PCA is a statistical analysis that derives feature vectors from a set of input stimuli—in this case, a set of faces. The first measures used were simply pixel values extracted from face images. Applied to a population of faces, PCA creates a representation of individual faces in terms of a set of global features derived from the statistics of the face set. These global features consist of images that can be combined linearly to construct individual faces. From a psychological perspective, the connection of the feature set to the face population analyzed defines the “experience” of the computational model. The global features act as face descriptors that can specify categorical (e.g., sex, race) and identity information about a face. The computational components of PCA, therefore, implement a psychologically grounded face[Page 348]space model of recognition. By this account, faces are represented as points in a space, with the axes of the space defining features. The computationally implemented face space accounts for important findings in human recognition. The other-race effect, for example, is the well-known advantage people have in recognizing faces of their own race over faces of other races. This occurs because the space optimally represents faces similar to those used to derive the feature set (i.e., own race faces), consequently constraining the representation of other-race faces to be less than optimal.

Morphable Models

An important computational innovation came from changing the PCA input from images to image-based representations that are aligned with an average or prototype face. This representation supports morphing between individual faces and is therefore able to create faces that have particular properties useful for experimental manipulations. The alignment of faces is carried out using a set of landmark feature points (e.g., corners of the eye, tip of the nose). The shape of the face is coded in terms of the deformation of its landmark features from the average face. The reflectance or pigmentation information is analyzed separately in an aligned (shape-free) image space. A PCA is applied independently to the shape and reflectance information from a set of faces. This representation supports morphing between faces both in the reflectance and shape spaces allowing for the creation of synthetic morphed faces at arbitrary points in the space. The approach has been extended to operate on laser scans of faces that measure the three-dimensional shape of a head directly along with its reflectance map.

...