Comprehensive Database for Facial Expression Analysis
Takeo Kanade
The Robotics Institute
Carnegie Mellon University
Pittsburgh, PA, USA 15213
http://www.cs.cmu.edu/~face
Jeffrey F. Cohn
Department of Psychology
University of Pittsburgh
The Robotics Institute
Carnegie Mellon University
4015 O'Hara Street
Pittsburgh, PA, USA 15260
Yingli Tian
The Robotics Institute
Carnegie Mellon University
Pittsburgh, PA, USA 15213
Abstract
Within the past decade, significant effort has occurred in
developing methods of facial expression analysis.
Because most investigators have used relatively limited
data sets, the generalizability of these various methods
remains unknown. We describe the problem space for
facial expression analysis, which includes level of
description, transitions among expression, eliciting
conditions, reliability and validity of training and test
data, individual differences in subjects, head orientation
and scene complexity, image characteristics, and
relation to non-verbal behavior. We then present the
CMU-Pittsburgh AU-Coded Face Expression Image
Database, which currently includes 2105 digitized image
sequences from 182 adult subjects of varying ethnicity,
performing multiple tokens of most primary FACS action
units. This database is the most comprehensive test-bed
to date for comparative studies of facial expression
analysis.
1. Introduction
Within the past decade, significant effort has
occurred in developing methods of facial feature tracking
and analysis. Analysis includes both measurement of
facial motion and recognition of expression. Because
most investigators have used relatively limited data sets,
the generalizability of different approaches to facial
expression analysis remains unknown. With few
exceptions [10, 11], only relatively global facial
expressions (e.g., joy or anger) have been considered,
subjects have been few in number and homogeneous
with respect to age and ethnic background, and recording
conditions have been optimized. Approaches to facial
expression analysis that have been developed in this way
may transfer poorly to applications in which expressions,
subjects, contexts, or image properties are more variable.
In addition, no common data exist with which multiple
laboratories may conduct comparative tests of their
methods. In the absence of comparative tests on common
data, the relative strengths and weaknesses of different
approaches is difficult to determine. In the areas of face
and speech recognition, comparative tests have proven
valuable [e.g., 17], and similar benefits would likely
accrue in the study of facial expression analysis. A
large, representative test-bed is needed with which to
evaluate different approaches.
We first describe the problem space for facial
expression analysis. This space includes multiple
dimensions: level of description, temporal organization,
eliciting conditions, reliability of manually coded
expression, individual differences in subjects, head
orientation and scene complexity, image acquisition, and
relation to non-facial behavior. We note that most work
to date has been confined to a relatively restricted region
of this space. We then describe the characteristics of
databases that map onto this problem space, and evaluate
Phase 1 of the CMU-Pittsburgh AU-Coded Facial
Expression Database against these criteria. This
database provides a large, representative test-bed for
comparative studies of different approaches to facial
expression analysis.
2 Problem space for face expression
analysis
2.1 Level of description
Most of the current work in facial expression
analysis attempts to recognize a small set of prototypic
expressions. These prototypes occur relatively
