Acquiring Linear Subspaces for Face
Recognition under Variable Lighting
Kuang-Chih Lee, Student Member, IEEE, Jeffrey Ho, Member, IEEE, and
David Kriegman, Senior Member, IEEE
Abstract—Previous work has demonstrated that the image variation of many objects (human faces in particular) under variable
lighting can be effectively modeled by low-dimensional linear spaces, even when there are multiple light sources and shadowing. Basis
images spanning this space are usually obtained in one of three ways: A large set of images of the object under different lighting
conditions is acquired, and principal component analysis (PCA) is used to estimate a subspace. Alternatively, synthetic images are
rendered from a 3D model (perhaps reconstructed from images) under point sources and, again, PCA is used to estimate a subspace.
Finally, images rendered from a 3D model under diffuse lighting based on spherical harmonics are directly used as basis images. In
this paper, we show how to arrange physical lighting so that the acquired images of each object can be directly used as the basis
vectors of a low-dimensional linear space and that this subspace is close to those acquired by the other methods. More specifically,
there exist configurations of k point light source directions, with k typically ranging from 5 to 9, such that, by taking k images of an object
under these single sources, the resulting subspace is an effective representation for recognition under a wide range of lighting
conditions. Since the subspace is generated directly from real images, potentially complex and/or brittle intermediate steps such as
3D reconstruction can be completely avoided; nor is it necessary to acquire large numbers of training images or to physically construct
complex diffuse (harmonic) light fields. We validate the use of subspaces constructed in this fashion within the context of face
recognition.
Index Terms—Illumination subspaces, illumination cone, face recognition, harmonic images, harmonic subspaces, ambient lighting.
æ
1INTRODUCTION
T
O build a robust and efficient face recognition system,
the problem of lighting variation is one of the main
technical challenges facing system designers. In the past
few years, many appearance-based methods have been
proposed to handle this problem, and new theoretical
insights, as well as good recognition results, have been
reported [1], [2], [3], [5], [7], [9]. The main insight gained
from these results is that there are both empirical and
analytical justifications for using low-dimensional linear
subspaces to model image variations of human faces under
different lighting conditions. Early work showed that the
variability of images of a Lambertian surface in fixed pose,
but under variable lighting, where no surface point is
shadowed, is a three-dimensional linear subspace [9], [12],
[17], [22]. What has been perhaps more surprising is that,
even with cast and attached shadows, the set of images is
still well approximated by a relatively low-dimensional
subspace, albeit with a bit higher dimension [5].
Under the Lambertian assumption and accounting for
shadowing and multiple light sources, the set of images of
an object under all possible lighting conditions forms a
polyhedral cone, the illumination cone, in the image space
[3]. Therefore, the illumination cone contains all the image
variations of an object in fixed pose, and an accurate
representation of the cone would be a powerful tool for
recognizing objects across a wide range of illumination
variations. Indeed, successful work on applying this theory
to face recognition has been reported, e.g., [7]. For most
objects, the exact illumination cone is very difficult to
compute due to the large number of extreme rays that make
up their cones, e.g., for a convex, Lambertian surface, there
are Oðn
2
Þ extreme rays, where n is the number of pixels.
This complicates both quantitative and qualitative studies
of the illumination cone.
However, several recent results have indicated that,
although it provides a theoretical basis for discussions on
illumination problems, the computation of the full illumina-
tion cone may be unnecessary. Using a few “primary
images,” [22] proposes an analytical formula for computing
the covariance matrix that accounts for global illumination
effects. More recent ly, usi ng spherical harmonics and
techniques from signal-processing, Basri and Jacobs have
shown that for a convex Lambertian surface, its illumination
cone can be accurately approximated by a nine-dimensional
linear subspace that they called the harmonic plane [2], [14],
[15 ]. The major contribution of their work is to treat
Lambertian reflection as a convolution process between
two spherical harmonics representing the lighting condition
and the Lambertian kernel. By observing that the Lambertian
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 27, NO. 5, MAY 2005 1
. K.-C. Lee is with the Beckman Institute and Computer Science Department,
University of Illinois at Urbana-Champaign, Urbana, IL 61801.
E-mail: klee10@uiuc.edu.
. J. Ho is with Computer and Information Science and Engineering,
University of Florida at Gainesville, FL 32611. E-mail: jho@cise.ufl.edu.
. D. Kriegman is with the Computer Science and Engineering Department,
University of California at San Diego, La Jolla, CA 92093.
E-mail: kriegman@cs.ucsd.edu.
Manuscript received 30 Dec. 2002; revised 5 Feb. 2004; accepted 19 Oct. 2004;
published online 11 Mar. 2005.
Recommended for acceptance by A. Yuille.
For information on obtaining reprints of this article, please send e-mail to:
tpami@computer.org, and reference IEEECS Log Number 118062.
0162-8828/05/$20.00 ß 2005 IEEE Published by the IEEE Computer Society
