Author's personal copy
A local region based approach to lip tracking
Yiu-ming Cheung
a,
n
, Xin Liu
a
, Xinge You
b
a
Department of Computer Science, Hong Kong Baptist University, Hong Kong SAR, China
b
Department of Electronics and Information Engineering, Huazhong University of Science and Technology, Wuhan, China
article info
Article history:
Received 11 May 2011
Received in revised form
9 February 2012
Accepted 20 February 2012
Available online 5 March 2012
Keywords:
Lip tracking
Localized color active contour model
Semi-ellipse
Local region
Deformable model
abstract
Lip tracking has played a significant role in a lip reading system. In this paper, we present a local region
based approach to lip tracking, which consists of two phases: (i) lip contour extraction for the first lip
frame, and followed by (ii) lip tracking in the subsequent lip frames. Initially, we construct a loc alized
color active color model provided that the foreground and background regions around the object are
locally different in color space. In th e first phase, we find a combined semi-ellipse around the lip as the
initial evolving curve and compute the localized energies for curve evolution such that the lip image is
separated into lip and non-lip regions. Then, we utilize a 16-point deformable model (Wang et al., 2004
[20]) with geometric constraint to achieve lip contour extraction. In the second phase, we present a
dynamic selection of the radius of local regions associated with the extracted lip contour of the
previous frame to realize lip tracking. The proposed approach not only adapts to the lip movement, but
it is also robust against the appearance of teeth, tongue and black hole. Extensive experiments show the
efficiency of the proposed lip tracking algorithm in comparison with the existing methods.
& 2012 Elsevier Ltd. All rights reserved.
1. Introduction
Lip contour tracking (simply called lip tracking hereinafter)
has received wide attention in recent years because of its
potential applications in a variety of areas such as audio–visual
speech recognition (AVSR) [1], lip reading [2,3], facial expression
analysis [4], human computer interfaces [5] and so forth.
Although various visual tracking methods have been developed
in the literature, e.g., see [6], these methods are usually utilized to
track the object positions, which may not be suitable for deter-
mining the variations of the lip contours. In fact, it is a non-trivial
task to track the lip movements accurately due to its elastic shape
and non-rigid motion, the large variations caused by different
speakers, lighting conditions, low contrast between the lip and
skin, teeth or tongue effect, and so forth.
In the past years, a few techniques have been proposed
towards lip tracking with the focus on segmentation of lip regions
or extraction of lip contours, which can be roughly classified into
two categories: the edge-based approaches and the region-based
approaches. The former basically utilizes the low level spatial
cues such as edge and color information to track the lip move-
ment. For instance, Zhang et al. [7] applied hue and edge
information to achieve the mouth localization and segmentation.
Eveon et al. [8] detected six key points, through which the fitting
shapes connecting these points were obtained according to the
edge information and color cues. In general, these two techniques
work well under a desired environment, but their performances
may deteriorate if the lips are glossy or their exists image noise.
Moreover, Kass et al. [9], Delmas et al. [10] and Freedman et al. [5]
introduced the applications of active contour model (ACM, i.e.,
snake) to detect the edge of the lip boundary via gradient descent
technique. Unfortunately, this type of active contours often
converge to the wrong result when the lip edges are indistinct
or the lip is very similar to the skin region. Subsequently, Barnard
et al. [11] integrated the edge-based ACM with 2D pattern
matching technique to drive the energy minimizing spline onto
the expected lip contours. However, such a method just employs a
combination of two semi-elliptical shapes to model the lip shape,
which may not fit the actual lip boundary quite well.
In contrast, the region-based approaches mainly utilize the
regional statistic characteristics to realize lip tracking. Typical
examples include deformable template (DT) [12–14], region-
based ACM [15,16], active shape model (ASM) [17–19], and active
appearance model (AAM) [2]. The DT algorithm utilizes a regional
cost function to partition a lip image into the lip and non-lip
regions via a parametric template, which represents the lip shape
properly. The pioneering work introduced by Yuille [12] shows a
lip template specified by a set of parameters, and these para-
meters are altered via an energy minimizing process so that the
lip template can match the lip boundary gradually. Later, Liew
et al. [13] addressed a different lip template and extended Yuille’s
Contents lists available at SciVerse ScienceDirect
journal homepage: www.elsevier.com/locate/pr
Pattern Recognition
0031-3203/$ - see front matter & 2012 Elsevier Ltd. All rights reserved.
doi:10.1016/j.patcog.2012.02.024
n
Corresponding author. Tel.: þ852 34115155.
E-mail addresses: ymc@comp.hkbu.edu.hk (Y.-m. Cheung),
xliu@comp.hkbu.edu.hk (X. Liu), youxg@hust.edu.cn (X. You).
Pattern Recognition 45 (2012) 3336–3347
剩余11页未读，继续阅读
资源评论
