头部姿态估计文献_头部姿态估计资源-CSDN文库

共5个文件

pdf：4个

xml：1个

headpose

word

需积分: 24 200 浏览量 2018-12-21 20:53:36 上传评论收藏 13.11MB ZIP 举报

资源推荐

资源详情

资源评论

收起资源包目录

头部姿态估计相关文献.zip （5个子文件）

头部姿态估计相关文献

（2018）Fine-Grained Head Pose Estimation Without Keypoints.pdf 2.19MB

（2015）Face Alignment Assisted by Head Pose.pdf 5.49MB

(2017 Deepgaze)Head pose estimation in the wild using Convolutional Neural Networks and adaptive gradient methods.xml 3KB

（2018 QuatNet) Quaternion-based Head Pose Estimation.pdf 3.78MB

(2017 Deepgaze)Head pose estimation in the wild using Convolutional Neural Networks and adaptive gradient methods.pdf 2.72MB

YANG, MOU, ZHANG ET AL.: FACE ALIGNMENT ASSISTED BY HEAD POSE ESTIMATION 1

Face Alignment Assisted by Head Pose

Estimation

Heng Yang

heng.yang@cl.cam.ac.uk

Wenxuan Mou

w.mou@qmul.ac.uk

Yichi Zhang

yichizhang@fas.harvard.edu

Ioannis Patras

i.patras@qmul.ac.uk

Hatice Gunes

h.gunes@qmul.ac.uk

Peter Robinson

peter.robinson@cl.cam.ac.uk

Computer Laboratory

University of Cambridge

Cambridge, UK

School of EECS

Queen Mary University of London

London, UK

Faculty of Arts & Sciences

Harvard University

Cambridge, MA, US

Abstract

In this paper we propose supervised initialisation scheme for cascaded face alignment

based on explicit head pose estimation. We ﬁrst investigate the failure cases of most

state of the art face alignment approaches and observe that these failures often share

one common global property, i.e. the head pose variation is usually large. Inspired by

this, we propose a deep convolutional network model for reliable and accurate head pose

estimation. Instead of using a mean face shape, or randomly selected shapes for cascaded

face alignment initialisation, we propose two schemes for generating initialisation: the

ﬁrst one relies on projecting a mean 3D face shape (represented by 3D facial landmarks)

onto 2D image under the estimated head pose; the second one searches nearest neighbour

shapes from a training set according to head pose distance. By doing so, the initialisation

gets closer to the actual shape, which enhances the possibility of convergence and in

turn improves the face alignment performance. We demonstrate the proposed method on

the benchmark 300W dataset and show very competitive performance in both head pose

estimation and face alignment.

1 Introduction

Both head pose estimation and face alignment have been well studied in recent years given

their wide application in human computer interaction, avatar animation, and face recogni-

tion/veriﬁcation. These two problems are very correlated and putting them together will en-

able mutual beneﬁts. Head pose estimation from 2D images remains a challenging problem

due to the high diversity of face images [13, 18]. Recent methods [10] attempt to estimate

the head pose by using depth data. On the contrary, face alignment has made signiﬁcant

 2015. The copyright of this document resides with its authors.

It may be distributed unchanged freely in print or electronic forms.

arXiv:1507.03148v2 [cs.CV] 18 Jul 2015

2YANG, MOU, ZHANG ET AL.: FACE ALIGNMENT ASSISTED BY HEAD POSE ESTIMATION

Face detection

2D mean face shape

or random shape

Initialisation

Output

3D mean face shape

Head pose

Figure 1: Our proposed head pose based cascaded face alignment procedure (path in cyan

color) vs. conventional cascaded face alignment procedure (path in red color).

progress and several methods [2, 20, 30, 36] have reported good performance on images in

the wild. However, they also show some failures. When we look into their failures cases, we

ﬁnd that those samples share one signiﬁcant property, i.e., the head (face) in such images is

usually rotated from frontal pose in big angles.

The best performing face alignment methods proposed in recent years ([30], [2] and [36])

also share a similar cascaded pose regression framework, i.e., face alignment starts from a

raw shape (a vector representation of the landmark locations), and updates the shape in a

coarse to ﬁne manner. The methods in this framework are usually initialisation dependent.

Therefore, the ﬁnal output of one cascaded face alignment system might change if a different

initialisation is provided to the same input image. Moreover, each model has a convergence

radius, i.e., if the initialisation lies within the range of the actual shape, the model will be

able to output a reasonable alignment result, otherwise it might lead the shape to a wrong

location, as shown in Fig. 1. The methods like [2, 30] perform initialisation using a mean

shape within the face bounding box or from a randomly selected shape from training set.

There is no guarantee the initialisation lies within the convergence radius, especially when

head pose variation is large.

In this paper, we aim to address the above discussed problems and make cascaded face

alignment perform better under large head pose variations. The difference between our

proposed method and the conventional cascaded method procedure is illustrated in Fig. 1.

In contrast to using mean shape or random shapes for initialisation by other methods, our

proposed method aims to produce better initialisation schemes for cascaded face alignment

based on explicit head pose estimation. This is motivated by two facts: 1) most current meth-

ods fail on face images with large head pose variation-as we will demonstrate later; 2) most

recent face alignment methods work in a cascaded fashion and perform initialisation with

mean shape. More speciﬁcally, we ﬁrst estimate the head pose using a deep Convolutional

Network (ConvNet) directly from face image. Given the estimated head pose, we propose

two schemes of producing the initialisations. The ﬁrst scheme projects a canonical 3D face

shape under the estimated head pose to the detected face bounding box. The second scheme

searches shape(s) for initialisation from the training set by nearest neighbour method in the

head pose space. We build on our proposed scheme on the Robust Cascaded Pose Regression

(RCPR) to demonstrate the effectiveness of supervised initialisation. We note that the pro-

posed initialisation scheme can be naturally applied to any other cascaded face alignment.

In summary, we make the following contributions:

YANG, MOU, ZHANG ET AL.: FACE ALIGNMENT ASSISTED BY HEAD POSE ESTIMATION 3

• We investigate the failure cases of several state of the art face alignment approaches

and ﬁnd that the head pose variation is a common issue across those methods.

• Based on the above observation, we propose a ConvNet framework for explicit head

pose estimation. It is able to achieve an accuracy of 4

◦

absolute mean error of head

pose estimation for face images acquired in unconstrained environment.

• We propose two initialisation schemes based on reliable head pose estimation. They

enable face alignment method (RCPR) perform better and reduce large head pose fail-

ures by 50% when using only one initialisation.

To summarise, we propose better initialisation schemes based on explicit head pose estima-

tion for cascaded face alignment, to improve the performance, especially in the case of large

head pose variation.

2 Related Work

Face alignment has made considerable progress in the past years and a large number of meth-

ods have been proposed. There are two different sources of information typically used for

face alignment: face appearance (i.e., texture of the face image) and the shape information.

Based on how the spatial shape information is used, the methods are usually categorized

into local-based methods and holistic-based methods. The methods in the former category

usually rely on discriminative local detection and use explicit deformable shape models to

regularize the local outputs while the methods in the latter category directly regress the shape

(the representation of the facial landmarks) in a holistic way, i.e. the shape and appearance

are modelled together.

2.1 Local-based methods

Local based methods usually consist of two parts. One is for local facial feature detection,

which is also called local experts and the other is for spatial shape models. The former

describes how image around each facial landmark looks like in terms of local intensity or

color patterns while the latter describes how face shape, that is the relative location of the

face parts, varies. This captures variations such as wide forehead, narrow eyes, long nose

etc.

There are three types of local feature detection. (1) Classiﬁcation methods include Sup-

port Vector Machine (SVM) classiﬁer [4, 19] based on various image features such as Ga-

bor [28], SIFT [15, 30], HOG [31] and multichannel correlation ﬁlter responses [11]. (2)

Regression-based approaches are also widely used. For instance, Support Vector Regres-

sors (SVRs) are used in [16] with a probabilistic MRF-based shape model and Continuous

Conditional Neural Fields (CCNF) are used in [3]. (3) Voting-based approaches are also

introduced in recent years, including regression forests based voting methods [6, 8, 32] and

exemplar based voting methods [22, 23].

One typical shape model is the Constrained Local Model (CLM) [7]. The CLM steps can

be summarised as follows: ﬁrst, sample a region from the image around the current estimate

and project it into a reference frame; second, for each point, generate a “response image"

giving a cost for having the point at each pixel; third, searching for a combination of points

which optimises the total cost, by manipulating the statistical shape model parameters. The

4YANG, MOU, ZHANG ET AL.: FACE ALIGNMENT ASSISTED BY HEAD POSE ESTIMATION

methods built on CLM mainly differ from each other in terms of local experts, for instance

CCNF in [3] and the Discriminative Response Map Fitting (DRMF) in [1]. There are many

other local based methods either using CLM or other models such as RANSAC in [4], graph-

matching in [38], Gaussian Newton Deformable Part Model (GNDPM) [26] and mixture of

trees [39].

2.2 Holistic-based methods

Table 1: Holistic methods and their properties.

Methods SDM [30] RCPR [5] IFA [2] LBF [20] CFAN [36] TCDCN [37]

initialisation mean pose random mean pose mean pose supervised supervised

features SIFT pixel HOG pixel auto-encoder ConvNet feature

regressor linear regression random ferns linear regression random forests linear regression ConvNet

Holistic methods have gained high popularity in recent years and most of them work in

a cascaded way like SDM [30] and RCPR [5]. We list very recent holistic methods as well

as their properties in Table 1. The methods following the cascaded framework differ from

each other mainly in three aspects. First, how to set up the initial shape; Second, how to cal-

culate the shape-indexed features; Third, what type of regressor is applied at each iteration.

For initialisation, there are mainly three strategies are proposed in literature: random, mean

pose, and supervised. In order to make it less sensitive to initialisation, previous approaches

such as [5, 29] propose to run multiple different initialisations and pick the median of all

the predictions as the ﬁnal output. Each initialisation is treated independently way until the

output is calculated. However, such a strategy has several issues, ﬁrst the theoretical support

for selecting the median value is not well understood; second, there is no guidance on how

to choose the multiple initialisations; third, using multiple initialisations is computationally

expensive. A similar supervised initialisation scheme was proposed in [35] where the ini-

tialisation shapes were selected by using an additional regression forest model for sparse

facial landmarks estimation. A recent work [33] proposed a re-initialisation scheme based

on mirrorability to improve the face alignment performance.

3 Data preparation

In this section we describe how the data is prepared in order to support our further discussion.

More speciﬁcally, we discuss how we provide ground truth head pose and face bounding

boxes from different face detectors for the benchmark dataset.

We use face image data from the benchmark face alignment in the wild dataset, 300W

[21]. Since their testing samples are not publicly available, we follow the partition of recent

methods [20] to set up the experiments. More speciﬁcally, we use face images from AFW

[39], HELEN [25], LFPW [4] and iBug [21], which include 3148 training images and 689

test images in total. 3148 training images are from AFW (337 images), HELEN training set

(2000 images) and LFPW training set (811 images), and 689 test images are from HELEN

test set (330 images), LFPW test set (224 images) and iBug (135 images).

It is intractable to get the ground truth 3D head pose for face images collected in uncon-

strained conditions. In order to generate reasonable head pose (Pitch, Yaw and Roll) values,

we use the pose estimator provided by Supervised Descent Method (SDM) [30]. Note that,

when calculating the head pose, we feed the ground truth facial landmark locations instead of

评论收藏

内容反馈

南国1粒小红豆

粉丝: 31
资源: 8

头部姿态估计文献

Python-基于OpenCVdlib的实时头部姿态估计源码.zip

头部姿态估计

头部姿态估计数据集

基于深度学习的头部姿态估计.pdf

Fine-Grained Head Pose Estimation Without Keypoints

计算机视觉中头部姿态估计的研究综述Head Pose Estimation in Computer Vision: A Survey（中文）

基于Kinect的头部姿态估计+两篇文档说明

头部动作图像数据库 HeadPoseImageDatabase

Python-基于OpenCVdlib的实时头部姿态估计

用微软的kinect对头部姿态进行估计的论文

基于AAM和POSIT的三维头部姿态估计

头部姿态估计技术研究综述

计算机视觉中头部姿态估计的研究综述Head Pose Estimation in Computer Vision: A Survey（中文）word

头部姿态估计的经典总结性论文

多尺度卷积神经网络的头部姿态估计_梁令羽.pdf

HeadPoseEstimation:使用OpenCV解决PNP的头部姿势估计

HeadPose Estimation头部姿态估计头部朝向Android Demo

多尺度卷积神经网络的头部姿态估计

FacePose_pytorch:pytorch的头姿势估计（偏航，侧倾，俯仰）和情感检测的pytorch工具具有SOTA实时性能，易于部署，易于使用且精度高，一次解决了面部检测的所有问题。（极简，极快，高效是我们的共同）

SVR头部姿态估计

AFLW2000头部姿态估计数据集

基于深度图像的头部姿态估计

深度图像 头部姿态估计

PnP问题-头部姿态估计.zip

基于结构光三维重建的头部姿态估计算法

头部姿态估计和情绪检测

最新资源

深度图像头部姿态估计