2019-3D面部重建-Joint 3D Face Reconstruction and Dense Face Alignmen
需积分: 0 38 浏览量
2022-08-03
13:24:37
上传
评论
收藏 1.96MB PDF 举报
Joint 3D Face Reconstruction and Dense Face Alignment from A Single Image
with 2D-Assisted Self-Supervised Learning
Xiaoguang Tu
1∗
, Jian Zhao
2,3
, Zihang Jiang
4
, Yao Luo
1
, Mei Xie
1
, Yang Zhao
3
, Linxiao He
5
, Zheng Ma
1
and Jiashi Feng
2
1
University of Electronic Science and Technology of China,
2
National University of Singapore,
3
National University of Defense Technology,
4
University of Science and Technology of China,
5
CASIA.
Figure 1: Dense face alignment (odd rows) and 3D face reconstruction (even rows) results from our proposed method. For alignment, only
68 key points are plotted for clear display; for 3D reconstruction, reconstructed shapes are rendered with head light for better view. Our
method offers strong robustness and good performance even in presence of large poses (the 3th, 4th and 5th columns) and occlusions (the
6th, 7th and 8th columns). Best viewed in color.
Abstract
3D face reconstruction from a single 2D image is a chal-
lenging problem with broad applications. Recent methods
typically aim to learn a CNN-based 3D face model that re-
gresses coefficients of 3D Morphable Model (3DMM) from
2D images to render 3D face reconstruction or dense face
alignment. However, the shortage of training data with 3D
annotations considerably limits performance of those meth-
ods. To alleviate this issue, we propose a novel 2D-assisted
self-supervised learning (2DASL) method that can effec-
tively use “in-the-wild” 2D face images with noisy land-
mark information to substantially improve 3D face model
learning. Specifically, taking the sparse 2D facial land-
marks as additional information, 2DSAL introduces four
novel self-supervision schemes that view the 2D landmark
and 3D landmark prediction as a self-mapping process, in-
cluding the 2D and 3D landmark self-prediction consis-
tency, cycle-consistency over the 2D landmark prediction
and self-critic over the predicted 3DMM coefficients based
on landmark predictions. Using these four self-supervision
schemes, the 2DASL method significantly relieves demands
on the the conventional paired 2D-to-3D annotations and
gives much higher-quality 3D face models without requir-
ing any additional 3D annotations. Experiments on multi-
ple challenging datasets show that our method outperforms
state-of-the-arts for both 3D face reconstruction and dense
face alignment by a large margin.
1
arXiv:1903.09359v1 [cs.CV] 22 Mar 2019
评论0
最新资源