About the course notes
Previous versions of this course have been offered at SIGGRAPH 2013, EG 2014 and
SGP 2015. These authors have all contributed to the creation of these course notes:
Dr. Sofien Bouaziz (me@sofienbouaziz.com, http://sofienbouaziz.com)
obtained his PhD degree in 2015 in the Computer Graphics and Geometry Labora-
tory (LGG) at EPFL. He received his MSc degree in Computer Science from EPFL
in 2009. His research interests include computer graphics, computer vision, and
machine learning. In 2012, he co-founded faceshift, an EPFL spin-off that brings
high-quality markerless facial motion capture to the consumer market.
Dr. Andrea Tagliasacchi (ataiya@uvic.ca, http://gfx.uvic.ca)
is an assistant professor at the University of Victoria and PI on the NSERC Dis-
covery grant “Real-Time Modeling and Registration of Dynamic Geometry”. He
was a post doctoral scholar at EPFL, and obtained his PhD from Simon Fraser
University as an NSERC Alexander Graham Bell scholar. He received his MSc
from Politecnico di Milano (cum laude, faculty gold medalist).
Dr. Hao Li (hao@hao-li.com, http://hao.li)
is an assistant professor at the University of Southern California, Director of the
Vision and Graphics Lab at the USC Institute for Creative Technologies, and CEO
of Pinscreen. He was a postdoctoral researcher at Columbia and Princeton and a
research lead at Industrial Light&Magic. He obtained his PhD from ETH Zurich
and his MSc degree from the University of Karlsruhe.
Dr. Mark Pauly (mark.pauly@epfl.ch, http://lgg.epfl.ch)
is a professor of computer science at EPFL. Prior to joining EPFL he was an
assistant professor at ETH Zurich and a post doctoral scholar at Stanford. He
received his Ph.D. degree in 2003 from ETH Zurich. His research interests include
computer graphics and animation, geometry processing, and architectural design.
Introduction
Recent technological advances in RGB-D sensing devices, such as the Microsoft Kinect,
facilitate numerous new and exciting applications, for example in 3D scanning [44] and
human motion tracking [39]. While affordable and accessible, consumer-level RGB-D
devices typically exhibit high noise levels in the acquired data. Moreover, difficult light-
ing situations and geometric occlusions commonly occur in many application settings,
potentially leading to a severe degradation in data quality. This necessitates a particular
emphasis on the robustness of image and geometry processing algorithms. The combina-
tion of geometry (3D) and image (2D) registration is one important aspect in the design
of robust applications based on RGB-D devices. This course introduces the main con-
cepts of 2D and 3D registration and explains how to combine them efficiently. To enable
dense correspondence computation and non-rigid registration between shapes of signifi-
cant deformations and shape variations, we present a deep learning framework based on
convolutional neural networks.
2