Tutorial on Variational Autoencoders
CARL DOERSCH
Carnegie Mellon / UC Berkeley
August 16, 2016
Abstract
In just three years, Variational Autoencoders (VAEs) have emerged
as one of the most popular approaches to unsupervised learning of
complicated distributions. VAEs are appealing because they are built
on top of standard function approximators (neural networks), and
can be trained with stochastic gradient descent. VAEs have already
shown promise in generating many kinds of complicated data, in-
cluding handwritten digits [
1
,
2
], faces [
1
,
3
,
4
], house numbers [
5
,
6
],
CIFAR images [
6
], physical models of scenes [
4
], segmentation [
7
], and
predicting the future from static images [
8
]. This tutorial introduces the
intuitions behind VAEs, explains the mathematics behind them, and
describes some empirical behavior. No prior knowledge of variational
Bayesian methods is assumed.
Keywords: variational autoencoders, unsupervised learning, structured
prediction, neural networks
1 Introduction
“Generative modeling” is a broad area of machine learning which deals with
models of distributions
P(X)
, defined over datapoints
X
in some potentially
high-dimensional space
X
. For instance, images are a popular kind of data
for which we might create generative models. Each “datapoint” (image) has
thousands or millions of dimensions (pixels), and the generative model’s job
is to somehow capture the dependencies between pixels, e.g., that nearby
pixels have similar color, and are organized into objects. Exactly what it
means to “capture” these dependencies depends on exactly what we want
to do with the model. One straightforward kind of generative model simply
allows us to compute
P(X)
numerically. In the case of images,
X
values
1
arXiv:1606.05908v2 [stat.ML] 13 Aug 2016
剩余22页未读，继续阅读
资源评论
