Gaussian processes
需积分: 9 200 浏览量
2018-08-11
10:34:31
上传
评论
收藏 160KB PDF 举报
Gaussian processes
Chuong B. Do (updated by Honglak Lee)
November 22, 2008
Many of the classical machine learning algorithms that we talked about during the first
half of this course fit the following pattern: given a training set of i.i.d. examples sampled
from some unknown distribution,
1. solve a convex optimization problem in order to identify the single “best fit” mo del for
the data, a nd
2. use this estimated model to make “best guess” predictions for future test input points.
In these notes, we will talk about a different flavor of learning algorithms, known as
Bayesian methods. Unlike classical learning algorithm, Bayesian algorithms do not at-
tempt to identify “best-fit” models of the data (or similarly, make “best guess” predictions
for new test inputs). Instead, they compute a p osterior distribution over models (or similarly,
compute posterior predictive distributions for new test inputs). These distributions provide
a useful way to quantify our uncertainty in model estimates, and to exploit our knowledge
of this uncertainty in order to make more robust predictions on new test points.
We focus on regression problems, where the goal is to learn a mapping from some input
space X = R
n
of n-dimensional vectors to an output space Y = R of real-valued targets.
In particular, we will talk about a kernel-based fully Bayesian regression algorithm, known
as Gaussian process regression. The material covered in these notes draws heavily on many
different topics that we discussed previously in class (namely, the probabilistic interpretation
of linear regression
1
, Bayesian methods
2
, kernels
3
, and properties of multivariate Gaussians
4
).
The organization of these notes is as follows. In Section 1, we provide a brief review
of multivaria t e Gaussian distributions and their properties. In Section 2 , we briefly review
Bayesian methods in the context of probabilistic linear regression. The centr al ideas under-
lying Gaussian processes are presented in Section 3, and we derive the full Gaussian process
regression model in Section 4 .
1
See course lecture notes on “Supervised Learning, Discriminative Algorithms.”
2
See course lecture notes on “Regularization and Model Selection.”
3
See course lecture notes on “Supp ort Vector Machines.”
4
See course lecture notes on “Factor Analysis.”
1
剩余13页未读，继续阅读
评论0
最新资源