Mixtures of Gaussians
1
Mixtures of Gaussians
A Tutorial for the Course Computational Intelligence
http://www.igi.tugraz.at/lehre/CI
Barbara Resch
Signal Processing and Speech Communication Laboratory
Inffeldgasse 16c
Abstract
This tutorial treats mixtures of Gaussian probability distribution functions. Gaussian mixtures are combinations
of a finite number of Gaussian distributions. They are used to model complex multi-dimensional distributions.
When there is a need to learn the parameters of the Gaussian mixture, the EM algorithm is used. In the second
part of this tutorial mixtures of Gaussian are used to model the emission probability distribution function in
Hidden Markov Models.
Usage
To make full use of this tutorial you should
1. Download the file MixtGaussian.zip which contains this tutorial and the accompanying Matlab pro-
grams.
2. Unzip MixtGaussian.zip which will generate a subdirectory named MixtGaussian/matlab where you can
find all the Matlab programs.
3. Add the folder MixtGaussian/matlab and the subfolders to the Matlab search path with a command like
addpath(’C:\Work\MixtGaussian\matlab’) if you are using a Windows machine or addpath(’/home/jack/MixtGaussian/matlab’)
if you are using a Unix/Linux machine.
1 Mixtures of Gaussians
1.1 Formulas and Definitions
Gaussian Mixtures are combinations of Gaussian, or ‘normal’, distributions. A mixture of Gaussians can be
written as a weighted sum of Gaussian densities.
Recall the d-dimensional Gaussian probability density function (pdf):
g
(µ,Σ)
(x) =
1
√
2π
d
p
det (Σ)
e
−
1
2
(x−µ)
T
Σ
−1
(x−µ)
, (1)
with mean vector µ and covariance matrix Σ.
A weighted mixture of K Gaussians can be written as
gm(x) =
K
X
k=1
w
k
· g
(µ
k
,Σ
k
)
(x), (2)
where the weights are all positive and sum to one:
w
k
≥ 0 and
K
X
k=1
w
k
= 1 for k ∈ {1, . . . , K}. (3)
In Figure
1 an example is given for an one dimensional Gaussian mixture, consisting of three single Gaussians.
By varying the number of Gaussians K, the weights w
k
, and the parameters µ
k
and Σ
k
of each Gaussian
density function, Gaussian mixtures can be used to describe any complex probability density function.
2
- 1
- 2
- 3
- 4
- 5
- 6
前往页