Lecture 9
Support Vector
Machines
(Chapter 8)
Contents
Formulation
Extension – Linearly non-separable data
sets
Extension – Kernel Trick
Justification
Said to start in 1979 with Vladimir Vapnik’s
paper.
Major developments throughout 1990’s
Elegant theory – statistical learning theory
Have been applied to diverse problems very
successfully in the last 10-15 years.
One of the most important developments in
machine learning in the last 20 years until
today.
Background
Vladimir Vapnik
What is the most important criterion to
design a good classifier/funct. Approximator?
Perceptron + percepton Learning Rule
Linear Neuron + MSE minimization
MLP + BP, NINNs, Convnets
----- Reduce (training) ERROR.
Why not consider generalization issue in the
beginning when design a classifier?
What is a good Decision Boundary?
Consider a two-class, linearly
separable classification
problem
Many decision boundaries!
The Perceptron can be used to
find such a boundary
Class 1
Class 2
Hyperplanes created by
perceptron
