CS229 Lecture notes
Andrew Ng
Part V
Support Vector Machines
This set of notes presents the Support Vector Machine (SVM) learning al-
gorithm. SVMs are among the best (and many believe are indeed the best)
“off-the-shelf” sup ervised learning algorithm. To tell the SVM story, we’ll
need to first talk about margins and the idea of separating data with a large
“gap.” Next, we’ll talk about the optimal margin classifier, which will lead
us into a digression on L agrange duality. We’ll also see kernels, which give
a way t o apply SVMs efficiently in very hig h dimensional (such as infinite-
dimensional) feature spaces, and finally, we’ll close off the story with the
SMO algorithm, which gives an efficient implementation of SVMs.
1 Margins: Intuition
We’ll start our story on SVMs by talking about margins. This section will
give the intuitions about margins and about the “ confidence” of our predic-
tions; these ideas will be made formal in Section 3.
Consider logistic regression, where the probability p(y = 1|x; θ) is mod-
eled by h
θ
(x) = g(θ
T
x). We would then predict “1” on an input x if and
only if h
θ
(x) ≥ 0.5, or equivalently, if and only if θ
T
x ≥ 0. Consider a
positive training example (y = 1 ) . The lar ger θ
T
x is, the larger also is
h
θ
(x) = p(y = 1|x; w, b), and thus a lso the higher our degree of “confidence”
that the label is 1. Thus, informally we can think of our prediction as being
a very confident one that y = 1 if θ
T
x ≫ 0. Similarly, we think of logistic
regression as making a very confident prediction of y = 0, if θ
T
x ≪ 0. Given
a training set, again informally it seems that we’d have found a good fit to
the training data if we can find θ so that θ
T
x
(i)
≫ 0 whenever y
(i)
= 1, and
1
剩余24页未读，继续阅读
资源评论
