A Practical Guide to Support Vector Classification
Chih-Wei Hsu, Chih-Chung Chang, and Chih-Jen Lin
Department of Computer Science
National Taiwan University, Taipei 106, Taiwan
http://www.csie.ntu.edu.tw/
~
cjlin
Initial version: 2003 Last updated: April 15, 2010
Abstract
The support vector machine (SVM) is a popular classification technique.
However, beginners who are not familiar with SVM often get unsatisfactory
results since they miss some easy but significant steps. In this guide, we propose
a simple procedure which usually gives reasonable results.
1 Introduction
SVMs (Support Vector Machines) are a useful technique for data classification. Al-
though SVM is considered easier to use than Neural Networks, users not familiar with
it often get unsatisfactory results at first. Here we outline a “cookbook” approach
which usually gives reasonable results.
Note that this guide is not for SVM researchers nor do we guarantee you will
achieve the highest accuracy. Also, we do not intend to solve challenging or diffi-
cult problems. Our purpose is to give SVM novices a recipe for rapidly obtaining
acceptable results.
Although users do not need to understand the underlying theory behind SVM, we
briefly introduce the basics necessary for explaining our procedure. A classification
task usually involves separating data into training and testing sets. Each instance
in the training set contains one “target value” (i.e. the class labels) and several
“attributes” (i.e. the features or observed variables). The goal of SVM is to produce
a model (based on the training data) which predicts the target values of the test data
given only the test data attributes.
Given a training set of instance-label pairs (x
i
, y
i
), i = 1, . . . , l where x
i
∈ R
n
and
y ∈ {1, −1}
l
, the support vector machines (SVM) (Boser et al., 1992; Cortes and
Vapnik, 1995) require the solution of the following optimization problem:
min
w,b,ξ
1
2
w
T
w + C
l
X
i=1
ξ
i
subject to y
i
(w
T
φ(x
i
) + b) ≥ 1 − ξ
i
, (1)
ξ
i
≥ 0.
1
