Preface
I was shocked to see a student’s report on performance comparisons between
support vector machines (SVMs) and fuzzy classifiers that we had developed
with our best endeavors. Classification performance of our fuzzy classifiers was
comparable, but in most cases inferior, to that of support vector machines.
This tendency was especially evident when the numbers of class data were
small. I shifted my research efforts from developing fuzzy classifiers with high
generalization ability to developing support vector machine–based classifiers.
This book focuses on the application of support vector machines to pat-
tern classification. Specifically, we discuss the properties of support vector
machines that are useful for pattern classification applications, several mul-
ticlass models, and variants of support vector machines. To clarify their ap-
plicability to real-world problems, we compare performance of most models
discussed in the book using real-world benchmark data. Readers interested in
the theoretical aspect of support vector machines should refer to books such
as [109, 215, 256, 257].
Three-layer neural networks are universal classifiers in that they can clas-
sify any labeled data correctly if there are no identical data in different classes
[3, 279]. In training multilayer neural network classifiers, network weights are
usually corrected so that the sum-of-squares error between the network out-
puts and the desired outputs is minimized. But because the decision bound-
aries between classes acquired by training are not directly determined, clas-
sification performance for the unknown data, i.e., the generalization ability,
depends on the training method. And it degrades greatly when the number
of training data is small and there is no class overlap.
On the other hand, in training support vector machines the decision
boundaries are determined directly from the training data so that the sepa-
rating margins of decision boundaries are maximized in the high-dimensional
space called feature space. This learning strategy, based on statistical learning
theory developed by Vapnik [256, 257], minimizes the classification errors of
the training data and the unknown data.
