viii Preface to the Second Edition
Chapter What’s new
1. Introduction
2. Overview of Supervised Learning
3. Linear Methods for Regression LAR algorithm and generalizations
of the lasso
4. Linear Methods for Classification Lasso path for logistic regression
5. Basis Expansions and Regulariza-
tion
Additional illustrations of RKHS
6. Kernel Smoothing Methods
7. Model Assessment and Selection Strengths and pitfalls of cross-
validation
8. Model Inference and Averaging
9. Additive Models, Trees, and
Related Methods
10. Boosting and Additive Trees New example from ecology; some
material split off to Chapter 16.
11. Neural Networks Bayesian neural nets and the NIPS
2003 challenge
12. Support Vector Machines and
Flexible Discriminants
Path algorithm for SVM classifier
13. Prototype Methods and
Nearest-Neighbors
14. Unsupervised Learning Spectral clustering, kernel PCA,
sparse PCA, non-negative matrix
factorization archetypal analysis,
nonlinear dimension reduction,
Google page rank algorithm, a
direct approach to ICA
15. Random Forests New
16. Ensemble Learning New
17. Undirected Graphical Models New
18. High-Dimensional Problems New
Some further notes:
• Our first edition was unfriendly to colorblind readers; in particular,
we tended to favor
red/green contrasts which are particularly trou-
blesome. We have changed the color palette in this edition to a large
extent, replacing the above with an orange/blue contrast.
• We have changed the name of Chapter 6 from “Kernel Methods” to
“Kernel Smoothing Methods”, to avoid confusion with the machine-
learning kernel method that is discussed in the context of support vec-
tor machines (Chapter 11) and more generally in Chapters 5 and 14.
• In the first edition, the discussion of error-rate estimation in Chap-
ter 7 was sloppy, as we did not clearly differentiate the notions of
conditional error rates (conditional on the training set) and uncondi-
tional rates. We have fixed this in the new edition.
