密歇根大学安阿尔分校机器学习研究生课程EECS 545的讲座材料【国外】.zip

# EECS 545, Winter 2016 This repository contains the lecture materials for EECS 545, a graduate course in Machine Learning, at the University of Michigan, Ann Arbor. ## [Formatted Lecture Materials](./Lectures.md) The link above gives a list of all of the available lecture materials, including links to ipython notebooks (via [Jupyter's nbviewer](http://nbviewer.jupyter.org/)), the slideshow view, and PDFs. ## Lecture Readings We will make references to the following textbooks throughout the course. The only required textbook is Bishop, *PRML*, but the others are very well-written and offer unique perspectives. - Bishop 2006, [*Pattern Recognition and Machine Learning*](http://research.microsoft.com/en-us/um/people/cmbishop/prml/) - Murphy 2012, [*Machine Learning: A Probabilistic Perspective*](https://www.cs.ubc.ca/~murphyk/MLbook/) #### Lecture 01: Introduction to Machine Learning *Wednesday, January 6, 2016* No required reading. #### Lecture 02: Linear Algebra & Optimization *Monday, January 11, 2016* - There are lots of places to look online for linear algebra help! - Juan Klopper has a [nice online review](http://www.juanklopper.com/opencourseware/mathematics-2/ipython-lecture-notes/), based on Jupyter notebooks. #### Lecture 03: Convex Functions & Probability *Wednesday, January 13, 2016*   ([Notebook Viewer](http://nbviewer.jupyter.org/github/thejakeyboy/umich-eecs545-lectures/blob/master/lecture03_convex-functions-optimization/lecture03_Convex-Functions-and-Optimization.ipynb), [PDF File](https://github.com/thejakeyboy/umich-eecs545-lectures/raw/master/lecture03_convex-functions-optimization/lecture03_Convex-Functions-and-Optimization.pdf), [Slide Viewer](http://nbviewer.jupyter.org/format/slides/github/thejakeyboy/umich-eecs545-lectures/blob/master/lecture03_convex-functions-optimization/lecture03_Convex-Functions-and-Optimization.ipynb)) Required: - **Bishop, §1.2:** Probability Theory - **Bishop, §2.1-2.3:** Binary, Multinomial, and Normal Random Variables Optional: - **Murphy, Chapter 2:** Probability #### Lecture 04: Linear Regression, Part I *Wednesday, January 20, 2016*   ([Notebook Viewer](http://nbviewer.jupyter.org/github/thejakeyboy/umich-eecs545-lectures/blob/master/lecture04_linear-regression-part1/lecture04_final_linear-regression-part-1.ipynb), [PDF File](https://github.com/thejakeyboy/umich-eecs545-lectures/raw/master/lecture04_linear-regression-part1/lecture04_final_linear-regression-part-1.pdf), [Slide Viewer](http://nbviewer.jupyter.org/format/slides/github/thejakeyboy/umich-eecs545-lectures/blob/master/lecture04_linear-regression-part1/lecture04_final_linear-regression-part-1.ipynb)) Required: - **Bishop, §1.1:** Polynomial Curve Fitting Example - **Bishop, §3.1:** Linear Basis Function Models Optional: - **Murphy, Chapter 7:** Linear Regression #### Lecture 05: Linear Regression, Part II *Monday, January 25, 2016*   ([Notebook Viewer](http://nbviewer.jupyter.org/github/thejakeyboy/umich-eecs545-lectures/blob/master/lecture05_linear-regression-part2/lecture05_final_linear-regression-part-2.ipynb), [PDF File](https://github.com/thejakeyboy/umich-eecs545-lectures/raw/master/lecture05_linear-regression-part2/lecture05_final_linear-regression-part-2.pdf), [Slide Viewer](http://nbviewer.jupyter.org/format/slides/github/thejakeyboy/umich-eecs545-lectures/blob/master/lecture05_linear-regression-part2/lecture05_final_linear-regression-part-2.ipynb)) Required: - **Bishop, §3.2:** The Bias-Variance Decomposition - **Bishop, §3.3:** Bayesian Linear Regression Optional: - **Murphy, Chapter 7:** Linear Regression #### Lecture 06: Probabilistic Models & Logistic Regression *Wednesday, January 27, 2016*   ([Notebook Viewer](http://nbviewer.jupyter.org/github/thejakeyboy/umich-eecs545-lectures/blob/master/lecture06_generative-models-gda/lecture06_final_probability-models-logistic-regression.ipynb), [PDF File](https://github.com/thejakeyboy/umich-eecs545-lectures/raw/master/lecture06_generative-models-gda/lecture06_final_probability-models-logistic-regression.pdf), [Slide Viewer](http://nbviewer.jupyter.org/format/slides/github/thejakeyboy/umich-eecs545-lectures/blob/master/lecture06_generative-models-gda/lecture06_final_probability-models-logistic-regression.ipynb)) Required: - **Bishop, §4.2:** Probabilistic Generative Models - **Bishop, §4.3:** Probabilistic Discriminative Models Optional: - **Murphy, Chapter 8:** Logistic Regression #### Lecture 07: Linear Classifiers *Monday, February 1, 2016*   ([Notebook Viewer](http://nbviewer.jupyter.org/github/thejakeyboy/umich-eecs545-lectures/blob/master/lecture07_nb-lda-perceptron/lecture07_final_nb-gda-lda.ipynb), [PDF File](https://github.com/thejakeyboy/umich-eecs545-lectures/raw/master/lecture07_nb-lda-perceptron/lecture07_final_nb-gda-lda.pdf), [Slide Viewer](http://nbviewer.jupyter.org/format/slides/github/thejakeyboy/umich-eecs545-lectures/blob/master/lecture07_nb-lda-perceptron/lecture07_final_nb-gda-lda.ipynb)) Required: - **Bishop, §4.1:** Discriminant Functions Recommended: - **Murphy §3.5:** Naive Bayes Classifiers - **Murphy §4.1:** Gaussian Models - **Murphy §4.2:** Gaussian Discriminant Analysis Optional: - **CS 229:** Notes on [Generative Models](http://cs229.stanford.edu/notes/cs229-notes2.pdf) - **Paper:** Zhang, H., 2004. ["The optimality of naive Bayes"](http://www.cs.unb.ca/profs/hzhang/publications/FLAIRS04ZhangH.pdf). AA, 1(2), p.3. - **Paper:** Domingos, P. and Pazzani, M., 1997. ["On the optimality of the simple Bayesian classifier under zero-one loss"](http://link.springer.com/article/10.1023/A:1007413511361). Machine learning, 29(2-3), pp.103-130. #### Lecture 08: Kernel Methods I, Kernels *Monday, February 8, 2016* Required: - **Bishop, §6.1:** Dual Representation - **Bishop, §6.2:** Constructing Kernels - **Bishop, §6.3:** Radial Basis Function Networks Optional: - **Murphy, §14.2:** Kernel Functions #### Lecture 09: Kernel Methods II, Duality & Kernel Regression *Wednesday, February 10, 2016* Required: - **Bishop, §6.1:** Dual Representation - **Bishop, §6.3:** Radial Basis Function Networks Optional: - Eric Kim, [Everything You Wanted to Know about the Kernel Trick (But were too Afraid to Ask)](http://www.eric-kim.net/eric-kim-net/posts/1/kernel_trick.html) #### Lecture 10: Kernel Methods III, Support Vector Machines & Gaussians *Monday, February 15, 2016* Required: - **Bishop, §7.1:** Maximum Margin Classifiers - **Bishop, §2.3.0-2.3.1:** Gaussian Distributions Optional: - CS229: [Support Vector Machines](http://cs229.stanford.edu/notes/cs229-notes3.pdf) #### Lecture 11: Kernel Methods III, Bayesian Linear Regression & Gaussian Processes *Wednesday, February 17, 2016* Required: - **Bishop, §3.3:** Bayesian Linear Regression - **Bishop, §6.4:** Gaussian Processes Recommended: - **Murphy, §7.6.1-7.6.2:** Bayesian Linear Regression - **Murphy, §4.3:** Inference in Joinly Gaussian Distributions Further Reading: - **Rasmussen & Williams**, Gaussian Processes for Machine Learning. (available [free online](http://www.gaussianprocess.org/gpml/)) #### Lecture 12: Machine Learning Advice *Monday, February 22, 2016* No required reading. #### Lecture 13: Information Theory & Exponential Families *Monday, March 7, 2016* Required: - **Bishop, §1.6:** Information Theory - **Bishop, §2.4:** The Exponential Family Recommended: - **Murphy, §2.8:** Information Theory - **Murphy, §9.2:** Exponential Families Further Reading: - **David Blei,**, [*Notes on Exponential Families*](https://www.cs.princeton.edu/courses/archive/fall11/cos597C/lectures/exponential-families.pdf). 2011. #### Lecture 14: Probabilistic Graphical Models *Wednesday, March 9, 2016* Required: - **Bishop, §8.1:** Bayesian Networks - **Bishop, §8.2:** Conditional Independence - **Bishop, §8.3:** Markov Random Fields Recommended: - **Murphy, §10