Foundations and Trends
R
in
Machine Learning
Vol. 2, No. 1 (2009) 1–127
c
2009 Y. Bengio
DOI: 10.1561/2200000006
Learning Deep Architectures for AI
By Yoshua Bengio
Contents
1 Introduction 2
1.1 How do We Train Deep Architectures? 5
1.2 Intermediate Representations: Sharing Features and
Abstractions Across Tasks 7
1.3 Desiderata for Learning AI 10
1.4 Outline of the Paper 11
2 Theoretical Advantages of Deep Architectures 13
2.1 Computational Complexity 16
2.2 Informal Arguments 18
3 Local vs Non-Local Generalization 21
3.1 The Limits of Matching Local Templates 21
3.2 Learning Distributed Representations 27
4 Neural Networks for Deep Architectures 30
4.1 Multi-Layer Neural Networks 30
4.2 The Challenge of Training Deep Neural Networks 31
4.3 Unsupervised Learning for Deep Architectures 39
4.4 Deep Generative Architectures 40
4.5 Convolutional Neural Networks 43
4.6 Auto-Encoders 45
- 1
- 2
- 3
- 4
- 5
- 6
前往页