东南大学 崇志宏:Deep Learning参考书

1星(超过10%的资源)
所需积分/C币:9 2017-08-03 07:00:19 21.89MB PDF
35
收藏 收藏
举报

东南大学数据与智能实验室暑期讨论班采用的深度学习参考书,全书分为三个部分,其中的第三部分针对研究生的研究综述了深度学习的主要技术方法和理论基础,对研究生入门很有帮助的一本参考书。
CONTENTS 3.2 Random Variables 56 3.3 Probability distributions 56 3.4 Marginal Probability 3.5 Conditional Probability 59 3. 6 The chain rule of conditional Probabilities 59 3.7 Independence and Conditional Independence 60 3.8 Expectation, Variance and Covariance 60 3.9 Common Probability Distributions 62 3.10 Useful Properties of Common Functions 67 3.11 Bayes' Rule 70 3.12 Technical Details of continuous variables 71 3.13 Information Theory 3.14 Structured Probabilistic models 75 4 Numerical Computation 80 4. 1 Overflow and Underflow 4.2 Poor Conditioning 4.3 Gradient-Based OptiInization 022 4.4 Constrained Optimization 93 4.5 Example: Linear Least Squares 96 5 Machine learning basics 98 5.1 Learning Algorithms 99 5. 2 Capacity, Overfitting and Underfitting 5.3 Hyperparameters and Validation Sets .120 5.1 Estimators Bias and Variance 122 5.5 Maximum likelihood estimation 131 5.6 Bayesian Statistics .135 5.7 Supervised Learning Algorithms 140 5.8 Unsupervised Learning Algorithms 146 5.9 Stochastic gradient Descent 151 5.10 Building a Machine Learning Algorithm 153 5.11 Challenges Motivating Deep Learning II Deep Networks: Modern Practices 166 6 Deep Feedforward Networks 168 6.1 Example: Learning XOR 171 6.2 Gradient-Based Learning 177 CONTENTS 6.3 Hiddon units .191 6.4 Architecture de esion 197 6.5 Back-Propagation and Other Differentiation Algorithms 204 6.6 Historical notes 224 7 Regularization for Deep Learning 228 7.1 Parameter Norm Penalties 7.2 Norm Penalties as Constrained Optimization 237 7.3 Regularization and under-Constrained problems Q 7. 4 Dataset Augmentation 240 7.5 Noise Robustness 242 7.6 Semi-Supervised Learning 243 7.7 Multi- Task Learning .214 7.8 Early Stopping 246 7.9 Parameter Tying and Parameter Sharing .253 7.10 Sparse Representations .254 7. 11 Bagging and Other Ensemble Methods .,,256 7.12 Dropout 258 7. 13 Adversarial Trainin 268 7.14 Tangent Distance, Tangent Prop, and Manifold Tangent Classifier 270 8 Optimization for Training Deep Models 274 8. 1 How Learning Differs from Pure Optimization 275 8.2 Challenges in Neural Network Optimization .282 8.3 Basic algorithms 294 8.1 Parameter Initialization Strategies 30l 8.5 Algorithms with Adaptive Learning rates 306 8.6 Approximate Second-Order Methods 310 8.7 Optimization Strategies and Meta-algorithms 317 9 Convolutional nctworks 330 9.1 The Convolution Operation .331 9.2 Motivation .335 9.3 Pooling 339 9.4 Convolution and Pooling as an Infinitely Strong Prior .345 9.5 Variants of the basic convolution Function 347 9.6 Structured Outputs 358 9. 7 Data Types 360 9.8 Efficient Convolution Algorithms ..362 9.9 Random or Unsupervised Features 363 CONTENTS 9.10 The Ncuroscicntific Basis for Convolutional Nctworks ..364 9. 11 Convolutional Networks and the History of Deep Learning 371 10 Sequence Modeling: Recurrent and Recursive Nets 373 10.1 Unfolding Computational graphs 10.2 Recurrent Neural Networks 378 10.3 Bidirectional rnns .:94 10.4 Encoder-Decoder Sequence-to-Sequence Architectures .396 10.5 Deep Recurrent Networks 398 10.6 Recursive Neural Networks 10.7 The Challenge of Long- Term Dependencies 401 10.8 Echo State Networks 404 10.9 Leaky Units and Other Strategies for Multiple Time Scales 106 10.10 The Long Short-Term Memory and Other Gated RNNS 408 10.11 Optimization for Long-Term Dependencies 413 10.12 Explicit Memory 416 11 Practical Methodology 421 11.1 Performance Metrics 422 11.2 Default baseline models 425 11.3 Determining Whether to Gather More Data .,,426 11.4 Selecting Hyperparameters .427 11.5 Debugging Strategies 436 11.6 Example: Multi-Digit Number recognition 440 12 Applications 443 12.1 Large-Scale Deep Learnin 443 12.2 Computer Vision 452 12.3 Speech Recognition 58 12.4 Natural Language Processing 46l 12.5 Other Applications 478 III Deep Learning Research 486 13 Linear Factor Models 489 13.1 Probabilistic PCA and Factor Analysis 490 13.2 Independent Component Analysis(ICA) .491 13.3 Slow Feature Analysis 493 13.1 Sparse Coding .496 CONTENTS 13.5 Manifold Interprctation of PCA .499 14 Autoencoders 502 14.1 Undercomplete Autoencoders 503 14.2 Regularized Autoencoders .504 11.3 Representational Power, Layer Size and Depth 508 14.4 Stochastic Encoders and decoders 14.5 Denoising Autoencoders .510 14.6 Learning Manifolds with Autoencoders 515 14.7 Contractive Autoencoders 521 14.8 Predictive Sparse Decomposition 523 14.9 Applications of Autoencoders 524 15 Representation Learning 526 15.1 Greedy Layer-Wise Unsupervised Pretraining 528 15.2 Transfer Learning and Domain Adaptation .536 15.3 Semi-Supervised Disentangling of Causal Factors 541 15.4 Distributed Representation 546 15.5 Exponential Gains from Depth 553 15.6 Providing Clues to Discover Underlying Causes 554 16 Structured Probabilistic Models for Deep Learning 558 16.1 The Challenge of Unstructured Modeling .559 16.2 Using Graphs to Describe Model Structure............. 563 16.3 Sampling from Graphical Models 580 16.4 Advantages of Structured Modeling .582 16.5 Learning about Dependencies 582 16.6 Inference and Approximate Inference 584 16.7 The Deep Learning Approach to Structured Probabilistic Models 585 17 Monte Carlo methods 590 17.1 Sampling and Monte Carlo Methods 590 17.2 Importance Sampling .592 17.3 Markov Chain Monte Carlo Methods 595 17.4 Gibbs sampling .599 17. The Challenge of mixing between Separated Modes 599 18 Confronting the Partition Function 605 18.1 The Log-Likelihood Gradient 606 18.2 Stochastic Maximum Likelihood and Contrastive Divergence... 607 CONTENTS 18.3 Pscudolikclihood ..615 18.4 Score Matching and Ratio Matching .617 18.5 Denoising Score Matching 619 18.6 Noise-Contrastive Estimation 620 18.7 Estimating the Partition Function 623 19 Approximate Inference 631 19.1 Inference as Optimization 19.2 Expectation Maximization 634 19.3 MAP Inference and Sparse Coding 635 19.4 Variational Inference and Learning .638 19.5 Learned Approximate Inference 651 20 Deep Generative Models 654 20.1 Boltzmann Machines 654 20.2 Restricted boltzmann machines .656 20.3 Deep Belief Networks 660 20.4 Deep Boltzmann Machines 663 20.5 Boltzmann Machines for real-Valued data .676 20.6 Convolutional boltzmann machines 20.7 Boltzmann Machines for Structured or Sequential Outputs .685 20.8 Other Boltzmann Machines 6 20.9 Back-Propagation through Random Operations 687 20.10 Directed Generative Nets 692 20 11 Drawing Samples from Autoencoders ..711 20.12 Generative Stochastic Networks 714 20.13 Other Generation schemes .716 20.14 Evaluating generative Models 717 20.15 Conclusion .720 Bibliography 721 Index 777 Website www.cleeplearlinlgboOk.org This book is accompanied by the above website. The website provides a variety of supplementary material, including exercises, lecture slides, corrections of mistakes, and other resources that should be useful to both readers and instructors Acknowledgments This book would not have been possible without the contributions of many people We would like to thank those who commented on our proposal for the book and helped plan its contents and organization: Guillaume Alain, Kyunghyun Cho Caglar gilcehre, David Krueger, Hugo larochelle, razvan Pascanu and Thomas Rohee We would like to thank the people who offered feedback on the content of the book itself. Some offered feedback on many chapters: Martin Abadi, Guillaume Alain, Ion Androutsopoulos, Fred Bertsch, Olexa Bilaniuk, Ufuk Can Bicici, Matko BoSnjak, John boersma, Greg brockman, Alexandre de brebisson, Pierre Luc Carrier, Sarath Chandar, Pawel Chilinski, Mark Daoust, Oleg Dashevskii, Laurent Dinh, Stephan Dreseitl, Jim Fan, Miao Fan, Meire Fortunato, Frederic francis Nando de Freitas, Caglar gulcehre, Jurgen Van Gael, Javier Alonso Garcia Jonathan Hunt, gopi Jeyaram, Chingiz Kabytayev, Lukasz Kaiser, Varun Kanade Asifullah Khan, Akiel Khall, John King, Diederik P Kingma, Yann LeCunl, Rudolf Mathey, Matias Mattamala, Abhinav maurya, Kevin Murphy, Oleg Mirk, Roman Novak, Augustus Q. Odena, Simon Pavlik, Karl Pichotta, Eddie Pierce, Kari Pulli Roussel rahman, Tapani Raiko, Anurag Ranjan, Johannes Roith, Mihaela rosca, Halis sak, Cosar Salgado, Grigory Sapunov, Yoshinori Sasaki, Mikc Schuster, Julian Serban, Nir Shabat, Ken Shirriff, Andre simpelo, Scott stanley, David Sussillo, Ilya Sutskever, Carles gelada saez, Graham Taylor, Valentin Tolmer Massimiliano Ttomassoli, An Tran. Shubhendu Trivedi, Alexey umno, Vincent Vanhoucke. Marco visentini-Scarzanella. Martin Vita. David Warde-Farlev dustin Webb, Kelvin Xu. Wei Xue, Ke Yang, Li Yao, Zygmunt Zajac and Ozan Caglayan We would also like to thank those who provided us with useful feedback on individual chapters Notation: Zhang Yuanhang e Chapter 1, Introduction: Yusuf Akgul, Sebastien Braticres, Samira ebrahimi CONTENTS Charlic gorichanaz. Brendan loudermilk. Eric morris Cosmin parvuloscu and alfredo solano Chapter 2, Linear Algebra: Amjad almahairi, Nikola banic, Kevin Bennett Philippe castonguay, Oscar Chang, Eric Fosler-Lussier, Andrey Khalyavin ergey Oreshkov, Istvan Petras, Dennis Prangle, Thomas rohee, gitanjali Gulve sehgal, Colby toland, Alessandro vitale and Bob welland Chapter 3, Probability and Information Theory: John Philip anderson, Kai Arulkumaran, Vincent Dumoulin, Rui Fa, Stephan Gouws. Artem Oboturov Antti rasmus, Alexey Surkov and Volker Tresp e Chapter 4, Numerical Computation: Tran Lam Anlan Fischer and hu Yuhuang Chapter 5, Machine Learning Basics: Dzmitry Bahdanau, Justin domingue Nikhil garg, Makoto Otsuka, Bob Pepin, Philip Popien, Emmanuel rayner Peter Shepard, Kee-Bong Song, Zheng Sun and Andy Wu e Chapter 6, Deep Feedforward Networks: Uriel Berdugo, Fabrizio Bottarel Elizabeth Burl, Ishan Durugkar, Jeff Hlywa, Jong Wook Kim, David Krueger and Aditya Kumar praharaj Chapter 7, Regularization for Deep Learning: Morten Kolbaek, Kshitij lauria Inkyu Lee, Sunil Mohan, Hai Phong Phan and Joshua salisbury Chapter 8, Optimization for Training Deep Models: Marcel Ackermann, Peter Armitage, Rowel Atienza, Andrew Brock, Tegan Maharaj, James Martens Kashif rasul. Klaus strobl and nicholas turner e Chapter 9, Convolutional Net works: Martin Arjovsky, Eugene Brevdo, Kon stantin Divilov, Eric Jensen, Mehdi mirza, Alex Paino, Marjorie Sayer, Ryan Stout and wentao wu Chapter 10, Sequence Modeling: Recurrent and Recursive Nets: Gokcen Eraslan, Steven Hickson, Razvan Pascanu, Lorenzo von Ritter, Rui rodrigues Dmitriy Serdyuk, Dongyu Shi and Kaiyu Yang e Chapter 11, Practica.I Methodology: Daniel Beckstein Chapter 12, Applications: George Dahl, Vladimir Nekrasov and Ribana R oscher Chapter 13, Linear Factor Models: Jayanth Koushik IX

...展开详情
试读 127P 东南大学 崇志宏:Deep Learning参考书
立即下载 身份认证后 购VIP低至7折
一个资源只可评论一次,评论内容不能少于5个字
老黑头 没必要下载了,就是Deep Learning,网上很多下载的地方。
2017-10-19
回复
您会向同学/朋友/同事推荐我们的CSDN下载吗?
谢谢参与!您的真实评价是我们改进的动力~
  • 分享宗师

关注 私信
上传资源赚钱or赚积分
最新推荐
东南大学 崇志宏:Deep Learning参考书 9积分/C币 立即下载
1/127
东南大学 崇志宏:Deep Learning参考书第1页
东南大学 崇志宏:Deep Learning参考书第2页
东南大学 崇志宏:Deep Learning参考书第3页
东南大学 崇志宏:Deep Learning参考书第4页
东南大学 崇志宏:Deep Learning参考书第5页
东南大学 崇志宏:Deep Learning参考书第6页
东南大学 崇志宏:Deep Learning参考书第7页
东南大学 崇志宏:Deep Learning参考书第8页
东南大学 崇志宏:Deep Learning参考书第9页
东南大学 崇志宏:Deep Learning参考书第10页
东南大学 崇志宏:Deep Learning参考书第11页
东南大学 崇志宏:Deep Learning参考书第12页
东南大学 崇志宏:Deep Learning参考书第13页
东南大学 崇志宏:Deep Learning参考书第14页
东南大学 崇志宏:Deep Learning参考书第15页
东南大学 崇志宏:Deep Learning参考书第16页
东南大学 崇志宏:Deep Learning参考书第17页
东南大学 崇志宏:Deep Learning参考书第18页
东南大学 崇志宏:Deep Learning参考书第19页
东南大学 崇志宏:Deep Learning参考书第20页

试读结束, 可继续阅读

9积分/C币 立即下载