deep learning

所需积分/C币:23 2019-03-19 09:33:11 66KB PDF

table of contents from deep learning book published by MIT, it is very famous in DNN.
CONTENTS 3.2 Random Variables 6 3.3 Probability Distributions 56 3.4 Marginal Probability 58 3.5 Conditional Probability 59 3.6 The Chain Rule of Conditional Probabilities .,.59 3.7 Independence and Conditional Independencc 3.8 Expectation, Variance and Covariance 60 3.9 Common Probability Distributions 62 3.10 Useful Properties of Common Functions 67 3.11 Bayes’Rule.. 70 3.12 Technical Details of Continuous Variables 3.13 Information Theory 3.14 Structured Probabilistic Models 4 Numerical Computation 80 4.1 Overfow and underflow 80 4.2 Poor Conditioning 82 4.3 Gradicnt-Bascd Optimization .82 4.4 Constrained Optimization 93 4.5 Example: Linear Least Squares 5 Machine Learning Basics 98 5.1 Learning Algorithms 99 5.2 Capacity, Overfitting and Underfitting 110 5.3 Hyperparameters and Validation Sets 120 5.4 Estimators Bias and variance 122 5.5 Maximum likelihood estimation 131 5.6 Bayesian Statistics .135 5.7 Supervised Learning algorithms 140 5.8 Unsupervised Learning Algorith .116 5.9 Stochastic Gradient Descent 151 5.10 Building a Machine Learning Algorithm 153 5.11 Challenges Motivating Deep Learning 155 II Deep Networks: Modern Practices 166 6 Deep Feedforward Networks 168 6.1 Examplc: Learning XOR 171 6.2 Gradient-Based Learning 177 CONTENTS 6. 3 Hidden Units 6.4 Architecture Design 197 6.5 Back-Propagation and Other Differentiation Algorithms 204 6.6 Historical notes 224 7 Regularization for Deep Learning 228 7.1 Parameter Norm Penalties 230 7.2 Norm Penalties as Constrained Optimization 237 7.3 Regularization and Under-Constrained Problems 239 7.4 Dataset Augmentation 240 7.5 Noise robustness 242 7.6 Semi-Supervised Learning 243 7.7 Multi-Task Learning 244 7.8 Early Stopping 246 7.9 Parameter Tying and Parameter Sharing 253 7.10 Sparse Representations .254 7.11 Bagging and Other Ensemble Methods 256 7. 12 Dropout 258 7.13 Adversarial Training 268 7. 14 Tangent Distance, Tangent Prop, and Manifold Tangent Classifier 270 8 Optimization for Training Deep Models 274 8. 1 How Learning Differs from Pure Optimization 275 8.2 Challenges in Neural Network Optimization 282 8.3 Basic Algorithms 294 8. 4 Parameter Initialization Strategies 301 8.5 Algorithms with Adaptive Learning rates .306 8.6 Approximate Second-Order Methods 310 8.7 Optimization Strategies and Meta-Algorithms 317 9 Convolutional networks 330 Q The Convolution Operation .331 9.2 Motivation 335 9.3 Pooling 339 9.4 Convolution and Pooling as an Infinitely Strong pric Or 345 9.5 Variants of the Basic Convolution function 347 9.6 Structured Outputs 358 9. 7 Data Types 360 9.8 Efficient Convolution Algorithms ..362 9.9 Random or Unsupervised Features 363 CONTENTS 9.10 The Neuroscientific Basis for Convolutional Networks .364 9.11 Convolutional Networks and the History of Deep Learning 371 10 Sequence Modeling: Recurrent and Recursive Nets 373 10.1 Unfolding Computational graphs .375 10.2 Recurrent Neural networks ,,378 10.3 Bidirectional RNNs 394 10.4 Encodcr-Dccoder Scquencc-to-Scqucncc Architectures 396 10.5 Deep Recurrent Networks 398 10.6 Rccursivc Neural nctworks 400 10.7 The Challenge of Long-Term Dependencies 401 10.8 Echo state networks 404 10.9 Leaky Units and Other Strategies for Multiple Time Scales 406 10.10 The Long short-Term Memory and Other Gated RNNS 408 10.11 Optimization for Long- Term Dependencies 413 10.12 Explicit Memory 416 1 Practical Methodology 421 11.1 Performance Metrics 422 11.2 Default baseline models 4125 11.3 Determining Whether to Gather More Data 426 11.1 Selecting Hyperparameters 427 11.5 Debugging strategies ..436 11.6 Example: Multi-Digit Number Recognition 440 12 Applications 443 12. 1 Large-Scale Deep Learning 443 12.2 Computer Vision ..152 12.3 Speech Recognition 458 12.1 Natural Language Processing .461 12.5 Other Applications 478 III Deep Learning Research 486 1 3 Linear Factor Models 489 13.1 Probabilistic PCA and Factor Analysis 490 13.2 Independent Component Analysis(ICA) 491 13.3 Slow Feature Analysis 493 13.4 Sparse Coding 496 CONTENTS 13.5 Manifold Interpretation of PCA 499 14 Autoencoders 502 14.1 Undercomplete Autoencoders 503 14.2 Regularized Autoencoders 504 14.3 Representational Power, Layer Size and Depth .508 14.4 Stochastic Encoders and Decoders ..509 14.5 Denoising Autocncodcrs .510 14.6 Learning manifolds with Autoencoders 515 14.7 Contractive Autocncodcrs 521 14.8 Predictive Sparse Decomposition 523 14.9 Applications of Autoencoders 524 15 Representation Learning 526 15.1 Greedy Layer-Wise Unsupervised Pretraining 528 15.2 Transfer Learning and Domain Adaptation 536 15.3 Semi-Supervised Disentangling of Causal Factors 541 15.4 Distributed Representation 546 15.5 Exponential Gains from Depth 15.6 Providing Clues to Discover Underlying causes 553 554 16 Structured Probabilistic Models for Deep Learning 558 16.1 The Challenge of Unstructured Modeling 559 16.2 Using Graphs to Describe Model Structure 563 16.3 Sampling from Graphical models 580 16.4 Advantages of Structured Modeling 582 16.5 Learning about Dependencies 582 16.6 Inference and Approximate Inference 584 16. 7 The Deep Learning Approach to Structured Probabilistic Models 585 17 Monte carlo methods 590 17.1 Sampling and Monte Carlo Methods 590 17.2 Importance Sampling 592 17.3 Markov Chain Monte Carlo Methods 595 17.4 Gibbs Sampling 599 17.5 The Challenge of Mixing between Separated Modes 59 18 Confronting the Partition Function 605 18.1 The Log-Likelihood Gradient .606 18.2 Stochastic Maximum Likelihood and Contrastive Divergence 607 CONTENTS 18.3 Pseudolikelihood 61 18.4 Score Matching and Ratio Matching 61 18.5 Denoising Score Matching 619 18.6 Noise-Contrastive Estimation 620 18.7 Estimating the Partition Function 623 19 Approximate Inference 631 19. 1 Inference as Optimization 633 19.2 Expectation Maximization 634 19.3 MAP Inference and Sparsc Coding 635 19.4 Variational Inference and Learning 638 19.5 Learned Approximate Inference 651 20 Deep Generative Models 654 20.1 Boltzmann machines 654 20.2 Restricted boltzmann machines 656 20.3 Deep Belief Networks 660 20.4 Deep Boltzmann Machines 663 20.5 Boltzmann Machines for Real-Valued Data 676 20.6 Convolutional boltzmann machines 683 20.7 Boltzmann Machines for Structured or Sequential Outputs.... 685 20.8 Other Boltzmann machines .686 20.9 Back-Propagation through Random Operations 687 20.10 Directed Generative Nets 692 20.11 Drawing Samples from Autoencoders 711 20.12 Generative Stochastic Networks 714 20.13 Other Generation Schemes 716 20.14 Evaluating Generative Models 717 20.15 Conclusion 720 Bibliography 721 Index 777 Website www.deeplearnilgbook.org This book is accompanied by the above website. The website provides a variety of supplementary material, including exercises, lecture slides, corrections of mistakes, and other resources that should be useful to both readers and instructors VI

...展开详情
试读 8P deep learning
img
Setizen

关注 私信 TA的资源

上传资源赚积分,得勋章
    最新推荐
    deep learning 23积分/C币 立即下载
    1/8
    deep learning第1页
    deep learning第2页
    deep learning第3页

    试读已结束,剩余5页未读...

    23积分/C币 立即下载 >