深度学习500问资料整理

# 500 Deep Learning Questions <p align="center"> <strong>A comprehensive body of knowledge on Deep learning and Artificial Intelligence</strong> </p> `500 Deep Learning Questions` is a collection of articles on the mathematical and technical aspects of Deep Learning and Artificial Intelligence that will help you build a strong foundation of knowledge in this domain. **Notes** - Please help us improve this resource as mentioned in the [section on contributing](#contributing) - Please respect the authors' intellectual property rights, and copyright. Piracy will be investigated - Feel free to share the location of this content with others or fork the repository for contributing, but please ask for permission before copying and forwarding this content 2018/06/27 TanJiyong, 2020/10/28 Arjun Krishna-University of Waterloo This was translated with Google Translate and partial human translation. Please help translate more of this resource to English as mentioned in the [section on contributing](#contributing) **Please star and share this project if you like it!** # Sections of README 1. [Sections of README](#sections-of-readme) 2. [How to use this Resource](#how-to-use-this-resource) 3. [Contributing](#contributing) 1. [How to help](#how-to-help) 2. [Notes on contributing](#notes-on-contributing) 4. [Communication](#communication) 5. [Table of Contents](#table-of-contents) # How to Use This Resource 1. Go to the Language section that you are most comfortable with (e.g. English ) 2. Look at the Table of Contents and go the topics that you are most interested in or start from Chapter 1 3. The best way to truly learn something is to apply it in practice 1. If you have just read the section on Recurrent Neural Networks, ask yourself this: 1. In what situations would I apply this? 1. Speech recognition 2. Translation 2. What benefits does this machine learning technique have over others? 3. What disdvantages does it have? 4. Code a recurrent neural network in MatLab, or using Tensorflow or PyTorch # Contributing ## How to help 1. Translate articles (especially to English) 2. Edit previous translations for grammar and technical errors 3. Suggest or create new articles 4. Share this project with your friends who want to contribute 5. It is recommended to use the typora-Markdown reader: https://typora.io/ as mentioned in Notes on Contributing ## Steps to contribute 1. Fork the project 2. Make changes 1. Reflect your name and affiliation when contributing in the format of 'Name-Affiliation' - eg: Daxie-West Lake University, or Arjun Krishna-University of Waterloo 3. Review your changes for grammatical or technical errors 4. Preview formatting using the Typora Markdown Editor and Reader 1. Recommended settings are in the section [Notes on Contributing](##notes-on-contributing) 5. Commit changes to your fork of the project 6. Create a pull request, clearly state what changes you made and why 7. Your changes will be reviewed 8. Congratulations! ## Notes on contributing - We recommend that you use the Typora Markdown Editor and Reader to preview your changes before you create a pull request - Recommended settings for Typora - Setting: File->Preference - Syntax Support - Inline Math - Subscript - Superscript - Highlight - Diagrams ## Example Contribution ### 3.3.2 How to find the optimal value of the hyperparameter? (Contributor: Daxie - Stanford University) There are always some difficult hyperparameters when using machine learning algorithms. For example, weight attenuation size, Gaussian kernel width, and so on. The algorithm does not set these parameters, but instead requires you to set their values. The set value has a large effect on the result. Common practices for setting hyperparameters are: 1. Guess and check: Select parameters based on experience or intuition, and iterate over. 2. Grid Search: Let the computer try to evenly distribute a set of values within a certain range. 3. Random search: Let the computer randomly pick a set of values. 4. Bayesian optimization: Using Bayesian optimization of hyperparameters, it is difficult to meet the Bayesian optimization algorithm itself. 5. Perform local optimization with good initial guessing: this is the MITIE method, which uses the BOBYQA algorithm and has a carefully chosen starting point. Since BOBYQA only looks for the nearest local optimal solution, the success of this method depends largely on whether there is a good starting point. In the case of MITIE, we know a good starting point, but this is not a universal solution, because usually you won't know where the good starting point is. On the plus side, this approach is well suited to finding local optimal solutions. I will discuss this later. 6. The latest global optimization method for LIPO. This method has no parameters and is proven to be better than a random search method. ## Communication 1. Create an issue using the Github Bug Tracker 2. Email the project at: scutjy2015@163.com 3. Join the project's WeChat: 1. It is easier to enter the group if you have made contributions and they have been integrated into this project 2. "Deep Learning 500 Questions" 1. please add WeChat Client 1: HQJ199508212176 Client 2: Xuwumin1203 Client 3: tianyuzy # Table of Contents Chapter 1 Mathematical Foundation 1 1.1 The relationship between scalars, vectors, and tensors 1 1.2 What is the difference between a tensor and matrix? 1 1.3 Matrix and vector multiplication results 1 1.4 Vector and matrix norm induction 1 1.5 How to judge a matrix to be positive? 2 1.6 Derivative Bias Calculation 3 1.7 What is the difference between 1.7 derivatives and partial derivatives? 3 1.8 Eigenvalue decomposition and feature vector 3 1.9 What is the relationship between singular values and eigenvalues? 4 1.10 Why should machine learning use probabilities? 4 1.11 What is the difference between a variable and a random variable? 4 1.12 Common probability distribution? 5 1.13 Example Understanding Conditional Probability 9 1.14 What is the difference between joint probability and edge probability? 10 1.15 Chain Law of Conditional Probability 10 1.16 Independence and conditional independence 11 1.17 Summary of Expectations, Variances, Covariances, Correlation Coefficients 11 Chapter 2 Fundamentals of Machine Learning 14 2.1 Various common algorithm illustrations 14 2.2 What is supervised learning, unsupervised learning, semi-supervised learning, and weak supervised learning? 15 2.3 What are the steps insupervised learning? 16 2.4 What is multi-instance learning? 17 2.5 What is the difference between classification networks and regression? 17 2.6 What is a neural network? 17 2.7 Advantages and Disadvantages of Common Classification Algorithms? 18 2.8 Is the correct rate good for evaluating classification algorithms? 20 2.9 How to evaluate a classification algorithm? 20 2.10 What kind of classifier is the best? 22 2.11 The relationship between big data and deep learning 22 2.12 Understanding Local Optimization and Global Optimization 23 2.13 Understanding Logistic Regression 24 2.14 What is the difference between logistic regression and a Naive Bayes Classifier? twenty four 2.15 Why do you need a cost function? 25 2.16 Principle of the function of the cost function 25 2.17 Why is the cost function non-negative? 26 2.18 Common cost function? 26 2.19 Why use cross entropy instead of quadratic cost function? 28 2.20 What is a loss function? 28 2.21 Common loss function 28 2.22 Why does logistic regression use a logarithmic loss function? 30 2.22 How does the logarithmic loss function measure loss? 31 2.23 Why do gradients need to be redu