大创项目 制造“隐形斗篷”.zip

# PyTorch-Deep-Learning ## 1.Overview **Requirements:** * Algebra + Probability * Python **Relationship:**  **Polular deep learning frameworks:** * Theano/Tensor Flow * Caffe/Caffe2 * Torch/PyTorch ## 2.线性模型 **Linear Model:** $\hat{y} = x * w + b$ **Training Loss(Error):** $loss = (\hat{y}-y)^2 = (x * w - y)^2$ **Mean Square Error:** $cost = \frac{1}{N}\sum\limits^{N}_{n=1}(\hat{y_n}-y_n)^2$ ## 3.梯度下降算法 **Optimization Problem:** $w^* = \underset{w}{arg min cost}(w)$ ### Gradient Descent **Gradient:** $\frac{\partial cost}{\partial w} = \frac{1}{N}\sum\limits^{N}_{n=1}2\cdot x_n\cdot (x_n \cdot w-y_n)$ **Update:** $w = w - \alpha \frac{\partial cost}{\partial w} = w - \alpha \frac{1}{N}\sum\limits^{N}_{n=1}2\cdot x_n\cdot (x_n \cdot w-y_n)$ ### Stochastic Gradient Descent **Gradient:** $\frac{\partial loss_n}{\partial w} = 2\cdot x_n\cdot (x_n \cdot w-y_n)$ **Update:** $w = w - \alpha \frac{\partial loss_n}{\partial w} = w - \alpha \cdot 2\cdot x_n\cdot (x_n \cdot w-y_n)$ Update weight by every grad of sample of train set. It is possible to jump over local optimum point when compared with GD algorithm. But the time complexity is larger. ## 4.反向传播  ### Tensor in PyTorch **Definition of TensorFlow:** A tensor is a generalization of vectors and matrices to potentially higher dimensions. * 点——标量(scalar)——零阶张量 * 线——向量(vector)——一阶张量 * 面——矩阵(matrix)——二阶张量 * 体——张量(tensor) In PyTorch, ***Tensor*** is the important component in constructing dynamic computational graph. Actually, it is a class object. It contains ***data*** and ***grad***, which storage the value of node and gradient w.r.t(abbreviation:with respect to) loss respectively.  ## 5.用PyTorch实现线性回归 In PyTorch, the computational graph is mini-batch fashion. **Steps:** 1. Prepare dataset 2. Design model using Class inherit from nn.module 3. Construct loss and optimizer using PyTorch API 4. Training cycle forward, backward, update **Different Optimizer in Linear Regression:** * torch.optim.Adagrad * torch.optim.Adam * torch.optim.Adamax * torch.optim.ASGD * torch.optim.LBFGS * torch.optim.RMSprop * torch.optim.Rprop * torch.optim.SGD ## 6.Logistics回归 **Logistic Function:** $\sigma(x) = \frac{1}{1 + e^{-x}}$ **Loss Function:** $loss = -(ylog\hat{y} + (1-y)log(1-\hat{y}))$ **Binary Classification Error:** $cost = -\frac{1}{N}\sum\limits_{n=1}^N y_nlog\hat{y_n}+(1-y_n)log(1-\hat{y_n})$ **Other Sigmoid Function:**  ## 7.处理多维特征的输入 ## 8.加载数据集 ## 9.多分类问题 ## 10.卷积神经网络（基础篇） ## 11.卷积神经网络（高级篇） ## 12.循环神经网络（基础篇） ## 13.循环神经网络（高级篇）