cs230讲义-super-cheatsheet-deep-learning
需积分: 10 40 浏览量
2018-11-29
14:43:43
上传
评论
收藏 5.28MB PDF 举报
CS 230 – Deep Learning Shervine Amidi & Afshine Amidi
Super VIP Cheatsheet: Deep Learning
Afshine Amidi and Shervine Amidi
November 25, 2018
Contents
1 Convolutional Neural Networks 2
1.1 Overview ................................. 2
1.2 Types of layer .............................. 2
1.3 Filter hyperparameters .......................... 2
1.4 Tuning hyperparameters ......................... 3
1.5 Commonly used activation functions ................... 3
1.6 Object detection ............................. 4
1.6.1 Face verification and recognition ................. 5
1.6.2 Neural style transfer ....................... 5
1.6.3 Architectures using computational tricks ............ 6
2 Recurrent Neural Networks 7
2.1 Overview ................................. 7
2.2 Handling long term dependencies .................... 8
2.3 Learning word representation ...................... 9
2.3.1 Motivation and notations . . . . . . . . . . . . . . . . . . . 9
2.3.2 Word embeddings . . . . . . . . . . . . . . . . . . . . . . . 9
2.4 Comparing words . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.5 Language model ............................. 10
2.6 Machine translation ........................... 10
2.7 Attention ................................. 10
3 Deep Learning Tips and Tricks 11
3.1 Data processing ............................. 11
3.2 Training a neural network ........................ 12
3.2.1 Definitions ............................ 12
3.2.2 Finding optimal weights ..................... 12
3.3 Parameter tuning ............................ 12
3.3.1 Weights initialization ...................... 12
3.3.2 Optimizing convergence ..................... 12
3.4 Regularization .............................. 13
3.5 Go od practices .............................. 13
1 Convolutional Neural Networks
1.1 Overview
r Architecture of a traditional CNN –Convolutionalneuralnetworks,alsoknownasCNNs,
are a specific type of neural networks that are generally composed of the following layers:
The convolution layer and the pooling layer can b e fine-tuned with respect to hyperparameters
that are described in the next sections.
1.2 Types of layer
r Convolutional layer (CONV) –Theconvolutionlayer(CONV)usesfiltersthatperform
convolution op erations as it is scanning the input I with respect to its dimensions. Its hyperpa-
rameters include the filter size F and stride S.TheresultingoutputO is called feature map or
activation map.
Remark: the convolution step can be generalized to the 1D and 3D cases as well.
r Pooling (POOL) –Thepoolinglayer(POOL)isadownsamplingoperation,typicallyapplied
after a convolution layer, which does some spatial invariance. In particular, max and average
pooling are special kinds of pooling where the maximum and average value is taken, respectively.
Stanford University 1 Winter 2019
剩余12页未读，继续阅读
评论0
最新资源