CaffeCN 深度学习社区 | http://caffecn.cn
1
目 录
前言........................................................................................................................................... 4
第一章 Blobs, Layers, and Nets：Caffe 模型解析 ........................................................ 5
1.1 Blob 的存储与交换 ................................................................................................. 5
1.1.1 实现细节 ......................................................................................................... 6
1.2 Layer 的计算和连接 ................................................................................................ 7
1.3 Net 的定义和操作 ...................................................................................................... 8
1.3.1 模型格式 ....................................................................................................... 11
第二章 Forward and Backward（前传/反传） .................................................................. 13
2.1 前传.......................................................................................................................... 13
2.2 反传.......................................................................................................................... 14
2.3 Caffe 中前传和反传的实现...................................................................................... 14
第三章 Loss .......................................................................................................................... 16
3.1 Loss weights .............................................................................................................. 16
第四章 Solver ......................................................................................................................... 18
4.1 Solver 简介 ............................................................................................................. 18
4.2 Methods .................................................................................................................... 18
4.2.1 SGD ................................................................................................................. 19
4.2.2 AdaDelta ......................................................................................................... 21
4.2.3 AdaGrad.......................................................................................................... 21
4.2.4 Adam .............................................................................................................. 21
4.2.5 NAG ................................................................................................................ 22
4.2.6 RMSprop ......................................................................................................... 23
第五章 Layer Cataloge ........................................................................................................... 24
5.1 视觉层 Vision Layers ............................................................................................... 24
5.1.1 卷积 Convolution ......................................................................................... 24
5.1.2 池化 Pooling ................................................................................................. 26
5.1.3 局部响应值归一化 Local Response Normalization (LRN) ............................ 27
5.1.4 im2col ............................................................................................................. 28
5.2 损失层 Loss Layers .................................................................................................. 28
5.2.1 Softmax 损失 ............................................................................................. 28
5.2.2 平方和/欧式损失 Sum-of-Squares / Euclidean ........................................... 28
5.2.3 Hinge / Margin 损失 .................................................................................. 29
5.2.3 交叉熵损失 Sigmoid Cross-Entropy ............................................................ 30
5.2.4 信息熵损失 Infogain .................................................................................... 30
5.2.5 准确率 Accuracy and Top-k ......................................................................... 30
5.3 激活层 Activation / Neuron Layers ....................................................................... 30
5.3.1 ReLU / Rectified-Linear and Leaky-ReLU ........................................................ 30
5.3.2. Sigmoid .......................................................................................................... 31
5.3.3 TanH / Hyperbolic Tangent ............................................................................. 32
5.3.4 Absolute Value ............................................................................................... 32
