---
title: LeNet MNIST Tutorial
description: Train and test "LeNet" on the MNIST handwritten digit data.
category: example
include_in_docs: true
priority: 1
---
# Training LeNet on MNIST with Caffe
We will assume that you have Caffe successfully compiled. If not, please refer to the [Installation page](/installation.html). In this tutorial, we will assume that your Caffe installation is located at `CAFFE_ROOT`.
## Prepare Datasets
You will first need to download and convert the data format from the MNIST website. To do this, simply run the following commands:
cd $CAFFE_ROOT
./data/mnist/get_mnist.sh
./examples/mnist/create_mnist.sh
If it complains that `wget` or `gunzip` are not installed, you need to install them respectively. After running the script there should be two datasets, `mnist_train_lmdb`, and `mnist_test_lmdb`.
## LeNet: the MNIST Classification Model
Before we actually run the training program, let's explain what will happen. We will use the [LeNet](http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf) network, which is known to work well on digit classification tasks. We will use a slightly different version from the original LeNet implementation, replacing the sigmoid activations with Rectified Linear Unit (ReLU) activations for the neurons.
The design of LeNet contains the essence of CNNs that are still used in larger models such as the ones in ImageNet. In general, it consists of a convolutional layer followed by a pooling layer, another convolution layer followed by a pooling layer, and then two fully connected layers similar to the conventional multilayer perceptrons. We have defined the layers in `$CAFFE_ROOT/examples/mnist/lenet_train_test.prototxt`.
## Define the MNIST Network
This section explains the `lenet_train_test.prototxt` model definition that specifies the LeNet model for MNIST handwritten digit classification. We assume that you are familiar with [Google Protobuf](https://developers.google.com/protocol-buffers/docs/overview), and assume that you have read the protobuf definitions used by Caffe, which can be found at `$CAFFE_ROOT/src/caffe/proto/caffe.proto`.
Specifically, we will write a `caffe::NetParameter` (or in python, `caffe.proto.caffe_pb2.NetParameter`) protobuf. We will start by giving the network a name:
name: "LeNet"
### Writing the Data Layer
Currently, we will read the MNIST data from the lmdb we created earlier in the demo. This is defined by a data layer:
layer {
name: "mnist"
type: "Data"
transform_param {
scale: 0.00390625
}
data_param {
source: "mnist_train_lmdb"
backend: LMDB
batch_size: 64
}
top: "data"
top: "label"
}
Specifically, this layer has name `mnist`, type `data`, and it reads the data from the given lmdb source. We will use a batch size of 64, and scale the incoming pixels so that they are in the range \[0,1\). Why 0.00390625? It is 1 divided by 256. And finally, this layer produces two blobs, one is the `data` blob, and one is the `label` blob.
### Writing the Convolution Layer
Let's define the first convolution layer:
layer {
name: "conv1"
type: "Convolution"
param { lr_mult: 1 }
param { lr_mult: 2 }
convolution_param {
num_output: 20
kernel_size: 5
stride: 1
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
}
}
bottom: "data"
top: "conv1"
}
This layer takes the `data` blob (it is provided by the data layer), and produces the `conv1` layer. It produces outputs of 20 channels, with the convolutional kernel size 5 and carried out with stride 1.
The fillers allow us to randomly initialize the value of the weights and bias. For the weight filler, we will use the `xavier` algorithm that automatically determines the scale of initialization based on the number of input and output neurons. For the bias filler, we will simply initialize it as constant, with the default filling value 0.
`lr_mult`s are the learning rate adjustments for the layer's learnable parameters. In this case, we will set the weight learning rate to be the same as the learning rate given by the solver during runtime, and the bias learning rate to be twice as large as that - this usually leads to better convergence rates.
### Writing the Pooling Layer
Phew. Pooling layers are actually much easier to define:
layer {
name: "pool1"
type: "Pooling"
pooling_param {
kernel_size: 2
stride: 2
pool: MAX
}
bottom: "conv1"
top: "pool1"
}
This says we will perform max pooling with a pool kernel size 2 and a stride of 2 (so no overlapping between neighboring pooling regions).
Similarly, you can write up the second convolution and pooling layers. Check `$CAFFE_ROOT/examples/mnist/lenet_train_test.prototxt` for details.
### Writing the Fully Connected Layer
Writing a fully connected layer is also simple:
layer {
name: "ip1"
type: "InnerProduct"
param { lr_mult: 1 }
param { lr_mult: 2 }
inner_product_param {
num_output: 500
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
}
}
bottom: "pool2"
top: "ip1"
}
This defines a fully connected layer (known in Caffe as an `InnerProduct` layer) with 500 outputs. All other lines look familiar, right?
### Writing the ReLU Layer
A ReLU Layer is also simple:
layer {
name: "relu1"
type: "ReLU"
bottom: "ip1"
top: "ip1"
}
Since ReLU is an element-wise operation, we can do *in-place* operations to save some memory. This is achieved by simply giving the same name to the bottom and top blobs. Of course, do NOT use duplicated blob names for other layer types!
After the ReLU layer, we will write another innerproduct layer:
layer {
name: "ip2"
type: "InnerProduct"
param { lr_mult: 1 }
param { lr_mult: 2 }
inner_product_param {
num_output: 10
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
}
}
bottom: "ip1"
top: "ip2"
}
### Writing the Loss Layer
Finally, we will write the loss!
layer {
name: "loss"
type: "SoftmaxWithLoss"
bottom: "ip2"
bottom: "label"
}
The `softmax_loss` layer implements both the softmax and the multinomial logistic loss (that saves time and improves numerical stability). It takes two blobs, the first one being the prediction and the second one being the `label` provided by the data layer (remember it?). It does not produce any outputs - all it does is to compute the loss function value, report it when backpropagation starts, and initiates the gradient with respect to `ip2`. This is where all magic starts.
### Additional Notes: Writing Layer Rules
Layer definitions can include rules for whether and when they are included in the network definition, like the one below:
layer {
// ...layer definition...
include: { phase: TRAIN }
}
This is a rule, which controls layer inclusion in the network, based on current network's state.
You can refer to `$CAFFE_ROOT/src/caffe/proto/caffe.proto` for more information about layer rules and model schema.
In the above example, this layer will be included only in `TRAIN` phase.
If we change `TRAIN` with `TEST`, then this layer will be used only in test phase.
By default, that is without layer rules, a layer is always included in the network.
Thus, `lenet_train_test.prototxt` has two `DATA` layers defined (with different `batch_size`), one for the training phase and one for the testing phase.
Also, there is an `Accuracy` layer which is included only in `TEST` phase for reporting the model accuracy every 100 iteration, as defined in `lenet_solver.prototxt`.
## Define t
没有合适的资源?快使用搜索试试~ 我知道了~
资源推荐
资源详情
资源评论
收起资源包目录
填完坑,配置好并写完bat的caffe深度学习 (325个子文件)
检测.bat 166B
获取binaryproto文件.bat 93B
分类器生成.bat 82B
生成leveldb.bat 50B
train.bin 146.58MB
test.bin 29.32MB
data_batch_2.bin 29.31MB
data_batch_5.bin 29.31MB
data_batch_4.bin 29.31MB
data_batch_3.bin 29.31MB
data_batch_1.bin 29.31MB
test_batch.bin 29.31MB
mean.binaryproto 12KB
cifar10_quick_iter_4000.caffemodel 570KB
CaffeTargets.cmake 4KB
CaffeTargets-release.cmake 3KB
CaffeConfig.cmake 2KB
classification.cpp 8KB
convert_mnist_data.cpp 4KB
convert_mnist_siamese_data.cpp 4KB
convert_cifar_data.cpp 4KB
gpu_util.cuh 1KB
CURRENT 16B
CURRENT 16B
caffe.VC.db 220KB
libopenblas.dll 36.56MB
libopenblas.dll 36.56MB
opencv_imgproc310.dll 24.39MB
opencv_imgproc310.dll 24.39MB
opencv_core310.dll 10.54MB
opencv_core310.dll 10.54MB
python35.dll 3.74MB
python35.dll 3.74MB
opencv_imgcodecs310.dll 2.63MB
opencv_imgcodecs310.dll 2.63MB
caffehdf5.dll 2.2MB
caffehdf5.dll 2.2MB
libgfortran-3.dll 1.22MB
libgfortran-3.dll 1.22MB
CONCRT140.dll 327KB
CONCRT140.dll 327KB
libquadmath-0.dll 324KB
libquadmath-0.dll 324KB
boost_python-vc140-mt-1_61.dll 276KB
boost_python-vc140-mt-1_61.dll 276KB
gflags.dll 137KB
gflags.dll 137KB
boost_filesystem-vc140-mt-1_61.dll 136KB
boost_filesystem-vc140-mt-1_61.dll 136KB
boost_thread-vc140-mt-1_61.dll 113KB
boost_thread-vc140-mt-1_61.dll 113KB
glog.dll 112KB
glog.dll 112KB
caffehdf5_hl.dll 102KB
caffehdf5_hl.dll 102KB
VCRUNTIME140.dll 86KB
VCRUNTIME140.dll 86KB
caffezlib1.dll 81KB
libgcc_s_seh-1.dll 81KB
caffezlib1.dll 81KB
libgcc_s_seh-1.dll 81KB
boost_chrono-vc140-mt-1_61.dll 33KB
boost_chrono-vc140-mt-1_61.dll 33KB
boost_system-vc140-mt-1_61.dll 24KB
boost_system-vc140-mt-1_61.dll 24KB
caffe.exe 3.01MB
extract_features.exe 2.98MB
classification.exe 2.98MB
convert_imageset.exe 2.96MB
compute_image_mean.exe 2.96MB
upgrade_solver_proto_text.exe 2.96MB
convert_mnist_data.exe 2.96MB
convert_cifar_data.exe 2.96MB
convert_mnist_siamese_data.exe 2.96MB
upgrade_net_proto_text.exe 2.96MB
upgrade_net_proto_binary.exe 2.95MB
train_net.exe 2.95MB
test_net.exe 2.95MB
finetune_net.exe 2.95MB
net_speed_benchmark.exe 2.95MB
device_query.exe 2.95MB
caffe.vcxproj.filters 826B
flickr_style.csv.gz 2.08MB
caffe.pb.h 937KB
signal_handler.h 640B
layer.hpp 17KB
net.hpp 12KB
test_gradient_check_util.hpp 11KB
filler.hpp 11KB
math_functions.hpp 9KB
blob.hpp 9KB
recurrent_layer.hpp 7KB
base_conv_layer.hpp 7KB
infogain_loss_layer.hpp 6KB
common.hpp 6KB
include_symbols.hpp 6KB
lstm_layer.hpp 6KB
softmax_loss_layer.hpp 6KB
sigmoid_cross_entropy_loss_layer.hpp 5KB
cudnn.hpp 5KB
共 325 条
- 1
- 2
- 3
- 4
资源评论
- yunzhou20152017-11-09烂就一个字
_寒潭雁影
- 粉丝: 2425
- 资源: 41
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
最新资源
- 基于STM8S103F3P6+STM8S207C8T6+STM32F103 单片机三合一最小系统开发板硬件(原理图+PCB)工程
- 基于C语言实现的打印杨辉三角
- 基于ASIO的插件式服务器,支持TCP,UDP,串口,Http,Websocket统一化的数据接口,隔离开发人员和IO之间的操作
- stm32 usb接口通信
- Chessmate是一款完全免费的国际象棋学习软件,支持引擎分析,学开局、残局、棋书解读、大数据分析等功能
- 总结整理的Android面试Java基础知识点面试资料精编汇总文档资料合集.zip
- .android_lq
- FDN5632N-VB一款SOT23封装N-Channel场效应MOS管
- 毛老板-2404250902.amr
- Java类加载流程(双亲委派)流程图.zip
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功