卷积神经网络资源-CSDN文库

4星 · 超过85%的资源需积分: 10 83 浏览量 2012-04-19 09:14:05 上传评论收藏 140KB PDF 举报

介绍关于卷积神经网络的文章 Convolutional neural networks (CNNs) have been successfully applied to many tasks such as digit and object recognition. ### 卷积神经网络（CNN）详解 #### 引言卷积神经网络(Convolutional Neural Networks,简称CNN)作为深度学习领域中的一个重要分支，在计算机视觉任务中展现出了卓越的表现能力，尤其是在数字识别、物体识别等领域。本文将详细介绍CNN的基本原理、结构特性以及其在图像处理中的应用。 #### 一、经典全连接网络中的反向传播在深入讨论CNN之前，我们首先回顾一下在全连接网络中如何进行反向传播。全连接网络是指网络中的每一层的每一个神经元都与前一层的所有神经元相连。这种网络结构虽然简单但却是构建更复杂模型的基础。 - **正向传播**：输入数据经过各层的权重矩阵乘法、偏置相加以及激活函数处理后，得到最终的输出。 - **损失计算**：通过比较网络的输出与实际目标值之间的差异来计算损失函数的值。 - **反向传播**：根据损失函数的梯度，自输出层向输入层逐层更新权重和偏置参数，以便最小化损失函数。在全连接网络中，每层的权重更新公式可以通过链式法则推导得出： \[ \Delta w_{ij} = -\eta \frac{\partial E}{\partial w_{ij}} \] 其中，\(E\)是损失函数，\(w_{ij}\)是第\(i\)个神经元到第\(j\)个神经元之间的权重，\(\eta\)是学习率。 #### 二、卷积神经网络的架构与原理与全连接网络不同，CNN利用了输入数据的空间结构特性，特别是在处理二维图像时。CNN的核心组成部分包括卷积层、池化层和全连接层等。 1. **卷积层**：通过一组可学习的滤波器对输入数据进行卷积操作，提取特征图(feature maps)。这些滤波器能够在空间上重复利用，减少了参数数量并实现了位置不变性。 2. **池化层**：通常位于卷积层之后，用于减少特征图的空间尺寸，从而降低计算量。常见的池化方法有最大池化(max pooling)和平均池化(average pooling)。 3. **全连接层**：在网络的末端，将卷积层和池化层提取的特征进一步处理，最后输出分类结果或回归预测。 #### 三、卷积神经网络中的反向传播在CNN中，反向传播的过程更加复杂，因为需要考虑到卷积和池化的特殊性。下面分别介绍这两个部分的反向传播过程： 1. **卷积层反向传播**：对于卷积层，需要计算每个滤波器的梯度，并更新滤波器的权重。由于卷积层的特殊结构，可以利用FFT快速傅里叶变换来加速计算。 2. **池化层反向传播**：池化层的反向传播主要关注于传递误差信号回上游层。例如，在最大池化中，只有产生最大值的位置会向后传递误差。 #### 四、高效实现技巧在实际应用中，为了提高CNN的训练效率，通常会采用以下几种技术手段： - **批归一化(Batch Normalization)**：通过归一化每批次的数据来加速训练过程，并减少过拟合。 - **残差连接(Residual Connections)**：引入残差块，使深层网络能够更好地学习恒等映射，从而避免梯度消失问题。 - **权重量化(Weight Quantization)**：减少模型大小，加快推理速度。 - **稀疏表示(Sparse Representation)**：学习如何自动组合先前层的特征图，尤其是学习稀疏组合，这有助于提高模型的解释性和泛化能力。 #### 结论通过对卷积神经网络原理及其实现细节的探讨，我们可以看到CNN不仅在理论上有坚实的数学基础，在实践中也展现了强大的性能优势。随着硬件技术的进步和算法优化，CNN将在更多的领域得到应用和发展。

资源推荐

资源详情

资源评论

Notes on Convolutional Neural Networks

Jake Bouvrie

Center for Biological and Computational Learning

Department of Brain and Cognitive Sciences

Massachusetts Institute of Technology

Cambridge, MA 02139

jvb@mit.edu

November 22, 2006

1 Introduction

This document discusses the derivation and implementation of convolutional neural networks

(CNNs) [3, 4], followed by a few straightforward extensions. Convolutional neural networks in-

volve many more connections than weights; the architecture itself realizes a form of regularization.

In addition, a convolutional network automatically provides some degree of translation invariance.

This particular kind of neural network assumes that we wish to learn ﬁlters, in a data-driven fash-

ion, as a means to extract features describing the inputs. The derivation we present is speciﬁc to

two-dimensional data and convolutions, but can be extended without much additional effort to an

arbitrary number of dimensions.

We begin with a description of classical backpropagation in fully connected networks, followed by a

derivation of the backpropagation updates for the ﬁltering and subsampling layers in a 2D convolu-

tional neural network. Throughout the discussion, we emphasize efﬁciency of the implementation,

and give small snippets of MATLAB code to accompany the equations. The importance of writing

efﬁcient code when it comes to CNNs cannot be overstated. We then turn to the topic of learning

how to combine feature maps from previous layers automatically, and consider in particular, learning

sparse combinations of feature maps.

Disclaimer: This rough note could contain errors, exaggerations, and false claims.

2 Vanilla Back-propagation Through Fully Connected Networks

In typical convolutional neural networks you might ﬁnd in the literature, the early analysis consists of

alternating convolution and sub-sampling operations, while the last stage of the architecture consists

of a generic multi-layer network: the last few layers (closest to the outputs) will be fully connected

1-dimensional layers. When you’re ready to pass the ﬁnal 2D feature maps as inputs to the fully

connected 1-D network, it is often convenient to just concatenate all the features present in all the

output maps into one long input vector, and we’re back to vanilla backpropagation. The standard

backprop algorithm will be described before going onto specializing the algorithm to the case of

convolutional networks (see e.g. [1] for more details).

2.1 Feedforward Pass

In the derivation that follows, we will consider the squared-error loss function. For a multiclass

problem with c classes and N training examples, this error is given by

n=1

k=1

− y

)

Here t

is the k-th dimension of the n-th pattern’s corresponding target (label), and y

is similarly

the value of the k-th output layer unit in response to the n-th input pattern. For multiclass classiﬁ-

cation problems, the targets will typically be organized as a “one-of-c” code where the k-th element

of t

is positive if the pattern x

belongs to class k. The rest of the entries of t

will be either zero

or negative depending on the choice of your output activation function (to be discussed below).

Because the error over the whole dataset is just a sum over the individual errors on each pattern, we

will consider backpropagation with respect to a single pattern, say the n-th one:

k=1

− y

)

− y

. (1)

With ordinary fully connected layers, we can compute the derivatives of E with respect to the net-

work weights using backpropagation rules of the following form. Let ` denote the current layer,

with the output layer designated to be layer L and the input “layer” designated to be layer 1. Deﬁne

the output of this layer to be

= f(u

), with u

= W

`−1

+ b

(2)

where the output activation function f (·) is commonly chosen to be the logistic (sigmoid) function

f(x) = (1 + e

−βx

)

−1

or the hyperbolic tangent function f(x) = a tanh(bx). The logistic function

maps [−∞, +∞] → [0, 1], while the hyperbolic tangent maps [−∞, +∞] → [−a, +a]. Therefore

while the outputs of the hyperbolic tangent function will typically be near zero, the outputs of a

sigmoid will be non-zero on average. However, normalizing your training data to have mean 0 and

variance 1 along the features can often improve convergence during gradient descent [5]. With a

normalized dataset, the hyperbolic tangent function is thus preferrable. LeCun recommends a =

1.7159 and b = 2/3, so that the point of maximum nonlinearity occurs at f(±1) = ±1 and will thus

avoid saturation during training if the desired training targets are normalized to take on the values

±1 [5].

2.2 Backpropagation Pass

The “errors” which we propagate backwards through the network can be thought of as “sensitivities”

of each unit with respect to perturbations of the bias

. That is to say,

∂E

∂b

∂E

∂u

∂b

= δ (3)

since in this case

∂u

∂b

= 1. So the bias sensitivity and the derivative of the error with respect to a

unit’s total input is equivalent. It is this derivative that is backpropagated from higher layers to lower

layers, using the following recurrence relation:

= (W

`+1

)

`+1

◦ f

) (4)

where “◦” denotes element-wise multiplication. For the error function (1), the sensitivities for the

output layer neurons will take a slightly different form:

= f

) ◦ (y

− t

Finally, the delta rule for updating a weight assigned to a given neuron is just a copy of the inputs

to that neuron, scaled by the neuron’s delta. In vector form, this is computed as an outer product

between the vector of inputs (which are the outputs from the previous layer) and the vector of

sensitivities:

∂E

∂W

= x

`−1

(δ

)

(5)

∆W

= −η

∂E

∂W

(6)

with analogous expressions for the bias update given by (3). In practice there is often a learning rate

parameter η

speciﬁc to each weight (W)

This nifty interpretation is due to Sebastian Seung

剩余7页未读，继续阅读

评论收藏

内容反馈

锦轩

2014-02-20

基础学习还是不错的。
潮浪大地

2014-07-11

入门了解还是可以的。
s06221098

2013-05-14

这篇论文一般，lecun的一些文章比较好
lihbo0523

2013-08-05

卷积神经网络的介绍，英文的，正在研读。
hspHit

2013-12-04

挺好，正在读。适合有一定基础的。

前往

页

Bomber

粉丝: 15
资源: 12

卷积神经网络

卷积神经网络（CNN）

卷积神经网络概述

卷积神经网络pdf讲义超详细

卷积神经网络ppt.pptx

卷积神经网络讲解PPT

深度学习——卷积神经网络

Awesome-Graph-Neural-Networks卷积神经网络_卷积神经网络_卷积网络_

task05–卷积神经网络

CNN卷积神经网络

卷积神经网络入门

卷积神经网络（CNN）.pdf

CNN卷积神经网络介绍

卷积神经网络结构设计及其在目标检测中的应用

Visio画卷积神经网络

用卷积滤波器matlab代码-PsiDONet:卷积神经网络用于有限角度层析成像重建

1.4卷积神经网络

卷积神经网络报告

机器学习之卷积神经网络学习

标准CNN（卷积神经网络）

卷积神经网络的工作原理

最新资源