迁移学习相关最新SCI文献资源-CSDN文库

共10个文件

pdf：10个

深度学习

迁移学习

SCI

需积分: 14 359 浏览量 2018-06-14 08:55:38 上传评论收藏 15.79MB ZIP 举报

身份认证购VIP最低享 7 折!

领优惠券(最高得80元）

资源推荐

资源详情

资源评论

收起资源包目录

Transfer learning.zip （10个子文件）

Transfer Learning for Brain Decoding using Deep.pdf 193KB

Transfer learning approach for classiﬁcation and noise reduction on noisy web data.pdf 3.76MB

A Survey on Transfer Learning.pdf 2.47MB

nature_deep-learning.pdf 1.99MB

Deep learning algorithms for human activity recognition .pdf 2.31MB

Unsupervised Transfer Learning via Multi-Scale.pdf 1.44MB

A deeply supervised residual network for HEp-2 cell classiﬁcation via cross-modal transfer learning.pdf 3.91MB

Real-time human activity recognition from accelerometer data using cnn.pdf 975KB

Learning with rethinking Recurrently improving convolutional neural networks through feedback.pdf 2.88MB

Do CIFAR-10 Classifiers Generalize to CIFAR-10.pdf 965KB

Pattern Recognition 79 (2018) 290–302

Contents lists available at ScienceDirect

Pattern Recognition

journal homepage: www.elsevier.com/locate/patcog

A deeply supervised residual network for HEp-2 cell classiﬁcation via

cross-modal transfer learning

Haijun Lei

, Tao Han

, Feng Zhou

, Zhen Yu

, Jing Qin

c , d

, Ahmed Elazab

c , e

, Baiying Lei

c , ∗

Guangdong Province Key Laboratory of Popular High-performance Computers, School of Computer and Software Engineering, Shenzhen University,

Shenzhen 518060, China

Industrial and Manufacturing Systems Engineering, University of Michigan at Dearborn, 4901 Evergreen Road, Dearborn, MI 48128, United States

School of Biomedical Engineering, Health Science Center, Shenzhen University, National-Regional Key Technology Engineering Laboratory for Medical

Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, Nanhai Ave 3688, Shenzhen, Guangdong 518060, China

Centre for Smart Health, School of Nursing, The Hong Kong Polytechnic University, Hong Kong

Computer Science Department, Misr Higher Institute for Commerce and Computers, Mansoura 35516, Egypt

a r t i c l e i n f o

Article history:

Received 1 October 2017

Revised 4 January 2018

Accepted 10 February 2018

Available online 15 February 2018

Keywords:

HEp-2 cell classiﬁcation

Residual network

Deeply supervised ResNet

Cross-modal transfer learning

a b s t r a c t

Accurate Human Epithelial-2 (HEp-2) cell image classiﬁcation plays an important role in the diagnosis

of many autoimmune diseases and subsequent treatment. One of the key challenges is huge intra-class

variations caused by inhomogeneous illumination. To address it, we propose a framework based on very

deep supervised residual network (DSRN) to classify HEp-2 cell images. Speciﬁcally, we adopt a resid-

ual network of 50 layers (ResNet-50) that is substantially deep to extract rich and discriminative fea-

tures. The deep supervision is imposed on the ResNet-based framework to further boost the classiﬁcation

performance by directly guiding the training of the lower and upper levels of the network. The pro-

posed method is evaluated using two publicly available datasets (i.e., International Conference on Pattern

Recognition (ICPR) 2012 and ICPR2016-Task1 cell classiﬁcation contest datasets). Different from the pre-

vious deep learning models learned from scratch, a cross-modal transfer learning strategy is developed.

Namely, we pretrain ICPR2012 dataset to ﬁne-tune ICPR2016 dataset based on our DSRN model since

both datasets are similar. Extensive experiments show that the proposed method delivers state-of-the-

art performance and outperforms the traditional methods based on deep convolutional neural network

(DCNN).

1. Introduction

Indirect immunoﬂuorescence detection technology is mainly

used for the analysis of anti-nuclear antibody (ANA) [1–4] , autoim-

mune disease diagnosis and treatment. For the ANA analysis, the

HEp-2 cell is able to divide and produce a large number of anti-

gens. Human experts usually employ a ﬂuorescence microscope to

discriminate the nuclear antibody via visual inspection. As shown

in Fig. 1 , the inhomogeneous illumination leads to huge intra-

class variations, which make the HEp-2 cell classiﬁcation chal-

lenging. In order to address such challenge, traditional methods

mainly involve three steps: feature extraction, feature encoding,

and classiﬁcation [5–26] . However, these methods mainly focus

on hand-crafted features, which suffer from limited classiﬁcation

∗

Corresponding author.

E-mail addresses: lhj@szu.edu.cn (H. Lei), 2151230219@email.szu.edu.cn (T. Han),

fezhou@umich.edu (F. Zhou), yishon@email.unc.edu (Z. Yu), harry.qin@polyu.edu.hk

(J. Qin), leiby@szu.edu.cn (B. Lei).

performance. To further improve the feature representation capa-

bilities for classiﬁcation enhancement, it is important to effectively

extract more discriminative and informative features.

In this respect, deep convolutional neural network (DCNN)

has attracted considerable interests. The DCNN has strong fea-

ture representation capabilities by end-to-end learning using var-

ious network layers. Due to the availability of large-scale anno-

tated datasets (e.g. ImageNet) [27] and their powerful representa-

tion capabilities, DCNN is able to improve the classiﬁcation per-

formance signiﬁcantly [28,29] . For example, Phan et al. [30] pro-

posed a classiﬁcation system to extract features using a pre-trained

DCNN model based on AlexNet architecture. Multi-class support

vector machine (SVM) is used as a classiﬁer to evaluate Interna-

tional Conference on Pattern Recognition (ICPR) 2012 dataset. Gao

et al. [31] proposed a DCNN model to recognize HEp-2 cells using

the classical LeNet-5 [32] . Bayramoglu et al. [17] proposed a pre-

training strategy to ﬁne-tune the CNN network. However, features

learned automatically by traditional DCNN algorithms are still lim-

ited [33] . Besides, some features in the early layers demonstrate

https://doi.org/10.1016/j.patcog.2018.02.006

H. Lei et al. / Pattern Recognition 79 (2018) 290–302 291

Homogeneous Speckled Golgi

Centromere

Nucleolar NuMem

CentromereHomogeneous

Nucleolar

Coarse-speckled

Fine-speckled

Cytoplasmic

RPCI2012 RPCI2016-Task1

Fig. 1. Examples of HEp-2 (ICPR2012 and ICPR2016-Task1 datasets) cell with various patterns.

certain degree of opacity [34] . It is also observed that, different ini-

tializations of the feature learning at the early layers make negligi-

ble difference to the ﬁnal classiﬁcation [20] . In addition, the pres-

ence of vanishing gradients also makes the training via DCNN slow

and ineffective [21] , which are not discriminative enough for HEp-2

cell classiﬁcation [35] . To address this limitation and further boost

feature representation ability of DCNN, numerous studies with very

deep representations are proposed such as 19 layers of VGG-Net

[36] , 30 layers of batch normalization (BN) Net [37] , and even 152

layers of ResNet [25] . As a main structure of ResNet, residual con-

nection can solve the degradation problem effectively. Instead of

simply stacking layer by layer, a residual mapping is developed to

explicitly ﬁt a desired underlying mapping by directly using these

layers. The reason is that optimizing the residual mapping is easier

than optimizing the original and unreferenced mapping. If an iden-

tity mapping is optimal, it would be easier to push the residual to

zero than to ﬁt an identity mapping by a stack of nonlinear layers

[38,39] .

Despite the great success achieved by ResNet based framework,

it is argued that deeply supervised mechanism [35] is able to fur-

ther improve the classiﬁcation results. The main reason is that

this strategy formulates an objective function to directly guide the

training of both upper and lower layers, which can facilitate the

propagation of gradient ﬂows within the network. Therefore, more

powerful and representative features can be learned. The objective

function in the individual hidden layers is incorporated as an ad-

ditional constraint (i.e., a new regularization model). For this rea-

son, we utilize a very deeply supervised residual network (DSRN)

to classify HEp-2 cell.

Due to the deep representation and a small number of la-

beled medical images available, the overﬁtting problem likely oc-

curs. To address it, the transfer learning strategy is an effective

way. Different from the traditional transfer learning techniques us-

ing natural images [30,40] , we propose a new cross-modal trans-

fer learning technique with a similar medical dataset. That is,

we pre-train the network on ICPR2012 dataset to boost HEp-2

classiﬁcation performance on ICPR2016-Task1 dataset due to the

similarity between ICPR2012 and ICPR 2016 -Task1 datasets. A

cross-modal based transfer learning via ResNet-50 network is de-

sired because not only the expression characteristics in the low-

level representation are similar, but also high-level features are

alike. Finally, our proposed method is evaluated using two pub-

licly available datasets, ICPR2012 and ICPR2016-Task1 datasets, and

extensive experiments show that our proposed method outper-

forms the traditional methods. Also, we use different visualization

methods to demonstrate visual patterns learned by our proposed

network.

2. Related work

2.1. Hand-crafted feature representation and classiﬁcation

The traditional hand-crafted features for the classiﬁcation of

HEp-2 cell generally consist of three steps: feature extraction, fea-

ture encoding, and feature classiﬁcation. Carefully designed data-

speciﬁc features are particularly important in the classiﬁcation.

For example, Shen et al. [41] proposed a rotation invariant co-

occurrence among adjacent local binary pattern (RIC-LBP) image

feature and a linear support vector machine (SVM), Phan et al.

[30] proposed a morphological features of stained patterns (MF),

Grangnaniello et al. [42] proposed a dense local descriptor invari-

ant to scale changes and rotations (DLDI), and Zhao et al. [32] pro-

posed a clustered multi-task learning.

2.2. Deep convolutional neural network

DCNN uses two-dimensional discrete convolution for image

processing to mimic human neural network operation, which has

a similar structure to the hierarchical model of the visual nervous

system. The CNN was ﬁrst proposed by Rumelhart et al. [43] us-

ing the back propagation (BP) algorithm, which renders the train-

ing of neural networks simple and feasible. Subsequently, LeCun

et al. [44] used the BP algorithm to train the model parameters

of CNNs and achieved an excellent performance in hand-written

digit recognition. Generally, DCNN model learns different levels of

features in different layers to obtain highly distinct and effective

deep representation [45] automatically without any manual work.

The DCNN integrates a local connection strategy with a weight

sharing mechanism to reduce the number of network parameters.

Therefore, DCNN has achieved signiﬁcant performance in a myriad

of applications [46,47] due to its powerful representation. For in-

stance, Gao et al. [31] proposed a DCNN model to recognize HEp-

2 cell based on the architecture of the classical LeNet-5 [44] . Xi

et al. [41] proposed an improvement of the DCNN with 22 layers.

Recently, very deep representations of neural networks have been

proposed in order to further improve the performance, such as 19

layers of network of VGG-Net [48] , Bengio [40] proposed method

by utilizing very deep convolutional networks (VGGNet), and 30

layers of the BN Net [49] , and even 152 layers ResNet (ResNet-152)

[50] .

2.3. Residual network

When the number of network layers increases, there is a prob-

lem of vanishing/exploding gradients [51] , which makes it hard for

292 H. Lei et al. / Pattern Recognition 79 (2018) 290–302

the network to converge. Even though such problem is alleviated

by batch normalization [49] , its accuracy gets saturated and then

deteriorates rapidly when the network starts to converge. To deal

with this issue [49,52] , He et al. [50] proposed a deep residual

learning framework by adding identity shortcuts between layers

of traditional CNNs. Within a residual block, it is easier to opti-

mize the residual mapping than to optimize the original one. Due

to the short-cut connections, the data ﬂow between the networks

is smooth [50] . However, ResNet still has some limitations. For ex-

ample, the ordinary depth feed forward network is diﬃcult to op-

timize. In addition, the additional layer increases the error rate of

training and validation because of the depth. Also, the residual net-

work structure can increase the sum of the outputs in the stacked

layer. Furthermore, the response variance of residual network in

the lower layers is approximately zero, which is too small to be

stable. Namely, the corresponding residual structure may be sim-

ilar to the unit mapping. However, with the increase of network

layers, the network may lose some features for feature representa-

tion. Although residual mapping can avoid the problem of vanish-

ing/exploding gradients [51] , it still cannot solve the problem. For

this reason, a deep supervision mechanism is added to the residual

network to increase the expression of features.

2.4. Cross-modal transfer learning

The most common problem in the domain of medical imaging

processing is that, the number of the samples in a training dataset

is limited. Hence, overﬁtting likely occurs in fully supervised deep

architectures, which makes it hard to generalize the classiﬁcation

model [53] . To solve the overﬁtting problem, transfer learning is

an effective way by sharing parameters from other large datasets.

Hence, it has been widely applied and has achieved promising

performance [53] . Transfer learning can accelerate model training,

save memory, and avoid repetition of existing similar problems

[54,55] as well. The traditional transfer learning is usually used in

the new dataset to update the last layer of the network weights of

VGG-Net [48] or Google-Net [54,56] . Most previous models were

pre-trained on the natural image dataset, i.e. ImageNet.

3. Methodology

Fig. 2 shows the overall architecture of our proposed method.

In the following, we describe it in detail.

3.1. Deep residual network

ResNet is a CNN-based network with deep hierarchical architec-

ture and residual connection. Due to the effectiveness of ResNet, it

has achieved promising performance in many applications. Com-

pared to the traditional DCNN network structure, ResNet utilizes a

residual connection to address the degradation problem by train-

ing a very deep network. Different from the previous studies, a

large number of layers are added to the residual network to en-

hance the performance. The residual network has lower conver-

gence loss without overﬁtting. The residual connection can acceler-

ate the convergence of deep layers [38] . The ResNet maintains pre-

cision by substantially increasing the network depth. Speciﬁcally, a

deep residual network is composed of a series of residual blocks

where each is composed of several stacked convolutional layers

(we regard rectiﬁed linear unit layer (ReLu) and the BN layers as

the component of convolutional layer). A residual block is formu-

lated as

l+1

= ReLu

(

+ f

(

, w

) )

, (1)

where h

and h

l+1

represent the input and the output of the l th

residual block, respectively. Relu ( · ) is the rectiﬁed linear unit func-

Table 1

Architecture of ResNet-50 used in this work.

Layer/Residual block Parameter (kernels, channels)

Conv1 7 × 7, 64

Max pooling, 3 × 3, 64

Conv2_x ResBlock1–3

⎡

⎣

1 × 1 , 64

3 × 3 , 64

1 × 1 , 256

⎤

⎦

× 3

Conv3_x ResBlock4–7

⎡

⎣

1 × 1 , 128

3 × 3 , 128

1 × 1 , 512

⎤

⎦

× 4

Conv4_x ResBlock8–13

⎡

⎣

1 × 1 , 256

3 × 3 , 256

1 × 1 , 1024

⎤

⎦

× 6

Conv5_x ResBlock14–16

⎡

⎣

1 × 1 , 512

3 × 3 , 512

1 × 1 , 2048

⎤

⎦

× 3

Average pooling, 7 × 7, 2048

FC (fully connected), softmax output, 1 × 1, 10 0 0

tion, f ( · ) is the residual mapping function, and w

are the param-

eters of the residual block. When f ( h

, w

) and h

are unequal, a

linear mapping A

is used to match the dimension and Eq. (2) is

reformulated as

l+1

= ReLu

(

+ f

(

, w

) )

. (2)

The most important component of the residual block is the con-

volutional layer. It contains a series of neurons, and each contains

a set of learnable weights and a bias. These weights are constantly

updated in the network training. The convolutional layers take lo-

cal receptive ﬁelds of feature map in the previous layer as input to

preserve spatial information. Suppose y

is the j th feature map in

the i th layer, and y

i −1

( m = 1,…, M ) is the m th feature map in the

( i −1)th layer, then y

can be computed as

= σ





m =1

∗y

i −1

+ b



, (3)

where w

is the weight of the m th feature map in the previ-

ous layer, b

is the j th bias in the i th layer, and σ ( · ) is the non-

linear activation function of the convolutional layer. The rectiﬁed

linear unit ReLU σ (x ) = max ( x, 0 ) is used as the activation func-

tion in our experiment. The detailed information of the ResNet-

50 architecture is shown in Table 1 . Considering the computational

cost, the residual block is computed and optimized to replace two

3 × 3 convolutional layers as 1 × 1 + 3 × 3 + 1 × 1. The dimen-

sion of the intermediate convolutional layer is reduced from 3 × 3

to 1 × 1. It is restored by another 1 × 1 convolutional layer to

maintain the accuracy and reduce the computational complexity.

The basic architecture of residual block is shown in Fig. 2 (a). The

interested readers can refer to [38] for more details. The residual

network is linked by the following function

= h

L −1



i = l

(

, w

)

, (4)

where the input of the L residual unit can be expressed by the sum

of the inputs of a shallow residual unit and all the middle complex

mappings. Let θ be the loss function, the backpropagation can be

derived as

∂θ

∂ h

∂θ

∂ h

∂θ

∂ h



1 +

∂

∂ h

L −1



i −l

(

, w

)



. (5)

Gradient optimization in neural network is based on the back-

propagation algorithm, and the backpropagation for hidden layer

H. Lei et al. / Pattern Recognition 79 (2018) 290–302 293

Conv 7×7 ResNet 1-3 ResNet 4-7 ResNet 8-13 ResNet 14-16

ICPR2016-Task1

Cross Modal Transfer

(a)

Average pooling

7×7

Identity

transformation

ResNet-50 pre-trained on ICPR2012

(b)

(c)

ResNet-50 trained on

ICPR2016-Task1

Centromere

Cytoplasmic

Homogenous Nucleolar

Input ( )

1×1Conv, d/4

Batch Norm

ReLu

3×3Conv, d/4

Batch Norm

Addition

ReLu

1×1Conv, d

Batch Norm

Output ( )

+1l

Homogeneous

Speckled

Golgi

Centromere

Nucleolar

NuMem

Coarse-speckled Fine-speckled

Max pooling

3×3

Side prediction

Deep supervision

Final prediction

(d)

Homogeneous

Speckled

Golgi

CentromereNucleolar

NuMem

Classification

(e)

Fig. 2. The ﬂowchart of HEp-2 cell classiﬁcation. (a) Residual block, where d represents the depth of the feature map. (b) Raw training dataset of ICPR2012 dataset. (c) Raw

training dataset of ICPR2016-Task1 dataset. (d) The deep supervision is added to the hidden layer of ResNet-50. (e) Classiﬁcation results.

Table 2

Location of different deeply supervised mechanisms in the ResNet-50.

Model Description

ResNet-50-T The ResNet-50 with transfer learning by using ImageNet dataset

ResNet-50-1D A deep supervised mechanism in the 4th convolutional layer

ResNet-50-2D A deep supervised mechanism in the 3rd and 4th convolutional layers

ResNet-50-3D A deep supervised mechanism in the 2nd, 3rd and 4th convolutional layers

ResNet-50-3DT A deep supervised mechanism in

the 2nd, 3rd and 4th convolutional layers with transfer learning from ResNet-50 by using ImageNet dataset

ResNet-50-3DCT A deep supervised mechanism in the 2nd, 3rd and 4th convolutional layers with transfer learning from ResNet-50 by using ICPR2012 dataset

gradient is fulﬁlled by the chain rule. The gradient values are a

series of products causing the shallow gradients of hidden layer.

Overall, the residual network has the following characteristics: 1)

The network controls the number of parameters; 2) There are sev-

eral levels of feature graphs to ensure the expressive power of the

output features; 3) Numerous down-sampling techniques are used

to improve the propagation eﬃciency with less pooling layers; 4)

BN accelerates regularization and global mean pooling and speeds

up training without dropout.

3.2. Deeply supervised ResNet (DSRN)

To counteract the adverse effects of the unstable gradients and

learn more powerful and representative features, we propose to

explore the supervision in the training of hidden layers of ResNet-

50. If the feature is hidden in the deep network layer, the feature

maps in these hidden layers are trained discriminatively. By prop-

erly using this feature feedback in each hidden layer of the net-

work, the hidden layer’s weight/ﬁlter is directly updated. Accord-

Table 3

Comparison results using ICPR2012 dataset (%).

Method Accuracy Precision Recall F1-score

AlexNet 77.52 79.31 79.96 79.64

VGG-16 63.49 62.73 64.05 63.38

VGG-19 73.32 75.05 76.17 75.61

ResNet-50 88.96 89.93 90.09 90.01

AlexNet -T 93.46 93.57 93.74 93.67

VGG-16-T 92.51 92.87 93.03 92.95

VGG-19-T 94.39 95.05 95.13 95.09

ResNet-50-T 95.63 96.08 96.11 96.10

ResNet-50-1D 91.01 91.53 91.80 91.67

ResNet-50-2D 92.92 93.93 94.13 94.03

ResNet-50-3D 93.46 93.42 93.59 93.51

ResNet-50-1DT 95.91 95.70 95.79 95.75

ResNet-50-2DT 96.46 96.45 96.51 96.48

ResNet-50-3DT 97.1 4 97.42 97.45 97.52

ingly, we can take advantage of the discriminative feature maps.

The feature maps of the output layer are used for ﬁnal classiﬁ-

评论收藏

内容反馈

tox33

粉丝: 64
资源: 305

迁移学习相关最新SCI文献

SCI论文撰写与发表

面向自然语言处理的神经网络迁移学习，(Ruder博士论文)

sci scientific writing

SCI英文写作指导

机器学习迁移学习简明手册

（迁移成分分析TCA）迁移学习算法程序实现_TCA迁移学习_TCA_迁移学习_迁移成分分析算法代码_

迁移学习简明手册

迁移学习手册.pdf

迁移学习研究进展

迁移学习相关论文

迁移学习进展研究

迁移学习相关代码.rar

(2021PIEEE) TransferLearning_关于迁移学习的最新和最权威综述_

基于深度迁移学习模型的花卉种类识别.pdf

迁移学习解析，教你快速上手迁移学习

迁移学习研究介绍

迁移学习介绍.pptx

机器学习和迁移学习

20180411_迁移学习1

迁移学习_transferlearning_迁移学习_迁移学习ppt_

迁移学习 目标检测

迁移学习手册V1.1版本

用PyTorch实现MNIST手写数字识别对应源码文件

基于行人检测的YOLOv8算法+全套代码

行人跌倒数据集（VOC格式）

最新资源

迁移学习目标检测