基于形态学扩张卷积神经网络的高光谱图像分类.pdf资源-CSDN文库

需积分: 12 6 浏览量 2021-12-23 19:05:56 上传评论收藏 3.08MB PDF 举报

资源详情

资源评论

资源推荐

Signal Processing: Image Communication 101 (2022) 116549

Contents lists available at ScienceDirect

Signal Processing: Image Communication

journal homepage: www.elsevier.com/locate/image

Morphologically dilated convolutional neural network for hyperspectral

image classification

Vinod Kumar

∗

, Ravi Shankar Singh, Yaman Dua

Department of Computer Science and Engineering, Indian Institute of Technology (BHU), Varanasi, Uttar Pradesh, 221005, India

A R T I C L E I N F O

Keywords:

Mathematical morphology

Hyperspectral image (HSI)

Dilated convolution

Binarization

Convolutional neural network (CNN)

A B S T R A C T

The use of hyperspectral images is expanding rapidly with the advancement of remote sensing technologies.

The precise classification of features for mapping land cover through hyperspectral images is a major

research topic and a major focus. In the classification of hyperspectral images, several methods provided

good classification results. Among all, convolutional neural network (CNN) is a widely used deep neural

network due to its robust feature extraction capabilities. It can enhance the hyperspectral image classification

accuracy. Mathematical morphology (MM) is a robust and straightforward spatial feature descriptor, which

can reduce the computational workload. We proposed a novel model morphologically dilated Convolutional

Neural Network (MDCNN), which can extract more robust spectral–spatial features. MDCNN adopt a process

to concatenate the morphological feature maps with original hyperspectral data. CNN structure uses both

traditional and dilated convolution. Replacing the dilated convolution in traditional convolution layers expands

the receptive field without boosting parameters and thus improves network performance without increasing

network complexity. The dilation layer does not reduce the number of parameters but reduces the size of the

output feature map, which leads to the overall reduction in the number of parameters. 3D convolution extracts

spectral–spatial features and maintains the correlation of spectral data. 2D CNN extracts spatial features and

reduces the model’s complexity, which can occur if only 3D convolution is used. Experimental findings show

that the proposed approach can provide better classification results than traditional deep learning models and

other state-of-the-art models on the Indian Pines, University of Pavia, and Salinas Scene data.

1. Introduction

Hyperspectral images (HSIs) stack several hundred spatial data

of the same scene with distinct spectral wavelengths. The in-depth

spectral and spatial information present in hyperspectral data improves

the ability to distinguish objects of interest precisely with improved

classification accuracy. As the advancement in hyperspectral sensors,

spatial resolution is getting high. It assists in the classification of small

spatial structures in hyperspectral images [1]. With positively enrich

spatial and spectral information and advancement, HSIs have been

useful in a variety of applications fields, such as geological science [2],

ecological science [3], military [4], agriculture [5], and environment

monitoring [6]. Over the last several years, many researchers have

turned attention to the land cover classification task [7]. Land-cover

classification’s primary objective is to allocate one thematic class to

each pixel existing on the scene. HSI classification tasks are still very

challenging because of hyperspectral data complexity. Initial classifica-

tion methods were purely based on spectral features. They were using

linear transform to extract dominant features in the spectral domain [8–

10]. But, HSI images are non-linear because of the non-linearity of

∗

Corresponding author.

E-mail addresses: vinodkumar.rs.cse18@iitbhu.ac.in (V. Kumar), ravi.cse@iitbhu.ac.in (R.S. Singh), yamandua.rs.cse18@iitbhu.ac.in (Y. Dua).

the ground scattering mechanism in natural objects [11]. Besides, the

massive number of spectral channels will add up noise into the data,

and the high data dimensionality contributes to the phenomenon of

the Hughes effect [12]. High data dimensionality is often handled

by various compression techniques [13,14]. Thus, methods of HSI

classification that uses spectral features explicitly are not appropriate

for the classification task [15]. And it is challenging to classify objects

into different classes with high precision using spectral information

alone when the same type of object poses a slightly different spectrum

and different types of objects having the same spectrum using spec-

tral information alone [16]. To address these, classification methods

based on both spatial and spectral features have been proposed [17].

Fang et al. [18] given a technique that uses multiple kernel learning,

which is based on both spatial and spectral information to improve

the SVM classifier accuracy. Many other approaches also come in the

picture, which uses spatial characteristics of the objects, like Markov

random fields [19], 3-D Gabor feature extraction [20], morphological

component analysis [21]. One of the most popular ways to extract

spatial features is mathematical morphology(MM), like morphological

profile [22], which turned out very good for the HSI classification task.

https://doi.org/10.1016/j.image.2021.116549

Received 6 March 2021; Received in revised form 20 August 2021; Accepted 10 October 2021

Available online 17 November 2021

V. Kumar, R.S. Singh and Y. Dua Signal Processing: Image Communication 101 (2022) 116549

But in recent times, Deep learning-based methods are found more

accurate than traditional machine learning-based approaches as they

try to mimic the human visual cortex. These methods contain more than

one layer in which features are extracted at various levels, which results

in high accuracy in image classification and target detection [23,24].

Factors like atmospheric scattering and intraclass variability create

extreme difficulty in extracting features from HSI data effectively.

To resolve such issues, deep learning models provide an effective

option since it can pull out hierarchical features, which are invariant

and robust [25]. Initially, the deep learning models proposed for HSI

classification were stacked autoencoder (SAE) [26,27] and deep belief

network (DBN) [28]. In terms of feature extraction power and classifi-

cation precision, these models were able to extract more robust features

and outperform other conventional machine learning approaches. But

these approaches mentioned above demand to train a massive amount

of parameters because of the full connection of different layers. Besides,

SAE and DBN cannot even extract spatial data efficiently since training

samples consisting of image patches must be translated into a one-

dimensional vector. This conversion results in the spatial information

lost, which might be present in the input image. To resolve these

problems, the Convolutional neural network (CNN) comes into the

picture, as layers are sparsely connected in the relative local region

to extract features and shared weights, which results in a significant

reduction in the number of trainable parameters.

Many studies find out that the techniques based on profound

spectral–spatial properties or features yield very high accuracy. Li

et al. [29] proposed a model in which a pixel-pair strategy is introduced

to address the shortage of training samples. Each pixel is paired with

its surrounding pixel, re-labeled with a voting strategy, and serves as

a new sample. With an increase in the number of training samples,

CNN will make full use of the features for the classification task. Li

et al. [30] proposed a multiscale deep middle-level feature fusion net-

work (MMFN). It extracts discriminative features through the fusion of

complementary and related information among different scale features.

Yang et al. [31] model use a two-path system, one is 1D-CNN to extract

the spectral features, and the other is 2D-CNN to extract the spatial

features. In the end, they are combined to get a classification result.

But this model lacks in the spectral–spatial relationship, which leads to

a decrease in accuracy as the model gets deeper. But 3D-CNN can be

used to extract in-depth spectral–spatial features directly [32]. Sellami

et al. [33] reduced spectral dimensions to select relevant channels.

Extracting the features using 3D CNN encoder–decoder architecture.

Yang et al. [34] design the R-3-D-CNN, which uses two different sizes

of patches with the same central pixel, which reduces the problem of a

noisy patch. Mirzaei et al. [35] used non negative tensor factorization

to extract discriminant features and designed 3D CNN model. Zhong

et al. [36] presented a 3D-convolution based supervised spectral–spatial

residual Network (SSRN), the concept of identity mapping in residual

blocks lessen the gravity of the declining accuracy phenomenon and

promotes the back-propagation of gradients. But using 2D-CNN or

3D-CNN alone has some demerits like 2D-CNN unable to extract high-

quality spectral features, and 3D-CNN makes the model complex. So,

Roy et al. [37] proposed a model for stacking 3D-CNN and 2D-CNN

layers to use the full spectrum of spectral and spatial features maps

to achieve optimum accuracy. These models are bound to perform

well when they have a sufficient amount of training samples. But, an

insufficient amount of training samples is a bottleneck in HSI analysis,

which reflects in the accuracy of test samples due to overfitting. So,

Wei et al. [38] come up with a model which uses local binary pattern

to reduce the demand of a vast amount of training samples and also

lessen the workload from the model.

In [39], the author proposes an enhanced hybrid-graph discrimi-

nant learning (EHGDL) method as a hybrid-graph learning method for

revealing the HSI’s complex high-order relationships. Instead of just

focusing on only binary relationships author develop a DR model based

on the intraclass hypergraph, the interclass hypergraph, the supervised

locality graph, and the weighted neighborhood margin to improve

the intraclass sample compactness and separability. A novel feature

learning algorithm, termed spatial–spectral hypergraph discriminant

analysis (SSHGDA), has been proposed to represent the complex intrin-

sic spatial–spectral of HSI [40]. To capture the inherent features of HSI,

SSHGDA generates an intraclass spatial–spectral hypergraph and an

interclass spatial–spectral hypergraph. Then a low-dimensional feature

learning model compacts intraclass information and separates interclass

information. This model yields an optimal projection matrix for extract-

ing HSI’s spatial–spectral features. For HSI classification, in [41], the

authors propose a novel unsupervised dimensionality reduction method

called LNSPE. First, LNSPE reconstructs each sample’s spectral neigh-

bors and obtains the optimal weights for constructing the adjacency

graph by modifying its loss function. Then, to find the training samples’

scatter information, LNSPE minimizes the scatter between pixels and

their neighbors and maximizes the total HSI scatter. Finally, it uses

scatter data and a dual graph structure to improve HSI aggregation. As a

result, LNSPE can effectively reveal the intrinsic structure and improve

HSI data classification. Thus, LNSPE can effectively demonstrate the

inherent structure and improve HSI data classification performance. All

of the methods mentioned earlier talk about dimensionality reduction

of HSI using graphical methods. However, this method requires an

inordinate time to design the hypergraph, graph, and margin models.

The author in [42] provides a deep survey on feature extraction

techniques over the years. This article provides a detailed overview

of hyperspectral FE techniques, divided into shallow and deep cat-

egories. It tells us whether DL-based FE approaches are better than

conventional shallow FE techniques for Earth Observation. The factors

that should be taken into account when developing reliable shallow

and deep FE techniques. In [43], the author proposes a cascaded

RNN model using gated recurrent units to explore HSIs redundant and

complementary information. It is consists of two RNN layers. The first

RNN layer seeks to eliminate redundancies among adjacent spectral

bands, whereas the second RNN layer seeks to learn complementary

information from dissimilar spectral bands. Furthermore, because HSIs

contain rich spatial data, we extend the proposed model to include

some convolutional layers. Locally semantic change, such as scene

composition, object position, illumination, atmospheric effects, and ma-

terial mix, has been studied less in spatial information modeling [44].

The author proposes a solution by extracting invariant features from

hyperspectral imagery (HSI) in spatial and frequency domains (IAPs).

A continuous histogram of oriented gradients constructed in a Fourier

polar coordinate also model invariant behaviors (e.g., shift, rotation).

This yields a combinatorial representation of HSI features for HSI clas-

sification. Deep learning models struggle in complex scenes requiring

good classification due to limited information homogeneity. To address

the earlier issue, the author improves multimodal deep learning (MDL).

Cross-modality learning (CML) is a popular application of MML in

RS image classification. The author shows different fusion strategies

and how to train deep neural networks and construct the network

architecture. Five fusion frameworks were developed and integrated

into the MDL framework. They used convolutional neural networks

to model spatial information [45]. To classify HS images, the author

in [46] studied CNNs and GCNs (qualitatively). The construction of

the adjacency matrix on all data is computationally expensive, es-

pecially in large-scale remote sensing (RS) problems. So they build

a minibatch GCN that trains large-scale GCNs. Because CNN’s and

GCNs extract different types of HS features, combining them makes

sense. To combine miniGCNs and CNNs, they investigate three fusion

strategies: additive, elementwise multiplicative, and concatenation. For

HSI spectral–spatial classification, MTGAN [47] employs CNNs for both

the generator and the discriminator. The generator is reconstructing

and classifies input cubes simultaneously. Unlabeled samples are suffi-

ciently exploited to improve CNN classification performance as a result

of the reconstruction process. The author uses skip-layer connections

in both the reconstruction and classification tasks. The spatial and

V. Kumar, R.S. Singh and Y. Dua Signal Processing: Image Communication 101 (2022) 116549

discriminative information in the shallow layers is transferred via these

connections to aid reconstruction and classification tasks. In [48],

the author proposes a two-branch spectral–spatial attention network

for hyperspectral image classification, with one branch dedicated to

spectral attention and the other to spatial attention. Each convolutional

layer includes attention modules. Thus, making CNN prioritize discrim-

inative channels and spatial positions while suppressing irrelevant ones.

Furthermore, two-branch results are fused in the classification phase

using an adaptively weighted summation method.

Based on the literature’s core problems, as mentioned above, and an

extension of work from Roy et al. [37], we proposed a novel morpho-

logically dilated convolutional network (MDCNN). MDCNN uses both

the 3D convolution layer and the 2D convolution layer, morphological

feature maps, and both standard as well as dilated convolution. In

MDCNN, the principal component analysis (PCA) algorithm is used

to transform high-dimensional input data into low-dimensional data

in order to minimize computational costs. CNN has a flaw, which is

the inaccuracy of boundary location, resulting in partial object shapes.

As a result, information obtained by another type of spatial extractor

can improve deep network feature representation [49]. Then different

mathematical morphology operations are applied on new low dimen-

sional input data to extract discriminant spatial feature maps. Further,

these morphological feature maps concatenated with the previous low

dimensional input data. Then, input patches are extracted around each

pixel, which will send to the network. Both standard 3D convolution

and Dilated-3D convolution were applied to extract spectral–spatial

features at the same time. Then Dilated-2D convolution was used to

extract discriminant spatial features. The dilated convolution applied

in this paper by substituting the convolution layers in traditional CNN

with dilated convolution layers, which expands the receptive field

without boosting parameters and thus improves network performance

without increasing network complexity [50–52]. The dilation layer

does not reduce the number of parameters but reduces the size of the

output feature map, which leads to the overall reduction in the number

of parameters. Spectral–spatial attributes are then transmitted to fully

connecting layers to extract abstract high-level features.

This papers primary contributions can be summarized as follows:

1. Mathematical morphological operations are applied to input

hyperspectral data to extract spatial feature map as output.

2. This output is concatenated with the input and fed into neural

network to reduce the workload of CNN and provide better

spatial features.

3. The neural network contains both 3D convolutional layers as

well as a 2D convolutional layer. We use a mix of traditional and

dilated convolution to increase the receptive field power, reduce

trainable parameters, and reduce overfitting, which in turn re-

sults in the reduction of overall complexity of the model. The

dilated convolution layer’s output feature map size is slightly

less than the output of the traditional convolution layer, which

reduces overall trainable parameters.

4. The model simultaneously extracts the discriminatory spectral–

spatial attributes or properties to achieve high classification

accuracy by utilizing the spectral–spatial relationship.

5. Experiments were performed on three different publicly avail-

able datasets to evaluate the performance of MDCNN model with

other state-of-the-art methods.

The rest of this paper is structured in Section 2, a detailed overview

of the current MDCNN structure. Section 3 discusses experimental

evidence, setup, findings, and interpretation. Finally, some conclusions

are drawn in Section 4.

2. Problem formulation

HSI data is also known as a hypercube. This hypercube can be

represented as 𝐈 ∈ 𝐑

[𝐇×𝐖×𝐂]

, where I is the original HSI input, H is

the height, W is the width of the input, and C is the total number

of spectral bands. HSI provides a tremendous amount of information

through a large number of spectral bands, but their high dimensionality

increases the computational burden. So, we use PCA, which can reduce

the high dimensional data into a low dimension data with minimum

loss of useful information. The reduced data cube after applying PCA is

𝐈𝐏 ∈ 𝐑

[𝐇×𝐖×𝐏]

, where P is the number of the principal component. Bina-

rization operation applies to low dimensional data and gives an output

𝐈𝐁 ∈ 𝐑

[𝐇×𝐖×𝐊]

. Only K initial bands are selected for the Binarization

process. Three mathematical morphological operations apply on Binary

data cube IB and give an output of size [𝐻 × 𝑊 × 𝐾] each. Data cube

IP, 𝐼

𝐸𝑟𝑜𝑠𝑖𝑜𝑛

, 𝐼

𝐷𝑖𝑙𝑎𝑡𝑖𝑜𝑛

and 𝐼

𝐺𝑟𝑎𝑑𝑖𝑒𝑛𝑡

are concatenated and form an output

𝐈𝐂 ∈ 𝐑

[𝐇×𝐖×𝐁]

, where 𝐁 = (𝐏 + 𝟑𝐊). Input patches 𝐈

𝐏𝐚𝐭𝐜𝐡

∈ 𝐑

[𝐒𝑥𝐒𝑥𝐁]

are

extracted from IC and fed to the CNN for deep spectral–spatial features

extraction and classification task.

3. Proposed framework

Fig. 1 describe the workflow of the proposed model MDCNN, which

includes different steps like (a) Dimension reduction of the hyper-

spectral cube in spectral direction using principal component analy-

sis (PCA), (b) Binarization of the low-dimension hyperspectral cube,

Concatenation of low-dimension hyperspectral cube and morpholog-

ical feature maps (Erosion, Closing, and Gradient), (e) Extraction of

patches for the input of the convolutional neural network, (f) Deep

spectral–spatial features extraction using standard 3D convolution and

Dilated-3D convolution, (g) Discriminative spatial feature extraction

using Dilated-2D convolution, and (h) Prediction of classification map

using softmax classifier.

3.1. Binarization process

Binarization is a process of converting any input vector’s value to

a spectrum of 0 to 1. In the case of HSI data, at the first pixel values,

transform into the range of 0 to 255. The expression for the conversion

is as follows:

𝛹

𝑖,𝑗

255 ∗ (𝐼𝑝

𝑖,𝑗

− 𝑚𝑖𝑛(𝐼 𝑝))

𝐼𝑝

𝑖,𝑗

(1)

Where Ip is the image, 𝐈𝐩

𝐢,𝐣

is the pixel value at the position (i, j). After

the range conversion, a threshold (Θ) is selected using the expression

as follow:

𝛩 =

∑

𝐻−1

𝑖=0

∑

𝑊 −1

𝑗=0

𝛹

𝑖,𝑗

𝐻 × 𝑊 × 𝐾

(2)

Where H and W are the height and width of the input cube, and K is

the number of bands selected for binarization. Now, the value of 𝜳

𝐢,𝐣

will be one if it is greater than or equal to Θ, otherwise zero. The new

image formed after thresholding is given by:

𝐵𝐼

𝑖,𝑗

{

1, 𝐢𝐟 𝛹

𝑖,𝑗

≥ 𝛩

0, 𝐢𝐟 𝛹

𝑖,𝑗

< 𝛩

(3)

3.2. Mathematical morphology

The high spatial resolution possessed by hyperspectral images makes

a very less number of mixed pixels and provides clear boundaries

between different objects in the case of land covers data. So, the

discriminative spatial features like morphological features can provide

results with better accuracy. Serra [53] first introduced morpholog-

ical analysis in 1982 and used Structuring Elements(SEs) to collect

information on the shape, boundary, and skeleton of an image. It is a

three-step process to acquire morphological feature map, (a) selection

of structuring element, (b) The conversion of image into a binary image,

since through binary image only the structuring element can follow the

剩余10页未读，继续阅读

评论收藏

内容反馈

qq_22771997

粉丝: 0
资源: 3

基于形态学扩张卷积神经网络的高光谱图像分类.pdf

评论0

最新资源

基于形态学扩张卷积神经网络的高光谱图像分类.pdf

评论0

利用卷积神经网络的高光谱图像分类.pdf

基于卷积神经网络的高光谱图像分类.pdf

基于多尺度卷积神经网络的高光谱图像分类.pdf

基于加权K近邻和卷积神经网络的高光谱图像分类.pdf

基于三维全卷积神经网络的高光谱图像分类.pdf

基于卷积神经网络的高光谱图像分类研究.pdf

基于扩张卷积注意力神经网络的高光谱图像分类

基于卷积神经网络的高光谱图像分类研究

基于卷积神经网络的多光谱图像多标签场景分类.pdf

基于卷积神经网络的高光谱图像谱-空联合分类.pdf

基于深度卷积神经网络的高光谱遥感图像分类.pdf

多层局部感知卷积神经网络的高光谱图像分类.pdf

基于卷积神经网络与主动学习的高光谱图像分类.pdf

卷积神经网络CNN进行图像分类_matlab

基于卷积神经网络的多光谱图像多标签场景分类

基于卷积神经网络的壁画颜料多光谱图像分类

python基于卷积神经网络的高光谱图像分类

基于BP神经网络高光谱图像分类研究.pdf

基于卷积神经网络的柑橘病叶高光谱分类.pdf

卷积神经网络与MRF相结合的高光谱图像分类.pdf

论文研究-基于加权K近邻和卷积神经网络的高光谱图像分类.pdf

基于局部保留降维卷积神经网络的高光谱图像分类算法.pdf

基于深度卷积神经网络的高光谱图像分类算法研究.pdf

高光谱图像分类.pdf

密集连接扩张卷积神经网络的单幅图像去雾.pdf

基于偏差估计卷积神经网络恒星光谱数据自动分类.pdf

ChatGPT教程（终极版）最全整理

博客中Kmeans以及FCM算法数据（免积分）

最新资源