IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 54, NO. 3, MARCH 2016 1349
Unsupervised Deep Feature Extraction for Remote
Sensing Image Classification
Adriana Romero, Carlo Gatta, and Gustau Camps-Valls, Senior Member, IEEE
Abstract—This paper introduces the use of single-layer and
deep convolutional networks for remote sensing data analysis.
Direct application to multi- and hyperspectral imagery of super-
vised (shallow or deep) convolutional networks is very challenging
given the high input data dimensionality and the relatively small
amount of available labeled data. Therefore, we propose the use
of greedy layerwise unsupervised pretraining coupled with a highly
efficient algorithm for unsupervised learning of sparse features.
The algorithm is rooted on sparse representations and enforces
both population and lifetime sparsity of the extracted features,
simultaneously. We successfully illustrate the expressive power of
the extracted representations in several scenarios: classification
of aerial scenes, as well as land-use classification in very high
resolution or land-cover classification from multi- and hyperspec-
tral images. The proposed algorithm clearly outperforms standard
principal component analysis (PCA) and its kernel counterpart
(kPCA), as well as current state-of-the-art algorithms of aerial
classification, while being extremely computationally efficient at
learning representations of data. Results show that single-layer
convolutional networks can extract powerful discriminative fea-
tures only when the receptive field accounts for neighboring pixels
and are preferred when the classification requires high resolution
and detailed results. However, deep architectures significantly
outperform single-layer variants, capturing increasing levels of
abstraction and complexity throughout the feature hierarchy.
Index Terms—Aerial image classification, classification, deep
convolutional networks, deep learning, feature extraction, hyper-
spectral (HS) image, multispectral (MS) images, segmentation,
sparse features learning, very high resolution (VHR).
I. INTRODUCTION
E
ARTH observation (EO) through remote sensing tech-
niques is a research field where a huge variety of physical
signals are measured from instruments on board space and
airborne platforms. A wide diversity of sensor characteristics
are available nowadays, ranging from medium and very high
resolution (VHR) multispectral (MS) imagery to hyperspectral
(HS) images that sample the electromagnetic sp ectrum with
high detail. These myriad of sensors serve to particularly
Manuscript received November 14, 2014; re vised April 24, 2015 and
June 29, 2015; accepted August 3, 2015. Date of publication October 6, 2015;
date of current version February 24, 2016.
A. Romero is with the Department of Applied Mathematics and Analysis,
Uni versitat de Barcelona, 08007 Barcelona, Spain (e-mail: adriana.romero@
ub.edu).
C. Gatta is with the Computer Vision Center, Univ ersitat Autònoma de
Barcelona, 01873 Barcelona, Spain (e-mail: cgatta@cvc.uab.es).
G. Camps-Valls is with the Image Processing Laboratory, Universitat de
València, 46980 València, Spain (e-mail: gcamps@uv.es).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TGRS.2015.2478379
different objectives, focusing either on obtaining quantitative
measurements and estimations of geobiophysical variables or
on the identification of materials by the a nalysis of the acquired
images [1]–[3]. Among all the different products that can be
obtained from the acquired images, classification maps
1
are
perhaps the most relevant ones. The remote sensing image clas-
sification problem is very challenging and ubiquitous because
land-cover and land-use maps a re mandatory in multitemporal
studies and constitute useful inputs to other processes.
Despite the h igh number of advanced, robust, and accurate
existing classifiers [4], the field faces very important challenges,
as follows.
1) Complex statistical characteristics of images. The statis-
tical properties of the acquired images place important
difficulties for automatic classifiers. The analysis of these
images turns out to be very challenging, particularly
because of the high dimensionality of the pixels, the
specific noise and uncertainty sources observed, the high
spatial and spectral redundancy and collinearity, and their
potentially nonlinear nature.
2
Beyond these well-known
data characteristics, we should highlight that spatial and
spectral redundancy also suggests that the acquired signal
may be better described in sparse representation spaces,
as recently reported in [4] and [6]–[8].
2) High-computational problems involved. We are witness-
ing the advent of a Big Data Era, particularly in remote
sensing data processing. The upcoming constellations of
satellite sensors will acquire a large variety of hetero-
geneous images of different spatial, spectral, angular,
and temporal resolutions. In fact, we are witnessing an
ever-increasing amount of data gathered with current and
upcoming EO satellite missions, from MS sensors such
as Landsat-8 [9] to VHR sensors such as WorldView-III
[10], the superspectral Copernicus’ Sentinel-2 [11] and
Sentinel-3 missions [12], as well as the planned Envi-
ronmental Mapping and Analysis Program [13], Hyper-
spectral Infrared Imager [14], and the European Space
Agency’s candidate Fluorescence Explorer [15] imag-
ing spectrometer missions. This data flux will require
1
In the remote sensing community, the term “classification” is often preferred
to the term “semantic segmentation.” We use the term classification to classify
full images in the first application of aerial image classification, as well as to
describe the process of attributing each pixel (or segment) to a single semantic
class in subsequent applications.
2
Factors such as multiscattering in the acquisition process, heterogeneities at
subpixel level, and atmospheric and geometric distortions lead to distinct non-
linear feature relations, since pixels lie in high-dimensional curved manifolds
[4], [5].
0196-2892 © 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
