This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING 1
Image Registration With Fourier-Based Image
Correlation: A Comprehensive Review of
Developments and Applications
Xiaohua Tong , Senior Member, IEEE,ZhenYe , Yusheng Xu, Member, IEEE,SaGao,HuanXie, Member, IEEE,
Qian Du, Fellow, IEEE, Shijie Liu, Xiong Xu, Member, IEEE, Sicong Liu, Member, IEEE,
Kuifeng Luan, and Uwe Stilla, Senior Member, IEEE
Abstract—Fourier-based image correlation is a powerful area-
based image registration technique, which involves aligning images
based on a translation model or similarity model by means of
the image information and operation in the frequency domain. In
recent years, Fourier-based image correlation has made significant
progress and attracted extensive research interest in a variety of
applications, especially in the field of photogrammetry and remote
sensing, leading to the development of a number of subpixel meth-
ods that have improved the accuracy and robustness. However, to
date, a detailed review of the literature related to Fourier-based
image correlation is still lacking. In this review, we aim at providing
a comprehensive overview of the fundamentals, developments, and
applications of image registration with Fourier-based image cor-
relation methods. Specifically, this review introduces the principal
laws underlying these methods, presents a survey of the existing
subpixel methods calculated both in the spatial domain and in
the frequency domain, summarizes the major applications from
three aspects, and discusses the challenges and possible directions
of future research. This review is expected to be beneficial for
researchers working in the relevant fields to obtain an insight into
the current state of the art, to develop new variants, to explore
Manuscript received March 9, 2019; revised June 18, 2019; accepted August
7, 2019. This work was supported in part by the National Natural Science
Foundation of China under Grant 41631178, in part by the National Key
Research and Development Project of China under Grant 2018YFB0505400, in
part by the National High Resolution Ground ObservationSystemofChinaunder
Grant 11-Y20A12-9001-17/18, in part by the Shanghai Science and Technology
Innovation Action Plan Program under Grant 18511102100, and in part by the
Fundamental Research Funds for the Central Universities. (Xiaohua Tong and
Zhen Ye contributed equally to this work.) (Corresponding authors: Xiaohua
Tong; Zhen Ye.)
X. Tong, S. Gao, H. Xie, S. Liu, X. Xu, and S. Liu are with the College of
Surveying and Geo-Informatics, Tongji University, Shanghai 200092, China (e-
mail: xhtong@tongji.edu.cn; 1410900@tongji.edu.cn; huanxie@tongji.edu.cn;
liusjtj@tongji.edu.cn; xvxiong@tongji.edu.cn; sicong.liu@tongji.edu.cn).
Z. Ye is with the College of SurveyingandGeo-Informatics,Tongji University,
Shanghai 200092, China, and also with the Department of Photogrammetry and
Remote Sensing, Technische Universitaet Muenchen, 80333 Munich, Germany
(e-mail: yezhen0402@126.com; z.ye@tum.de).
Y. Xu and U. Stilla are with the Department of Photogrammetry and Remote
Sensing, Technische Universitaet Muenchen, 80333 Munich, Germany (e-mail:
yusheng.xu@tum.de; stilla@tum.de).
Q. Du is with the Department of Electrical and Computer Engineering, Missis-
sippi State University, Starkville, MS 39759 USA (e-mail: du@ece.msstate.edu).
K. Luan is with the College of Marine Sciences and the Shanghai Engineering
Research Center of Estuarine and Oceanographic Mapping, Shanghai Ocean
University, Shanghai 201306, China (e-mail: lkf0129@163.com).
Color versions of one or more of the figures in this article are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/JSTARS.2019.2937690
potential applications, and to suggest promising future trends of
image registration with Fourier-based image correlation.
Index Terms—Fourier-based image correlation, Fourier-Mellin
(FM) transform, image registration, phase correlation (PC), pho-
togrammetry and remote sensing, subpixel image matching.
I. INTRODUCTION
I
MAGE registration is the process of overlaying images of
the same scene, which can be generally divided into two
categories: Feature-based methods and area-based methods [1].
The feature-based methods are normally performed in three
steps, i.e., feature detection [2], feature description [3], and
feature matching[4]. The salient features, including corner,blob,
region, edge, line segment, and shape, are initially extracted
from images, and matched using similarity measures and spa-
tial relationships calculated from their descriptors or geometric
attributes to establish the geometric correspondence between
images. The area-based methods directly utilize the intensity
information to match areas or regions, which have advantages
in precision, distribution, stability, and other aspects. The focus
of the area-based methods is the similarity measure, and the
commonly used similarity measures include normalized cross
correlation (NCC) [5], sum of squared differences [6], mu-
tual information [7], cross-cumulative residual entropy [8], etc.
Fourier-based image correlation is a specific type of area-based
image registration technique, which involves aligning images
based on a translation model or similarity model by means of
the image information and operation in the frequency domain.
Image registration with Fourier-based image correlation pos-
sesses the merits of high theoretical accuracy, high computa-
tional efficiency, and insensitivity to the frequency-dependent
noise and intensity contrast. As a result, it has received a lot
of attention in the computer vision, photogrammetry, and re-
mote sensing communities. Due to its remarkable advantages,
Fourier-based image correlation has been recommended in many
review papers and books concerning image registration [1],
[9]–[13]. Since the earliest use of the fast Fourier transform
(FFT) for image registration and the introduction of the phase
correlation (PC) method in the 1970s [14], [15], Fourier-based
image correlation has experienced rapid development, and a
1939-1404 © 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.