2 Introduction
In the photogrammetry community, more manually intensive meth-
ods based on surveyed ground control points or manually registered tie
points have long been used to register aerial photos into large-scale
photo-mosaics [181]. One of the key advances in this community was
the development of bundle adjustment algorithms that could simultane-
ously solve for the locations of all of the camera positions, thus yielding
globally consistent solutions [207]. One of the recurring problems in cre-
ating photo-mosaics is the elimination of visible seams, for which a vari-
ety of techniques have been developed over the years [1,50,135,136,148].
In film photography, special cameras were developed at the turn
of the century to take ultra wide-angle panoramas, often by expos-
ing the film through a vertical slit as the camera rotated on its
axis [131]. In the mid-1990s, image alignment techniques were started
being applied to the construction of wide-angle seamless panoramas
from regular hand-held cameras [43, 124, 193, 194]. More recent work
in this area has addressed the need to compute globally consistent
alignments [167, 178, 199], the removal of “ghosts” due to parallax
and object movement [1, 50, 178, 210], and dealing with varying expo-
sures [1, 116, 124, 210]. (A collection of some of these papers can be
found in [19].) These techniques have spawned a large number of com-
mercial stitching products [43, 168], for which reviews and comparison
can be found on the Web.
While most of the above techniques work by directly minimizing
pixel-to-pixel dissimilarities, a different class of algorithms works by
extracting a sparse set of features and then matching these to each
other [7,30,35, 38,129, 227]. Feature-based approaches have the advan-
tage of being more robust against scene movement and are potentially
faster, if implemented the right way. Their biggest advantage, how-
ever, is the ability to “recognize panoramas,” i.e., to automatically dis-
cover the adjacency (overlap) relationships among an unordered set of
images, which makes them ideally suited for fully automated stitching
of panoramas taken by casual users [30].
What, then, are the essential problems in image alignment and
stitching? For image alignment, we must first determine the appro-
priate mathematical model relating pixel coordinates in one image
to pixel coordinates in another. Section 2 reviews these basic motion
