Warping Residual Based Image Stitching for Large Parallax
12 浏览量
2024-03-19
21:15:10
上传
评论
收藏 1.44MB PDF 举报
Warping Residual Based Image Stitching for Large Parallax
Kyu-Yul Lee and Jae-Young Sim
School of Electrical and Computer Engineering, UNIST, Ulsan, Korea
ever1135@unist.ac.kr and jysim@unist.ac.kr
Abstract
Image stitching techniques align two images captured at
different viewing positions onto a single wider image. When
the captured 3D scene is not planar and the camera base-
line is large, two images exhibit parallax where the relative
positions of scene structures are quite different from each
view. The existing image stitching methods often fail to work
on the images with large parallax. In this paper, we propose
an image stitching algorithm robust to large parallax based
on the novel concept of warping residuals. We first estimate
multiple homographies and find their inlier feature matches
between two images. Then we evaluate warping residual for
each feature match with respect to the multiple homogra-
phies. To alleviate the parallax artifacts, we partition input
images into superpixels and warp each superpixel adap-
tively according to an optimal homography which is com-
puted by minimizing the error of feature matches weighted
by the warping residuals. Experimental results demonstrate
that the proposed algorithm provides accurate stitching re-
sults for images with large parallax, and outperforms the
existing methods qualitatively and quantitatively.
1. Introduction
Image stitching is an important technique for diverse
computer vision applications which aligns multiple im-
ages captured from different viewing positions onto a com-
mon coordinate domain to generate an image with wider
field of view. Recently, many commercial products us-
ing image stitching techniques have been released such as
360° panorama camera
1
and surround-view monitoring sys-
tems
2
. Also, image stitching software products were pro-
vided to synthesize multiple images, e.g., Adobe Photoshop
Photomerge™ and Autostitch [2].
Most of the conventional image stitching methods fol-
low similar procedures [19]. Feature points are first de-
tected from a pair of input images, and their correspondence
matches are found between the images. Then parametric
1
https://www.panono.com/
2
https://www.bmwblog.com/2019/04/18/video-bmw/
image warping models are estimated by using the detected
feature matches, which warp a target image onto a reference
image domain. Finally, we composite an output stitched im-
age by determining the pixel values in the overlapped areas
between the warped target image and the reference image.
One of the most crucial and challenging steps of image
stitching is image warping. Homography is a simple and
traditional image warping model which describes the para-
metric planar transformation based on the planar scene as-
sumption [9]. However, when the captured scene is not pla-
nar including foreground objects at different scene depths
and the camera baseline is large, we observe the parallax
phenomenon where the relative positions of the objects are
different from two images. In such cases, the stitching re-
sults using planar transformation models such as homog-
raphy often exhibit parallax artifacts in the vicinity of the
object boundaries.
To alleviate the parallax artifacts of image stitching,
adaptive warping algorithms have been proposed which par-
tition an image into regular grid cells or pixels and warp
the partitions by different models [7, 10, 11, 15, 22, 24]. En-
ergy minimization frameworks were applied to optimize the
adaptive warps to prevent the distortion in the warped im-
ages [11,15,24]. Local alignment techniques were proposed
which align only a specific image region while hiding the
artifacts in other misaligned regions based on seam-cutting
methods [8, 14, 23]. However, for images with large par-
allax, a group of neighboring pixels in one image may not
have the corresponding pixels adjacent each other in another
image, which causes severe parallax artifacts in resulting
stitched images obtained by the existing smooth warping
based methods [7, 11, 15, 22, 24]. A video stitching method
has been proposed which addresses the large parallax prob-
lem based on the epipolar geometry [10], however it cannot
be directly applied to image stitching due to the lack of tem-
poral motion information of video sequences.
In this paper, we propose a warping residual based stitch-
ing algorithm for images with large parallax. We first parti-
tion input images into superpixels and warp the superpixels
adaptively, since the parallax phenomenon usually occurs in
the vicinity of object boundaries. We detect feature points
8195
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
2575-7075/20/$31.00 ©2020 IEEE
DOI 10.1109/CVPR42600.2020.00822
Authorized licensed use limited to: HUNAN UNIVERSITY. Downloaded on March 19,2024 at 12:58:23 UTC from IEEE Xplore. Restrictions apply.
资源评论
