Image Completion using Planar Structure Guidance
Jia-Bin Huang
1
Sing Bing Kang
2
Narendra Ahuja
1
Johannes Kopf
2
1
University of Illinois at Urbana-Champaign
2
Microsoft Research
Figure 1: Our image completion algorithm automatically extracts mid-level constraints (perspective and regularity) and uses them to guide
the filling of missing regions in a semantically meaningful way. Our method is capable of completing challenging scenes such as multiple
building facades (left), strong perspective distortion (middle) and large regular repetitive structures (right). We significantly outperform three
representative state-of-the-art image completion techniques for these images (see Figure 2). Image credits (left to right): Flickr users micromegas, Theen
Moy, Nicu Buculei.
Abstract
We propose a method for automatically guiding patch-based image
completion using mid-level structural cues. Our method first esti-
mates planar projection parameters, softly segments the known re-
gion into planes, and discovers translational regularity within these
planes. This information is then converted into soft constraints for
the low-level completion algorithm by defining prior probabilities
for patch offsets and transformations. Our method handles multi-
ple planes, and in the absence of any detected planes falls back to
a baseline fronto-parallel image completion algorithm. We validate
our technique through extensive comparisons with state-of-the-art
algorithms on a variety of scenes.
CR Categories: I.3.8 [Computer Graphics]: Applications;
Keywords: Patch-based synthesis, image completion, mid-level
analysis, guided synthesis
Links:
DL PDF WEB DATA CODE
1 Introduction
Replacing or filling regions in images with plausibly synthesized
content is a common image editing operation. This task, known as
image completion, is used in applications ranging from the removal
of unwanted objects in personal photos to movie post-production.
It is also an important step in many graphics algorithms, e.g., for
generating a clean background plate or reshuffling image contents.
While much progress has been made, image completion remains a
challenging problem. This is because some amount of higher level
understanding of the scene is often required. The state-of-the-art
automatic algorithms typically rely on low-level cues; they syn-
thesize the missing region as a field of overlapping patches copied
from the known region [Wexler et al. 2007]. Here, they attempt
to synthesize an image that locally appears like the known input
everywhere, and such that overlapping patches agree as much as
possible. Barnes et al. [2009] showed how this algorithm can be
sped up using a random search and propagation scheme.
Most of these algorithms have two important limitations. First,
since they only directly copy translated patches from the input, the
performance degrades with scenes that are not fronto-parallel. They
would not be able to effectively handle the perspective foreshorten-
ing as shown in Figure 1. The other limitation is in the tendency of
converging to local minima, due to the strong non-convexity of the
objective. This second problem is somewhat alleviated by applying
the algorithm in a coarse-to-fine manner.
Recent approaches handle the fronto-parallel limitation by consid-
ering patch transformations such as rotation, scale, and gain/bias
color adjustments [Mansfield et al. 2011; Darabi et al. 2012]. While
评论0