适用于移动 AIGC 的统一数据驱动抠图引擎.pdf
版权申诉
107 浏览量
2024-03-29
07:39:38
上传
评论
收藏 675KB PDF 举报
Matting Moments: A Unified Data-Driven Matting Engine for Mobile AIGC in
Photo Gallery
Yanhao Zhang
1
, Fanyi Wang
1
, Weixuan Sun
1,2
, Jingwen Su
1
, Peng Liu
1
,
Yaqian Li
1
, Xinjie Feng
1
, Zhengxia Zou
3
1
OPPO Research Institute
2
Australian National University
3
Beihang University
{zhangyanhao, wangfanyi}@oppo.com
Abstract
Image matting is a fundamental technique in vi-
sual understanding and has become one of the most
significant capabilities in mobile phones. Despite
the development of mobile storage and comput-
ing power, achieving diverse mobile Artificial In-
telligence Generated Content (AIGC) applications
remains a great challenge. To address this issue,
we present an innovative demonstration of an auto-
matic system called ”Matting Moments” that en-
ables automatic image editing based on matting
models in different scenarios. Coupled with ac-
curate and refined matting subjects, our system
provides visual element editing abilities and back-
end services for distribution and recommendation
that respond to emotional expressions. Our sys-
tem comprises three components: 1) photo con-
tent structuring, 2) data-driven matting engine, and
3) AIGC functions for generation, which automat-
ically achieve diverse photo beautification in the
gallery. This system offers a unified framework that
guides consumers to obtain intelligent recommen-
dations with beautifully generated contents, help-
ing them enjoy the moments and memories of their
present life.
1 Introduction
Precise matting of salient subjects in an image becomes piv-
otal in phone applications. And matting results can facili-
tate secondary creation including intelligent editing and Ar-
tificial Intelligence Generated Content (AIGC)
[
Du et al.,
2023
]
. As a key capability of visual systems, matting sub-
stantially impacts the editing efficiency and user experience
of mobile phone products. To realize diverse AIGC ap-
plications on mobile terminals, matting serves as the foun-
dation for translating technical capabilities into commercial
value
[
Sun et al., 2022; Chen et al., 2022; Hu et al., 2023;
Li et al., 2022
]
. Integrating the visual interface enables users
to select and extract desired elements by holding, copying
and dragging them. This expedites and simplifies editing or
generation.
Flash Matting
Sky Matting
Stylization&
Colorization
Magic Eraser
Quality
Content-based
recommendation
Data
sync
Label
scanning
Saliency
Portrait
Scenery
Encoder
Basic
Parsing
Semantic
Parsing
Time GPS
Size
Scene Subject Person
Address
OCR
Aesthetic
Composition
Assessment
Recommendation
personality
Mobile AIGC Application
Decoder
Hard Matting Samples
Simple Segment Samples
‘phone’,
‘making
phone call’
Text-Image
/Unlabeled Samples
(c)(b)
(a)
Region
Training
Feedback
Policy
Figure 1: Overview of our unified data-driven system. (a) Photo
content structuring. (b) Data-driven matting engine. (c) AIGC func-
tions for generation.
Within the broader context of AIGC, matting functions and
capabilities appear to be in high demand among users. They
can be extended in several ways, such as through scene pars-
ing and portrait editing. Combining matting abilities with
super-resolution and stylization models could enable diverse
intelligent creations. To maximize the effectiveness of mat-
ting functions, it would be useful to automatically identify
matting subjects based on visual and contextual content. This
could enable recommending relevant generated galleries to
users. This would not only help users showcase the beauty
of life’s moments, but also improve the user experience by
recommending recent and relevant content.
Advances in AI have enabled tremendous potential for au-
tomatic generation and creation. A wide range of innovative
applications have emerged in this field, particularly AIGC ap-
plications on mobile devices. Matting technology can help
users quickly and accurately isolate parts of images for edit-
ing, such as people, objects, and scenes. This makes it eas-
ier for users to perform various editing operations includ-
ing beautification, retouching, and compositing of specific
objects, thereby enhancing entertainment and visual effects.
However, automatic recommendation of AIGC in conjunc-
tion with matting engines remain underdeveloped. This could
improve the overall user experience and promote the develop-
ment of more efficient and effective creation and generation
applications on mobile devices.
Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence (IJCAI-23)
Demonstrations Track
7183
资源评论
百态老人
- 粉丝: 1184
- 资源: 2万+
下载权益
C知道特权
VIP文章
课程特权
VIP享7折，此内容立减5.97元 开通VIP
