A CNN-based In-loop Filtering Approach for AV1
Video Codec
Dandan Ding Guangyao Chen
Hangzhou Normal University
Hangzhou, China
DandanDing@hznu.edu.cn
Debargha Mukherjee Urvang Joshi Yue Chen
Google Inc.
Mountain View, United States
debargha@google.com
Abstract—In-loop filter using Convolutional Neural Network
(CNN) has lately attracted lots of attention in video coding.
CNN models may be trained to learn how to restore degradation
introduced by compression in pictures, and hence effectively help
improve the coding efficiency. State-of-the-art work in this field
generally employs a single network to enhance reconstructed
frames mainly in intra coding. In this paper, we develop a depth-
variable network handling both intra and inter coding. The depth
of our network is varied with the distortion levels of reconstructed
frames. Moreover, we leverage a skip enhancing strategy for
inter coding, which improves both the coding efficiency and the
resulting visual quality, while maintaining low computational
complexity. We apply our approach to AV1, a newly released
video coding standard from AOM. Experimental results show
that our approach achieves an average BD-rate reduction of
7.27% and 5.57% for intra and inter modes, respectively, com-
pared to AV1 anchor. The code and model of our approach are
published in our Github website [1].
Index Terms—CNN, video coding, AV1, in-loop filter, enhance-
ment, restoration.
I. INTRODUCTION
Modern video coding standards, such as H.264/AVC,
H.265/HEVC, and AV1, all adopt block-based prediction,
transform, and quantization. As a result, the reconstructed
frame suffers from visually annoying artifacts, such as block-
ing, ringing, and blurring artifacts, particularly at low bitrates.
In order to alleviate these artifacts, in-loop filters are usually
applied on the reconstructed frame, with an aim to improve
the video quality both subjectively and objectively.
Taking AV1 as an example, we will review the conven-
tional approach of designing in-loop filters. AV1 is a recently
released open source video coding standard designed and
developed by the Alliance of Open Media (AOM) [2]. It
adopts three kinds of in-loop filters, including the Deblocking
Filter, the Constrained Directional Enhancement Filter (CDEF)
[3], and the Loop-Restoration (LR) [4]. The three tools work
in a cascade mode. Firstly, the Deblocking filter is applied
to attenuate discontinuities at prediction and transform block
boundaries. Afterwards, CDEF conducts a direction-adaptive
nonlinear filtering for removing ringing and basis noise around
sharp edges and patterns in an image. The last filter, namely
LR, employs a switchable framework, considering that the
same restoration scheme may not be suitable to restore all parts
The work was supported by Google Chrome University Research Program.
of a frame. Two schemes are provided within the framework:
one is the separable symmetric Wiener filter and the other is
the dual self-guided filtering with subspace projection. Such
tools have achieved a bitrate saving as large as 1.5% to
2.0% [4]. The three types of filters introduced above can be
flexibly selected for each frame during encoding. Auxiliary
side-information required by each tool is sent in bitstream to
facilitate the decoding.
It can be seen that the traditional in-loop filtering tools
are generally originated from signal processing theory and
designed manually. Due to the limited adaptive capacity of
these tools, they may not be able to handle a variety of video
contents. Recently, there are many research efforts employing
Convolutional Neural Network (CNN) to solve the in-loop
filtering problem through learning. Significant gains have been
obtained over traditional methods, as summarized in Section
II. The basic idea is to employ a single powerful network to
address the in-loop filtering problem. Most existing work has
recognized that it is beneficial to train different CNN models
for different levels of Quantization Parameter (QP), indicating
that the distortion is relative to the selection of CNN model.
As such, we can further design different network structures
for different QPs to deal with different levels of distortion.
In addition, most previous work only integrates CNN into
intra coding process. More coding gains are expected from
inter coding since the frame enhanced by CNN will serve as
reference in subsequent coding.
In this work, we develop a depth-variable CNN filter for
both intra and inter video coding. We develop a novel ap-
proach from the perspective of automatic learning rather than
handcrafted design, aiming to provide further coding gains.
We apply the CNN-based in-loop filter on AV1 codec. It is
found that although the use of conventional in-loop filtering
technology has been extensively investigated in AV1, CNN
can still achieve superior performance.
II. RELATED WORK
Recently, there have been many research efforts employing
CNN to solve the in-loop [5] - [11] or out-loop filtering [12]-
[16] problems and significant gains have been obtained over
traditional methods.
Some CNN models are designed to replace the traditional
in-loop filters, and they have achieved significant gains [5]
978-1-7281-4704-8/19/$31.00 ©2019 IEEE
2019 Picture Coding Symposium (PCS)
Nov. 12-15, 2019, Ningbo, China
资源评论
码流怪侠
- 粉丝: 2w+
- 资源: 155
最新资源
- 白色简洁风格的户外旅行服务整站网站源码下载.zip
- 白色简洁风格的互联网推广企业网站源码下载.zip
- 白色简洁风格的户外自助游企业网站源码下载.zip
- 白色简洁风格的灰白相册网站模板下载.zip
- 白色简洁风格的花园草坪护理整站网站源码下载.zip
- 白色简洁风格的婚礼活动展示信息源码下载.zip
- 白色简洁风格的婚介服务动态源码下载.zip
- 白色简洁风格的婚礼电子请柬整站网站模板.zip
- 白色简洁风格的家居建材网站模板下载.zip
- 白色简洁风格的计划实现倒计时页源码下载.zip
- 白色简洁风格的技能展示企业网站模板.zip
- 白色简洁风格的家居装饰设计企业网站源码下载.zip
- 白色简洁风格的家居装修企业网站模板.zip
- 白色简洁风格的家具装饰品商城整站网站源码下载.zip
- 白色简洁风格的建筑工地企业网站模板.zip
- 白色简洁风格的建筑施工建设整站网站源码下载.zip
资源上传下载、课程学习等过程中有任何疑问或建议，欢迎提出宝贵意见哦~我们会及时处理！
点击此处反馈
