Slimmable Video Codec
Zhaocheng Liu
1
, Luis Herranz
2
*
, Fei Yang
2
, Saiping Zhang
4
, Shuai Wan
1
, Marta Mrak
3
and Marc G
´
orriz Blanch
3
1
School of Electronics and Information, Northwestern Polytechnical University, Xi’an, China
2
Computer Vision Center, Universitat Autonoma de Barcelona, 08193 Barcelona, Spain
3
BBC Research & Development, The Lighthouse, White City Place, 201 Wood Lane, London, UK
4
State Key Laboratory of Integrated Services Networks, Xidian University, Xi’an, China
liuzhaocheng@mail.nwpu.edu.cn
Abstract
Neural video compression has emerged as a novel
paradigm combining trainable multilayer neural net-
works and machine learning, achieving competitive rate-
distortion (RD) performances, but still remaining imprac-
tical due to heavy neural architectures, with large memory
and computational demands. In addition, models are usu-
ally optimized for a single RD tradeoff. Recent slimmable
image codecs can dynamically adjust their model capacity
to gracefully reduce the memory and computation require-
ments, without harming RD performance. In this paper we
propose a slimmable video codec (SlimVC), by integrating a
slimmable temporal entropy model in a slimmable autoen-
coder. Despite a significantly more complex architecture,
we show that slimming remains a powerful mechanism to
control rate, memory footprint, computational cost and la-
tency, all being important requirements for practical video
compression.
1. Introduction
During the last two decades, video has become the domi-
nant form of communication of the digital society. This has
led to an explosive growth where video content accounts
for more than 80% of global data traffic. The basic (lossy)
video compression objective consists of transmitting as few
bits as possible (i.e. minimize rate) while representing the
input sequence at a certain level of fidelity (i.e. distortion).
Video is now consumed using heterogeneous devices rang-
ing from TV sets to smartphones. Furthermore, real-time
video conferencing has become a household technology,
pervasive in work and educational environments. These
practical scenarios imposes additional constraints to the de-
*
L.H. acknowledges the support of the Ram
´
on y Cajal grant RYC2019-
027020-I (MICINN, Spain).
sign of video codec in practice, such as dynamically con-
trollable rate, low computational and memory footprint, and
low latency. Together with the previous rate and distortion
objectives, they conform the more challenging problem of
practical video compression.
In parallel, the deep learning revolution has motivated a
new compression paradigm based on parametric encoders
and decoders implemented as deep neural networks which
are optimized with data. This compression approach has
been applied successfully first in images [4, 5, 7] and then
videos [6, 13]. This paradigm contrasts with the tradi-
tional hybrid video coding paradigm, based on block-based
linear transforms and carefully engineered coding tools
(e.g. H.264/AVC, H.265/HEVC). Focusing on improving
rate-distortion performance, most neural image and video
codecs are impractical, since require heavy and complex
networks. Practical aspects have been always carefully con-
sidered in the design of traditional codecs. In contrast to
previous works, our paper focuses chiefly on those practical
constraints, proposing a lightweight and flexible design for
practical neural video compression.
Our design is based on a slimmable autoencoder aug-
mented with a slimmable temporal entropy model. This
design is motivated by two recent works. Motivated by
the empirical observation that lower rates do not require
the use of full capacity, Yang et al. [12] proposed the
slimmable compressive autoencoder (SlimCAE) architec-
ture, where the slimming becomes a flexible mechanism to
both vary the rate-distortion tradeoff and control the com-
plexity. However, extending SlimCAE to video by includ-
ing temporal prediction is not trivial, since most designs re-
quire additional modules to estimate and compensate mo-
tion (e.g. optical flow nets, motion compensation nets).
Slimmable designs of such modules are not straightforward,
nor the potential interplay with other elements in the com-
pression framework. Recently, Sun et al. [9] proposed spa-
tiotemporal entropy model (STEM), a motion-free frame-
资源评论
码流怪侠
- 粉丝: 2w+
- 资源: 432
最新资源
- 3b015大学生创业项目管理系统_springboot+vue0.zip
- 3b014宠物猫店管理系统_springboot+vue.zip
- 3b016个性化课程推荐系统_springboot+vue.zip
- 3b017旅游景区预约服务系统_springboot+vue0.zip
- 考务报名平台论--论文pf-springboot毕业项目，适合计算机毕-设、实训项目、大作业学习.zip
- 疫情居家办公系统--论文pf-springboot毕业项目，适合计算机毕-设、实训项目、大作业学习.zip
- 考勤管理系统代码-springboot毕业项目，适合计算机毕-设、实训项目、大作业学习.zip
- 打印室预约小程序-毕业设计，基于Python+Django+Vue+MySql开发，前后端分离，源码+数据库+毕业论文+视频演示
- 海淀数学七上期末， 海淀数学七上期末
- 基于C#人事管理系统项目(源代码+论文+答辩PPT)
- 3b019企业人事管理系统_springboot+vue.zip
- 3b018企业人力资源管理系统_springboot+vue.zip
- 地理位置点定义，基于Qt封装，方便使用
- 学校防疫物资管理平台的设计与实现boot--论文pf-springboot毕业项目，适合计算机毕-设、实训项目、大作业学习.zip
- 3b020汽车配件销售管理系统_springboot+vue.zip
- 3b022投票系统_springboot+vue.zip
资源上传下载、课程学习等过程中有任何疑问或建议，欢迎提出宝贵意见哦~我们会及时处理！
点击此处反馈
