i
Copyright © 2016 Google, Inc. All Rights Reserved.
VP9$Bitstream$&$Decoding$Process$Specification$
!
Version!0.6!
!
!
Adrian!Grange,!Google!
Peter!de!Rivaz,!Argon!Design!
Jonathan!Hunt,!Argo n !De sign!
ABSTRACT$
This! document! defines! the! bitstream! format! and! decoding! process! for! the! Google!
VP9!video!codec.!
VP9 Bitstream & Decoding Process Specification - v0.6 31st March 2016
ii Copyright © 2016 Google, Inc. All Rights Reserved.
Contents Page
1 Scope ..................................................................................................................................................... 1!
2 Terms and definitions ............................................................................................................................. 2!
3 Symbols (and abbreviated terms) ........................................................................................................... 5!
4 Conventions ............................................................................................................................................ 7!
4.1 Arithmetic operators ............................................................................................................................... 7!
4.2 Logical operators .................................................................................................................................... 7!
4.3 Relational operators ............................................................................................................................... 7!
4.4 Bitwise operators .................................................................................................................................... 7!
4.5 Assignment ............................................................................................................................................. 8!
4.6 Mathematical functions ........................................................................................................................... 8!
4.7 Method of describing bitstream syntax ................................................................................................... 8!
4.8 Functions .............................................................................................................................................. 10!
4.9 Descriptors ........................................................................................................................................... 11!
4.9.1 f(n) ........................................................................................................................................................ 11!
4.9.2 s(n) ....................................................................................................................................................... 11!
4.9.3 B(p) ....................................................................................................................................................... 11!
4.9.4 L(n) ....................................................................................................................................................... 11!
4.9.5 T ........................................................................................................................................................... 11!
5 Overview of the decoding process (Informative) .................................................................................. 12!
5.1 Purpose of VP9 .................................................................................................................................... 12!
5.2 Compressing image data ...................................................................................................................... 12!
5.3 Quantization and lossy compression .................................................................................................... 13!
5.4 Predicting image data ........................................................................................................................... 13!
5.5 Inter prediction ...................................................................................................................................... 14!
5.6 Superblocks .......................................................................................................................................... 15!
5.7 Multiple transforms ............................................................................................................................... 16!
5.8 Inverse DCT structure .......................................................................................................................... 16!
5.9 Inverse ADST structure ........................................................................................................................ 19!
5.10 Reference frames ................................................................................................................................. 19!
5.11 Hidden frames ...................................................................................................................................... 20!
5.12 Compound prediction ........................................................................................................................... 20!
5.13 Motion vector prediction ....................................................................................................................... 20!
5.14 Tiles ...................................................................................................................................................... 20!
5.15 Segmentation map ............................................................................................................................... 21!
5.16 Reference frame scaling ....................................................................................................................... 21!
5.17 Arithmetic coding .................................................................................................................................. 21!
5.18 Probability updates ............................................................................................................................... 22!
5.19 Chroma format ...................................................................................................................................... 22!
5.20 High bit depth ....................................................................................................................................... 23!
5.21 Probability Contexts .............................................................................................................................. 23!
5.22 Zigzag ordering ..................................................................................................................................... 23!
5.23 Loop filter .............................................................................................................................................. 24!
5.24 Loop filter ordering and filters ............................................................................................................... 24!
5.25 Frame structure .................................................................................................................................... 25!
5.26 Superframes ......................................................................................................................................... 26!
6 Bitstream syntax ................................................................................................................................... 27!
6.1 Frame syntax ........................................................................................................................................ 27!
6.1.1 Trailing bits syntax ................................................................................................................................ 27!
VP9 Bitstream & Decoding Process Specification - v0.6 31st March 2016
Copyright © 2016 Google, Inc. All Rights Reserved
iii
6.1.2 Refresh probs syntax ........................................................................................................................... 27!
6.2 Uncompressed header syntax .............................................................................................................. 28!
6.2.1 Frame sync syntax ............................................................................................................................... 30!
6.2.2 Color config syntax ............................................................................................................................... 30!
6.2.3 Frame size syntax ................................................................................................................................ 30!
6.2.4 Render size syntax ............................................................................................................................... 31!
6.2.5 Frame size with refs syntax .................................................................................................................. 31!
6.2.6 Compute image size syntax ................................................................................................................. 31!
6.2.7 Interpolation filter syntax ...................................................................................................................... 31!
6.2.8 Loop filter params syntax ..................................................................................................................... 32!
6.2.9 Quantization params syntax ................................................................................................................. 32!
6.2.10 Delta quantizer syntax .......................................................................................................................... 33!
6.2.11 Segmentation params syntax ............................................................................................................... 33!
6.2.12 Probability syntax ................................................................................................................................. 34!
6.2.13 Tile info syntax ..................................................................................................................................... 34!
6.2.14 Tile size calculation .............................................................................................................................. 34!
6.3 Compressed header syntax ................................................................................................................. 35!
6.3.1 Tx mode syntax .................................................................................................................................... 35!
6.3.2 Tx mode probs syntax .......................................................................................................................... 36!
6.3.3 Diff update prob syntax ........................................................................................................................ 36!
6.3.4 Decode term subexp syntax ................................................................................................................. 36!
6.3.5 Inv remap prob syntax .......................................................................................................................... 37!
6.3.6 Inv recenter noneg syntax .................................................................................................................... 37!
6.3.7 Coef probs syntax ................................................................................................................................ 38!
6.3.8 Skip probs syntax ................................................................................................................................. 38!
6.3.9 Inter mode probs syntax ....................................................................................................................... 38!
6.3.10 Interp filter probs syntax ....................................................................................................................... 38!
6.3.11 Intra inter probs syntax ......................................................................................................................... 38!
6.3.12 Frame reference mode syntax ............................................................................................................. 39!
6.3.13 Frame reference mode probs syntax ................................................................................................... 39!
6.3.14 Y mode probs syntax ............................................................................................................................ 40!
6.3.15 Partition probs syntax ........................................................................................................................... 40!
6.3.16 MV probs syntax ................................................................................................................................... 40!
6.3.17 Update mv prob syntax ........................................................................................................................ 40!
6.3.18 Setup compound reference mode syntax ............................................................................................. 41!
6.4 Decode tiles syntax .............................................................................................................................. 41!
6.4.1 Get tile offset syntax ............................................................................................................................. 42!
6.4.2 Decode tile syntax ................................................................................................................................ 42!
6.4.3 Decode partition syntax ........................................................................................................................ 42!
6.4.4 Decode block syntax ............................................................................................................................ 43!
6.4.5 Mode info syntax .................................................................................................................................. 44!
6.4.6 Intra frame mode info syntax ................................................................................................................ 44!
6.4.7 Intra segment id syntax ........................................................................................................................ 44!
6.4.8 Skip syntax ........................................................................................................................................... 45!
6.4.9 Segmentation feature active syntax ..................................................................................................... 45!
6.4.10 Tx size syntax ....................................................................................................................................... 45!
6.4.11 Inter frame mode info syntax ................................................................................................................ 45!
6.4.12 Inter segment id syntax ........................................................................................................................ 46!
6.4.13 Is inter syntax ....................................................................................................................................... 46!
6.4.14 Get segment id syntax .......................................................................................................................... 47!
6.4.15 Intra block mode info syntax ................................................................................................................. 47!
6.4.16 Inter block mode info syntax ................................................................................................................. 48!
6.4.17 Ref frames syntax ................................................................................................................................ 49!
6.4.18 Assign MV syntax ................................................................................................................................. 49!
6.4.19 MV syntax ............................................................................................................................................. 50!
6.4.20 MV component syntax .......................................................................................................................... 50!
6.4.21 Residual syntax .................................................................................................................................... 50!
6.4.22 Get uv size syntax ................................................................................................................................ 52!
6.4.23 Get plane block size syntax .................................................................................................................. 52!
VP9 Bitstream & Decoding Process Specification - v0.6 31st March 2016
iv Copyright © 2016 Google, Inc. All Rights Reserved.
6.4.24 Token syntax ........................................................................................................................................ 52!
6.4.25 Get scan syntax .................................................................................................................................... 53!
6.4.26 Coef syntax ........................................................................................................................................... 54!
6.5 Motion vector prediction ....................................................................................................................... 55!
6.5.1 Find MV refs syntax .............................................................................................................................. 55!
6.5.2 Is inside syntax ..................................................................................................................................... 57!
6.5.3 Clamp mv ref syntax ............................................................................................................................. 57!
6.5.4 Clamp mv row syntax ........................................................................................................................... 57!
6.5.5 Clamp mv col syntax ............................................................................................................................ 58!
6.5.6 Add mv ref list syntax ........................................................................................................................... 58!
6.5.7 If same ref frame add syntax ................................................................................................................ 58!
6.5.8 If diff ref frame add syntax .................................................................................................................... 58!
6.5.9 Scale mv syntax ................................................................................................................................... 59!
6.5.10 Get block mv syntax ............................................................................................................................. 59!
6.5.11 Get sub block mv syntax ...................................................................................................................... 59!
6.5.12 Find best ref mvs syntax ....................................................................................................................... 59!
6.5.13 Use mv hp syntax ................................................................................................................................. 60!
6.5.14 Append sub8x8 mvs syntax ................................................................................................................. 60!
7 Bitstream semantics ............................................................................................................................. 62!
7.1 Frame semantics .................................................................................................................................. 62!
7.1.1 Trailing bits semantics .......................................................................................................................... 62!
7.1.2 Refresh probs semantics ...................................................................................................................... 62!
7.2 Uncompressed header semantics ........................................................................................................ 62!
7.2.1 Frame sync semantics .......................................................................................................................... 64!
7.2.2 Color config semantics ......................................................................................................................... 64!
7.2.3 Frame size semantics ........................................................................................................................... 65!
7.2.4 Render size semantics ......................................................................................................................... 65!
7.2.5 Frame size with refs semantics ............................................................................................................ 65!
7.2.6 Compute image size semantics ............................................................................................................ 66!
7.2.7 Interpolation filter semantics ................................................................................................................. 66!
7.2.8 Loop filter semantics ............................................................................................................................. 66!
7.2.9 Quantization params syntax ................................................................................................................. 67!
7.2.10 Segmentation params syntax ............................................................................................................... 67!
7.2.11 Tile info semantics ................................................................................................................................ 68!
7.3 Compressed header semantics ............................................................................................................ 68!
7.3.1 Tx mode semantics .............................................................................................................................. 68!
7.3.2 Diff update prob semantics ................................................................................................................... 69!
7.3.3 Decode term subexp semantics ........................................................................................................... 69!
7.3.4 Inv remap prob semantics .................................................................................................................... 69!
7.3.5 Coef prob semantics ............................................................................................................................. 69!
7.3.6 Frame reference mode semantics ........................................................................................................ 69!
7.3.7 Update mv prob semantics ................................................................................................................... 69!
7.4 Tile level ............................................................................................................................................... 70!
7.4.1 Decode tiles semantics ......................................................................................................................... 70!
7.4.2 Decode tile semantics .......................................................................................................................... 70!
7.4.3 Decode partition semantics .................................................................................................................. 70!
7.4.4 Decode block semantics ....................................................................................................................... 71!
7.4.5 Intra frame mode info semantics .......................................................................................................... 71!
7.4.6 Intra and inter segment id semantics .................................................................................................... 71!
7.4.7 Skip semantics ..................................................................................................................................... 72!
7.4.8 Tx size semantics ................................................................................................................................. 72!
7.4.9 Is inter semantics .................................................................................................................................. 72!
7.4.10 Intra block mode info semantics ........................................................................................................... 72!
7.4.11 Inter block mode info semantics ........................................................................................................... 72!
7.4.12 Ref frames semantics ........................................................................................................................... 73!
7.4.13 MV semantics ....................................................................................................................................... 73!
7.4.14 MV component semantics .................................................................................................................... 73!
VP9 Bitstream & Decoding Process Specification - v0.6 31st March 2016
Copyright © 2016 Google, Inc. All Rights Reserved
v
7.4.15 Residual semantics .............................................................................................................................. 74!
7.4.16 Token semantics .................................................................................................................................. 74!
7.4.17 Coef semantics ..................................................................................................................................... 75!
8 Decoding process ................................................................................................................................. 76!
8.1 General ................................................................................................................................................. 76!
8.2 Frame order constraints ....................................................................................................................... 76!
8.3 Clear counts process ............................................................................................................................ 76!
8.4 Probability adaptation process ............................................................................................................. 77!
8.4.1 Merge prob process ............................................................................................................................. 77!
8.4.2 Merge probs process ............................................................................................................................ 77!
8.4.3 Coefficient probability adaption process ............................................................................................... 78!
8.4.4 Non coefficient probability adaption process ........................................................................................ 79!
8.5 Prediction processes ............................................................................................................................ 80!
8.5.1 Intra prediction process ........................................................................................................................ 80!
8.5.2 Inter prediction process ........................................................................................................................ 83!
8.6 Reconstruction and dequantization ...................................................................................................... 89!
8.6.1 Dequantization functions ...................................................................................................................... 89!
8.6.2 Reconstruct process ............................................................................................................................. 94!
8.7 Inverse transform process .................................................................................................................... 94!
8.7.1 1D Transforms ...................................................................................................................................... 94!
8.7.2 2D Inverse Transform ......................................................................................................................... 100!
8.8 Loop filter process .............................................................................................................................. 100!
8.8.1 Loop filter frame init process .............................................................................................................. 101!
8.8.2 Superblock loop filter process ............................................................................................................ 101!
8.8.3 Filter size process .............................................................................................................................. 103!
8.8.4 Adaptive filter strength process .......................................................................................................... 104!
8.8.5 Sample filtering process ..................................................................................................................... 105!
8.9 Output process ................................................................................................................................... 109!
8.10 Reference frame update process ....................................................................................................... 110!
9 Parsing Process ................................................................................................................................. 111!
9.1 Parsing process for f(n) ...................................................................................................................... 111!
9.2 Parsing process for Boolean decoder ................................................................................................ 111!
9.2.1 Initialization process for Boolean decoder .......................................................................................... 111!
9.2.2 Boolean decoding process ................................................................................................................. 111!
9.2.3 Exit process for Boolean decoder ...................................................................................................... 112!
9.2.4 Parsing process for read_literal .......................................................................................................... 112!
9.3 Parsing process for tree encoded syntax elements ........................................................................... 112!
9.3.1 Tree selection process ....................................................................................................................... 113!
9.3.2 Probability selection process .............................................................................................................. 116!
9.3.3 Tree decoding process ....................................................................................................................... 125!
9.3.4 Syntax element counting process ...................................................................................................... 125!
10 Additional tables ................................................................................................................................. 127!
10.1 Scan tables ......................................................................................................................................... 127!
10.2 Conversion tables ............................................................................................................................... 131!
10.3 Pareto probability table ....................................................................................................................... 135!
10.4 Fixed probability tables ....................................................................................................................... 138!
10.5 Default probability tables .................................................................................................................... 141!
Annex A Levels ............................................................................................................................................... 162!
A.1 Overview ............................................................................................................................................ 162!
Annex B Superframes ..................................................................................................................................... 163!
B.1 Overview ............................................................................................................................................ 163!
B.2 Superframe syntax ............................................................................................................................. 163!
B.2.1 Superframe index ............................................................................................................................... 163!
B.2.2 Superframe header syntax ................................................................................................................. 163!
B.3 Superframe semantics ....................................................................................................................... 163!
B.4 Superframe parsing ............................................................................................................................ 164!
