h264协议(中英文对照)H264解码手册资源-CSDN文库

需积分: 9 42 浏览量 2012-05-17 16:17:30 上传评论收藏 2.25MB PDF 举报

资源详情

资源评论

资源推荐

DRAFT ISO/IEC 14496-10 : 2002 (E)

DRAFT ITU-T Rec. H.264 (2002 E) 沈燕飞译 0 . 5 版 i

Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG

(ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6)

Meeting: Pattaya, Thailand, 7-14 March, 2003

Document: JVT-G050

Filename: JVT-G050d35.doc

Title:

Draft ITU-T Recommendation and Final Draft International Standard

of Joint Video Specification (ITU-T Rec. H.264 | ISO/IEC 14496-10 AVC)

Status: Approved Output Document of JVT

Purpose: Text

Author(s) or

Contact(s):

Thomas Wiegand

Heinrich Hertz Institute (FhG),

Einsteinufer 37, D-10587 Berlin,

Germany

Gary Sullivan

Microsoft Corporation

One Microsoft Way

Redmond, WA 98052 USA

Tel:

Fax:

Email:

Tel:

Fax:

Email:

+49 - 30 - 31002 617

+49 - 30 - 392 72 00

wiegand@hhi.de

+1 (425) 703-5308

+1 (425) 706-7329

garysull@microsoft.com

Source: Editor

_____________________________

This document is an output document to the March 2003 Pattaya meeting of the JVT.

Title page to be provided by ITU-T | ISO/IEC

DRAFT INTERNATIONAL STANDARD

DRAFT ISO/IEC 14496-10 : 2002 (E)

DRAFT ITU-T Rec. H.264 (2002 E)

DRAFT ITU-T RECOMMENDATION

TABLE OF CONTENTS

Foreword............................................................................................................................................................................xi

0 Introduction ..............................................................................................................................................................xii

0.1 Prologue..............................................................................................................................................................xii

0.2 Purpose ...............................................................................................................................................................xii

0.3 Applications........................................................................................................................................................xii

0.4 Profiles and levels...............................................................................................................................................xii

0.5 Overview of the design characteristics ..............................................................................................................xiii

0.5.1 Predictive coding .......................................................................................................................................xiii

0.5.2 Coding of progressive and interlaced video...............................................................................................xiii

0.5.3 Picture partitioning into macroblocks and smaller partitions.....................................................................xiii

0.5.4 Spatial redundancy reduction.....................................................................................................................xiv

0.6 How to read this specification ...........................................................................................................................xiv

1 Scope............................................................................................................................................................................1

2 Normative references..................................................................................................................................................1

3 Definitions ...................................................................................................................................................................1

4 Abbreviations..............................................................................................................................................................7

5 Conventions.................................................................................................................................................................8

5.1 Arithmetic operators .............................................................................................................................................8

5.2 Logical operators ..................................................................................................................................................8

5.3 Relational operators..............................................................................................................................................9

DRAFT ISO/IEC 14496-10 : 2002 (E)

ii DRAFT ITU-T Rec. H.264 (2002 E) 沈燕飞译 0 . 5 版

5.4

Bit-wise operators.................................................................................................................................................9

5.5 Assignment operators ...........................................................................................................................................9

5.6 Range notation......................................................................................................................................................9

5.7 Mathematical functions ........................................................................................................................................ 9

5.8 Variables, syntax elements, and tables ............................................................................................................... 10

5.9 Text description of logical operations.................................................................................................................11

5.10 Processes............................................................................................................................................................. 12

6 Source, coded, decoded, output data formats, scanning processes, and neighbouring relationships................12

6.1 Bitstream formats ...............................................................................................................................................12

6.2 Source, decoded, and output picture formats......................................................................................................12

6.3 Spatial subdivision of pictures and slices ...........................................................................................................14

6.4 Inverse scanning processes and derivation processes for neighbours................................................................. 15

6.4.1 Inverse macroblock scanning process......................................................................................................... 15

6.4.2 Inverse macroblock partition and sub-macroblock partition scanning process...........................................15

6.4.2.1 Inverse macroblock partition scanning process ......................................................................................16

6.4.2.2 Inverse sub-macroblock partition scanning process ...............................................................................16

6.4.3 Inverse 4x4 luma block scanning process...................................................................................................17

6.4.4 Derivation process of the availability for macroblock addresses................................................................ 17

6.4.5 Derivation process for neighbouring macroblock addresses and their availability..................................... 17

6.4.6 Derivation process for neighbouring macroblock addresses and their availability in MBAFF frames ...... 18

6.4.7 Derivation processes for neighbouring macroblocks, blocks, and partitions..............................................19

6.4.7.1 Derivation process for neighbouring macroblocks .................................................................................19

6.4.7.2 Derivation process for neighbouring 8x8 luma block.............................................................................20

6.4.7.3 Derivation process for neighbouring 4x4 luma blocks ...........................................................................20

6.4.7.4 Derivation process for neighbouring 4x4 chroma blocks .......................................................................21

6.4.7.5 Derivation process for neighbouring partitions.......................................................................................21

6.4.8 Derivation process for neighbouring locations ........................................................................................... 22

6.4.8.1 Specification for neighbouring luma locations in fields and non-MBAFF frames ................................. 22

6.4.8.2 Specification for neighbouring luma locations in MBAFF frames.........................................................23

7 Syntax and semantics ...............................................................................................................................................25

7.1 Method of describing syntax in tabular form......................................................................................................25

7.2 Specification of syntax functions, categories, and descriptors............................................................................26

7.3 Syntax in tabular form ........................................................................................................................................ 28

7.3.1 NAL unit syntax ......................................................................................................................................... 28

7.3.2 Raw byte sequence payloads and RBSP trailing bits syntax ...................................................................... 29

7.3.2.1 Sequence parameter set RBSP syntax..................................................................................................... 29

7.3.2.2 Picture parameter set RBSP syntax......................................................................................................... 30

7.3.2.3 Supplemental enhancement information RBSP syntax........................................................................... 31

7.3.2.3.1 Supplemental enhancement information message syntax ................................................................31

7.3.2.4 Access unit delimiter RBSP syntax ........................................................................................................31

7.3.2.5 End of sequence RBSP syntax................................................................................................................ 31

7.3.2.6 End of stream RBSP syntax....................................................................................................................32

7.3.2.7 Filler data RBSP syntax.......................................................................................................................... 32

7.3.2.8 Slice layer without partitioning RBSP syntax......................................................................................... 32

7.3.2.9 Slice data partition RBSP syntax ............................................................................................................ 32

7.3.2.9.1 Slice data partition A RBSP syntax..................................................................................................32

7.3.2.9.2 Slice data partition B RBSP syntax..................................................................................................32

7.3.2.9.3 Slice data partition C RBSP syntax..................................................................................................33

7.3.2.10 RBSP slice trailing bits syntax..............................................................................................................33

7.3.2.11 RBSP trailing bits syntax ......................................................................................................................33

7.3.3 Slice header syntax ..................................................................................................................................... 34

7.3.3.1 Reference picture list reordering syntax .................................................................................................35

7.3.3.2 Prediction weight table syntax................................................................................................................ 36

7.3.3.3 Decoded reference picture marking syntax............................................................................................. 37

7.3.4 Slice data syntax ......................................................................................................................................... 38

7.3.5 Macroblock layer syntax............................................................................................................................. 39

7.3.5.1 Macroblock prediction syntax................................................................................................................. 40

7.3.5.2 Sub-macroblock prediction syntax .........................................................................................................41

7.3.5.3 Residual data syntax ...............................................................................................................................42

7.3.5.3.1 Residual block CAVLC syntax........................................................................................................ 43

7.3.5.3.2 Residual block CABAC syntax........................................................................................................44

7.4 Semantics............................................................................................................................................................ 45

7.4.1 NAL unit semantics ....................................................................................................................................45

DRAFT ISO/IEC 14496-10 : 2002 (E)

DRAFT ITU-T Rec. H.264 (2002 E) 沈燕飞译 0 . 5 版 iii

7.4.1.1

Encapsulation of an SODB within an RBSP (informative) ....................................................................47

7.4.1.2 Order of NAL units and association to coded pictures, access units, and video sequences....................48

7.4.1.2.1 Order of sequence and picture parameter set RBSPs and their activation .......................................48

7.4.1.2.2 Order of access units and association to coded video sequences ..................................................... 48

7.4.1.2.3 Order of NAL units and coded pictures and association to access units..........................................49

7.4.1.2.4 Detection of the first VCL NAL unit of a primary coded picture ....................................................50

7.4.1.2.5 Order of VCL NAL units and association to coded pictures............................................................50

7.4.2 Raw byte sequence payloads and RBSP trailing bits semantics .................................................................51

7.4.2.1 Sequence parameter set RBSP semantics ............................................................................................... 51

7.4.2.2 Picture parameter set RBSP semantics ...................................................................................................53

7.4.2.3 Supplemental enhancement information RBSP semantics .....................................................................55

7.4.2.3.1 Supplemental enhancement information message semantics...........................................................55

7.4.2.4 Access unit delimiter RBSP semantics ................................................................................................... 55

7.4.2.5 End of sequence RBSP semantics........................................................................................................... 56

7.4.2.6 End of stream RBSP semantics............................................................................................................... 56

7.4.2.7 Filler data RBSP semantics..................................................................................................................... 56

7.4.2.8 Slice layer without partitioning RBSP semantics ...................................................................................56

7.4.2.9 Slice data partition RBSP semantics....................................................................................................... 56

7.4.2.9.1 Slice data partition A RBSP semantics ............................................................................................56

7.4.2.9.2 Slice data partition B RBSP semantics ............................................................................................ 56

7.4.2.9.3 Slice data partition C RBSP semantics ............................................................................................ 57

7.4.2.10 RBSP slice trailing bits semantics.........................................................................................................57

7.4.2.11 RBSP trailing bits semantics.................................................................................................................57

7.4.3 Slice header semantics................................................................................................................................ 57

7.4.3.1 Reference picture list reordering semantics ............................................................................................ 62

7.4.3.2 Prediction weight table semantics........................................................................................................... 63

7.4.3.3 Decoded reference picture marking semantics .......................................................................................64

7.4.4 Slice data semantics.................................................................................................................................... 66

7.4.5 Macroblock layer semantics .......................................................................................................................66

7.4.5.1 Macroblock prediction semantics ...........................................................................................................73

7.4.5.2 Sub-macroblock prediction semantics ....................................................................................................73

7.4.5.3 Residual data semantics..........................................................................................................................76

7.4.5.3.1 Residual block CAVLC semantics................................................................................................... 76

7.4.5.3.2 Residual block CABAC semantics ..................................................................................................76

8 Decoding process.......................................................................................................................................................77

8.1 NAL unit decoding process ................................................................................................................................ 78

8.2 Slice decoding process........................................................................................................................................78

8.2.1 Decoding process for picture order count................................................................................................... 78

8.2.1.1 Decoding process for picture order count type 0 ....................................................................................80

8.2.1.2 Decoding process for picture order count type 1 ....................................................................................80

8.2.1.3 Decoding process for picture order count type 2 ....................................................................................81

8.2.2 Decoding process for macroblock to slice group map................................................................................82

8.2.2.1 Specification for interleaved slice group map type................................................................................. 83

8.2.2.2 Specification for dispersed slice group map type ...................................................................................83

8.2.2.3 Specification for foreground with left-over slice group map type .......................................................... 83

8.2.2.4 Specification for box-out slice group map types.....................................................................................84

8.2.2.5 Specification for raster scan slice group map types................................................................................ 84

8.2.2.6 Specification for wipe slice group map types .........................................................................................85

8.2.2.7 Specification for explicit slice group map type.......................................................................................85

8.2.2.8 Specification for conversion of map unit to slice group map to macroblock to slice group map ........... 85

8.2.3 Decoding process for slice data partitioning...............................................................................................85

8.2.4 Decoding process for reference picture lists construction ..........................................................................86

8.2.4.1 Decoding process for picture numbers....................................................................................................86

8.2.4.2 Initialisation process for reference picture lists ...................................................................................... 87

8.2.4.2.1 Initialisation process for the reference picture list for P and SP slices in frames............................. 87

8.2.4.2.2 Initialisation process for the reference picture list for P and SP slices in fields............................... 88

8.2.4.2.3 Initialisation process for reference picture lists for B slices in frames............................................. 88

8.2.4.2.4 Initialisation process for reference picture lists for B slices in fields............................................... 89

8.2.4.2.5 Initialisation process for reference picture lists in fields..................................................................90

8.2.4.3 Reordering process for reference picture lists......................................................................................... 90

8.2.4.3.1 Reordering process of reference picture lists for short-term pictures ..............................................91

8.2.4.3.2 Reordering process of reference picture lists for long-term pictures ...............................................92

8.2.5 Decoded reference picture marking process ...............................................................................................92

DRAFT ISO/IEC 14496-10 : 2002 (E)

iv DRAFT ITU-T Rec. H.264 (2002 E) 沈燕飞译 0 . 5 版

8.2.5.1

Sequence of operations for decoded reference picture marking process ................................................ 92

8.2.5.2 Decoding process for gaps in frame_num...............................................................................................93

8.2.5.3 Sliding window decoded reference picture marking process..................................................................93

8.2.5.4 Adaptive memory control decoded reference picture marking process .................................................. 93

8.2.5.4.1 Marking process of a short-term picture as “unused for reference”................................................. 94

8.2.5.4.2 Marking process of a long-term picture as “unused for reference”..................................................94

8.2.5.4.3 Assignment process of a LongTermFrameIdx to a short-term reference picture............................. 94

8.2.5.4.4 Decoding process for MaxLongTermFrameIdx...............................................................................95

8.2.5.4.5 Marking process of all reference pictures as “unused for reference” and setting

MaxLongTermFrameIdx to “no long-term frame indices”................................................................................. 95

8.2.5.4.6 Process for assigning a long-term frame index to the current picture .............................................. 95

8.3 Intra prediction process.......................................................................................................................................95

8.3.1 Intra_4x4 prediction process for luma samples .......................................................................................... 96

8.3.1.1 Derivation process for the Intra4x4PredMode........................................................................................96

8.3.1.2 Intra_4x4 sample prediction ...................................................................................................................98

8.3.1.2.1 Specification of Intra_4x4_Vertical prediction mode ...................................................................... 98

8.3.1.2.2 Specification of Intra_4x4_Horizontal prediction mode..................................................................98

8.3.1.2.3 Specification of Intra_4x4_DC prediction mode .............................................................................99

8.3.1.2.4 Specification of Intra_4x4_Diagonal_Down_Left prediction mode ................................................ 99

8.3.1.2.5 Specification of Intra_4x4_Diagonal_Down_Right prediction mode..............................................99

8.3.1.2.6 Specification of Intra_4x4_Vertical_Right prediction mode ......................................................... 100

8.3.1.2.7 Specification of Intra_4x4_Horizontal_Down prediction mode .................................................... 100

8.3.1.2.8 Specification of Intra_4x4_Vertical_Left prediction mode ...........................................................100

8.3.1.2.9 Specification of Intra_4x4_Horizontal_Up prediction mode ......................................................... 101

8.3.2 Intra_16x16 prediction process for luma samples ....................................................................................101

8.3.2.1 Specification of Intra_16x16_Vertical prediction mode....................................................................... 102

8.3.2.2 Specification of Intra_16x16_Horizontal prediction mode................................................................... 102

8.3.2.3 Specification of Intra_16x16_DC prediction mode .............................................................................. 102

8.3.2.4 Specification of Intra_16x16_Plane prediction mode...........................................................................102

8.3.3 Intra prediction process for chroma samples ............................................................................................103

8.3.3.1 Specification of Intra_Chroma_DC prediction mode ........................................................................... 104

8.3.3.2 Specification of Intra_Chroma_Horizontal prediction mode................................................................105

8.3.3.3 Specification of Intra_Chroma_Vertical prediction mode.................................................................... 105

8.3.3.4 Specification of Intra_Chroma_Plane prediction mode........................................................................ 105

8.3.4 Sample construction process for I_PCM macroblocks ............................................................................. 106

8.4 Inter prediction process.....................................................................................................................................106

8.4.1 Derivation process for motion vector components and reference indices................................................. 108

8.4.1.1 Derivation process for luma motion vectors for skipped macroblocks in P and SP slices.................... 108

8.4.1.2 Derivation process for luma motion vectors for B_Skip, B_Direct_16x16, and B_Direct_8x8........... 109

8.4.1.2.1 Derivation process for the co-located 4x4 sub-macroblock partitions ........................................... 109

8.4.1.2.2 Derivation process for spatial direct luma motion vector and reference index prediction mode ... 112

8.4.1.2.3 Derivation process for temporal direct luma motion vector and reference index prediction mode113

8.4.1.3 Derivation process for luma motion vector prediction .........................................................................115

8.4.1.3.1 Derivation process for median luma motion vector prediction ...................................................... 116

8.4.1.3.2 Derivation process for motion data of neighbouring partitions......................................................116

8.4.1.4 Derivation process for chroma motion vectors ..................................................................................... 117

8.4.2 Decoding process for Inter prediction samples.........................................................................................118

8.4.2.1 Reference picture selection process ......................................................................................................118

8.4.2.2 Fractional sample interpolation process................................................................................................ 119

8.4.2.2.1 Luma sample interpolation process................................................................................................ 120

8.4.2.2.2 Chroma sample interpolation process ............................................................................................122

8.4.2.3 Weighted sample prediction process ....................................................................................................123

8.4.2.3.1 Default weighted sample prediction process.................................................................................. 124

8.4.2.3.2 Weighted sample prediction process..............................................................................................124

8.5 Transform coefficient decoding process and picture construction process prior to deblocking filter process..126

8.5.1 Specification of transform decoding process for residual blocks..............................................................126

8.5.2 Specification of transform decoding process for luma samples of Intra_16x16 macroblock prediction

mode 127

8.5.3 Specification of transform decoding process for chroma samples............................................................ 128

8.5.4 Inverse scanning process for transform coefficients.................................................................................129

8.5.5 Derivation process for the quantisation parameters and scaling function................................................. 129

8.5.6 Scaling and transformation process for luma DC transform coefficients for Intra_16x16 macroblock type

130

DRAFT ISO/IEC 14496-10 : 2002 (E)

DRAFT ITU-T Rec. H.264 (2002 E) 沈燕飞译 0 . 5 版 v

8.5.7

Scaling and transformation process for chroma DC transform coefficients ............................................. 131

8.5.8 Scaling and transformation process for residual 4x4 blocks.....................................................................131

8.5.9 Picture construction process prior to deblocking filter process ................................................................ 133

8.6 Decoding process for P macroblocks in SP slices or SI macroblocks .............................................................. 133

8.6.1 SP decoding process for non-switching pictures ...................................................................................... 134

8.6.1.1 Luma transform coefficient decoding process ...................................................................................... 134

8.6.1.2 Chroma transform coefficient decoding process...................................................................................135

8.6.2 SP and SI slice decoding process for switching pictures .......................................................................... 136

8.6.2.1 Luma transform coefficient decoding process ...................................................................................... 137

8.6.2.2 Chroma transform coefficient decoding process...................................................................................137

8.7 Deblocking filter process.................................................................................................................................. 138

8.7.1 Filtering process for block edges .............................................................................................................. 141

8.7.2 Filtering process for a set of samples across a horizontal or vertical block edge ..................................... 142

8.7.2.1 Derivation process for the luma content dependent boundary filtering strength .................................. 143

8.7.2.2 Derivation process for the thresholds for each block edge ...................................................................144

8.7.2.3 Filtering process for edges with bS less than 4..................................................................................... 145

8.7.2.4 Filtering process for edges for bS equal to 4 ........................................................................................146

9 Parsing process .......................................................................................................................................................147

9.1 Parsing process for Exp-Golomb codes............................................................................................................147

9.1.1 Mapping process for signed Exp-Golomb codes ...................................................................................... 149

9.1.2 Mapping process for coded block pattern................................................................................................. 149

9.2 CAVLC parsing process for transform coefficient levels................................................................................. 151

9.2.1 Parsing process for total number of transform coefficient levels and trailing ones .................................. 151

9.2.2 Parsing process for level information ....................................................................................................... 154

9.2.3 Parsing process for run information.......................................................................................................... 156

9.2.4 Combining level and run information.......................................................................................................158

9.3 CABAC parsing process for slice data ............................................................................................................. 159

9.3.1 Initialisation process ................................................................................................................................. 160

9.3.1.1 Initialisation process for context variables............................................................................................ 160

9.3.1.2 Initialisation process for the arithmetic decoding engine......................................................................170

9.3.2 Binarization process.................................................................................................................................. 170

9.3.2.1 Unary (U) binarization process.............................................................................................................172

9.3.2.2 Truncated unary (TU) binarization process ..........................................................................................172

9.3.2.3 Concatenated unary/ k-th order Exp-Golomb (UEGk) binarization process ........................................ 172

9.3.2.4 Fixed-length (FL) binarization process................................................................................................. 173

9.3.2.5 Binarization process for macroblock type and sub-macroblock type ................................................... 173

9.3.2.6 Binarization process for coded block pattern........................................................................................ 176

9.3.2.7 Binarization process for mb_qp_delta .................................................................................................. 176

9.3.3 Decoding process flow..............................................................................................................................176

9.3.3.1 Derivation process for ctxIdx................................................................................................................ 177

9.3.3.1.1 Assignment process of ctxIdxInc using neighbouring syntax elements......................................... 179

9.3.3.1.1.1 Derivation process of ctxIdxInc for the syntax element mb_skip_flag...................................179

9.3.3.1.1.2 Derivation process of ctxIdxInc for the syntax element mb_field_decoding_flag.................. 179

9.3.3.1.1.3 Derivation process of ctxIdxInc for the syntax element mb_type...........................................180

9.3.3.1.1.4 Derivation process of ctxIdxInc for the syntax element coded_block_pattern ....................... 180

9.3.3.1.1.5 Derivation process of ctxIdxInc for the syntax element mb_qp_delta.................................... 181

9.3.3.1.1.6 Derivation process of ctxIdxInc for the syntax elements ref_idx_l0 and ref_idx_l1 .............. 181

9.3.3.1.1.7 Derivation process of ctxIdxInc for the syntax elements mvd_l0 and mvd_l1 ....................... 182

9.3.3.1.1.8 Derivation process of ctxIdxInc for the syntax element intra_chroma_pred_mode................182

9.3.3.1.1.9 Derivation process of ctxIdxInc for the syntax element coded_block_flag ............................ 183

9.3.3.1.2 Assignment process of ctxIdxInc using prior decoded bin values ................................................. 184

9.3.3.1.3 Assignment process of ctxIdxInc for syntax elements significant_coeff_flag,

last_significant_coeff_flag, and coeff_abs_level_minus1 ................................................................................ 184

9.3.3.2 Arithmetic decoding process.................................................................................................................185

9.3.3.2.1 Arithmetic decoding process for a binary decision........................................................................ 186

9.3.3.2.1.1 State transition process............................................................................................................187

9.3.3.2.2 Renormalization process in the arithmetic decoding engine.......................................................... 189

9.3.3.2.3 Bypass decoding process for binary decisions............................................................................... 189

9.3.3.2.4 Decoding process for binary decisions before termination ............................................................ 190

9.3.4 Arithmetic encoding process (informative) .............................................................................................. 191

9.3.4.1 Initialisation process for the arithmetic encoding engine (informative) ............................................... 191

9.3.4.2 Encoding process for a binary decision (informative) ..........................................................................191

9.3.4.3 Renormalization process in the arithmetic encoding engine (informative)........................................... 192

剩余359页未读，继续阅读