OverviewoftheH.264/AVCvideocodingstandard资源-CSDN文库

共1个文件

pdf：1个

h.264

需积分: 3 196 浏览量 2008-07-07 09:58:29 上传评论收藏 2.71MB ZIP 举报

资源详情

资源评论

资源推荐

收起资源包目录

csvt_overview.zip （1个子文件）

csvt_overview.pdf 2.92MB

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, JULY 2003 1

Abstract— H.264/AVC is newest video coding standard of the

ITU-T Video Coding Experts Group (VCEG) and the

ISO/IEC Moving Picture Experts Group (MPEG). The main

goals of the H.264/AVC standardization effort have been

enhanced compression performance and provision of a

"network-friendly" video representation addressing

"conversational" (video telephony) and "non-conversational"

(storage, broadcast, or streaming) applications. H.264/AVC

has achieved a significant improvement in rate-distortion

efficiency relative to existing standards. This article provides

an overview of the technical features of H.264/AVC, describes

profiles and applications for the standard, and outlines the

history of the standardization process.

Index Terms—Video, Standards, MPEG-2, H.263, MPEG-4,

AVC, H.264, JVT.

I. INTRODUCTION

is the newest international

video coding standard [1]. By

the time of this publication, it

is expected to have been approved by ITU-T as

Recommendation H.264 and by ISO/IEC as International

Standard 14496-10 (MPEG-4 part 10) Advanced Video

Coding (AVC).

The MPEG-2 video coding standard (also known as ITU-T

H.262) [2], which was developed about 10 years ago

primarily as an extension of prior MPEG-1 video capability

with support of interlaced video coding, was an enabling

technology for digital television systems worldwide. It is

widely used for the transmission of standard definition (SD)

and High Definition (HD) TV signals over satellite, cable,

and terrestrial emission and the storage of high-quality SD

video signals onto DVDs.

However, an increasing number of services and growing

popularity of high definition TV are creating greater needs

for higher coding efficiency. Moreover, other transmission

media such as Cable Modem, xDSL or UMTS offer much

lower data rates than broadcast channels, and enhanced

coding efficiency can enable the transmission of more video

channels or higher quality video representations within

existing digital transmission capacities.

Video coding for telecommunication applications has

evolved through the development of the ITU-T H.261,

H.262 (MPEG-2), and H.263 video coding standards (and

later enhancements of H.263 known as H.263+ and

H.263++), and has diversified from ISDN and T1/E1

service to embrace PSTN, mobile wireless networks, and

LAN/Internet network delivery. Throughout this evolution,

continued efforts have been made to maximize coding

efficiency while dealing with the diversification of network

types and their characteristic formatting and loss/error

robustness requirements.

Recently the MPEG-4 Visual (MPEG-4 part 2) standard [5]

has also begun to emerge in use in some application

domains of the prior coding standards. It has provided video

shape coding capability, and has similarly worked toward

broadening the range of environments for digital video use.

In early 1998 the Video Coding Experts Group (VCEG –

ITU-T SG16 Q.6) issued a call for proposals on a project

called H.26L, with the target to double the coding

efficiency (which means halving the bit rate necessary for a

given level of fidelity) in comparison to any other existing

video coding standards for a broad variety of applications.

The first draft design for that new standard was adopted in

October of 1999. In December of 2001, VCEG and the

Moving Picture Experts Group (MPEG – ISO/IEC

JTC 1/SC 29/WG 11) formed a Joint Video Team (JVT),

with the charter to finalize the draft new video coding

standard for formal approval submission as H.264/AVC [1]

in March 2003.

The scope of the standardization is illustrated in Fig. 1,

which shows the typical video coding/decoding chain

(excluding the transport or storage of the video signal). As

has been the case for all ITU-T and ISO/IEC video coding

standards, only the central decoder is standardized, by

imposing restrictions on the bitstream and syntax, and

defining the decoding process of the syntax elements such

that every decoder conforming to the standard will produce

similar output when given an encoded bitstream that

conforms to the constraints of the standard. This limitation

of the scope of the standard permits maximal freedom to

optimize implementations in a manner appropriate to

specific applications (balancing compression quality,

implementation cost, time to market, etc.). However, it

provides no guarantees of end-to-end reproduction quality,

as it allows even crude encoding techniques to be

considered conforming.

Pre -Processing

Encoding

Source

Destination

Post -Processing

& Error Recovery

Decoding

Scope of Standard

Pre -Processing

Encoding

Source

Destination

Post -Processing

& Error Recovery

Decoding

Scope of Standard

Pre -Processing

Encoding

Source

Destination

Post -Processing

& Error Recovery

Decoding

Scope of Standard

Pre -Processing

Encoding

Source

Destination

Post -Processing

& Error Recovery

Decoding

Scope of Standard

Fig. 1: Scope of video coding standardization.

Overview of the H.264 / AVC

Video Coding Standard

Thomas Wiegand, Gary J. Sullivan, Gisle Bjontegaard, and Ajay Luthra

H.264/AVC

WIEGAND et al.: OVERVIEW OF THE H.264/AVC VIDEO CODING STANDARD 2

This paper is organized as follows. Section II provides a

high-level overview of H.264/AVC applications and

highlights some key technical features of the design that

enable improved operation for this broad variety of

applications. Section III explains the network abstraction

layer and the overall structure of H.264/AVC coded video

data. The video coding layer is described in Section IV.

Section V explains the profiles supported by H.264/AVC

and some potential application areas of the standard.

II. A

PPLICATIONS AND DESIGN FEATURE HIGHLIGHTS

The new standard is designed for technical solutions

including at least the following application areas

• Broadcast over cable, satellite, Cable Modem, DSL,

terrestrial, etc.

• Interactive or serial storage on optical and magnetic

devices, DVD, etc.

• Conversational services over ISDN, Ethernet, LAN,

DSL, wireless and mobile networks, modems, etc. or

mixtures of these.

• Video-on-demand or multimedia streaming services

over ISDN, Cable Modem, DSL, LAN, wireless

networks, etc.

• Multimedia Messaging Services (MMS) over ISDN,

DSL, Ethernet, LAN, wireless and mobile networks,

etc.

Moreover, new applications may be deployed over existing

and future networks. This raises the question about how to

handle this variety of applications and networks.

To address this need for flexibility and customizability, the

H.264/AVC design covers a Video Coding Layer (VCL),

which is designed to efficiently represent the video content,

and a Network Abstraction Layer (NAL), which formats the

VCL representation of the video and provides header

information in a manner appropriate for conveyance by a

variety of transport layers or storage media (see Fig. 2).

Video Coding Layer

Data Partitioning

Network Abstraction Layer

H.320 MP4FF H.323/IP MPEG-2

etc.

Control Data

Coded Macroblock

Coded Slice/Partition

Video Coding Layer

Data Partitioning

Network Abstraction Layer

H.320 MP4FF H.323/IP MPEG-2

etc.

Control Data

Coded Macroblock

Coded Slice/Partition

Fig. 2: Structure of H.264/AVC video encoder.

Relative to prior video coding methods, as exemplified by

MPEG-2 video, some highlighted features of the design that

enable enhanced coding efficiency include the following

enhancements of the ability to predict the values of the

content of a picture to be encoded:

• Variable block-size motion compensation with

small block sizes: This standard supports more

flexibility in the selection of motion compensation

block sizes and shapes than any previous standard,

with a minimum luma motion compensation block

size as small as 4x4.

• Quarter-sample-accurate motion

compensation: Most prior standards enable half-

sample motion vector accuracy at most. The new

design improves up on this by adding quarter-

sample motion vector accuracy, as first found in an

advanced profile of the MPEG-4 Visual (part 2)

standard, but further reduces the complexity of the

interpolation processing compared to the prior

design.

• Motion vectors over picture boundaries: While

motion vectors in MPEG-2 and its predecessors

were required to point only to areas within the

previously-decoded reference picture, the picture

boundary extrapolation technique first found as an

optional feature in H.263 is included in

H.264/AVC.

• Multiple reference picture motion

compensation: Predictively coded pictures (called

"P" pictures) in MPEG-2 and its predecessors used

only one previous picture to predict the values in

an incoming picture. The new design extends upon

the enhanced reference picture selection technique

found in H.263++ to enable efficient coding by

allowing an encoder to select, for motion

compensation purposes, among a larger number of

pictures that have been decoded and stored in the

decoder. The same extension of referencing

capability is also applied to motion-compensated

bi-prediction, which is restricted in MPEG-2 to

using two specific pictures only (one of these

being the previous intra (I) or P picture in display

order and the other being the next I or P picture in

display order).

• Decoupling of referencing order from display

order: In prior standards, there was a strict

dependency between the ordering of pictures for

motion compensation referencing purposes and the

ordering of pictures for display purposes. In

H.264/AVC, these restrictions are largely

removed, allowing the encoder to choose the

ordering of pictures for referencing and display

purposes with a high degree of flexibility

constrained only by a total memory capacity bound

imposed to ensure decoding ability. Removal of

the restriction also enables removing the extra

delay previously associated with bi-predictive

coding.

• Decoupling of picture representation methods

from picture referencing capability: In prior

standards, pictures encoded using some encoding

methods (namely bi-predictively-encoded pictures)

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, JULY 2003 3

could not be used as references for prediction of

other pictures in the video sequence. By removing

this restriction, the new standard provides the

encoder more flexibility and, in many cases, an

ability to use a picture for referencing that is a

closer approximation to the picture being encoded.

• Weighted prediction: A new innovation in

H.264/AVC allows the motion-compensated

prediction signal to be weighted and offset by

amounts specified by the encoder. This can

dramatically improve coding efficiency for scenes

containing fades, and can be used flexibly for other

purposes as well.

• Improved "skipped" and "direct" motion

inference: In prior standards, a "skipped" area of a

predictively-coded picture could not motion in the

scene content. This had a detrimental effect when

coding video containing global motion, so the new

H.264/AVC design instead infers motion in

"skipped" areas. For bi-predictively coded areas

(called B slices), H.264/AVC also includes an

enhanced motion inference method known as

"direct" motion compensation, which improves

further on prior "direct" prediction designs found

in H.263+ and MPEG-4 Visual.

• Directional spatial prediction for intra coding:

A new technique of extrapolating the edges of the

previously-decoded parts of the current picture is

applied in regions of pictures that are coded as

intra (i.e., coded without reference to the content

of some other picture). This improves the quality

of the prediction signal, and also allows prediction

from neighboring areas that were not coded using

intra coding (something not enabled when using

the transform-domain prediction method found in

H.263+ and MPEG-4 Visual).

• In-the-loop deblocking filtering: Block-based

video coding produces artifacts known as blocking

artifacts. These can originate from both the

prediction and residual difference coding stages of

the decoding process. Application of an adaptive

deblocking filter is a well-known method of

improving the resulting video quality, and when

designed well, this can improve both objective and

subjective video quality. Building further on a

concept from an optional feature of H.263+, the

deblocking filter in the H.264/AVC design is

brought within the motion-compensated prediction

loop, so that this improvement in quality can be

used in inter-picture prediction to improve the

ability to predict other pictures as well.

In addition to improved prediction methods, other parts of

the design were also enhanced for improved coding

efficiency, including:

• Small block-size transform: All major prior

video coding standards used a transform block size

of 8x8, while the new H.264/AVC design is based

primarily on a 4x4 transform. This allows the

encoder to represent signals in a more locally-

adaptive fashion, which reduces artifacts known

colloquially as "ringing". (The smaller block size

is also justified partly by the advances in the

ability to better predict the content of the video

using the techniques noted above, and by the need

to provide transform regions with boundaries that

correspond to those of the smallest prediction

regions.)

• Hierarchical block transform: While in most

cases, using the small 4x4 transform block size is

perceptually beneficial, there are some signals that

contain sufficient correlation to call for some

method of using a representation with longer basis

functions. The H.264/AVC standard enables this in

two ways: 1) by using a hierarchical transform to

extend the effective block size use for low-

frequency chroma information to an 8x8 array, and

2) by allowing the encoder to select a special

coding type for intra coding, enabling extension of

the length of the luma transform for low-frequency

information to a 16x16 block size in a manner very

similar to that applied to the chroma.

• Short word-length transform: All prior standard

designs have effectively required encoders and

decoders to use more complex processing for

transform computation. While previous designs

have generally required 32-bit processing, the

H.264/AVC design requires only 16-bit arithmetic.

• Exact-match inverse transform: In previous

video coding standards, the transform used for

representing the video was generally specified

only within an error tolerance bound, due to the

impracticality of obtaining an exact match to the

ideal specified inverse transform. As a result, each

decoder design would produce slightly different

decoded video, causing a "drift" between encoder

and decoder representation of the video and

reducing effective video quality. Building on a

path laid out as an optional feature in the H.263++

effort, H.264/AVC is the first standard to achieve

exact equality of decoded video content from all

decoders.

• Arithmetic entropy coding: An advanced entropy

coding method known as arithmetic coding is

included in H.264/AVC. While arithmetic coding

was previously found as an optional feature of

H.263, a more effective use of this technique is

found in H.264/AVC to create a very powerful

entropy coding method known as CABAC

(context-adaptive binary arithmetic coding).

• Context-adaptive entropy coding: The two

entropy coding methods applied in H.264/AVC,

termed CAVLC (context-adaptive variable-length

coding) and CABAC, both use context-based

adaptivity to improve performance relative to prior

standard designs.

WIEGAND et al.: OVERVIEW OF THE H.264/AVC VIDEO CODING STANDARD 4

Robustness to data errors/losses and flexibility for operation

over a variety of network environments is enabled by a

number of design aspects new to the H.264/AVC standard

including the following highlighted features.

• Parameter set structure: The parameter set

design provides for robust and efficient

conveyance header information. As the loss of a

few key bits of information (such as sequence

header or picture header information) could have a

severe negative impact on the decoding process

when using prior standards, this key information

was separated for handling in a more flexible and

specialized manner in the H.264/AVC design.

• NAL unit syntax structure: Each syntax structure

in H.264/AVC is placed into a logical data packet

called a NAL unit. Rather than forcing a specific

bitstream interface to the system as in prior video

coding standards, the NAL unit syntax structure

allows greater customization of the method of

carrying the video content in a manner appropriate

for each specific network.

• Flexible slice size: Unlike the rigid slice structure

found in MPEG-2 (which reduces coding

efficiency by increasing the quantity of header data

and decreasing the effectiveness of prediction),

slice sizes in H.264/AVC are highly flexible, as

was the case earlier in MPEG-1.

• Flexible macroblock ordering (FMO): A new

ability to partition the picture into regions called

slice groups has been developed, with each slice

becoming an independently-decodable subset of a

slice group. When used effectively, flexible

macroblock ordering can significantly enhance

robustness to data losses by managing the spatial

relationship between the regions that are coded in

each slice. (FMO can also be used for a variety of

other purposes as well.)

• Arbitrary slice ordering (ASO): Since each slice

of a coded picture can be (approximately) decoded

independently of the other slices of the picture, the

H.264/AVC design enables sending and receiving

the slices of the picture in any order relative to

each other. This capability, first found in an

optional part of H.263+, can improve end-to-end

delay in real-time applications, particularly when

used on networks having out-of-order delivery

behavior (e.g., internet protocol networks).

• Redundant pictures: In order to enhance

robustness to data loss, the H.264/AVC design

contains a new ability to allow an encoder to send

redundant representations of regions of pictures,

enabling a (typically somewhat degraded)

representation of regions of pictures for which the

primary representation has been lost during data

transmission.

• Data Partitioning: Since some coded information

for representation of each region (e.g., motion

vectors and other prediction information) is more

important or more valuable than other information

for purposes of representing the video content,

H.264/AVC allows the syntax of each slice to be

separated into up to three different partitions for

transmission, depending on a categorization of

syntax elements. This part of the design builds

further on a path taken in MPEG-4 Visual and in

an optional part of H.263++. Here the design is

simplified by having a single syntax with

partitioning of that same syntax controlled by a

specified categorization of syntax elements.

• SP/SI synchronization/switching pictures: The

H.264/AVC design includes a new feature

consisting of picture types that allow exact

synchronization of the decoding process of some

decoders with an ongoing video stream produced

by other decoders without penalizing all decoders

with the loss of efficiency resulting from sending

an I picture. This can enable switching a decoder

between representations of the video content that

used different data rates, recovery from data losses

or errors, as well as enabling trick modes such as

fast-forward, fast-reverse, etc.

In the following two sections, a more detailed description of

the key features is given.

III. N

ETWORK ABSTRACTION LAYER

The network abstraction layer (NAL) is designed in order to

provide "network friendliness" to enable simple and

effective customization of the use of the VCL for a broad

variety of systems.

The NAL facilitates the ability to map H.264/AVC VCL

data to transport layers such as

• RTP/IP for any kind of real-time wire-line and wireless

Internet services (conversational and streaming)

• File formats, e.g. ISO MP4 for storage and MMS

• H.32X for wireline and wireless conversational

services

• MPEG-2 systems for broadcasting services, etc.

The full degree of customization of the video content to fit

the needs of each particular application is outside the scope

of the H.264/AVC standardization effort, but the design of

the NAL anticipates a variety of such mappings. Some key

concepts of the NAL are NAL units, byte stream, and

packet format uses of NAL units, parameter sets, and access

units. A short description of these concepts is given below

whereas a more detailed description including error

resilience aspects is provided in [6] and [7].

A. NAL units

The coded video data is organized into NAL units, each of

which is effectively a packet that contains an integer

number of bytes. The first byte of each NAL unit is a

header byte that contains an indication of the type of data in

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, JULY 2003 5

the NAL unit, and the remaining bytes contain payload data

of the type indicated by the header.

The payload data in the NAL unit is interleaved as

necessary with emulation prevention bytes, which are bytes

inserted with a specific value to prevent a particular pattern

of data called a start code prefix from being accidentally

generated inside the payload.

The NAL unit structure definition specifies a generic format

for use in both packet-oriented and bitstream-oriented

transport systems, and a series of NAL units generated by

an encoder is referred to as a NAL unit stream.

B. NAL units in byte stream format use

Some systems (e.g., H.320 and MPEG-2 | H.222.0 systems)

require delivery of the entire or partial NAL unit stream as

an ordered stream of bytes or bits within which the

locations of NAL unit boundaries need to be identifiable

from patterns within the coded data itself.

For use in such systems, the H.264/AVC specification

defines a byte stream format. In the byte stream format,

each NAL unit is prefixed by a specific pattern of three

bytes called a start code prefix. The boundaries of the NAL

unit can then be identified by searching the coded data for

the unique start code prefix pattern. The use of emulation

prevention bytes guarantees that start code prefixes are

unique identifiers of the start of a new NAL unit.

A small amount of additional data (one byte per video

picture) is also added to allow decoders that operate in

systems that provide streams of bits without alignment to

byte boundaries to recover the necessary alignment from

the data in the stream.

Additional data can also be inserted in the byte stream

format that allows expansion of the amount of data to be

sent and can aid in achieving more rapid byte alignment

recovery, if desired.

C. NAL units in packet-transport system use

In other systems (e.g., internet protocol / RTP systems), the

coded data is carried in packets that are framed by the

system transport protocol, and identification of the

boundaries of NAL units within the packets can be

established without use of start code prefix patterns. In such

systems, the inclusion of start code prefixes in the data

would be a waste of data carrying capacity, so instead the

NAL units can be carried in data packets without start code

prefixes.

D. VCL and non-VCL NAL units

NAL units are classified into VCL and non-VCL NAL

units. The VCL NAL units contain the data that represents

the values of the samples in the video pictures, and the non-

VCL NAL units contain any associated additional

information such as parameter sets (important header data

that can apply to a large number of VCL NAL units) and

supplemental enhancement information (timing information

and other supplemental data that may enhance usability of

the decoded video signal but are not necessary for decoding

the values of the samples in the video pictures).

E. Parameter sets

A parameter set is supposed to contain information that is

expected to rarely change and offers the decoding of a large

number of VCL NAL units. There are two types of

parameter sets:

• sequence parameter sets, which apply to a series of

consecutive coded video pictures called a coded video

sequence, and

• picture parameter sets, which apply to the decoding of

one or more individual pictures within a coded video

sequence.

The sequence and picture parameter set mechanism

decouples the transmission of infrequently changing

information from the transmission of coded representations

of the values of the samples in the video pictures. Each

VCL NAL unit contains an identifier that refers to the

content of the relevant picture parameter set, and each

picture parameter set contains an identifier that refers to the

content of the relevant sequence parameter set. In this

manner, a small amount of data (the identifier) can be used

to refer to a larger amount of information (the parameter

set) without repeating that information within each VCL

NAL unit.

Sequence and picture parameter sets can be sent well ahead

of the VCL NAL units that they apply to, and can be

repeated to provide robustness against data loss. In some

applications, parameter sets may be sent within the channel

that carries the VCL NAL units (termed "in-band"

transmission). In other applications (see

Fig. 3) it can be

advantageous to convey the parameter sets "out-of-band"

using a more reliable transport mechanism than the video

channel itself.

1 2 123 3

Parameter Set #3:

•

Video format

PAL

•

Entr. Code

CABAC

•...

Reliable Parameter Set Exchange

NAL unit with VCL Data encoded

with PS #3 ( address in Slice Header )

NAL unit with VCL Data encoded

with PS #3 ( address in Slice Header )

1 2 12

H.264/AVC Encoder

H.264/AVC Decoder

3 3

Parameter Set #3:

•

Video format

PAL

•

Entr. Code

CABAC

•...

1 2 123 3

Parameter Set #3:

•

Video format

PAL

•

Entr. Code

CABAC

•...

Reliable Parameter Set Exchange

NAL unit with VCL Data encoded

with PS #3 ( address in Slice Header )

NAL unit with VCL Data encoded

with PS #3 ( address in Slice Header )

NAL unit with VCL Data encoded

with PS #3 ( address in Slice Header )

NAL unit with VCL Data encoded

with PS #3 ( address in Slice Header )

1 2 12

H.264/AVC Encoder

H.264/AVC Decoder

3 3

Parameter Set #3:

•

Video format

PAL

•

Entr. Code

CABAC

•...

Fig. 3: Parameter set use with reliable "out-of-band" parameter set

exchange.

F. Access units

A set of NAL units in a specified form is referred to as an

access unit. The decoding of each access unit results in one

decoded picture. The format of an access unit is shown in

Fig. 4.

评论收藏

内容反馈

libing881

粉丝: 4
资源: 1

Overview of the H.264/AVC video coding standard

评论0

最新资源

Overview of the H.264/AVC video coding standard

评论0

Overview of the H.264-AVC Video Coding Standard

Overview of the H.264_AVC Video Coding Standard

Overview_of_the_H.264_AVC_Video_Coding_Standard

Video coding using the H.264 MPEG-4 AVC compression standard

Overview of the Scalable Video Coding Extension of the H.264/AVC Standard

H264_AVC stardard

The H.264AVC Advanced Video Coding Standard Overview and Introduction to the Fidelity Range Extensions

给新手学习H.264的建议

AVC_overview_1.rar_IEC_The Network

Video coding using the H.264/MPEG-4 AVC compression standard

Video_coding_using_the_H.264_MPEG-4_AVC_compression_standard

Overview and Introduction to H.264AVC Fidelity Range Extensions

video-coding_mini-project_H.264.rar_264_H.264_H.264-matlab_h.264

Efficient watermarking technique for H.264/AVC compressed video

三阶段学习H.264

H264入门5部曲

H264学习指南介绍

h264进阶阅读打包下载（全）

H.264学习必备资料

ITU-T H.264：2019 Advanced video coding for generic audiovisual s

Rate-distortion analysis for H.264AVC video coding and its application to rate control

Hash key-based video encryption scheme for H.264AVC.pdf

Overview of the High Efficiency Video Coding(HEVC) Standard.pdf

H264学习指南书籍

H.264+MPEG4学习经典教程

最新资源