Basics of MPEG

Picture sizes: up to 4095 x 4095 Most algorithms are for the CCIR 601 format for
video frames
Y-Cb-Cr color space NTSC: 525 lines per frame at 60 fps, 720 x 480 pixel
luminance frame, 360 x 480 pixel chrominance frame PAL: 625 lines per frame at 50 fps, 720 x 576 pixel luminance frame, 360 x 576 pixel chrominance frame

SIF (source input format) for digital TV

Luminance resolution: 360 x 240 pixels at 30 fps or 360
x 288 pixels at 25 fps Chrominance resolution: half the luminance resolution in both dimensions

Detour: Motion Vectors with Subpixel Accuracy

Find motion vector (u,v) with integer pixel accuracy
Compute the MAE at its 4-neighbor pixels (m1 .. m4) Horizontal pixels
1 0

Let the MAE be m0

Model with the function p(i)=a|i-b|+c If 2(m3 m0) < (m4 m0), the i coordinate is to the left of
the center 4 If (m3 m0) > 2(m4 m0), the i coordinate is to the right of the center Otherwise it is along the center line

Similarly for the vertical direction

The Moving Picture Experts Group (MPEG) is a working group of experts that was formed by ISO and IEC to set standards for audio andvideo compression and transmission.[1] It was established in 1988 by the initiative of Hiroshi Yasuda (Nippon Telegraph and Telephone) andLeonardo Chiariglione[2], who has been from the beginning the Chairman of the group. The first MPEG meeting was in May 1988 in Ottawa, Canada.[3][4][5] As of late 2005, MPEG has grown to include approximately 350 members per meeting from various industries, universities, and research institutions. MPEG's official designation is ISO/IEC JTC1/SC29 WG11 Coding of moving pictures and audio (ISO/IEC Joint Technical Committee 1, Subcommittee 29, Working Group 11

Basics of MPEG
Types of pictures
I (intra) frame
compressed using only intraframe coding Moderate compression but faster random access

P (predicted) frame
Coded with motion compression using past I frames or P frames Can be used as reference pictures for additional motion
compensation

B (bidirectional) frame
Coded by motion compensation by either past or future I or P frames

D (DC) frame
Limited use: encodes only DC components of intraframe coding

MPEG: Video Encoding

The MPEG standards
do not define an encoding process define syntax of the coded stream define a decoding process

MPEG: Video Encoding

Regulator Frame Memory

+ Predictive frame

DCT

Quantizer (Q) Q-1 IDCT Motion vectors

VLC Encoder

Pre processing

Buffer

+
Motion Compensation
Motion Estimation Frame Memory

Output

Input

MPEG: Video Encoding

Some highlights
Interframe predictive coding (P-pictures)
For each macroblock the motion estimator produces the best
matching macroblock The two macroblocks are subtracted and the difference is DCT coded

Interframe interpolative coding (B-pictures)

The motion vector estimation is performed twice The encoder forms a prediction error macroblock from either
or from their average The prediction error is encoded using a block-based DCT

The encoder needs to reorder pictures because B-frames

always arrive late

MPEG: Structure of the Coded Bit-Stream

Sequence layer: picture dimensions,

pixel aspect ratio, picture rate, minimum buffer size, DCT GOP-1 GOP-2 GOP-n quantization matrices Sequence layer GOP layer: will have one I picture, start with I or B picture, end with I or P picture, has closed GOP flag, timing I B B B P B B.. GOP layer info, user data Picture layer: temporal ref number, picture type, synchronization info, Slice layer resolution, range of motion vectors Slice-1 Slices: position of slice in picture, mb-1 mb-2 mb-n quantization scale factor Slice-2 Macroblock: position, H and V motion Macroblock layer vectors, which blocks are coded and Slice-N 01 transmitted 23 4 5 Picture layer 8x8 block

MPEG: Macroblock Coding

Picture Type

I picture

P picture

B picture

change MQUANT

no change to MQUANT

motion comp.

motion vector set to 0

Fwd motion compensation

Bwd motion compensation

interpolated compensation

interframe

intraframe

coded

not coded

change MQUANT

no change to MQUANT

MQUANT= scale factor q

change MQUANT

no change to MQUANT

Quantization matrix

MPEG-2
Why another standard?
Support higher bit rates e.g., 80-100 Mbits/s for HDTV
instead of the 1.15 Mvits/s for SIF Support a larger number of applications The encoding standard should be a toolkit rather than a flat procedure

Interlaced and non-interlaced frame Different color subsampling modes e.g., 4:2:2, 4:2:0, 4:4:4 Flexible quantization schemes can be changed at picture level Scalable bit-streams Profiles and levels

MPEG-2: Effects of Interlacing

Fields or frame pictures can be encoded Prediction Modes and Motion Compensation
Frame prediction: current frame predicted from previous frame Field prediction:
Top and bottom fields of reference frame predicts first field Bottom field of previous frame and top field of current frame predicts
the bottom field of current frame

16 X 8 motion compensation mode

A macroblock may have two of them A B picture macroblock may have four! Dual prime motion compensation Top field of current frame is predicted from two motion vectors
coming from the top and bottom field of reference frame Works for P vectors

MPEG-2: Profiles and Levels

Profiles
Levels SNR 4:2:0 Lower High Bitrate Enhancement High-1440 Lower Bitrate Enhancement Main Lower Bitrate Enhancement Low Lower Bitrate 4, 3 15, 10 352 X 288/30 720 X 576/30 1440 X 1152/60 720 X 576/30 60, 40, 15 Spatial 4:2:0 High 4:2:0;4:2:2 Multiview 4:2:0

Enhancement

1920 X 1151/60
960 X 576/30 100, 80,25 1440 X 1152/60 720 X 576/30 80, 60, 20 720 X 576/30 352 X 288/30 20, 15, 4

1920 X 1151/60
1920 X 1151/60 130, 50, 80 1920 X 1152/60 1920 X 1152/60 100, 40, 60 720 X 576/30 720 X 576/30 25, 10, 15 352 X 288/30 352 X 288/30 8, 4, 4

MPEG-2 Applications
Digital Betacam: 90 Mbits/s video MPEG-2
Main Profile, Main Level, 4:2:0: 15 Mbits/s High Profile, High Level, 4:2:0: adequate, expensive Image quality preserved across generations of
processing Multiview Profile
Stereoscopic view disparity prediction Virtual walk-throughs composed from multiple viewpoints

MPEG-1 (1993): Coding of moving pictures and associated audio for digital storage media at up to about 1,5 Mbit/s (ISO/IEC 11172). The first MPEG compression standard -. It was basically designed to allow moving pictures and sound to be encoded into the bitrate of a Compact DiscTo meet the low bit requirement, MPEG-1 downsamples the images, as well as uses picture rates of only 2430 Hz, resulting in a moderate quality.[18] It includes the popular MPEG-1 Audio Layer III (MP3) audio compression format. MPEG-2 (1995): Generic coding of moving pictures and associated audio information (ISO/IEC 13818). Transport, video and audio standards for broadcast-quality television. MPEG-2 standard was considerably broader in scope and of wider appeal supporting interlacing and high definition. MPEG-2 is considered important because it has been chosen as the compression scheme for over-the-air digital television ATSC, DVB and ISDB, digital satellite TV services like Dish Network, digital cable television signals, SVCDand DVD Video.[18] It is also used on Blu-ray Discs,

MPEG-3 dealt with standardizing scalable and multi-resolution compression[18] and was intended for HDTV compression but was found to be redundant and was merged with MPEG-2, as a result there is no MPEG-3 standard.[18][19] . MPEG-4 (1998): Coding of audio-visual objects. (ISO/IEC 14496) MPEG-4 uses further coding tools with additional complexity to achieve higher compression factors than MPEG-2.[20] In addition to more efficient coding of video, MPEG-4 moves closer to computer graphics applications. In more complex profiles, the MPEG-4 decoder effectively becomes a rendering processor and the compressed bitstream describes three-dimensional shapes and surface texture.[20] MPEG-4 supports Intellectual Property Management and Protection (IPMP), which provides the facility to use proprietary technologies to manage and protect content like digital rights management.[21] It also supports MPEG-J, a fully programmatic solution for creation of custom interactive multimedia applications (Java application environment with a Java API) and many other features.[22][23][