You are on page 1of 34

CSEP 590 Data Compression

Autumn 2007

Video Compression

Human Perception of Video
• 30 frames per second seems to allow the visual system to integrate the discrete frames into continuous perception. Even 10 frames per second is acceptable. • If distorted, nearby frames in the same scene should have only small details wrong.
– A difference in average intensity is noticeable

• Compression choice when reducing bit rate
– skipped frames cause stop action – lower fidelity frames may be better
CSEP 590 - Lecture 13 - Autumn 2007 2

Applications of Digital Video
• Teleconference or video phone
– Real-time video – Very low delay (1/10 second is a standard)

• Live Broadcast Video
– Modest delay is tolerable (seconds is normal) – Error tolerance is needed.

• Video-in-a-can (DVD, Video-on-Demand)
– Random access to compressed data is desired – Encoding can take a lot of time

• Decoding must always be at at least the frame rate.

CSEP 590 - Lecture 13 - Autumn 2007

3

Lecture 13 .Autumn 2007 4 .Video Encoding Frame-by-Frame coding Encoder video frames decoding each frame requires the previous frame Group-of-Frames coding Group of 144 4 2Frames 444 3 Encoder video frames coded groups of frames no interdependencies CSEP 590 .

Lecture 13 .Coding Techniques • Frame-by-frame coding with prediction – Very low bit rates – low delay – Not error resilient • Group-of-frames coding – – – – Higher bit rates – within a group prediction is used Error resilient Random Access Higher delay CSEP 590 .Autumn 2007 5 .

CIF) is ¼ the data or 12.2.Digital Video Data • CCIR 601 (4.5 MHz sample rate for luminance channel 6.2 scheme) – – – – 13.2 Mb per second. CSEP 590 .Autumn 2007 6 .Lecture 13 .7 Mb per second – QCIF (Quarter .75 MHz sample rate for each of two chrominance channels 8 bits per sample is a bit rate of 27 x 8 = 216 Mb per second MPEG-SIF – ½ sample rate for luminance and ¼ for chrominance – 81 Mb per second • CIF (Common Interchange Format) – 288 x 352 pixels per frame for luminance channel – 144 x 176 pixels per frame for each of two chrominance – 8 bits per pixel and 30 frames per second gives 48.

– Video coders use motion compensation as part of prediction CSEP 590 .Autumn 2007 7 . Use the previous frame to predict the current one.High Compression Ratios Possible • Nearby frames are highly correlated.Lecture 13 . • Need to take advantage of the fact that usually objects move very little in 1/30 th of a second.

Autumn 2007 .Lecture 13 .Motion Compensation Previous Frame Frame 8 CSEP 590 .

Lecture 13 .Block Based Motion Compensation motion compensation blocks CSEP 590 .Autumn 2007 9 .

0) CSEP 590 .Lecture 13 .Autumn 2007 10 .Motion Vectors motion vector = (0.

Motion Vectors motion vector = (20.Autumn 2007 11 .Lecture 13 .5) 20 down and 5 to right CSEP 590 .

Lecture 13 . (intra block) – Otherwise code the difference (inter block) • The previous decoded frame is used because both the encoder and decoder have access to it.Motion Compensation • For each motion compensation block – Find the block in the previous decoded frame that gives the least distortion. – If the distortion is too high then code the block independently.Autumn 2007 12 . CSEP 590 .

Lecture 13 . quarter-pixel resolution – Higher resolution takes longer.Issues • Distortion measured in squared error or absolute error – Absolute error is quicker to calculate • Block size – Too small then too many motion vectors – Too large then there may be no good match • Searching range to find best block – Too large a search range is time consuming – Too small then may be better matches – Prediction can help. • Prediction resolution – Full pixel. half-pixel. but better prediction results CSEP 590 .Autumn 2007 13 .

Lecture 13 .Fractional Motion Compensation • Half or quarter pixel motion compensation may achieve better predictions. current frame previous frame Half pixel motion compensation CSEP 590 .Autumn 2007 14 . • Fractional motion compensation is achieved by linear interpolation.

Autumn 2007 15 . Half a1 a a3 a4 a2 a= a1 4 + a2 4 + a3 4 + a4 4 Quarter a1 a3 a a2 a4 a= 3a1 16 + 9a 2 16 + a3 16 + 3a 4 16 CSEP 590 . • Better interpolation methods exist.Linear Interpolation • Calculate an interpolated pixel as the average of overlapping pixels.Lecture 13 .

Rate Control Encoded frames buffer Decoder frame player • Buffer is filled at a constant rate (almost).Autumn 2007 16 .Lecture 13 . • Buffer is emptied at a variable rate. CSEP 590 .

bi is the number of bits in compressed frame i Initial occupancy of buffer B • Bi is the number of bits in the buffer at frame i B0 = B Bi+1 = Bi + C/F − bi • Buffer should never empty Bi > 0 for all i CSEP 590 .Lecture 13 .Autumn 2007 17 .Underflow Problem • Set up – – – – Constant rate channel at C bits per second Frame rate F in frames per second.

Lecture 13 .Autumn 2007 18 .Example C/F = 60 20 buffer Decoder 100 initial 60 bits in this frame not yet arrived Decoder 40 20 20 60 bits that arrived Decoder 100 20 CSEP 590 .

Lecture 13 .Autumn 2007 19 .Example Decoder 100 20 Decoder 80 20 20 20 100 Decoder 20 80 20 20 Decoder 20 20 100 20 CSEP 590 .

scene changes. • Causes of channel rate variability – Congestion on the internet CSEP 590 . – Encoder allocates more bits in frames that are hard to encode because they are not predicted well . • Causes of variability in bits per frame.Autumn 2007 20 .Lecture 13 .scalar quantization. – Encoder does not predict well how many bits will be used in a frame .Rate Control • The rate control buffer is seeded to allow for variability in number of bits per frame and variability in channel rate.

– Seed the buffer with enough frames to prevent underflow.Autumn 2007 21 . • Off-line solution – Attempt to allocate bits to frames to assure even fidelity. CSEP 590 . should the buffer threaten to empty start sending more frames at lower fidelity or skip frames (decoder will interpolate skipped frames).Lecture 13 .Rate Control Algorithms • On-line solution – Send a few frames to seed the buffer – Encoder simulates the buffer.

• Half pixel motion compensation CSEP 590 .Lecture 13 .H.261 • Application – low bit rate streaming video • Frame-by-frame encoder • DCT based with 8x8 coding block – Uses JPEG style coding • Motion compensation based on 16x16 macroblocks.Autumn 2007 22 .

Autumn 2007 23 .261 P-Frame I-Frame CSEP 590 .Lecture 13 .H.

Lecture 13 .H.Autumn 2007 24 .261(QCIF) 176 pixels Block Macroblock 144 pixels 99 motion vectors CSEP 590 .

H.Lecture 13 .261 • Within a group of blocks (GOB) prediction is used with motion vectors for coding.Autumn 2007 25 . GOB Also called a slice CSEP 590 .

261 Organization 176 Picture Header GOB 1 Picture Frame 144 GOB 2 … GOB 9 Group of Blocks Macroblock GOB Header MB 1 MB 2 … MB 11 MB Header Y1 Y3 Y2 Y4 Cb Cr JPEG style run length code 26 Block EOB CSEP 590 .Lecture 13 .Autumn 2007 .H.

P-Frame Inter Intra CSEP 590 .Autumn 2007 27 .Lecture 13 .

Lecture 13 .Intra-Macroblock Distribution Forman Video CSEP 590 .Autumn 2007 28 .

Autumn 2007 29 .Lecture 13 .MPEG-1 • Application – Video coding for random access • Group-of-frames encoder • DCT based with 8x8 coding block – Uses JPEG style coding • Motion compensation based on 16x16 macroblocks. • Forward and Backward Prediction within a group of frames CSEP 590 .

Autumn 2007 30 .MPEG-1 • P-frames predicted by I-frames or P-frames • B-frames predicted by I-frames and P-frames P-Frame B-Frame I-Frame CSEP 590 .Lecture 13 .

Lecture 13 .Orders Display Order 10 9 8 7 6 5 4 3 2 1 Coding/Decoding Order 9 8 10 6 5 7 3 2 4 1 Added delay is one frame time CSEP 590 .Autumn 2007 31 .

MPEG-1 Notes • Random access unit = Group-of-Frames – Called GOP for group-of-pictures • Error resilient – B-frames can be damaged without propagation • Added delay – Coding order different than display order • Encoding time consuming – Suitable for non-interactive applications CSEP 590 .Autumn 2007 32 .Lecture 13 .

Lecture 13 .Autumn 2007 33 .Beyond MPEG-1 • MPEG-2 – Application independent standard • MPEG-4 – Multimedia applications – Model based coding • H.263 – More error resilience CSEP 590 .

264 – – – – Just out in 2003. many new features Quarter pixel motion compensation Variable size motion blocks Multiple frame prediction • 3-D Wavelet Coding – Third dimension is time – 3-D SPIHT has been implemented – Delay is large because GOP is large • GTV – Group testing for video – Bits per frame can be controlled enabling off-line rate control to succeed.Newest Trends • H. CSEP 590 .Autumn 2007 34 .Lecture 13 .