"Digital Media Primer" Yue-Ling Wong,

Copyright (c)2013 by Pearson Education,

Inc. All rights reserved.

1
 Chapter 6
Fundamentals of Digital Video
Part 6
MPEG Compression
Streaming Video and Progressive
Download

2
 In this lecture, you will learn:
• Basic concepts of MPEG compression

• Types of MPEG

• Applications of MPEG

• What GOP (group of pictures) in MPEG is

• How GOP settings affect MPEG video file size

• How GOP settings affect MPEG picture quality

• What streaming video and progressive download are

3
 MPEG
• Moving Pictures Experts Group
Committee who derives standards for
encoding video

• Allow high compression

• MPEG-1, MPEG-2, MPEG-4

4
 What Happend to MPEG-3?
• NOT MP3 (which is audio format)

• Intended for HDTV

• HDTV specifications was merged into MPEG-2

5
 MPEG-1
• Video quality comparable to VHS

• Originally intended for Web and CD-ROM

playback

• Frame sizes up to 352  240 pixels

• Video format for VCD (VideoCD) before DVD

became widespread
6
 MPEG-2
• Supports DVD-video, HDTV, HDV standards

• For DVD video production:

Export video into DVD MPEG-2 format

• For HDV video production:

Export video into HDV's MPEG-2 format

7
 MPEG-4
• Newer standard of MPEG family

• Different encoding approach from MPEG-1

and MPEG-2
(will discuss after MPEG-1 and MPEG-2
compression in this lecture)

8
MPEG-1 and MPEG-2

How the Compression Works

9
 Properties of Typical Video
• Neighboring frames are very similar
– This property is called Temporal redundancy.

• MPEG compression exploits temporal

redundancy to reduce video file size
– by looking for motion difference from one frame
to the next
• This technique is called motion compensation.

10
 Basic Ideas of Motion Compensation
1. An video frame image is read in as a
reference frame
2. Read the next video frame image
3. Compare the image content between this
current frame (target frame) with the
reference frame
– one block of pixels at a time

11
 Comparing Reference and Target Frames

Case 1:
If a pixel block is identical at the same location
in both frames:
– No need to encode the target frame's
– Just save an instruction to refer to the block in the
reference image—requires less space than
encoding the whole pixel block

12
 Comparing Reference and Target Frames

Case 2:
If a pixel block at the same location in both
frames is not identical:
– Search for the reference image for a match
(because the content may be moved to another
location)
• Case 2a: No match is found
• Case 2b: A match is found

13
 Case 2a: No Match is Found
• The whole pixel block is encoded

• Thus, no saving in file size

14
 Case 2b: A Match is Found
• The displacement information of the block is
saved.

• Displacement information:
– A 2-dimensional value, indicating how much the block
moves horizontally and vertically from the reference
frame to the target frame
– Called motion vector
– Much smaller size than encoding the whole pixel
blocks
(thus the saving in file size)

15
 Illustration of the Ideas

Let's consider the first 4 frames of a video

Frame 1 Frame 2 Frame 3 Frame 4

The square grid cell in a frame represents a block of pixels.

16
 Illustration of the Ideas

Frame 1 is read in.

Frame 1 Frame 2 Frame 3 Frame 4

17
 Illustration of the Ideas

Frame 1 is used as a reference frame.

Frame 1 Frame 2 Frame 3 Frame 4

The red color highlights the pixel blocks that are encoded.
The whole frame of a reference frame is encoded. No saving in file size.

18
 Illustration of the Ideas

Frame 2 is read in.

Frame 1 Frame 2 Frame 3 Frame 4

19
 Illustration of the Ideas
Each pixel block in Frame 2 is searched in Frame 1
trying to find a match.

Frame 1 Frame 2 Frame 3 Frame 4

20
 Illustration of the Ideas
Each pixel block in Frame 2 is searched in Frame 1
trying to find a match.

Frame 1 Frame 2 Frame 3 Frame 4

The blue color highlights some pixel blocks that are found in Frame 1.

The yellow color highlights some pixel blocks that are found in Frame 1 (but
later in the process they are found in the next reference frame.)
21
 Illustration of the Ideas

Frame 3 is read in.

Frame 1 Frame 2 Frame 3 Frame 4

22
 Illustration of the Ideas
Each pixel block in Frame 3 is searched in Frame 1
trying to find a match.

Frame 1 Frame 2 Frame 3 Frame 4

The process is repeated similar to that for Frame 2.

23
Illustration of the Ideas about Motion Vector

Take Frames 1 and 2 as an example

Frame 1 Frame 2

24
Illustration of the Ideas about Motion Vector

Let's look at the pixel block showing the car's windshield

Frame 1 Frame 2

25
Illustration of the Ideas about Motion Vector
Let's arrange the frames vertically to see the displacement easier

Frame 1

Frame 2

26
Illustration of the Ideas about Motion Vector
Let's arrange the frames vertically to see the displacement easier

Frame 1

Frame 2

27
Illustration of the Ideas about Motion Vector
Let's arrange the frames vertically to see the displacement easier

Frame 1

Frame 2

28
Illustration of the Ideas about Motion Vector
Let's arrange the frames vertically to see the displacement easier

Frame 1

Frame 2

The red line indicates

the displacement of the
pixel block.
29
So how are reference frames chosen?

We will need to understand an

important concept of group of
pictures (GOP) first.

30
31
 Motivation of Understanding GOP
To understand:

• the terminology related to GOP because

– it is used by MPEG which in turns used by DVD video and HDTV

– it comes up in video production application dialog boxes when you

export video into MPEG format

• the impact of the GOP parameters on video file size and quality

• why some video editing programs may not support frame accurate
editing of MPEG

32
33
 Group of Pictures (GOP)
• Specifies the grouping structure of frames

• Frame types in a structure:

– I-frames
– P-frames
– B-frames

• An MPEG video contains 1 or more repeating

GOPs
34
 Number of Frames in a GOP
• Each GOP in a video has a fixed number of
frames

• The number is the N parameter

• Example:
– DVD-compliant MPEG-3: N = 15
– HDV: N = 15
35
 I-frames
• Stands for Intraframes

• Encoded only using the information within the frame--

intracoding.
– Use spatial compression
• Similar to JPEG compression
– No temporal compression

• Least compressed of the three types

• Reference frame in the previous 4-frame example

36
 I-frames
• Each GOP starts with an I-frame

• Each GOP has only one I-frame

37
 P-frames
• Stands for predicted frames

• Encoded using the information from the

previous I- or P-frames as the reference frame
if a match of a pixel block is found

• The 2nd and 3rd frames in the previous 4-video

example
38
 B-frames
• Stands for bidirectional frames

• Between I- and P-frames

• Encoded using information from the previous

and subsequent I- and/or P-frame as the
reference frames

39
 Example GOP
DVD-compliant MPEG-2:

I B B P B B P B B P B B P B B

N = 15
i.e., number of frames in a GOP = 15

40
 Example GOP
DVD-compliant MPEG-2:

I B B P B B P B B P B B P B B

M=3
i.e., number of frames between non-B-frames plus one = 2+1 = 3

41
Revisit the 4-frame video
example
Suppose N = 15, M = 3

42
 Example GOP:
N = 15, M = 3
I B B P B B P B B P B B P B B

Frame 1 Frame 2 Frame 3 Frame 4

43
 Example GOP:
N = 15, M = 3
I B B P B B P B B P B B P B B

Frame 1 Frame 2 Frame 3 Frame 4

I-frame:
Least compressed

44
 Example GOP:
N = 15, M = 3
I B B P B B P B B P B B P B B

Frame 1 Frame 2 Frame 3 Frame 4

P-frame:
Based on changes
from Frame 1
45
 Example GOP:
N = 15, M = 3
I B B P B B P B B P B B P B B

Frame 1 Frame 2 Frame 3 Frame 4

B-frame:
Most compressed.
Based on changes from
Frames 1 and 4 46
 Example GOP:
N = 15, M = 3
I B B P B B P B B P B B P B B

Frame 1 Frame 2 Frame 3 Frame 4

B-frame:
Most compressed.
Based on changes from
Frames 1 and 4 47
 How GOP Settings Affect File Size
• Shorter GOP (i.e., lower N):
– larger file size

• Rationales:
– A MPEG-2 consists of a repeating GOP structure
– Each GOP contains one I-frame
– I-frames are the least compressed among the three
types of frames, and thus take up more storage space
– Shorter GOP means more GOPs in a MPEG-2
– More GOPs means more I-frames in a MPEG-2

48
 How GOP Settings Affect Picture Quality

• Shorter GOP (i.e., lower N):

– better picture quality

• Rationales:
– I-frames are compressed based on the frame
information, rather than predicted from other
frames

49
 How GOP Structure Affects Frame-
Accurate Video Editing
• Some video editing programs may not support
frame-accurate video editing of MPEG-2 because
it is more complex than other video format

• Rationales:
– The information for a P-frame depends on the
information of its previous I-frame.
– A B-frame depends on the information of its previous
and subsequent I- or P-frames.
– Thus, it is more complex to trim out frames from a
MPEG-2 than other video format.

50
 Applications of MPEG-4
• Cover a wide range of data rate

• Low end of the data rate:

Video playback on mobile devices

• High end of the data rate:

– HDTV
– Handheld and portable game devices (e.g., Sony
PSP)
51
 MPEG-4 Coding Approach
• Uses media objects
– A scene may contain separate media objects

• Not the frame-based coding used in MPEG-1 and

MPEG-2

• Conventional frame-based video can be

converted to MPEG-4 because a frame can be
treated as a media object--a degenearted case.
52
Ways of Playing Video

53
 Two Ways of Playing Video
• Play from disk
• Play over a network

54
 Play from Disk
• An entire clip needs to be on disk before it can
be played

• Played from hard drive, CD, or DVD

55
 Play over a Network
• The video can be played while it is being
downloaded

• Can be played from disk

• Streaming video

• Progressive download

56
 Streaming Video
• Play video as soon as enough data has arrived

• Examples:
– Streaming QuickTime
– Real Video
– Window Media Video (WMV)

57
 Streaming Video
• Require a streaming server to stream video

• Allow saving several different compression levels

of a video in a single file

• The server chooses the compression level to

match the speed of network connection

• Buffering: Wait time depends on network speed

58
 Progressive Download
• Play video as soon as enough data has arrived

• Does not require special servers

• Example:
– QuickTime fast-start
• Created by saving the QuickTime movie as self-
contained using QuickTime Pro

59
 Review Questions
Note to instructor:
Depending on your preference, you may want to go
over the review questions at the end of this lecture as
an instant review or at the beginning of next lecture to
refresh students' memory of this lecture.

60
 Review Question
True/False: The MP3 audio is a MPEG-3.

61
 Review Question
___ provides a video quality comparable to VHS
and is the file format for VCD.
A.MPEG-1
B.MPEG-2
C.MPEG-3
D.MPEG-4

62
 Review Question
___ support the DVD-video, HDV, and HDTV
standards.
A.MPEG-1
B.MPEG-2
C.MPEG-3
D.MPEG-4

63
 Review Question
True/False: A typical MPEG-2 consists of a
repeating GOP structure.

64
 Review Question
Motion compensation is a key technique in ___.
A.asymmetric
B.lossless
C.lossy
D.spatial
E. temporal

65
 Review Question
___ is encoded using only the information
within that frame.
A.B-frame
B.I-frame
C.P-frame

66
 Review Question
___ is encoded using only the previous I- or P-
frame as the reference frame.
A.B-frame
B.I-frame
C.P-frame

67
 Review Question
___ is encoded using the previous and
subsequent I- and/or P-frame as the reference
frames.
A.B-frame
B.I-frame
C.P-frame

68
 Review Question
___ is the least compressed among the three
frame types.
A.B-frame
B.I-frame
C.P-frame

69
 Review Question
The N parameter of the GOP refers to ___.

A.the number of B-frames in a GOP

B. the number of I-frames in a GOP
C. the number of P-frames in a GOP
D.the total number of frames in a GOP
E. one plus the number of frames between the I-
and P-frame, the P- and P-frame, and the P- and
next GOP's I-frame

70
 Review Question
The M parameter of the GOP refers to ___.

A.the number of B-frames in a GOP

B. the number of I-frames in a GOP
C. the number of P-frames in a GOP
D.the total number of frames in a GOP
E. one plus the number of frames between the I-
and P-frame, the P- and P-frame, and the P- and
next GOP's I-frame

71