JPEG and MPEG

Introduction to JPEG and
MPEG
Ingemar J. Cox
University College London
Outline
 Elementary information theory
 Lossless compression
 Quantization
UCL Adastral Park Postgraduate Campus

 Fundamentals of images
 Discrete Cosine Transform (DCT)
 JPEG
 MPEG-1, MPEG-2
Nov 27th 2006 Ingemar J. Cox 2
Bibliography
 D. MacKay, “Information Theory, Inference and learning
Algorithms”, Cambridge University Press, 2003.
http://www.inference.phy.cam.ac.uk/itprnn/book.html
 W. B. Pennebaker and J. L. Mitchell, “JPEG Still Image

Data Compression Standard”, Chapman Hall, 1993
(ISBN 0-442-01272-1).
 G. K. Wallace, “The JPEG Still-Picture Compression
Standard”, IEEE Trans. On Consumer Electronics, 38,
1, 18-34, 1992.
 http://en.wikipedia.org/wiki/JPEG
Bibliography
 http://en.wikipedia.org/wiki/MPEG-2
 T. Sikora, “MPEG Digital Video-Coding
Standards”, IEEE Signal Processing Magazine,

82-100, September 1997

Elementary Information Theory
 How much information does a symbol convey?
 Intuitively, the more unpredictable or surprising
it is, the more information is conveyed.

 Conversely, if we strongly expected something,
and it occurs, we have not learnt very much

 If p is the probability that a symbol will occur
 Then the amount of information, I, conveyed is:
1

I  log 2  
 p
 The information, I, is measured in bits
 It is the optimum code length for the symbol

 The entropy, H, is the average information per
symbol
1
H   p( s) log 2 (

)
s p( s)
 Provides a lower bound on the compression

that can be achieved

Elementary Information theory
 A simple example. Suppose we need to
transmit four possible weather conditions:
1. Sunny

2. Cloudy
3. Rainy
4. Snowy
 If all conditions are equally likely, p(s)=0.25,

and H=2
 i.e. we need a minimum of 2 bits per symbol

Elementary information theory
 Suppose instead that it is:
1. Sunny 0.5 of the time
2. Cloudy 0.25 of the time

3. Rainy 0.125 of the time, and
4. Snowy 0.125 of the time
 Then the entropy is

1 1 1
H  0.5 log 2  0.25 log 2  2  0.125 log 2
0.5 0.25 0.125
H  0.5 1  0.25  2  2  0.125  3
H  0.5  0.5  0.75  1.75

 Variable length codewords
 Huffman code – integer code lengths
 Arithmetic codes – non-integer code lengths

 Huffman code
Weather Probability Information Integer code

Sunny 0.5 1 0
Cloudy 0.25 2 10
Rainy 0.125 3 110
Snowy 0.125 3 111

 Previous illustration is an example of a lossless
code
 I.e. we are able to recover the information exactly

 Note that we have assumed that each symbol
is independent of the other symbols
 I.e. the current symbol provides no information
regarding the next symbol

Quantization
 Quantization is the process of approximating a
continuous (or range of values) by a (much)
smaller range of values

 x  0.5 
Q( x, )  Round  
  
 Where Round(y) rounds y to the nearest integer
  is the quantization stepsize

Quantization
 Example: =2
-5 -4 -3 -2 -1 0 1 2 3 4 5

-2 -1 0 1 2
-4 -2 0 2 4

Quantization
 Quantization plays an important role in lossy
compression
 This is where the loss happens

Fundamentals of Images
Fundamentals of images
 An image consists of pixels (picture elements)
 Each pixel represents luminance (and colour)
 Typically, 8-bits per pixel

 Colour
 Colour spaces (representations)
 RGB (red-green-blue)
 CMY (cyan-magenta-yellow)

 YUV
• Y = 0.3R+0.6G+0.1B (luminance)
• U=R-Y
• V=B-Y
 Greyscale
 Binary
 A TV frame is about 640x480 pixels
 If each pixels is represented by 8-bits for each
colour, then the total image size is

 640×480*3=921,600 bytes or 7.4Mbits
 At 30 frames per second, this would be

  220Mbits/second

 Do we need all these bits?

 Here is an image represented with 8-bits per
pixel

 Here is the same image at 7-bits per pixel

 And at 6-bits per pixel



 Do we need all these bits?
 No!
 The previous example illustrated the eye’s

sensitivity to luminance
 We can build a perceptual model
 Only code what is important to the human visual
system (HVS)
 Usually a function of spatial frequency

Fundamentals of Images
 Just as audio has temporal frequencies
 Images have spatial frequencies
 Transforms

 Fourier transform
 Discrete cosine transform
 Wavelet transform
 Hadamard transform

Discrete cosine transform
 Forward DCT
C (u )N 1
 u 
S (u )   s(n) cos (n  0.5) 

2 n 0  8 
 Inverse DCT
C (u ) N 1  u 
s ( n)  
2 u 0
S (u ) cos
 8
(n  0.5) 

Basis functions
 DC term

Basis functions
 First term

Basis functions
 Second term

Basis functions
 Third term

Basis functions
 Fourth term

Basis functions
 Fifth term

Basis functions
 Sixth term

Basis functions
 Seventh term

DCT Example
Example
 Signal

Example
 DCT coefficients are:
 4.2426
 0


-3.1543
 0
 0
 0
 -0.2242
 0

Example: DCT decomposition
 DC term

 2nd AC term

 6th AC term

Example: summation of DCT terms
 First two non-zero coefficients

Example: summation of DCT terms
 All 3 non-zero coefficients

Example
 What if we quantize DCT coefficients?
 =1
 Quantized DCT coefficients are:



4
 0
 -3
 0
 0
 0
 0
 0

Example
 Approximate reconstruction

Example
 Exact reconstruction

2-D DCT Transform
 Let i(x,y) represent an image with N rows and
M columns
 Its DCT I(u,v) is given by

1 M N  (2 x  1)u   (2 y  1)v 
I (u, v)  C (u )C (v)  i( x, y) cos  cos 
4  x 1 y 1  16   16 
 where
1 C (u )  1
C (0) 
2
 Discrete cosine transform
 Coefficients are approximately uncorrelated
 Except DC term
 C.f. original 8×8 pixel block

 Concentrates more power in the low frequency
coefficients
 Computationally efficient
 Block-based DCT
 Compute DCT on 8×8 blocks of pixels

 Basis functions for the 8×8 DCT (courtesy
Wikipedia)

Fundamentals of JPEG
Encoder
DCT Quantizer Entropy coder

Compressed
image data
IDCT Dequantizer Entropy

decoder
Decoder

 JPEG works on 8×8 blocks
 Extract 8×8 block of pixels
 Convert to DCT domain

 Quantize each coefficient
 Different stepsize for each coefficient
 Based on sensitivity of human visual system
 Order coefficients in zig-zag order

 Entropy code the quantized values
 A common quantization table is
16 11 10 16 24 40 51 61
12 12 14 19 26 58 60 55

14 13 16 24 40 57 69 56
14 17 22 29 51 87 80 62
18 22 37 56 68 109 103 77
24 35 55 64 81 104 113 92
49 64 78 87 103 121 120 101
72 92 95 98 112 100 103 99
 Zig-zag ordering
0 1 5 6 14 15 27 28
2 4 7 13 16 26 29 42

3 8 12 17 25 30 41 43
9 11 18 24 31 40 44 53
10 19 23 32 39 45 52 54
20 22 33 38 46 51 55 60
21 34 37 47 50 56 59 61
35 36 48 49 57 58 62 63
 Entropy coding
 Run length encoding followed by
 Huffman
 Arithmetic

 DC term treated separately
 Differential Pulse Code Modulation (DPCM)
 2-step process
1. Convert zig-zag sequence to a symbol sequence
2. Convert symbols to a data stream
 Modes
 Sequential
 Progressive
 Spectral selection

• Send lower frequency coefficients first
 Successive approximation
• Send lower precision first, and subsequently refine
 Lossless
 Hierarchical
 Send low resolution image first

Fundamentals of MPEG-1/2
Fundamentals of MPEG
 A sequence of 2D images
 Temporal correlation as well as spatial
correlation

 TV broadcast
 Frame-based
 Field-based

MPEG
 Moving Picture Experts Group
 Standard for video compression
 Similarities with JPEG

MPEG
 Design is a compromise between
 Bit rate
 Encoder/decoder complexity
 Random access capability

MPEG
 Images
 Spatial redundancy
 Perceptual redundancy

 Video
 Spatial redundancy
 Intraframe coding
 Temporal redundancy
 Interframe coding
 Perceptual redundancy

MPEG
 Consider a sequence of n frames of video.
 It consists of:
 I-frames

 P-frames
 B-frames
 A sequence of one I-frame followed by P- and

B-frames is known as a GOP
 Group of Pictures
 E.g. IBBPBBPBBPBBP

MPEG
 I-frames
 Intraframe coded
 No motion compensation
 P-frames

 Interframe coded
 Motion compensation
• Based on past frames only
 B-frames
 Interframe coded
 Motion compensation
• Based on past and future frames

MPEG
 Motion-compensated prediction
 Divide current frame, i, into disjoint 16×16
macroblocks
 Search a window in previous frame, i-1, for closest

match
 Calculate the prediction error
 For each of the four 8×8 blocks in the macroblock,
perform DCT-based coding
 Transmit motion vector + entropy coded prediction
error (lossy coding)

MPEG
 Like JPEG, the DC term is treated separately
 DPCM
 B-frame compression high

 Need buffer and delay

JPEG and MPEG

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

JPEG and MPEG

Uploaded by

Copyright:

Available Formats

Introduction to JPEG and

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

Nov 27th 2006 Ingemar J. Cox 4

UCL Adastral Park Postgraduate Campus

Nov 27th 2006 Ingemar J. Cox 6

UCL Adastral Park Postgraduate Campus

Nov 27th 2006 Ingemar J. Cox 7

UCL Adastral Park Postgraduate Campus

 Provides a lower bound on the compression

Nov 27th 2006 Ingemar J. Cox 8

UCL Adastral Park Postgraduate Campus

 If all conditions are equally likely, p(s)=0.25,

Nov 27th 2006 Ingemar J. Cox 9

UCL Adastral Park Postgraduate Campus

 Then the entropy is

Nov 27th 2006 Ingemar J. Cox 10

UCL Adastral Park Postgraduate Campus

Weather Probability Information Integer code

UCL Adastral Park Postgraduate Campus

Nov 27th 2006 Ingemar J. Cox 12

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

Nov 27th 2006 Ingemar J. Cox 15

UCL Adastral Park Postgraduate Campus

Nov 27th 2006 Ingemar J. Cox 16

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

 At 30 frames per second, this would be

Nov 27th 2006 Ingemar J. Cox 21

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

 The previous example illustrated the eye’s

UCL Adastral Park Postgraduate Campus

Nov 27th 2006 Ingemar J. Cox 28

UCL Adastral Park Postgraduate Campus

Nov 27th 2006 Ingemar J. Cox 29

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

Nov 27th 2006 Ingemar J. Cox 41

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

UCL Adastral Park Postgraduate Campus

 Quantized DCT coefficients are: