Multimedia

 A digital camera samples the imaged scene

as discrete frames
 Each frame consists of sample values that
are discrete both horizontally and vertically.
 Let m and n be the integer indices in the
horizontal and vertical directions and let k
be the frame number.
 The actual spatial and temporal locations
are
x=mΔx, y=mΔy, and k=mΔt.
 We may use (m, n, k) to describe a digital
video.

Multimedia Coding and Processing 17/09/2018 2

  Let Nb be the number of bits to represent a
pixel’s colour value.
 Nb = 8 for monochrome video and Nb = 24 for
colour video.
 The data rate, R, is determined by

R  f s ,t f s , x f s , y N b
 where fs,t , fs,x , fs,y are the frame rate, samples
per line, and line number per frame respectively.

17/09/2018 Multimedia Coding and Processing 3

  Each pixel is render as a rectangular
region with a constant colour for the
pixel.
 The ratio of the width to the height of
this area is known as Pixel Aspect Ratio
(PAR).
 PAR relates to the Image Aspect Ratio by
PAR  IAR  f s , y / f s , x .

 The display device should conform to

this PAR to avoid distortions.
17/09/2018 Multimedia Coding and Processing 4
  ITU-R developed the BT.601 to
standardize the digital TV format.
 To convert a raster scan to a digital video
signal, one need only sample the 1D
waveform.
 If a total number of fs,x samples are taken
per line, the equivalent sampling rate is
f  f f f  f f.
s s,x s, y s ,t s,x l

17/09/2018 Multimedia Coding and Processing 5

 The sampling rate in BT.601 standard is
chosen to satisfy two constraints:
◦ The horizontal sampling resolution should
match the vertical sampling resolution as
close as possible. That is, Δx ≈ Δy.
◦ The same sampling rate should be used for
NTSC and PAL/SECAM and it should be a
multiple of respective line rates. Thus,
fs,x ≈ IAR* fs,y .

 Using fs = fs,x * fl and fl = fs,t * fs,y , we

have
fs ≈ IAR* f 2
s,y * fl .

Multimedia Coding and Processing 17/09/2018 6

 fs ≈ 11 and 13 MHz for NTSC and
PAL/SECAM.
 A number that is closest to both numbers
and satisfies the second criterion is then
chosen. We have
fs = 858* fl (NTSC) = 864* fl (PAL/SECAM) =13.5
MHz.

 This gives the 525/60 and 625/60 signals.

Multimedia Coding and Processing 17/09/2018 7

 NTSC PAL/SECAM

858 pels 864 pels

525 720 pels 625 720 pels

lines Active lines Active
480
Area 576
Area
lines lines
122 16 132 12
pels pel pels pel
s s

17/09/2018 Multimedia Coding and Processing 8

9/17/2018
Multimedia - Rdh 9
 Aspect ratio
◦ Conventional TV is 4:3 (1.33)
◦ HDTV is 16:9 (2.11)
◦ Cinema uses 1.85:1 or 2.35:1
 Frame Rate
◦ NTSC is 60Hz interlaced (actually 59.94Hz)
◦ PAL/SECAM is 50Hz interlaced
◦ Cinema is 24Hz non-interlaced

Multimedia - Rdh 9/17/2018 10

  Aspect ratio
◦ Conventional TV is
4:3 (1.33)
◦ HDTV is 16:9 (2.11)
◦ Cinema uses 1.85:1
or 2.35:1
 Frame Rate
◦ NTSC is 60Hz interlaced
(actually 59.94Hz)
◦ PAL/SECAM is 50Hz
interlaced
◦ Cinema is 24Hz non-
interlaced

9/17/2018
Multimedia - Rdh 11
Source: wikipedia
 Pixel aspect
Resolution ratio
Pixels Aspect Ratio Video Format Description
(WxH) (Standard
16:9)

Used on PDP
1024×768 786,432 16:9 4:3 720p/XGA
HDTV
Used on Digital
1280×720 921,600 16:9 1:1 (square) 720p/WXGA
television,
720p/WXGA
Used on
Approx. 1:1 - HDTV
1366×768 1,049,088 16:9 LCD/PDP HDTV
(square) standard
displays
format
Used on PDP
1024×1080 1,105,920 16:9 15:8 1080p
displays HDTV
Used on PDP
1280×1080 1,382,400 16:9 3:2 1080p
HDTV displays
1080p -
Used on all
HDTV
1920×1080 2,073,600 16:9 1:1 (square) types of HDTV
standard
technologies
format

3840x2160 8,294,400 16:9 1:1 (square) 2160p Ultra HDTV,

9/17/2018
Multimedia - Rdh 12
 Note that both formats have the same
number (720) of active pixels/line.
 Note that the pixel width-to-height ratio is
not 1.
 PAR= Δx /Δy = IAR*f 's,y/f 's,x
◦ PARNTSC = (4/3)*480/720 = 8/9
◦ PARPAL = (4/3)*576/720 = 16/15

Multimedia Coding and Processing 17/09/2018 13

  BT.601 also uses the YCbCr
representation.
 The YCbCr values (0-255) are related to
the RGB values (0-255) by
 Y   0.257 0.504 0.098   R   16 
C    0.148 0.291 0.439  G   128
 b     
Cr   0.439 0.368 0.071  B  128

17/09/2018 Multimedia Coding and Processing 14

  The inverse relation is

 R  1.164 0.000 1.596   Y  16 

G   1.164 0.392 0.813 C  128
    b 
 B  1.164 2.017 0.000  Cr  128

17/09/2018 Multimedia Coding and Processing 15

 Human eyes are less sensitive to colour
differenc than brightness.
 Chrominance values are sampled at lower
frequency than the luminance value.
 4:2:2 format. Each chrominance component, Cb
and Cr, are usually sampled at half of the
sampling rate of the luminance component.
 4:1:1 format. Each chrominance component is
sampled at ¼ of the sampling rate of the
luminance component.
 4:2:0 format. Each chrominance component is
sampled at ½ of the sampling rate both
horizontally and vertically.
 4:4:4 format. The chrominance values are
sampled at the same rate as the luminance
values.

Multimedia Coding and Processing 17/09/2018 16

 Y
Cb
Cr

4:2:2 format 4:1:1 format

Each 2×2 Y pixels Each 2×2 Y pixels
2Cb & 2Cr pixels 1Cb & 1Cr pixels
(horizontal subsampling) (4:1 horizontal subsampling)

Multimedia Coding and Processing 17/09/2018 17

 Y
Cb
Cr

4:2:0 format 4:4:4 format

Each 2×2 Y pixels Each 2×2 Y pixels
1Cb & 1Cr pixels 4Cb & 4Cr pixels
(2:1 subsampling both (no subsampling)
horizontally and vertically)

Multimedia Coding and Processing 17/09/2018 18

 Mean Square Error (MSE)

   m, n, k    m, n, k  
1 2
MSE   2
e 1 2
N
 PSNR k m,n

 max
2
 max
 Mean Absolute
PSNR Difference
 10log10 2(MAD)
 20log10
e e

1
MAD 
N
   m, n, k    m, n, k 
k m,n
1 2

Multimedia Coding and Processing 17/09/2018 19

 CGA – Colour Graphics Adapter
◦ Resolutions: 320 × 200 pixels with 4 colours (2
bits)
◦ (320x200) pixels × 2 bits/pixel = 15.625
KB/image
 EGA – Enhanced Graphics Adapter
◦ Resolutions: 640 × 350 pixels with 16 colours (4
bits)
◦ (640x350) pixels × 4 bits/pixel = 109.375
KB/image

Multimedia Coding and Processing 17/09/2018 20

 VGA – Video Graphics Array
◦ Resolution: 640 x 480 pixels with 256 colours (8 bits)
◦ (640x480) pixels x 8 bits/pixel = 300 KB/image
 XGA – Extended Graphics Array
◦ Resolution: 640 x 480 pixels with 65,536 colours (16
bits) or 1024 x 768 pixels with 256 colours
◦ (640x480) pixels x 16 bits/pixel = 600 KB/image
◦ (1024x768) pixels x 8 bits/pixel = 768 KB/image
 SVGA – Super VGA
◦ Resolution: 800 x 600 pixels with 16,777,216 colours
(24 bits) or 1024 x 768 pixels with 65,536 colours (16
bits)
◦ (800x600) pixels x 24 bits/pixel = 1.37 MB/image
◦ (1024x768) pixels x 16 bits/pixel = 1.5 MB/image

Multimedia Coding and Processing 17/09/2018 21

 Digitization of an analog waveform
 Take samples at different temporal
locations
 Frequency of taking samples: sampling
rate
 Amplitude of taken samples: quantization
 Objective: To maintain the information of
the original analog waveform

Multimedia Coding and Processing 17/09/2018 22

 Sampling rate is the no. of samples taken per unit
time
 Affects whether the no. of samples are enough to
reproduce the waveform.
Reproduced waveform
E.g. if only 1 sample per period = a straight line

time
0 1 2 3 4 5 6 7 8 9 10 11 12

Multimedia Coding and Processing 17/09/2018 23

 For lossless digitization, the sampling rate
must be at least twice of the maximum
frequency
 That is, at least 2 samples must be obtained
within each cycle. Otherwise, either the
maximum or the minimum amplitude in a
period is gone; leading to loss of
information in the digitization process.

time
0 1 2 3 4 5 6 7 8 9 10 11 12

Multimedia Coding and Processing 17/09/2018 24

 When the sampling rate is reduced,
◦ Fewer samples are accessed per unit time
◦ Lower temporal resolution
◦ Object size is reduced
◦ The information of the higher frequency waveforms
is also lost

Multimedia Coding and Processing 17/09/2018 25

 Consider using a decimal number to describe a value.
Each digit has ten different values.
 One digit specifies 10 different values (0-9)
 Two digits specifies 100 different values (0-99)
 Three digits specifies 1000 different values (0-999).
 Each additional digit increases the range by 10 times.

 Consider using a binary number to describe a

sample value.
 Each additional digit(bit) doubles the range of values.

Multimedia Coding and Processing 17/09/2018 26

 Quantization of a sample – the mapping of
values to integral values in describing a
sample value. Mathematically,

V ( x, y )  U ( x, y ) /  sgn U ( x, y )
 No. of bits, Nb = log2(no. of integral values)

Multimedia Coding and Processing 17/09/2018 27

 Analog: A  Digital: steps in
continuous slope the stairs

Step size

Multimedia Coding and Processing 17/09/2018 28

 More values(v) need more bits(b): v  2b
◦ 8 bits:  256 values
◦ 16 bits:  65,536 values
◦ 24 bits:  16,777,216 values
 Affects the step size in the reproduced
waveform
 More bits  smaller step size  better
sample quality

Multimedia Coding and Processing 17/09/2018 29

 Consider a sine value, original values in the sine
curve are
0.707, 1.0, 0.707, 0, -0.707, -1.0, -0.707, 0, ...
 Values in binary representation are
0.10110101, 1, 0.10110101, 0, -0.10110101, -
1, -0.10110101, 0, …

time
0 1 2 3 4 5 6 7 8 9 10 11 12

Multimedia Coding and Processing 17/09/2018 30

 Values in 7-bit quantization are
0.101101, 1.000, 0.101101, 0.000, -0.101101, -
1.000, -0.101101, 0.000, …
 Quantized value in decimal becomes
0.703125, 1, 0.703125, 0, -0.703125, -1, -
0.703125, 0, …

time
0 1 2 3 4 5 6 7 8 9 10 11 12

Multimedia Coding and Processing 17/09/2018 31

 Values in 4-bit quantization are
0.101, 1.000, 0.101, 0.000, -0.101, -1.000, -
0.101, 0.000, …
 Quantized value in decimal becomes
0.625, 1, 0.625, 0, -0.625, -1, -0.625, 0, …

time
0 1 2 3 4 5 6 7 8 9 10 11 12

Multimedia Coding and Processing 17/09/2018 32

 When the no. of integral values to represent a
sample is reduced,
◦ The number of bits per sample is reduced
◦ Lower sample resolution
◦ Object size is reduced
◦ The quality of each sample value is also reduced

Multimedia Coding and Processing 17/09/2018 33

 Digitization of the analog waveform
involves sampling and quantization.
 Sampling rate must be at least twice of
the highest frequency to avoid loss of
information
 No. of bits in quantization affects the
quality of each sample value
 Object size are reduced with loss of
information by lowering the sampling
rate and number of sample values.

Multimedia Coding and Processing 17/09/2018 34

 Computer graphics are represented using the
coordinates on the screen.
 Computer animations are done by updating changes
to the frame buffers and these changes are drawn on
the display
 Images are represented as 2D pixels. Each pixel can
be represented using RGB, YUV, YCbCr, or CMYK.
 A/D converters digitize an analog wave by taking
samples of amplitudes at fixed time intervals.
 A video is represented as an array of frames. 24 to
30 frames should be displayed per second to show
full motions.

Multimedia Coding and Processing 17/09/2018 35

 A sine curve of maximum amplitude of 16
is sampled at 6 times per cycle and the
sample values are quantized using 8 bits.
Alternatively, the sine curve may be
sampled at 8 times per cycle and the
sample values are quantized using 6 bits.
 Find the Mean Absolute Difference of each
quantized curves with the original sine
curve.
 Find the Mean Square Errors of the two
quantized curves.
 Compare the PSNR of the two quantized
curves.

Multimedia Coding and Processing 17/09/2018 36