Applying 3-D Methods to

Video for Compression

Salih Burak Gokturk
Anne Margot Fernandez Aaron
March 13, 2002
EE 392J Project Presentation
 OUTLINE
MOTIVATION
PREVIOUS WORK
ALGORITHMS
RESULTS
CONCLUSION
 BACKGROUND
Most common video compression scheme
Blockwise transform (i.e. 2D-DCT) for
spatial compression
Blockwise Motion Estimation and
Compensation for temporal redundancy
Used in MPEG
Motion Compensated Predictive Coding is
better than Image based coding since it uses
a good prediction from the previous frame.
 OBJECTIVE
Exploit Temporal Redundancies MORE!

Why?

How?
Use 3-D Transformations and Methods.
 3-D Transform Methods
Good for ideal case smooth motion,
global motion
Bad for non-uniform motion
Need large number of frame stores
 PREVIOUS WORK
3-D DCT for Video
Bauer and Sayood, Chan and Sui, Servais and
de Jager
3-D Wavelets for Video
Pearlman et al. , Taubman et al.,
Motion Compensated Coding
 ALGORITHMS IN OUR
EXPERIMENTS
1. 2D DCT
2. 3D DCT
3. (Block) Motion Compensated Coding
4. 3D (Volume) Motion Compensated
Coding
5. Other Transformation as a prediction
scheme (PCA, Wavelet)
 2-D versus 3-D DCT
2-D Blocks versus 3-D Blocks

2-D DCT versus 3-D DCT

Markov-1 chain
P(Xn=Xn-1) = p
Xn uniform [0, 255]

When would 3-D DCT not work?

 3D Volume Motion Compensated
Coding

depth
Current stack of frames
Previous stack of frames

For each volume block

Divide stack Find 3D
Calculate 3D DCT on
into volume motion Q
residual residual
blocks vectors
3D Volume Motion Compensated
Coding
Finding the motion vectors
Full Search
Measurement criterion
MAD
standard deviation of residual
When it does not work
No smooth motion (acceleration present)
Cases when 2D MC does not work (occlusion,
illumination, non-uniform motion)
Block size (depth) along time axis is too small or too big
Centroid Effect : Center of the block usually has a
better prediction than edges of the block.
 Initial Prediction Results

2D MC Prediction 3D MC Prediction
 3D Principal Component
Analysis Current stack of frames

1 Apply PCA on a large collection of blocks

[ ]= A = U V T

U= [ ]
2 Each Block is represented by a linear
combination of principal components:
= 1 + 2 + 3 +
For each volume block
Divide each Find PCA
Calculate 3D DCT on
stack into Q
Coefs residual residual
volume
blocks
 Experimental Setup
Techniques
1. 2D DCT by 8x8 Blocks
2. 3D DCT by 8x8x8 Blocks
3. Block Motion Compensated Coding (8x8 and 16x16 followed by
2-D DCT)
4. Volume Motion Compensated Coding (8x8x4, 8x8x6 and 8x8x8)
followed by 3-D DCT.
5. Principal Component Analysis (use principal components of
8x8x8 blocks as prediction followed by 3-D DCT)

Bit Rate vs PSNR Comparison

DCT coefficients were quantized using a uniform quantizer
PSNR was computed over the same support region.
Bit rate was calculated by finding the theoretical entropy of the
quantized DCT coefficients and motion vectors.
Miss Am
 0.14 bit/pixel
Miss Am
2D-DCT 2D-MC 3D-DCT

3D-MC 3D-PCA
Salesman 2D-MC vs 3D-MC
2D-MC

3D-MC

0.11 bit/pixel
Salesman 3D-DCT vs 3D-MC
3D-DCT

3D-MC

0.16 bit/pixel
 2D-MC 3D-MC

Foreman

3D-DCT

0.56 bit/pixel
Bus
 CONCLUSION
There is redundancy across temporal domain.
There is a lot of room for improvement if we use
3-D compression methods!
2-D motion compensated coding is good for
movies with A LOT of motion.
When movements are smooth, 3-D methods will
perform better than 2-D motion compensated
coding.

FUTURE WORK
Find Better Residual Coding Schemes for 3D MC.
(That deals with centroid effect better)
Other 3D Methods ?