Lule˚ tekniska universitet a Avd f¨r signalbehandling o Frank Sj¨berg o

Mediakodning SMS047 Laboration 4

Video Coding
Purpose: The goal of this lab is to experiment with video coding and getting better understanding of the properties of a hybrid coder structure similar to MPEG-1. The hybrid structure is motion estimation and prediction in the time domain, and 2 dimensional DCT in the spatial domain. You can do the lab individually or in groups of two. Deadline for the report is 060316.




In this lab you will experiment with video coding using a hybrid coder structure similar to MPEG-1 and ITU-T H.261. Motion estimation and prediction is used to reduce the correlation in the time domain, and transform coding with a 2 dimensional DCT is made to reduce the correlation in the spatial domain. On the homepage you can find a number of small short gray-scale video sequences that you can use in the lab. Video coding is quite computationally complex, so we wouldn’t want to work on long hi-resolution color video sequences. It is primarily the motion estimation part that takes time. To be able to do the lab with reasonable run-time you should use the supplied Mex-file for motion estimation.


Video in matlab

You can load video sequences of AVI-format into Matlab with the function aviread. The following example reads the first 10 frames from the file news.avi. >> Mov = aviread(’news.avi’,1:10); This movie is represented in Matlab as a (1 × N ) struct array, where N is the number of frames. The field Mov(1).cdata contains the actual image in the first frame, and Mov(1).colormap contains the corresponding color map (gray(256)). To write a Matlab movie to a file you can use the movie2avi command: >> movie2avi(Mov,’filename’,’fps’,25,’compression’,’none’); Don’t use any compression when you write the coded movie to file, because that would otherwise introduce extra coding artifacts. To add new frames/images to a movie you use im2frame. The following code shows an example where the whole movie is filtered with a median filter: >> for i=1:N >> Mov2(i) = im2frame( medfilt2(Mov(i).cdata), gray(256) ); >> end


Motion estimation

To do motion estimation you can use either of the two Mex-files MME and MME fast. The first one uses a full search and you can set the search depth: >> [vx, vy] = MME(x, xref, m, d); where x is the current image to be encoded, xref is the reference image, m is the block size and d is the search depth. The other function, MME fast, uses a three-step search method with fixed ±7 search depth: >> [vx, vy] = MME fast(x, xref, m); To make the actual motion compensation you can use the function MC: >> pred = MC(xref, vx, vy, m); Motion compensation can create additional high frequency components in the difference image. These can be reduced with a loop filter. To avoid edge effects you can do as follows: >> hloop = fspecial(’gaussian’,3); >> diff = x2-pred; >> diff(2:r-1,2:c-1) = filter2(hloop, diff,’valid’); where a gaussian 3-tap filter is used, and (r,c) is the image size. 2


Transform coding of images

In Lab2 you did JPEG-like transform encoding of images. We will use the same method here to encode the images, and difference images, of the video sequences. The function JPEG enc implements that JPEG-like coding and decoding of an image: >> [y,nrbits,X dct] = JPEG enc(x,Q); where x is the input image, Q is the quantization matrix, y is the reconstructed image, nrbits is the number of bits required for encoding, and X dct contains the un-quantized DCT-coefficients. You can of course use your own Matlab-code from Lab 2 (last exercise) if you want to. The function Qmpeg gives the quantization matrix used in MPEG for intra-frames. For predicted frames you should use the same step size for all coefficients, nominally 16.


Lab Exercises

Try different ways of encoding video sequences. For example: • Use only intra frames (no prediction). • Use simple prediction in time without motion compensation. • Use motion estimation and compensation. You can use either the last intra-frame as reference or the previous P-frame. Compare the different methods in terms of bitrate and visual quality. You can measure the signal-to-distortion ratio also if you want to. To change the quality you can scale the two different quantization matrices. When you calculate the average bitrate you should also include the side information needed for the motion vectors. When you do motion estimation you can vary the block size and see how that affects the result. Try also with and without loop filter on the difference image. Plot and compare the coefficient statistics for a difference image and normal image (use imstat). Use the plot command quiver to plot one of the motion vector fields.




In the table below is a list of useful Matlab-functions. Some of them are standard Matlab functions (or from a toolbox), and the others you can download from the course homepage. Use ”type” or ”help” in Matlab to learn more about how they work. Name imdct invimdct imstat JPEG enc MC MME MME fast Qmpeg aviread colormap gray imshow im2frame movie2avi Quiver Description 2-dimensional block-DCT. Inverse function to imdct Calculates mean value and variance of transform coefficients. JPEG-like coding/decoding of images (Lab2). Performs motion compensation. Mex-file for motion estimation. Mex-file for faster three-step motion estimation. MPEG quantization matrix. Reads an AVI-file. Sets the color map. Gray-scale color map. Plots an image. Adds an image to a Matlab-movie. Writes a Matlab-movie to AVI-file. Plots a 2D vector field.