This action might not be possible to undo. Are you sure you want to continue?

by

JIANCONG LUO

Presented to the Faculty of the Graduate School of The University of Texas at Arlington in Partial Fulfillment of the Requirements for the Degree of

DOCTOR OF PHILOSOPHY

THE UNIVERSITY OF TEXAS AT ARLINGTON August 2005

ACKNOWLEDGEMENTS I am deeply indebted to my supervisor Prof. Dr. Ishfaq Ahmad whose helps, stimulating suggestions and encouragement helped me in all the time of research for writing of this dissertation. Without his invaluable guidance through these years in HKUST and UTA, my doctorial studies would not have been completed. Dr. Alp Aslandogan, Dr. Hao Che, Dr. Hua-mei Chen and Dr. Jean Gao have spent their precious time in reviewing this dissertation and provided me many useful suggestions. I would like to express my gratitude to them for all their helps and supports. I would like to thank Dr. Weiguo Zheng, Dr. Yu Sun for their kind helps and useful discussions. I also would like to thank my colleagues in CSE Multimedia Lab for their advices, discussions and warm helps all these years. I dedicate this dissertation to my parents, who have provided me the greatest help and constant encouragement. Without their support, I would not have gone through all the difficult times. Especially, I would like to give my special thanks to my wife Yamin, whose patient love enabled me to complete this work. July 14, 2005

ii

ABSTRACT

GAME THEORETICAL MODELS AND ALGORITHMS FOR RATE CONTROL IN VIDEO COMPRESSION

Publication No. ______ Jiancong Luo, PhD. The University of Texas at Arlington, 2005 Supervising Professor: Ishfaq Ahmad This thesis investigates game theory based rate control algorithms for optimizing the bit allocation in video compression. The first algorithm utilizes the cooperative bargaining game in a MB level rate control algorithm to optimize the perceptual quality while guaranteeing “fairness” in bit allocation among macroblocks. The algorithm first allocates the target bits to frames based on their coding complexity; a method to estimate the coding complexity of the remaining frames is proposed. Next, macroblocks of a frame play cooperative games such that each macroblock competes for a share of resources (bits) to optimize its quantization scale while considering the iii

e. The major advantage of the proposed approach is that the cooperative game leads to an optimal and fair bit allocation strategy based on the Nash Bargaining Solution. macroblocks compete cooperatively under a global objective of achieving the best quality with the given bit constraint. The proposed algorithm achieves accurate bit rate regulation and smooth buffer occupancy. The game is non-deterministic in which the players’ strategies are bound to a probability distribution over the set of available actions. The second algorithm based on a non-cooperative strategic game is aimed for video object level bit allocation.human visual system (HVS) perceptual property. Since the whole frame is an entity perceived by viewers. Another advantage is that it allows multi-objective optimization with multiple decision makers (e.g.. macroblocks). in which the utilities of the players are pre-determined by a set of available strategies (i. and to maintain a stable buffer level. The outcome of the game is a mixed strategy Nash equilibrium. The algorithm achieves accurate bit rate with good perceptual quality.. the possible quantization parameters). iv . We formulate a two-player bi-matrix game.

......................................... 1........................... LIST OF ILLUSTRATIONS...............................................................................................................2 Digital Video Coding Standards................................................................. v ii iii viii x 1 3 9 16 16 19 21 23 26 27 27 30 32 .....................................................3 Rate Control Overview ...................................................3 MB Level Bit Allocation ..........................1 Video Coding Architecture....3 Related Works ..................3...... 1............1 Introduction....... INTRODUCTION ..................................................................................................................... 1.......................................................................................................................... 2................................................................................................. 2..................................................... ABSTRACT ......................TABLE OF CONTENTS ACKNOWLEDGEMENTS..............................................................2 Rate Control Architecture............... 1........................................3..................3....................... 1...................... 2......1 Principles .....................5 Organization of the Dissertation............................................... 2.... Chapter 1..2 Frame-Level Bit Allocation................. 1........................ LIST OF TABLES..................4 Research Methodology ...................... PERCEPTUALLY TUNED RATE CONTROL USING COOPERATIVE GAME THEORY...................................................................................................................................................... 1............. 1..........

....... 3.............5...................4 Algorithm Summary ........... 3.........................................1 Summary of Contributions ............5.......................................... 2............................. 2................................................................................................................... 3............................................... 3.........6 Summary .... 2...1 Joint Multiple Video Object Rate Control using Cooperative Game ................................. JOINT MULTIPLE VIDEO OBJECT RATE CONTROL USING NON-COOPERATIVE GAME THEORY.................... 2.................................................5.......................2 Solving Quantizer Optimization with Game Theory...................................2 Target Bit Estimation for a Time Instance ............ 2...........1 Quadratic Rate-Distortion Model ........................... SUMMARY AND FUTURE WORKS.....................3 PSNR and PSPNR ................................... 2........ 3.3 Perceptually Tuned Quantizer ..............................3...............................................................6 Experiments and Results........3......7 Summary ...........2 Frame Skipping........................................................2...... vi 33 33 39 43 44 46 47 47 59 60 60 62 63 65 73 74 94 95 95 96 96 ....... 4.......... 3..............................1 Bit Rate Accuracy and Percentage of Bits Saved.....................................................5 Quantization Level Calculation and Frame-skipping Control .. 3. 2.2.....3...3 Buffer Policy....................................... 2..........5 Experiments and Results.......................................................... 4..2 Future Works .....................4 VOP Level Bit Allocation ..................... 3............................. 4.................................................................................... 4...............................................................................................................................1 Introduction...............................................................

...3 Game Theory Applications on Joint Source-Channel Rate Control .................... 4.............. 97 98 REFERENCES ............. 100 BIOGRAPHICAL INFORMATION............ 111 vii .......4.......................................................2...............................................................................................2 Improved Utility Function in Game Theoretical Models .....2.............................................

. 56 Buffer occupancy in the coding of “Container” QCIF at 192kbps and 30Hz with buffer size = 24kbits................7 2............................. 17 Distortion-quantization relation of DCT-based video encoder .. 41 Buffer occupancy in the coding of “Foreman” QCIF at 64kbps and 30Hz with buffer size = 8kbits......................................4 1........................ (d) G4 .......................................... 14 MPEG-4 Decoder Architecture....... 5 Motion vector .............. 17 The proposed algorithm embedded in a DCT-based video encoder ..............3 1.......5 2.................................................... 8 MPEG-4 Encoder Architecture ................................................ 14 Rate-distortion relation of DCT-based video encoder ............................6 2... (b) G2 ...................... 55 PSPNR in the coding of “Stefan” QCIF at 12kbps and 30Hz with buffer size = 14kbits...................................4 2...................................... 53 PSPNR in the coding of “Foreman” QCIF QCIF at 64kbps and 30Hz with buffer size = 8kbits............................ (c) G3 ................ 4 Distribution of DCT coefficients ...............5 1.............................6 1.......... 41 Matrix Gk (a) G1 ....................... 57 viii ............ 54 Buffer occupancy in the coding of “Stefan” QCIF at 12kbps and 30Hz with buffer size = 14kbits.1 2................................................................................2 1...................................................... 29 Matrix B ................LIST OF ILLUSTRATIONS Figure 1...1 1....................................................7 2..............................................8 Page Block diagram of Hybrid DCT/DPCM coding scheme .3 2.....................2 2......

..........17 3.. 79 Buffer occupancy in the coding of “bream” at 30Hz and 256kbps........ 82 Buffer occupancy in the coding of “container” at 30Hz and 256kbps..... 93 ix .18 3.......... 87 PSNR of “coastguard” using VM8 at 256kbps ..... 85 PSNR of “bream” using VM8 at 256kbps .................. 68 The simplified matrices by eliminating the dominated rows and columns......... 64 A sample of bi-matrix game............ 66 The utility matrices (a) matrix of player 1 (b) matrix of player 2....................... 88 PSNR of “coastguard” using the proposed rate control at 256kbps......................... 91 PSNR of “container” using VM8 at 256kbps......13 3.........11 3.........................................................5 3......19 PSPNR in the coding of “Container” QCIF at 192kbps and 30Hz with buffer size = 24kbits................... for i<j..................7 3.................... where ai>aj>0 and bi> bj>0....... 83 PSNR of “news” using VM8 at 256kbps ............6 3...............16 3.................................................10 3.................................. 92 PSNR of “container” using the proposed rate control at 256kbps ....1 3............................. 81 Buffer occupancy in the coding of “children” at 30Hz and 256kbps .....................9 3.......... 86 PSNR of “bream” using the proposed rate control at 256kbps ............................................. 80 Buffer occupancy in the coding of “coastguard” at 30Hz and 256kbps ...............2........................... 72 Buffer occupancy in the coding of “news” at 30Hz and 256kbps ............. 89 PSNR of “children” using VM8 at 256kbps ..9 3................. 84 PSNR of “news” using the proposed rate control at 256kbps................... 90 PSNR of “children” using the proposed rate control at 256kbps.14 3......8 3.......................................3 3.15 3.......2 3.............12 3.................4 3............................. 58 PID buffer control diagram ........................................

............1 Bit rate accuracy.................1 The performance comparison between game theory algorithm and the VM8 reference algorithm ............. 51 3.. 49 2...LIST OF TABLES Table Page 2.... PSNR......................2 Bit rate accuracy.......... PSNR..... PSPNR and number of skipped frame comparison between VM8 and the GT algorithm (QCIF sequences).......... PSPNR and number of skipped frame comparison between VM8 and the GT algorithm (CIF sequences).. 77 x ....

5Gbits per second of video data need to be transmitted via a 6 MHz channel. video conferencing. such as HDTV system. which uses 24 bits for one pixel (8 bits for each red. Problem raises that the storage and bandwidth is not satisfied the requirement of storing and transmitting the raw video data. For instance. if the line condition is very good. Another example is video phone applications.CHAPTER 1 INTRODUCTION With the increasing demands of multimedia applications. and assuming that the frame rate is 30 frames per second.3Mbits. processed and transmitted. the size of one second video is 176×144×24×30=18. which can support only about 20Mbits per second. green and blue channel respectively). This requirement implies that 1. various digital videos need to be stored. For a 24-bit color sequence. which has a dimension of 176×144 pixels. A common format for such an application is the quarter common intermediate format (QCIF). video on demand and other kinds of video based interactive multimedia services. That means that it requires transmitting video data of bite 1 . The common modem available today for PSTN can transmit at a bit rate of 56kbps. the Federal Commission (FCC) requires that the HDTV transmitted signal fits inside the same 6 MHz channel spacing as is currently used for today’s NTSC transmissions.

rate at 18.3Mbps via a channel with a bandwidth of 56kbps. It is a 330 times larger bandwidth than what is available with PSTN. The storage and bandwidth requirements of the raw video data are nearly incomprehensible. For the sake of efficient bandwidth utilization, compression is inevitable. With the help of compression technology, the requirements can be reduced by a long way. Alternatively, instead of sending the raw video sequence at a given spatial and temporal resolution, one can send a higher resolution compressed video sequence in the same channel. Due to the high demands for video application, several international video compression standards have been or are being developed during the last two decades, such as MPEG-1 [15], MPEG-2 [16] , MPEG-4 [17], H.261 [20], H.263 [21] and H.264 [23]. A series of different disciplines of digital signal processing principles, such as coding theory, rate-distortion theory, prediction techniques and control theory, have been applied cooperatively to improve the compression ratio, coding efficiency, picture quality and complexity, etc. In this work, we will focus on the bit rate control problem and aim to design rate-distortion efficient rate control schemes for frame-based and object-based video coding. The exchange of video information between remote sites requires that the digital video be encoded and transmitted through specified network connections. Due to the variable amount of redundancy and irrelevancy of video contents, the amount of compressed video data varies in a rather unpredictable manner. Therefore, the compressed video data rate is inherently variable and inconsistent with the channel 2

bandwidth. It may be lower than the available bandwidth, which leads to unnecessary degrading of visual quality and waste of bandwidth; or higher than the bandwidth constraints, resulting in traffic congestion and loss of data. Rate control is a mechanism that regulates the compressed video data rate to meet the channel requirement. It is critical in determining the encoding efficiency and video quality. This chapter will give an overview of video compression schemes with an emphasis on the rate control techniques. 1.1 Video Coding Architecture Although many different video compression standards exist, they adopt a similar architecture: Hybrid DCT/DPCM coding scheme. This scheme employs Discrete Cosine Transform (DCT) and quantization to exploit the spatial correlation while utilizes inter-frame Differential Pulse Coding Modulation (DPCM) coding techniques to reduce the temporal redundancy. The common video processing functions in Hybrid DCT/DPCM coding are: DCT and Inverse DCT (DCT/ DCT-1), Quantization and De-quantization (Q/Q-1), Entropy Coding and Decoding (EC/ED), Motion Estimation (ME) and Motion Compensation (MC), and Rate Control (RC). Figure 1.1 shows a generic block diagram for hybrid DCT/DPCM coding scheme. “FS” in Figure 1.1 symbolizes the frame storage.

3

RC Input Video DCT Q Q-1 EC Buffer

Encoder

DCT-1

MC

FS

ME

Channel

Output Video DCT-1 Q-1 ED Buffer

FS

MC

Decoder

Figure 1.1 Block diagram of Hybrid DCT/DPCM coding scheme

4

which corresponds to the low spatial frequencies.2 Distribution of DCT coefficients 5 .Discrete Cosine Transform (DCT) is a technique converting signals to frequency components. the lower-right DCT coefficients relate to low spatial frequencies within the image block and the upper-left DCT coefficients relate to the high spatial frequencies. Figure 1. the human eyes are more sensitive to the components with low spatial frequencies than those with high spatial frequencies. Based on the human visual systems criteria. This property is used to remove the subjective redundancies. Figure 1. Most of the DCT coefficients are small enough to vanish in the quantization step and need not to be encoded and transmitted. Coefficients with small value are less significant for the reconstruction of the image blocks than coefficients with large variances. For 2D DCT transform.2 depicts the variance distribution of 8x8-block DCT coefficients. We can observe that the energy is concentrated around the upper-left corner.

The first element is the number of consecutive zeros while the second element represents the non-zero value immediately following the previous run of zeros. quantization reduces the amount of information required to represent the amplitudes. Huffman coding assigns a variable 6 . Entropy coding for video compression applications is a two step process: Zero RunLength Coding (RLC) and Huffman coding. For example. Quantization in the frequency domain has many advantages over quantization in pixel domain. Some other quantization schemes have been reported in [55] [59]. Although different types of quantization are proposed. 0. By converting amplitudes that fall in certain ranges to one in a set of predetermined levels. Entropy coding is a lossless process based on statistics of the image or the motion picture sequence to be compressed. In this manner. 0.Quantization is the major source of data loss in DCT based compression algorithms. for simplicity. 6) represents the sequence (0. data can be compressed by a factor of 3 or 4. Quantization in pixel domain leads to "contour" artifact where small amplitude changes in a smoothly gradient area cause step-sized changes in the reconstructed amplitude. 6) of numbers. Although there is a variety of different implementation of entropy coding in the existing video compression standards. RLC data is an intermediate symbolic representation of the quantized bins which utilizes a pair of numbers. the underlying theory of entropy coding is to encode the most frequently occurring patterns with the shortest code word and encode the less frequently appearing patterns with longer code words. the RLC code (3. all the standard image compression algorithms use linear quantization where the step size quantization levels are constant.

motion estimation is utilized to determine the displacement of an object. Forward motion vectors refer to correlation with previous pictures. Since entropy coding produces variable length code word. a frame is divided into 16x16 blocks. In the block matching motion estimation.. In contrast. A Huffman tables is pre-computed based on statistical properties of the image (as it is in JPEG) or pre-determined if a default table is to be used (as it is in H. In general. To address this issue.e. Backward motion vectors refer to correlation with future pictures. compression ratios for motion video sequences may be increased by encoding the difference between two or more successive frames. The same table as used in encoding is used to decode the bitstream data. which implies that the difference between these frames is small. motions of objects increase the difference between frames. Full search algorithm exhaustively evaluates all possible displacements 7 . i.261 and MPEG). For this reason. the digital compressed stream has no specific boundaries or fixed length. consecutive frames in a motion video tend to be highly correlated. The Huffman coding is a table-lookup process. Motion vectors encapsulate the displacements between the macro-blocks in the current frame and the best correlated macro-blocks in the past and/or future reference frames containing previously decoded pixels that are used to form the prediction and the error difference signal. the frames only change slightly over a small period of time. This implies that more bits are required to encode the sequence. Motion estimation (ME) is the process of finding motion vectors during the encoding process.length code to the RLC data. called macro-block. producing variable length code word.

In order to reduce the computational complexity of the encoding algorithm. The defect is also obvious.within the predetermined or adaptive search range to find out the best matching 16x16 block in the reference frame for a macro-block of the current frame. Reference frame Search Area Current frame Macroblock Comparison of macroblocks Motion vector (dx. Extensive survey of FME has been included in [31]. The computational complexity for a full search block matching algorithm is very heavy.3 Motion vector 8 . The advantage of full search block matching algorithm is accuracy. It always leads to the global minimum by looking at everywhere in the search area. The computational complexity of full search algorithm can be 60~80% of total computations in the encoding process involves in the purpose of full search algorithm [32] [55]. a number of fast ME (FME) algorithms are proposed.dy) Figure 1. The representative FME algorithms include [14] [24] [29] [36] [38] [40] [56] [70] [71] [79] [80].

ISO/IEC has published MPEG9 . video compression produces variable bit rate output streams due to the variable amount of redundancy and irrelevancy in the input sequences. It is not standardized and is an open issue for research. number of frame skipped. Meanwhile rate control algorithms optimize the visual quality subject to the available transmission data rate and certain quality of service (QoS) requirements. Furthermore. The objective of rate control algorithms is to regulate the output bit rate to meet the channel constrains.As mentioned above. such as delay. etc. Our research focuses on the rate control problem. Although the video coding standards designate the syntax of encoded presentation. Rate control is critical to the visual quality and rate-distortion performance. the constant bit rate (CBR) channels require that a constant number of bits per time unit be sent to the channel.2 Digital Video Coding Standards Standardization of video compression facilitates manipulation and storage of motion video as a form of computer data. quality fluctuation. interchange and interoperation of video contents become possible only when the coded representations of video contents conform to the standards. To this end. For example. In both cases. This characteristic of video encoders inherently mismatches the constrained media channels. and the variable bit rate (VBR) channels require that a set of specified traffic parameters be met since otherwise data losses may occur. and its transmission over existing and future networks or over terrestrial broadcast channels. 1. bit rate regulation is necessary. the methods of generating the standard conformed bitstream is not defined.

although MPEG compression scales well with increased bit rates. we will introduce the major video coding standards in the chronicle manner. transmitted as . The increasingly obsolete H. Each standard was designed for a specific application and bit rate.261 was specifically designed for transmission over ISDN lines in that data rates are multiples of 64kbit/s. which was ratified in 1990 by ITU. MPEG-1 is also a popular standard for video on the Internet. It was designed for CD-ROM video applications and a bit rate of up to 1. The standard supports CIF and QCIF video frames at resolutions of 352x288 and 176x144 respectively.261. In the sequel. MPEG-1 is the standard of compression 10 . The data rate of the coding algorithm was designed to be able to be set to between 40Kbits/s and 2Mbits/s.261/263/264. The coding algorithm is a hybrid of inter-picture prediction. such as video phone and video conference.mpg files. Five MPEG standards are being used or in development. and motion compensation. mainly aiming to real time video conferencing applications. Telecommunication Standardization Sector of the International Telecommunications Union (ITU-T) also defined its own video compression standards. MPEG-1 was completed in 1992 as the first MPEG standard for the compression of moving pictures and audio. transform coding.5 Mbit/sec. These standards are mainly focus on real time video applications. established in 1988 to develop standards for digital audio and video formats. Moving Picture Experts Group (MPEG) is an ISO/IEC working group. The first video coding standard is H.1/2/4 targeting on different applications and ITU also proposed H.

261. etc.k. data partitioning. SNR scalability.263v2 (a. optimized run length coding (VLC) table.263 has a substantial improvement in video quality and coding efficiency compared with H. It has been superseded by H. MPEG-1 employed some advanced techniques. including efficient compression of interlaced video. such as half-pixel motion compensation. H. spatial and temporal scalability. It was first designed to be utilized in H. MPEG-2 supports flexible picture size and bit rate. etc.5 and 15 Mbit/sec). advanced motion vector prediction.263 provides advanced coding modes.a. such as bidirectional prediction frame and half-pixel motion compensation. Comparing with H. MPEG-2 supports some significant enhancement from MPEG-1. MPEG-4 was initiated for multimedia and Web compression. arithmetic coding and enhanced error resilience.263 provides a suitable replacement for H. obviating the need for an MPEG-3. H. Ratified in 1994. PB frame mode.for Video CD. H. MPEG-2 is a standard on which Digital Television set top boxes and DVD compression is based. The motivation of 11 . H. but now is finding use in RTSP (streaming media) and SIP (Internet conferencing) solutions as well. MPEG-2 was based on and was a superset of MPEG-1 but specifically designed for the compression and transmission of digital broadcast television which applied higher bit rate (between 1. It scales well to HDTV resolution and bit rates.263 is a video compression standard designed by the ITU-T as a low bit-rate encoding solution for videoconferencing.261. H. unrestricted motion vector.263+ or H.323 based systems. the most popular video distribution format throughout much of Asia.263 1998).261 at all bit-rates.

compression and universal accessibility [53]: Content-based interactivity .Improved temporal random access Compression .Content-based manipulation and bitstream editing . TV and film industry. What distinguishes MPEG-4 from other video coding standards is the concept of object-based video coding. 3D image and video and telecommunications. The new or improved functionalities of MPEG4 have been clustered in three sets . last but not least. multimedia cooperative work. MPEG-4 supports a wide range of bit rates (5kbps-5Mbps) and applications such as video conferencing.Hybrid natural and synthetic data coding .261/263.MPEG-4 is the convergence various multimedia technologies. such as computer graphic.Improved coding efficiency . tele-teaching and. but has been extended to support 12 . mobile videophone.Coding of multiple concurrent data streams Universal access .Robustness in error-prone environments .content-based interactivity.Content-based multimedia data access tools . games. MPEG-2 and H.Content-based scalability The MPEG-4 video compression scheme is based on the block-based concept as used in the MPEG-1.

Alpha-plane can be gray-scale or binary image. It also allows developers to control objects independently in a scene. Visual objects can be natural video object or synthetic objects. The video object at a time instance is called video object plane (VOP). Arbitrarily-shaped video objects are split up into macroblocks within a bounding box. Individual objects within a scene are encoded and decoded separately by different encoder and decoder instance and multiplexed together to create an MPEG4 file. Every object is encoded and decoded by a difference encoder and decoder instance and may use different coding options. An alpha-plane represents the shape of a video object. This results in very efficient and very scalable compression. and therefore introduce interactivity.arbitrarily-shaped video objects. 13 .

4 MPEG-4 Encoder Architecture Encoding MUX Bitstream VOP 0 Decoding VOP 1 Display Scene Composition Decoding DEMUX VOP 2 Decoding … Figure 1.VOP 0 Encoding VOP 1 Input VOP Formation VOP 2 Encoding … Figure 1.5 MPEG-4 Decoder Architecture Input Bitstream 14 .

AVC employs a series of new technologies. rate-distortion optimized (RDO) mode decision and motion estimation. support both Context-based Adaptive Binary Arithmetic Coding and Variable Length Coding. and it is under consideration for other broadcast use -.for example. A number of broadcasters in Japan and Korea have announced future support for the codec. sub-pixel motion compensation (down to 1/8 pixel). The major enhancements include: multiple reference frame motion compensation prediction in inter mode. now under development by the DVD Forum. advanced in-loop filter. In the wireless world. AVC is already widely used for videoconferencing. etc. Intra prediction. It has also been preliminarily adopted as a mandatory part of the future DVD specification known as HD-DVD.264 and identical to ISO MPEG-4 part 10. As the latest video compression standard. This standard is also known as H.Advanced Video Coding (AVC) is a high compression digital video compression standard written by the ITU-T Video Coding Experts Group (VCEG) together with the ISO/IEC Moving Picture Experts Group (MPEG) as the product of a collective effort known as the Joint Video Team (JVT). it is under consideration for adoption by the 3rd-Generation Partnership Project (3GPP). 15 . multiple block size from 4x4 to 16x16. 4x4 integer DCT/IDCT. it is under consideration in the United States' Advanced Television Systems Committee (ATSC) and in Europe's Digital Video Broadcast (DVB) standards bodies. AVC offers significantly better coding efficiency than the previous ITU and MPEG standards.

and vice versa. In this channel mode. rate-distortion function can well embody the behavior of video encoder. In CBR channels. meanwhile.6 illustrates the rate-distortion relation of a generic video encoder. by choosing the proper operation point on the rate-distortion curve. 16 . the objective of rate control is to regulate the output bit rate of a video encoder to the bandwidth requirement. the output bit rate of and encoder can be controlled. a transmission buffer is used to accommodate the variable data rate nature of the compressed video.1. Though the network is capable to provide a wide rage service from narrow bandwidth to wide bandwidth. rate control is still required to avoid network congestion. the target of the rate control is to regulate the output bit rate to avoid buffer overflow and underflow.1 Principles Limited bandwidth network usually is used as the infrastructure of the digital video communication and transmission. depending on the required data rate to maintain a satisfactory reconstructed quality. But network congestion still can happen since the burst of peak rate of the video data may exceed the network capacity. Therefore.3 Rate Control Overview 1. the video data generally are transmitted over a constant bit rate (CBR) channels. The output bit rate can be increased by increasing the distortion level. Figure 1. Some network service. the possibility of buffer overflow or underflow exists. Overall. Therefore. In VBR channel. also supports variable bit rate (VBR) channel.3. The buffer is not necessary in VBR channel since the network can accommodate the variable rate video data. In lossy video coding. such as B-ISDN [9].

7 Distortion-quantization relation of DCT-based video encoder Quantization is the major source of data loss in DCT based video compression thus is closely related to the distortion level of the reconstructed image.6 Rate-distortion relation of DCT-based video encoder Operation points Distortion Quantization scale Figure 1. The output bit 17 .Operation points Bit rate Quantization level Figure 1.

macroblock-level rate control assigns less bits to encode the DCT 18 . Compare with frame-level rate control. The control unit is a frame. In the other word. 2) With macroblock-level rate control. which generally leads to better responds to the change in video characteristics and model parameters. A larger quantization scale masks more DCT coefficients. in the same bit rate. The control unit is a macroblock. saves more bits. but a uniform QP is used for all MBs within a VOP. rate control algorithms can be classified into the following categories: (1) Frame-level rate control: a uniform quantization parameter (QP) is assigned to all the MBs in a frame. but at the expense of picture quality.rate of a DCT-based video encoder is typically controlled by choosing a proper quantization scale. For the multiple-video-object (MVO) coding. therefore. since macroblock-level rate control use a smaller basic control unit. The defects of macroblock-level rate control include: 1) it requires extra bits to encode the change of QP. A finer quantization scale produces more bits but results in better perceptual quality. it is possible to compute QP for each macroblock to optimize the visual quality of a frame. whereas using a uniform QP may not be optimal in the rate-distortion (R-D) sense. Depend on the control granularity. VOP-level rate control can be employed where different QPs can be assigned to video object plane (VOP). (2) Macroblock-level rate control: QPs for macroblocks can be different. Macroblock-level rate control has the following advantages: 1) macroblock-level rate control is able to achieve more accurate bit rate.

the rate control algorithm consists of the following steps: 19 . Frame-based rate control algorithms are employed in the frame-based video encoding such as MPEG-1 and MPEG-2. especially in low bit rate coding environment. Multiple object encoders share a total data rate. different rate control strategies can be employed.2 Rate Control Architecture The following rate control architecture is commonly used in most of the current video applications: A buffer is placed between the encoder and the channel in order to absorb the variation of the output bit rate and stabilize the output bit rate. For the object-based video coding such as MPEG-4.coefficients compare with frame layer rate control. which may lead to the lower visual quality.3. In the independent rate control. such as MPEG-4. Compared with frame-based algorithms. 2) A macroblock-level rate control usually is more complicate that a frame-level rate control. The rate control algorithms can also be categorized frame-based rate control and object-based rate control. coding of scene description information and bit allocation for multiple objects. data rate of object encoders are jointly considered. 1. whilst the object-based rate control algorithms are utilized in the objectbased video coding framework which supports the encoding the arbitrary shape object. depending on the encoding infrastructure. each object encoder has its own rate controller. the object-based rate control algorithms deals with additional issues including data for shape coding (coding of alpha plane). Efficient resource allocation among objects is required. In the joint multiple-video-object (JMVO) rate control. The objects are encoded separately at independent constant bit rates or quality. Generally.

The popular approach of obtaining the quantization parameter is the utilization of R-D models. – Buffer occupancy. etc. Target bit allocation The target bit constraint allocated prior to the encoding of a frame. Depending on the control granularity.1. an object or a macroblock. the 20 . buffer status and the statistic of previous coded units. – Quality fluctuations. Buffer overflow occurs when the encoder output bit rate is too high such that the accumulate bits is more than the buffer capacity. 2. Quantization parameter determination A proper quantization parameter for the coding unit is obtained in this step. the encoder status is updated in this step. a VOP or a macroblock. 3. Several aspects strongly affect the performance of a rate control algorithm. Smaller buffer occupancy leads to lower delay. remaining frames. In this case. The allocation can be based on the available bits. R-D model parameters. – Buffer overflow. The quantization parameter is estimated such that the output bit rate meets the target bit constraints determined in step 1. buffer occupancy. The updated values are used for the encoding of the next coding unit. the quantization parameter can be for a frame. remaining coding unit. These values include available bits. Post-Encoding Processing After encoding a frame. Fluctuations in image quality between consecutive time instances can be visually annoying and thus should be reduced as much as possible. an object or a macroblock.

various rate-distortion models have been utilized to improve the accuracy of the quantization scale estimation. Thereafter. In VBR networks. With respect to the model-based quantization scale decision. MPEG Test Model 5 (TM5) [18] rate control employs a hierarchical bit allocation scheme. Target bits for I. P. such as H. It is not as critical as buffer overflow but should also be avoided. which results in sub-optimal use of the channel. Buffer underflow usually occurs when the encoder output bit rate is lower than the dripping bit rate of the buffer. The target bit budget for each frame is obtained by dividing the target bit rate by the frame rate. Constrained by the GOP bit budget. like ATM networks. – Buffer underflow. In this case.263 Test Model Near-term version 5 (TMN5) [22].3. – Traffic contract. Buffer overflow always causes loss of data. such as TM5. early rate control algorithms. packets transmitted outside the bounds established by the negotiated contract may be discarded if traffic congestion occurs resulting in loss of data which must be avoided. therefore must be avoided. the target number of bits is decided for the GOP layer first. a fixed bit allocation for each frame is employed.3 Related Works Several rate control algorithms have been proposed and utilized in video compression standards and applications. adopt a linear rate-distortion model. target number of bits for a frame is then calculated.excess bits will be dropped. In the early rate control algorithms. 1. Chiang and Zhang proposed a quadratic rate-distortion 21 . not enough bits are dripped to the channel. B frame are allocated according to the complexity of the previous frame.

Tsai and Hsieh have proposed to modify the encoding order of MBs to favor the more complex MBs [70]. An improvement of [4] has been proposed by Pan et al. Chiang and Zhang have proposed an algorithm that is scalable for various bit rates. Based on [62]. The model used in low bit rate maintains a near-constant quantization scales. Based on [4]. spatial and temporal resolution [33] [34]. and an optimized bit allocation has been proposed based on this model [10] [11] [12]. Lee et al. Ngan. which is the percentage of zeros among the quantized DCT coefficients. which was adopted in the H. He and Mitra have modeled the rate and distortion as the functions of ρ. The model used in high bit rate adapts the quantization scale with the energy of the block by using finer quantization for MB of flatter image regions. Lee and Zhang have proposed a scheme for MB level bit allocation and two distinct models for high bit rate and low bit rate situations respectively [3]. to fine tune the bit allocation according to the position of a frame in a GOP [52].263+ testing model TMN8 [62]. in order to minimize the overhead bits for DQUANT. has proposed models for coded frames and objects as well as skipped frame and objects [35]. Chiang. Another MB level rat-distortion model has been addressed by Oehler and Webb [50]. which is to define the change of quantization scale. For object-based video coding. Vetro and Sun also developed a scheme for multiple video objects in [73] [74]. Other rate control algorithms have 22 . Meier and Chen have introduced a new constraint for the least-mean-square estimation of the model parameter of the rate-distortion function [49]. Cheng and Huang have proposed an adaptive piecewise linear model [2].model that can be applied to both DCT and wavelet-based coders [4]. Ribas-Corbera and Lei have proposed a rate-distortion model. Lee.

Since the quantization scale can refer the distortion of the decoded frame. and propose an efficient solution. Although these algorithms address the rate-distortion optimization problem.been proposed in [27] [38] [68] [78]. Towards this goal. The objective of our research is to define a new criterion of optimizing the ratedistortion efficiency under the bit rate and “fairness” constraints. 23 . a few algorithms have considered the rate-distortion optimization problem. the “fairness” of bit allocation has not been considered. the perceptual redundancy of human vision system has not been efficiently exploited. E. some rate control algorithms seek optimal quantization to maximize the aggregate perceptual quality or minimize the aggregate distortion subject to the bit rate constraints. our study applied cooperative and non-cooperative game theory approaches for rate control.4 Research Methodology Although rate control is a well studied topic in video compression and several efficient schemes have been widely used in standards. What distinguishes our approach from these works is that we define a new criterion of optimizing the rate-distortion efficiency under the bit rate and “fairness” constraints. Moreover. the fairness of bit allocation among the macroblocks has not been discussed. there still are different methods that have not been explored and improvement can be achieved by using new approaches. using Lagrange multiplier or dynamic programming [58] [60] [62] [65] [77]. 1. the perceptual redundancy of human vision system has not been efficiently exploited. etc. The above rate control algorithms reportedly achieve efficient bit rate regulation.g. and propose an efficient solution.

called cooperative and non-cooperative. Players may or may not communicate and may make binding agreements. players can communicate and make binding commitments. It is widely thought to have originated in 24 .Game theory deals with the decision making problems. once or repetitively. who are called decision makers or players. They are supposed to be able to reach cooperatively some desirable gains (for the group and each player). thus you are playing with other bidders. In cooperative games. Two classes of games are distinguished. when you are bidding in an auction. when you are driving. sometimes estimation of other bidders’ bids. The units of analysis (primitives) are groups.. Players can act simultaneously or sequentially. Each player is treated as a “selfish” individual who cares about maximizing its own utility in presence of strategic interdependences. which is the decisions each player makes and their corresponding outcomes.g. Each player may of may not be fully informed about the other players’ choices. your decision is based on how you value the item and the money in your pocket. In noncooperative games. Game can be found in everyday life. no communications and binding commitments are allowed. The question in cooperative games is how the players share the gains from their cooperation. and is assumed to react only in his/her own interests. In every game we look for a “solution”. thus you are playing a game with other drivers. you need to make decisions depending on the behaviors of the other drivers on the road. The history of game theory is not long. A game is a mathematical model of an interactive decision problem – a situation in which the outcome is determined by the choices of two or more parties. E.

the early twentieth century. He also proposed that cooperative games were reducible to non-cooperative games. 25 . He accomplished that by pioneering the axiomatic bargaining theory and proved the existence of the Nash Bargaining Solution (NBS) for cooperative games (a notion similar to the Nash Equilibrium) [43]. Biology [13]. auction theory is being recognized as the emerging solution for problems in microeconomics [41]. Nash proved the existence of a strategic equilibrium for non-cooperative games (Nash Equilibrium). In [45]. and agent-based systems [64]. Some of the most pioneering results were reported within a year. However. that the world realized the power of game theory for studying the behaviors in Economy. a few applications of Game Theory applied to job scheduling and networking problems have been documented [61]. Later. Others had anticipated some of the ideas. In computer science. Game Theory has been used as a powerful method to solve and analyze problems that contain natural competition in several areas of social sciences [29]. when von Neumann laid the groundwork by proving the min-max theorem in a paper in 1928 [47]. when Nobel Laureate John Nash made seminal contributions to both cooperative and non-cooperative games. Recently. The remarkable property of Game Theory is its abstractly defined mathematics and notions of optimality. Political Science [42] Economics [41]. it was not until 1944 when Von Neumann and Morgenstern’s classic book The Theory of Game and Economic Behaviors [48] appeared. many economists and mathematicians made innovative contributions. etc. Social Science and Biology. Political Science. In no other branch of Sciences do we find so many understandable definitions and levels of optimality [51].

The perpetual visual quality is optimized under the bit rate and “fairness” constraint. a macroblock level rate control schemes using cooperative game theory is presented. 1.Cooperative and non-cooperative games are applied in our study of game theoretical rate control. Moreover.5 Organization of the Dissertation The dissertation organized as follows: In chapter 2. In chapter 3. 26 . a JMVO rate control schemes using non-cooperative game theory in object level bit allocation is proposed. Chapter 4 summarizes the studies and research presented in this dissertation. we incorporate the human visual property into the game theoretical frameworks. this work is the first attempt to apply game theory in video compression. We formulate the bit allocation problems in rate control to different types of games and derive the bit allocation strategies from the solutions. To our knowledge. and discusses the possible future research.

The proposed rate control algorithm has two stages. blocks).g.g. which is the mean absolute prediction error.. which is typically the perceptual quality or distortion of an entire frame. under which each decision maker has its own objective function.CHAPTER 2 PERCEPTUALLY TUNED RATE CONTROL USING COOPERATIVE GAME THEORY 2. target bits are allocated at the frame level. In the second stage. we propose a method to estimate the coding 27 .. While such an approach allows an overall good visual quality. while working under an overall constraint (e.1 Introduction Most rate-distortion optimization algorithms optimize a unique objective function. the quantization scale for each MB is determined by using a game theoretical approach. a given bit budget for a frame). This work presents a game theory based rate allocation strategy that allows multi-objective optimization with multiple decision makers (e. In the first stage. the algorithm allocates target bits to the current frame based on the coding complexity of the frame. The solution for the cooperative game yields the optimal bit allocation that is fair to each macroblock (MB) under the give constraints. The proposed approach is based on cooperative game theory. At the frame level. the same objective function may not yield the best results for the whole video frame or a sequence. which is its own perceptual quality measure. Since the remaining frames are unavailable.

The target number of bits of the current frame is optimized by using the current frame coding complexity as well as the estimated coding complexity of the remaining frames. Figure 2. we incorporate the human visual system (HVS) perceptual property in the game theory framework. Section 3 describes the proposed frame level bit allocation scheme and the proposed game theoretical formulation for MB layer quantizer optimization.complexity of the remaining frames from the encoded frames. Based on Nash Bargaining Solution [46]. At the MB level. which is guaranteed for each MB. Section 5 concludes the work with some final remarks. Each MB competes for a share of resources. 28 . which is the distortion that exceeds the Just-Noticeable-Distortion (JND) threshold [24]. The gray box in the diagram shows the modules of the proposed algorithm. The rest of this chapter is organized as follows: Section 2 presents the related work on rate control algorithms reported in the literature. which are the target bits for a frame. is determined proportional to the perceptible distortion. Finally. We formulate the bit allocation as a bargaining problem [44]. Section 4 presents simulation results of the proposed scheme including a comparison to the quadratic rate control proposed by [4] and recommended by MPEG-4 VM8 [19].1 shows the block diagram of the proposed rate control algorithm that can be embedded in a DCT-based video encoder. we derive a cooperative optimal quantization scale for each MB. Initial visual quality of the game setting. the algorithm utilizes game theory for bit allocation. Furthermore.

1 The proposed algorithm embedded in a DCT-based video encoder 29 .Frame Layer MAD Bit Allocation Buffer fullness Noticeable distortion computation Initial quality computation MB level Game theoretic QP computation Buffer Output bitstream Prediction Error Computation Input sequence Compensated frame DCT Q Q-1 DCT-1 Frame reconstruction VLC Motion Estimation & Motion Compensation Reconstructed frame Motion Vectors Frame Delay Figure 2.

y) − p( x.1) where (x. y) x =0 y =0 W H (2. The estimated coding complexity of the remaining frames is denoted by mad_rt. We measure the coding complexity of a frame using the mean absolute prediction error. Therefore. The target bit budget is allocated proportional to the coding complexity of the current coding frame. Since the encoder only contain the current frame and the reference frame due to the memory and delay constraints. denoted by mad: mad = 1 W ⋅H ˆ ∑∑ p( x. Since it takes more bits to encode a complex frame than a simple frame to obtain the same visual quality. W and H are the width and height of the frame in pixel. we estimate the coding complexity of the remaining frame by computing the weighted mean of the coding complexity of the previous coded frames. one needs to consider the distribution of the bit budget for each frame.2. we need to know the coding complexity of the remaining frames.y) is the pixel coordinate. we propose an approach to tune the frame level target bit budget according to the estimated coding complexity of the frame. To maintain stable quality throughout the video sequence. Suppose we are encoding a frame at time t. and given by: 30 .2 Frame-Level Bit Allocation The visual quality of an encoded frame is related to the bits it consumes.

We need to further adjust the target bits Tt’ to the current buffer fullness with the following equation: 31 .3) where Et-1 and Nt-1 are the numbers of remaining bits and remaining P frames after encoding the frame at time t-1. The number of bits allocated to a frame at time t is given by: Tt′ = Rt −1 madt ⋅× N t −1 mad _ rt (2.4) where Br is the target bit rate of the sequence. L is the total length of the sequence in unit of second. The number of the remaining bits after encoding the first I frame is: R0 = Br ⋅ L − R f (2.2) (t + 1) ⋅ t 2 ⋅ ∑ (i ⋅ mad i ) t −1 = (t − 1) 2 mad t ⋅ i =1 + (t + 1) t ⋅ (t − 1) t +1 (t − 1) 2 = mad t ⋅ mad _ rt −1 + (t + 1) t +1 where madi is the coding complexity of frame at time i. Initially. I frame at time 0 is encoded using an external input quantization scale. Rf is the number of bits used to encode the first I frame.mad _ rt = ∑ (i ⋅ mad i =1 t i ) ∑i i =1 t = 2 ⋅ ∑ (i ⋅ mad i ) i =1 t (t + 1) ⋅ t 2 ⋅ ∑ (i ⋅ mad i ) + 2t ⋅ mad t i =1 t −1 = (2.

Tt′′= Bt + 2 × ( B − Bt ) × Tt′ 2 × Bt + ( B − Bt ) (2. Br/F is the average bits for a frame (2. 32 . the target number of bits is bounded by: ⎛ 2 Br ⎛ Br ⎞⎞ Tt = min ⎜ ⎟ ⎜ F . We solve the problem by playing a multiple-player cooperative game. Otherwise. Each MB competes for a share of resources to optimize its own performance. Finally. An optimal and fair bit allocation strategy is derived based on the Nash bargaining solution.3 MB Level Bit Allocation The problem to identify the optimal quantization scale is equivalent to find an optimal allocation of the frame target bits to maximize the perceptual quality.6) where F is the frame rate. which is a resource optimization problem.5) where B is the buffer size and Bt is the buffer fullness at time t. Since the whole frame is an entity perceived by the viewers. If the buffer fullness is lower than the middle level. 2.6) intends to avoid allocating extremely large or extremely small number of target bits to a frame.5) intends to maintain the buffer fullness in the middle of the buffer. MBs need to work cooperatively. fewer bits will be allocated. in order to prevent the buffer overflow or under flow. max⎜ 4 F . Tt′′⎟ ⎟ ⎝ ⎠⎠ ⎝ (2. more bits will be allocated to the current frame. The upper bound of target number of bits for a frame is two times of average bits for a frame and the lower bound is a quarter of the average bits for a frame. (2.

Qi is the quantization step size for the ith MB. The initial value K0 is set to 3000. The ratedistortion model parameter K for the (i+1)th MB will be updated as the following: ⎧ (i − n s − 1) ' A Q2 Ki + α i i ⎪ = ⎨ i − ns mi (i − n s ) ⎪ K i × 0.9 ⎩ Ai > 0 Ai = 0 K i +1 (2.3. where Ai is the actual bits used to encode the ith MB. α is a constant (α=0. ns is the number of skipped MBs. 2.3. to determine the optimal quantization scale. mi is the standard deviation of prediction errors of the ith MB. The rate-distortion model is updated after the encoding of each MB. K are model parameters.2.8).2 Solving Quantizer Optimization with Game Theory In the proposed MB level bit allocation.1 Quadratic Rate-Distortion Model We propose a quadratic rate-distortion model to formulate the relation of the consumed bits and the quantization scale used to encode a MB.7) where ri is the number of bits spend to encode the ith MB. the bargaining game is configured as following: 33 .8) ⎧K and K i'+1 = ⎨ i' ⎩K i Ai > 0 Ai = 0 . The model is given by: ri K = 2 α mi Qi (2. The proposed model is plugged in to the game theoretical framework presented in section 0.

The initial quality is determined according to the perceptible distortion of each MB. d2. Since u>d.r2 . r20 . which means: 34 . uN(r)) is the utility of the game. where r0 = (r10 . the visual quality of a MB is related only to the number of bits it obtained. Preference: A utility function ui for each player i reflects its preference. is the initial perceptual quality that required to be guaranteed.…. u2(r2). Since in the DCT video coding.: ∑r ≤ R i =1 i N c (2. we have u>d. Each MB is regarded as a player in the game.e. Higher visual quality is more preferable. …. u = (u1(r). Given a combination of strategies carried out by all the MBs r = (r1 . rN0). the sum of the bits requested by the N remaining MBs should be no more than the remaining bits for a frame. the utility can be represented by u=(u1(r1). …. u2(r). which is defined in section 0. denoted by ri. Initial utility: The initial utility of the ith MB. dN). uN(rN)). Define the number of bits achieving di as ri0.Players: Each frame contains N uncoded MBs. we have r > r0 .rN). i. …. N players compete for the use of a fixed resource. Denote the initial quality of the game d=(d1.…. Since the target number of bits for a frame is constrained. We use the visual quality as a measure of utility. Strategies: The strategy of a player is the number of bits it requests for.9) where Rc is the remaining bits for the N MBs. which is the target bit budget Rt for the frame at time t. denoted by di.

The irrelevant alternatives axiom expressed that if we remove the irrelevant subset (the subset does not contain the Nash bargaining point). Let strategy r* be the NBS of the game <U. F (d ) >) = F (u * (< U . the corresponding utility u * (< U . and X ⊆ Y . then u * (< Y . d>. the NBS of the game will not change. d > ) .e.ri > ri0 (2. d > ) ∈ X . The last three axioms 35 . The efficiency axiom states that the Nash bargaining solution is cooperatively optimal. d > ) is called the Nash bargaining point. then exchange the two players will not affect the solution. i. d> represents the game setting. The linearity axiom implies that the NBS will not change if the player’s objective is linearly transformed. there is no other solution produces better utility for one player without hurting another player. Tuple <U. Symmetry: If U is symmetric with respect to any two players in the game. Nash bargaining solution (NBS) is a unique solution that satisfies a set of axioms for fair bargain [46].10) Define U the set of achievable utilities. d > ) = u * (< X . Nash bargaining point u* satisfies that: u * (< F (U ). Linearity: Given a monotonous increasing linear transform function F. The symmetry axiom says that the solution is only depended on the players’ initial utilities and their utility functions. and if their initial utility is equally preferable.. Y are sets of attainable utilities. d >) Independence of irrelevant alternatives: Let X. The following axioms define a unique NBS: Efficiency: The Nash bargaining solution is Pareto optimal. If u * (< Y .

we defined the visual quality of the ith MB by: ui = 1 12 12ri = 2 = Di Qi Kmiα (2. The above problem is equivalent with the following problem: max ∑ ln(u i (ri ) − d i ) i =1 N s. ri > ri0 .14) where λ and θi are the Lagrange multiplier. Therefore. NBS for a multiple players bargaining game is characterized by the following property [66]: The solution r* is a NBS if and only if ∏ (ui (r i * ) − d i ) ≥ ∏ (u i ( r ) − d i ) i for all r ∈ H . we need to solve the following maximization problem: max ∏ (u i (ri ) − d i ) i =1 N s. Therefore. The approximate mean square error (MSE) distortion of the ith MB is Di = Q 2 / 12 [8]. using the theorem of Kuhn and Tucker [67]. ln(ui) is strictly concave. where H is the set of all feasible combination of strategies.9) and (2.11) The conditions in (2. J = ∑ ln(u i − d i ) + λ ( Rc − ∑ ri ) + ∑θ i (ri − ri0 ) i =1 i =1 i =1 N N N (2.13) The above inequality constrained optimization problem can be solved by maximizing the following Lagrangean.10). ri > ri0 . 36 . to find the Nash bargaining solution. ∑r i =1 N i ≤ Rc (2.11) are constraints from (2.t.t. ∑r i =1 N i ≤ Rc (2.12) Since ui is concave and injective.are the axioms of fairness.

16) λ + K α mi d i 12 and R c − ∑ ri = 0 i =1 N K N ⇒ R c − − ∑ m iα d i = 0 λ 12 i =1 N (2. From (2. θ i = 0 .. there must be a solution r that strictly superior to r .17) Therefore.⎧ ∂J ∂ ln(ui − di ) − λ + θi = 0 ⎪ ∂r = ∂ri i ⎪ N ∂J ⎪ = Rc − ∑ ri ≥ 0 ⎪ ∂λ i =1 ⎪ N ∂J ⎪ = λ ( Rc − ∑ ri ) = 0 ⎨ λ ∂λ i =1 ⎪ ∂J ⎪ = ri − ri0 ≥ 0 ⎪ ∂θ i ⎪ ⎪ θ i ∂J = θ i (ri − ri0 ) = 0 ⎪ ∂θ i ⎩ ..15). Therefore.15) Since ∑r i =1 N 0 i < Rc .N ) (2. the NBS for the game is given by: ri = Rc K N α K − ∑ m j d j + 12 miα d i N 12 N j =1 (2. so that 0 ri − ri0 > 0 .18) And the optimal quantization step size is given by: 37 . we have ∂ ln(u i − d i ) =λ ∂ri ⇒ ri = 1 (2. where i ∈ (1.

we have ∑ i =1 N [ui (ri ' ) − d i ] − [ui (ri* ) − d i ] ≤ 0 u i (ri* ) − d i ⎡ 12ri ⎤ ⎡ 12ri* ⎤ − di ⎥ − ⎢ − di ⎥ ⎢ α N Kmiα ⎦ ⎣ Kmi ⎦ ≤0 ⇔∑⎣ * 12ri i =1 − di Kmiα ⇔∑ i =1 N N 12ri 12ri* − Kmiα Kmiα ≤0 12ri* 12ri0 − Kmiα Kmiα ri '−ri* ≤0 ri* − ri0 38 ⇔∑ i =1 (2.sN) is said to be proportionally fair with respect to a utility function f. the aggregate proportional changes is zero or negative [26]. where ri ' = ri* + Δri* .s2 . Consider a small changes of r* to r’.Qi = Kmiα (2.12). if for any other feasible allocation s’.19) ⎛ Rc K ⎜ ⎜ N − 12 N ⎝ ∑ mαj d j + j =1 N Kmiα d i 12 ⎞ ⎟ ⎟ ⎠ Proof of fairness: An assignment of resource s = (s1 . ∑ i =1 N f (s ' ) − f (si ) ≤0 f (si ) (2.….21) . In the context of our game setting.20) Denote r* the allocation from NBS. the proportional fairness criteria can be rewritten as ∑ i =1 N [u i (ri ' ) − d i ] − [u i (ri* ) − d i ] ≤ 0 u i (ri* ) − d i Replace u with (2.

It is found that the human visual system is insensitive to the signals in some spatial frequencies.23) ∑ u (r i =1 i N u i′ (ri ) r = r * * i ) − di Δri* ≤ 0 (2. a metric of just-noticeabledistortion (JND) is proposed in [24] to measure the perceptual lower bound of the signal distortion.13). Due to the above observation.24) Plug (2. the human vision is much easier to detect the luminance difference rather than the absolute intensity.Let vi (ri ) = ln(u i (ri ) − d i ) .12) into (2. Therefore.3 Perceptually Tuned Quantizer The perceptual redundancy is inherent in video signals.25) 2.y) is given by [5]: 39 . any change in r* will make a zero or negative change in ∑ vi (ri ) . Moreover.22) ⎡ ∂v ∑ ⎢ ∂ri i =1 ⎢ ⎣ i N ⎤ Δri* ⎥ ≤ 0 ⎥ ri = ri* ⎦ i i (2. And the sensitivity to the luminance contrast is depended on the average background intensity. we have i =1 N ∑ [v (r ' ) − v (r )] ≤ 0 N * i =1 i i i i (2. we have ∑r i =1 N Δri* * i − ri0 ≤0⇒∑ i =1 N ri '−ri* ri* − ri0 ≤0 (2.3. The computation of JND of a pixel at (x. JND is a threshold below which the distortion is imperceptible.24). Since r* is the optimal solution for (2.

y)) + β (bg ( x. y))} (2. y ) / 127)1 / 2 )) + 3 bg ( x.y).3. y)) = λ − bg ( x.4} is one of the following four directions. The calculations of bg(x.JND( x. y). y ) × 0.30) f 1 (bg ( x. j ) 32 i =1 j =1 for 0 ≤ x < H . 4: diagonal (upper-right to lower-left).2. y )) = ⎨ 0 γ ⋅ (bg ( x.y) and mg(x. 40 .0001 + 0.. mg ( x.27) (2.y) is the luminance of pixel at (x. 3: horizontal. grad k represents the gradient in direction k.33). grad k is computed with (2.y).2. y)) = mg( x. γ and λ are 17. 2: diagonal (upper-left to lower-right). y) = max{ f1 (bg ( x. y)) ⎧T ⋅ (1 − (bg ( x.01 where T0. y ) ≤ 127 f 2 (bg ( x. y ) − 127) + 3 bg ( x. (2. f 2 (bg ( x. bg(x. where k={1. which is shown in Figure 2. 1: vertical. mg is the maximum gradient among different directions. y ) = 1 5 5 ∑ ∑ p( x − 3 + i. y) ⋅ α (bg ( x.28) (2. B(i.5).j).115 β (bg ( x. 3/128 and 1/2.y) and mg(x. respectively.0 ≤ y < W . y)..26) (2. y )) = bg ( x.y) represent the average background luminance and the maximum weighted average of luminance differences around the pixel at (x. (i.j=1.31) where p(x. y ) × 0..29) (2.y) are given below: bg ( x. y)). mg( x. is a matrix of weights. y ) > 127 ⎩ α (bg ( x. y − 3 + j ) ⋅ B(i.

2 Matrix B 0 1 0 -1 0 0 3 0 -3 0 0 8 0 -8 0 (a) 0 3 0 -3 0 0 1 0 -1 0 0 0 1 0 0 0 8 3 0 0 1 3 0 -3 -1 (b) 0 0 -3 -8 0 0 0 -1 0 0 0 0 -1 0 0 0 0 -3 -8 0 1 3 0 -3 -1 (c) 0 8 3 0 0 0 0 1 0 0 0 0 0 0 0 1 3 8 3 1 0 0 0 0 0 (d) -1 -3 -8 -3 -1 0 0 0 0 0 Figure 2. (b) G2 .1 1 1 1 1 1 2 2 2 1 1 2 0 2 1 1 2 2 2 1 1 1 1 1 1 Figure 2. (d) G4 41 . (c) G3 .3 Matrix Gk (a) G1 .

the total bits corresponding to the initial quality of the N uncoded MBs in a frame cannot exceed the remaining bits. y ) ≤ JND( x.mg ( x. y ) (2. y ) δ ( x. y ) = p ( x .. j ) 16 i =1 j =1 (2. y ) = ⎨ ⎧1. we bound the initial quality by a scale factor C. and ˆ h ( x . In the proposed algorithm. C is computed as follows: 42 .3. such that the total bits corresponding to the initial quality are bounded by a half of the remaining bits for the N uncoded MBs (0.e. n + v) − JND(m + u. h( x. a factor that affects the optimal quantization step size Qi is the initial quality di. ∑r i =1 N 0 i < Rc . y − 3 + j ) ⋅ G k (i.35) (2. The noticeable distortion in the ith MB is given by: ei = 1 16 16 ∑∑{ [h(m + u. y ) ⎩0. In (2. y ) − p ( x. v) is the top-left corner of the ith MB. which is given in Figure 2. n + v)]⋅ δ (m + u.19). the initial quality of a MB is proportional to its noticeable distortion e. n + v) 256 m=1 n=1 } (2. y ) > JND( x.34) where (u. y ) = 1 5 5 ∑ ∑ p( x − 3 + i. y) = max{ grad k ( x.36) Due to the bit rate constraint. Therefore.32) (2. y) } grad k ( x. h( x.33) where Gk is the matrices of weights in four directions.5Rc). i.

3). The noticeable distortion for each MB in the frame is calculated with (2.38) 2. as shown in Figure 2. The Buffer fullness feedback further tunes Tt’ with (2.34). The mean absolute prediction error of the current frame is computed with (2. H.263 or MPEG-4). Rc is initially set to Tt.6) to get Tt.⎧ 1 ⎪ ⎪ ⎪ 6 Rc C=⎨ N ⎪ α ⎪K ⋅∑mj ej ⎪ j =1 ⎩ K N α ∑mj ej 6 j =1 K N Rc < ∑ mα e j j 6 j =1 Rc ≥ (2.37) and (2. (2) Before encoding a frame.g. given by (2. (4) The quantization step size of current MB is computed based on the derived game theoretical formulation. in order to determine the initial estimation of the target number of bits Tt’.1.5). 43 .38). The functionality of each block in the diagram is introduced in the previous sections. is embedded in a DCT-based video encoder (e.2) and (2. We summarize the GT algorithm in the following steps: (1) Frame layer bit allocation will be performed prior to the encoding of a frame. name Game Theoretic (GT) algorithm. (3) Based on the noticeable distortion and remaining bits Rc.37) The initial quality for the ith MB is given below: di = C ⋅ ei (2. the initial qualities for the remaining MBs are computed with (2. Then Tt” is further adjusted with (4.19).4 Algorithm Summary The proposed algorithm.

(5) After encoding each MB. we compare the GT algorithm to the quadratic rate control algorithm suggested by MPEG-4 VM8 [19].5 Experiments and Results First. I. Rc is updated as Rc=Rc-Ai. Both comparing algorithms are implemented in Momusys encoder for MPEG-4 Verification Model. The peak-signal-to-perceptible-noise ratio (PSPNR) A good rate control algorithm should be able to control the actual bit rate as close as possible to the target bit rate. 2. (6) Update the buffer status after encoding each frame. the quadratic rate-distortion model is updated with (2. We measure the bit rate accuracy with the following equation: 44 . otherwise go to step 1. The performance of a rate control algorithm is evaluated by the following metrics. The initial buffer fullness is Br/16. The bit rate accuracy II. otherwise go to step 3. Percentage of bits saved III. To be consistent with the VM8 rate control algorithm. we set the buffer size to Br/8. which means the maximum delay is 125ms. The number of skipped frames IV. The peak-signal-to-noise ratio (PSNR) V. Go to step 6 if all the MBs in the frame is finished.8). Stop if all the frames are encoded.

Frame skip technique is employed to avoid the buffer overflow.y) and is defined in (2. PSNR is a widely adopted metric to measure visual quality. we also measure the percentage of bits saved of the GT algorithm compared to the VM8 algorithm.39) where Ractual and Rtarget are the actual bit rate and the target bit rate respectively. the encoding of the next frame will be skipped and will not be buffer in order to cut down the buffer fullness level. y )) x =0 y =0 H W 2 ⋅ δ ( x. Once the buffer fullness is over a threshold. For the same visual quality. the PSNR (and PSPNR) of the skipped frame is 45 . In the decoder side. PSNR averages the noise of all pixels in a frame. y ) (2. y ) − JND ( x. a frame skip will degrade the signal quality. regardless if it is perceptible to human visual system or not.y) and δ(x. A good rate control algorithm should be able to avoid the buffer overflow and minimize the number of skipped frames.36) respectively. lower bit consumption means the higher rate-distortion efficiency.40) where h(x.bit _ rate _ accuracy = 1 − Ractual − Rtarget Rtarget (2. PSPNR only take account of the perceptible noise and is defined as: PSPNR = 20 log 10 255 1 HW ∑ ∑ (h( x. PSPNR proposed by Chou and Li [5] measures the perceptible visual quality incorporating the human perceptual property.35) and (2. Besides the bit rate accuracy. According to the MPEG-4 core experiment. However. the skipped frame will be replaced by a duplication of the previous frame in order to maintain the continuity of the video decoding.

That means. 100 frames are encoded. To be consistent with VM8 in the comparison. comparing with VM8 that skips a certain number of frames 46 .1 Bit Rate Accuracy and Percentage of Bits Saved From Table 2. the GT algorithm saves 3.5.e.2 shows the results of CIF format. The temporal prediction structure used in the experiment is IPPP….57% and 3. It is to be noted that the GT algorithm is not limited to P frames.1 shows the results of QCIF format and Table 2. only the first frame is encoded as I-frame and the remaining frames are encoded as Pframe (IPP…). We encode QCIF and CIF format test sequences at various target bit rates. Compare to VM8.45% for CIF format and 96. One can note that the GT algorithm uses fewer bits in the situation that without frame skipped. i. The detailed simulation results and the comparisons are shown in Table 2.2. and easily extended to the frame level bit allocation for I frame and B frame. The reason of using this temporal prediction structure in the experiment is that the comparative algorithm (VM8) supports only this structure..1 and 2.61 for QCIF format while the GT algorithm achieves 99.93% for QCIF format.1. we adopt IPPP… structure in the experiment. 2. For each sequence. Table 2. The average bit rate accuracy of VM8 is 96. we observe that the GT rate control algorithm produces fewer bits and achieves more accurate bit rate than the VM8 algorithm in most test cases.95% for CIF format and 99. The frame rate in the experiments is 30 frames per second.39% bits for CIF format and QCIF format respectively.computed by considering the skipped frame as the duplication of the previous decoded frame in the decoded sequence [62] [63].

Figure 2. in the “Table” QCIF 64Kbps test and “Stefan” QCIF 96Kbps test.6 and Figure 2. VM8 skips 3 frames and 5 frames respectively.2 Frame Skipping On the number of skipped frames comparison. Different rate control algorithm may lead to different number of frame to be skipped.3 PSNR and PSPNR The overall visual quality is measure by the average PSNR and PSPNR. more bits are used to encode the non-skipped frame. The buffer fullness level is maintained within a safe margin to avoid frame skipping. Therefore. The advantage of the GT algorithm is especially prominent in the low bit rate tests. Figure 2. while the GT algorithm does not skip any frame. one can notice that various numbers of frames are skipped when using VM8 algorithm. the proposed algorithm averagely use even less bits to encode a frame. One can observe that the proposed algorithm has less fluctuation on buffer level and is able to control the buffer fullness around the middle of the buffer size. which thanks to the accurate estimation of frame level target bit budge and the high accuracy of MB level rate control.5.5.4. which will lead to a higher PSNR and PSPNR value on the non-skipped frame. The reason of improvement of frame skipping is because the GT can maintain a stable buffer level. there is no frame skips in the experiments when using the GT rate control algorithm. On the contrast.to achieve the target bit rate. 2. For example.8 illustrate the frame-to-frame comparison on buffer fullness. 2. If a rate control algorithm skips more frames than the other one. it is unfair to compare the average PSNR and PSPNR 47 .

It can be observed that the GT algorithm achieves higher PSNR than its counterpart in the PSNR comparison. In the MPEG-4 decoder.68dB in the “Container” qcif 384kbps test.23dB in the “Akiyo” qcif 384kbps test.9 show detailed results of buffer fullness level and PSPNR values of each frame achieved by the two algorithms.only taking into account the encoded frame. Hence. in the MPEG-4 rate control test the previous frame is used in the PSNR and PSPNR computation when a frame is skipped [62]. The GT algorithm outperforms the VM8 algorithm in terms of both the bit rate and the visual quality. the GT algorithm successfully masks the imperceptible distortion. Figure 2.4 to Figure 2. the PSPNR improves by 1. 48 . compared to the VM8 algorithm. although the GT algorithm is optimized for the PSPNR. This is due to the bit allocation based on the noticeable distortion. Table 2. Therefore. and by 2. The distortion of the skipped frame should be considered to for a fair comparison. a skipped frame will be repeated by the previous encoded frame.1 shows the PSNR and PSPNR comparison between VM8 and the proposed algorithm. the algorithm produces substantial improvement in perceptual quality. With regard to the perceptible distortion comparison. One can observe that the GT algorithm achieves higher PSPNR and maintains the buffer fullness level within a safe margin to avoid frame skipping. For example.

36 0.06 96.48 41.76 35.93 38.00% 2.03 28.36 34.63% 386.94 99.Table 2.24 96.1 Bit rate accuracy.96% 6.11 0.03% 111.53 32.15 0.22 49 .54 97.12 99.23% 96.98 99.95% 3.27 34.00% 3.98 26.79 1.14 0.7 91.88% 64.39 27.2 99.94 27.83% 128.09 99.96% -0.84 32.3 56.88% 127.29 47.15 38.03 1.17 0.88% 5.03 25.34 31.32 38.89% 1.00% 512.04 99.33 0.13% 69.95% 3.04 30.31 0.30% 191.84% 131.2 31.06 0.81% 199.97% 0. PSPNR and number of skipped frame comparison between VM8 and the GT algorithm (QCIF sequences) Bit rate video bit rate bits Average Average Frame PSNR PSPNR Algorithm gain gain sequence Target Actual accuracy saved PSNR_Y PSPNR_Y skipped (dB) (dB) VM8 GT Table VM8 GT VM8 GT VM8 GT Stefan VM8 GT VM8 GT VM8 GT Foreman VM8 GT VM8 GT VM8 GT container VM8 GT VM8 GT 64 64 96 96 128 128 96 96 112 112 128 128 64 64 128 128 192 192 192 192 384 384 512 512 66.25% 384.84 58.79 99.68 0.94% 8.09% 64.04% 28.23% 128 100.84% 3.34 25.18 33.77% 101.6 34.11 0.05 0.96 42.09 0.99 32.98 41.46 47.62 56.54 94.56 33.63% 131.38 32.46 29.93 0.37% 99.9 93.98% 6.26 30.07 0.48 35.78 3 0 3 0 2 0 5 0 6 0 5 0 8 0 2 0 1 0 3 0 0 0 0 0 1.51 24.08 0.87 26.45 24.7 37.64% 119.64 95.34 24.81 54.27 98.61 28.81 93.22 0.87 99.73 29.72% 512.15 41.01 100.14 0.99% 135.62 34.12 0.88 29.17% 195.12 99.95 41.23 40.22 0.32% 192.31 0.1 96 99.15 0.96% 100.86 25.05 99.99 96. PSNR.

25 55.31 42.41 99.23 0.39 46.22 96.00% 511.91 99.11 2.56 2.16 44.53 0.Table 2.87% 0.62 62.18 67.71 2 0 0 0 0 0 0.39% 511.88 99.85% 0.30% 384.32 99.76% 383.99 100.94% 3.97 58.72 45.42 44.07 50 .1 – Continued VM8 GT Akiyo VM8 GT VM8 GT 192 192 384 384 512 512 198.13% 42.86 3.76% 191.85 64.05 60.

93% 521.2 Bit rate accuracy.03 0.99% 2.30 0.03 100.40 27.21 0.02 0.64 38.98% 511.24 0.82% 400.44 34.41 99. PSNR.45% 522.09 98.39% 256.80 41.93 32.73 29.27 0.81 34.04 99.99 100.10 0.09% 383.95 99.98 39.88 37.57 33.14 0.12 0.12 99.14 98.09 VM8 GT VM8 GT VM8 GT VM8 GT VM8 GT VM8 GT VM8 GT VM8 GT foreman table 51 .05 0.89% 2.31% 191.69 28.10 94.82 99.64 30.00 96.67 32.03 0.07 32.40 0.86 30.02 0.20 0.27% 32.59% 395.31% 383.04 0 0 2 0 4 0 5 0 2 0 1 0 5 0 3 0 1 0 3 0 1 0 1 0 0.43 28.97% 398.03% 511.04 0.56 36.89 35.35 97.59 34.32 30.32 99.37 33.80 37.10 0.92% 4.88 32.15 36.93% 5.61% 384.13% 511.97 38.98% 5.29 36.17 96.33 37.35 33.Table 2.95% 2.36 94.19 97.94% 3.80 31.05 0.86% 3.89% 271.86 34.83% 522.32 97.10 34.69 34.05 0.88 99.79 29.55 0.00% 1.05 0.57 33.77 34.06 0.30% 270.32 99.42 38.10% 255.75 32.12 0.61% 383.26 41.82 99.16 0. PSPNR and number of skipped frame comparison between VM8 and the GT algorithm (CIF sequences) Bit rate bit rate bits Average Average Frame PSNR PSPNR video Algorithm skippe gain gain sequence Target Actual accuracy saved PSNR_Y PSPNR_Y d (dB) (dB) VM8 GT container VM8 GT VM8 GT VM8 GT Coastguard 192 192 256 256 384 384 256 256 384 384 512 512 256 256 384 384 512 512 384 384 512 512 640 640 193.84 95.77 99.97% 648.98% 1.90 27.78 32.73% 640.55 98.18 40.48 99.78% 263.00% 1.79 40.08 31.85 38.13% 397.13% 256.54 32.05 32.42 36.91% 0.

09 6 0 4 0 3 0 0.2 – Continued VM8 GT Mobile VM8 GT VM8 GT 512 512 640 640 768 768 548.51 92.06% 674.01 27.09 52 .10 24.79% 768.07 0.06 0.97% 5.19 99.20 94.95 25.66% 640.01 29.16 99.07 0.87% 512.Table 2.88 24.17 24.33 99.31% 800.19% 24.94% 7.98% 4.36 95.62 27.16 29.09 0.08 0.07 28.56 25.10 28.

4 Buffer occupancy in the coding of “Foreman” QCIF at 64kbps and 30Hz with buffer size = 8kbits 53 .8000 7000 6000 5000 Buffer 4000 level (bits) 3000 2000 1000 0 1 50 Frame 99 VM8 GT Figure 2.

40 35 30 25 PSPNR (dB) 20 15 10 5 0 1 50 Frame 99 VM8 GT Figure 2.5 PSPNR in the coding of “Foreman” QCIF QCIF at 64kbps and 30Hz with buffer size = 8kbits 54 .

6 Buffer occupancy in the coding of “Stefan” QCIF at 12kbps and 30Hz with buffer size = 14kbits 55 .14000 12000 10000 Buffer 8000 level (bits) 6000 4000 VM8 2000 0 1 50 Frame 99 GT Figure 2.

35 30 25 20 PSPNR (dB) 15 10 VM8 5 0 1 50 Frame 99 GT Figure 2.7 PSPNR in the coding of “Stefan” QCIF at 12kbps and 30Hz with buffer size = 14kbits 56 .

8 Buffer occupancy in the coding of “Container” QCIF at 192kbps and 30Hz with buffer size = 24kbits 57 .24000 20000 16000 Buffer level 12000 (bits) 8000 VM8 4000 0 99 1 50 Frame GT Figure 2.

9 PSPNR in the coding of “Container” QCIF at 192kbps and 30Hz with buffer size = 24kbits 58 .60 50 40 PSPNR (dB) 30 20 VM8 10 0 1 50 Frame 99 GT Figure 2.

The algorithm includes an efficient frame level bit allocation according to the frame coding complexity. The algorithm models the bit allocation problem on the MB level as a bargaining problem. This scheme has the best overall performance among the compared algorithms after considering the overhead introduced by the video analysis process. 59 . PSNR. The proposed rate control algorithm outperforms the VM8 rate control algorithm in terms of several aspects. while showing the adaptability to various types of scenes and abrupt scene change occasions. the proposed motion estimation scheme outscores the other predictive motion estimation algorithms in terms of computational cost and visual quality. including bit rate accuracy. This proposed rate control algorithm utilizes a game theoretical approach. Bit allocation and quantization scale of each MB are decided based on the Nash Bargaining Solution.6 Summary In this proposal. we propose adaptive and perceptually tuned motion estimation and rate control schemes for a new paradigm of video compression called content adaptive video compression. The proposed algorithm masks the imperceptible distortions by adjusting the initial quality of each MB based on noticeable distortion. By adapting to the video characteristics extracted by a online video analysis process.2. PSPNR and the buffer stability.

CHAPTER 3 JOINT MULTIPLE VIDEO OBJECT RATE CONTROL USING NON-COOPERATIVE GAME THEORY

3.1 Introduction As a standard for multimedia and Web compression, MPEG-4 supports objectbased interactivity, high compression, and universal accessibility [28]. In MPEG-4 standard, a scene is composed of one or more audio-video objects, which are separately tracked and compressed but multiplexed into a single MPEG-4 bit stream for transmission. This scheme leads to efficient and scalable compression, and makes MPEG-4 able to support interactive functionalities, such as the manipulation and control of individual objects. Because of the difference with the conventional framebased standards, such as MPEG-2 and H.263, the rate control algorithms for MPEG-4 need to be redesigned to accommodate to the MVO coding functionality, in order to provide the maximum efficiency. In [3] and [4], Chiang and Zhang have reported a scalable rate control algorithm for both DCT and wavelet-based coders. Vetro and Sun extended this framework with a scheme for JMVO rate control [73], [74]. Their scheme distributes the target bits to video objects proportional to the normalized factors of object size, motion and variance. Ronda and Eckert focused on rate control in real-time applications [63]. The goal was to minimize the average distortion of the objects, to

60

guarantee desired qualities to the most relevant ones, and to keep constant ratios among the object qualities. The core of JMVO rate control is the bit rate allocation among video objects subject to constraint data rate. The allocation scheme strongly impacts the coding efficiency, the quality of individual audio-visual objects and the overall bit rate accuracy. In this chapter, we propose a JMVO rate control algorithm that distributes bits to VOPs based on the mixed strategy Nash equilibrium of a non-cooperative bi-matrix game. For the reason of simplicity, most of the current rate control algorithm adopts a hierarchical bit allocation scheme, where the target bits are first distributed in coarser level then in finer level [3] [34] [35] [52] [73] [74]. In our algorithm, hierarchical structure is employed. The total available bits for a video sequence, which are computed based on the bit rate requirement, are firstly distributed for each time instance; then the bits for a time instance are further allocated to VOPs. The quantization parameter for each VOP is calculated from a rate-distortion model. In each level, we utilize efficient bit allocation schemes to regulate the bits in order to meet the bit rate requirement while optimizing the visual quality. In the time instance level, the number of bits allocated is depended on the number of remaining bits for the remaining VOPs as well as the number of bits used for the previous time instance. In the object level, a noncooperative matrix game is employed, where each VOP is regarded as a rational player. The bi-matrix strategic game simulates the behaviors of non-cooperative players competing for available bits to optimize their own performance. We look for the

61

“solution” of the game, the mixed strategy Nash Equilibrium, which is the probability distribution of the actions carried by players that optimizes their expected utility, the number of bits. The game is played iteratively. The expected utility of each play will be accumulated. The game will terminate when all available bits for the specified time instance are distributed to VOPs. 3.2 Target Bit Estimation for a Time Instance The bit allocation scheme in the proposed rate control algorithm is hierarchical. In this section, we describe an approach that estimate the target number of bits for all VOPs in a time instance. This simple yet efficient approach is first appeared in [33], [34]. First, the estimation of target number of bits for each VOP is initially estimated from the remaining bits by:

Ti =

R N × nr

(3.1)

where N is the number of video objects and nr is the number of remaining VOPs. Taking account of the actual bits consumption for the previous VOP, the initial estimation is further adjusted by Eq. (3.2) in order to maintain a smooth change in target bit budget:

Ti′ = 0.8 × Ti + 0.2 S i

where Si is the actual bits for the previous VOP of the ith video object.

(3.2)

Finally, the number of bits for a specific time instance is the sum of estimated bits for all VOPs:

62

3) 3.01 and 0. which is a half of the buffer size. It combines to the proportional. Ki and Kd are the proportional. The proportional feedback has the intension to reduce the error between the actual buffer level and the target buffer level. The feedback of the buffer status is used to adjust the target bit estimation.4) where Bs is the buffer size.1.4 respectively. The integral feedback intends to 63 . A block diagram of the PID buffer control system is shown in figure 3. Bt is the buffer fullness at time t. In order to maintain an accurate regulation and avoid buffer overflow and underflow. The buffer control system can be regarded as a tracking system where the actual buffer fullness level traces the target buffer level. the buffer control tries to keep buffer fullness in the middle level. we adopt the PID buffer feedback control in our related works [75].3 Buffer Policy A buffer is placed between encoder and the transmission channel to accommodate bit rate variation in the CBR channel mode. Proportional-Integral-Derivative (PID) control is widely used in the feedback control systems [7] [54]. In this work.T ′ = ∑ Ti′ i =1 N (3. integral and derivative feedback compensation components to improve the steady state response and transient response. The PID buffer feedback adjustment can be formulated as following: errt = 1 − 2 B t / B s T = T ′ + T ′ × K p (errt + K i ⋅ ∑ errt + K d ⋅ (errt − errt −1 )) i =0 t (3. 0. Close-loop control is widely used in tracking systems. Kp. They are set to 1. integral and derivative control parameters.

e. the number of target bits is bounded by [0. Br/Fr is 64 . and choosing proper control parameters. Generally. which makes the system slowly responding to the current error. To compensate it. integral and derivative compensator. Combining the proportional. Bit allocation Initial bit estimation VO1 Encoder VO2 Encoder … + Target Buffer level err - Ki Kp Feedback compensator Kd ∫ d dt Figure 3. The negative feedback will increase along the increasing of the accumulated error. the system will overshoot to the accumulated error.1 PID buffer control diagram After the PID buffer feedback adjustment.25Br/Fr. i. such that the feedback keeps increasing while the error is decreasing. a derivative compensator is used to add more stability to the system by reducing the overshoot. the buffer level can be accurately control to the desired level.eliminate the steady-state error. the integral feedback improves the steady-state response but degrades the system’s transient response. 2Br/Fr]. where Br is the target bit rate and Fr is the frame rate.

From another point of view to look at the bit allocation problem. According to the rate-quantization relation. such as video object size. we proposed a non-cooperative game approach for bit allocation among VOPs. Most current existing rate control algorithms for MVO coding distribute bits proportional to the linear combination of several factors. They need to consider the other VOPs’ choices. The objective of this adjustment is to avoid the buffer overflow and guarantee a minimum quality. Assuming each VOP is a rational player. residual data complexity. which is related to the encoded bits. Assume the VOPs can choose their quantization parameters independently. 65 .4 VOP Level Bit Allocation Given the bit target for a specified time instance. using a smaller quantization parameter can achieve higher visual quality but produce more bits. On the other hand. In the MVO coding.the average number of bits for a time instance. each VOP is encoded with a quantization parameter. The target is to obtain a balance visual quality over all objects. the behaviors of the VOPs can be modeled by a bi-matrix strategic game. then it will lose most of the bits. if a VOP chooses a large quantization parameter while the other VOPs choose small ones. bits are further distributed to VOPs at this time instance. motion vector length. 3. etc. it may happen that the other VOPs also choose small quantization parameter such that the total number of bits consume will exceed the target bit budget. This approach is based on the assumption that these factors are linear related to the bit consumption of encoding an object. If a VOP chooses a small quantization parameter in order to obtain a high quality.

b3 a2. L}. H H L L a1.b2 a4.b1 a3. A player is an individual or a group of individual making decision. The utility of a player is a function of the combination of actions that carry out by all players. One player’s actions are identified with the row and the other player’s actions with the column.b4 Figure 3. A strategic game is a game in which each player chooses his plan of action once and for all.2 shows an example of two player strategic game. Figure 3. Each player is “rational” in the sense that he is aware of his available strategies and the preference on these strategies. A strategic game can be described conveniently by a matrix.In all game theory models the basic entity is a player. When choosing an action.2 A sample of bi-matrix game An interpretation of game in Figure 3. thus he chooses to carry out an action that will maximize his utility. In each entry of the matrix are the utilities of player A and player B given they choose the corresponding actions. and all players’ decisions are made simultaneously. each player is not informed the actions chosen by other players.2 is that: Both player (player A in column and player B in row) have a strategy space {H. When player A chooses H and 66 .

For the ease of description. More objects can be applied in the same framework.6) where u1 and u2 are utilities of player 1 and player 2 respectively.player B choose H. When player A chooses H and player B chose L. The utility is defined as follow: ⎧ R (q ) if R1 (q1 ) + R2 (q 2 ) ≤ s u1 (q1 . q1 and q2 represent the strategies chosen by player 1 and player 2. q 2 ) = ⎨ 2 2 if R1 (q1 ) + R2 (q 2 ) > s ⎩ 0 (3. s = T − ∑ Thdr .. Different rate-quantization functions have been proposed in the literatures to model the rate quantization relation [6] [27] [69] [76]. In our work.. A strategic game can be utilized to model the behaviors of VOPs.5) (3.7) where Thdr. In the MPEG-4 environment. motion and header of the previous VOP of the ith video object.31}.. We define a strategic game as following. Each entry in the matrix contains the utilities of players given corresponding combination of actions taken by the two players.i i (3. s is the number of remaining bits for texture coding. the strategy space is {1.i is the total number of bits used for coding the shape. R1 and R2 are the rate quantization functions of player 1 and player 2. In this game. but the computational complexity will increase accordingly. we employ a 67 . the objects are regard as players. The strategy space of the player is defined on the set of quantization parameters. player A’s utility is a1 and player B’s is b1. player A’s utility is a2 while player B’s is b2. q 2 ) = ⎨ 1 1 if R1 (q1 ) + R2 (q 2 ) > s ⎩ 0 ⎧ R (q ) if R1 (q1 ) + R2 (q 2 ) ≤ s u 2 (q1 . we limit the number of objects to two.

due to its wide usage and accuracy.2) … … … … … … 31 u1 (1.31) u2 (2.1) u1(2.quadratic model which first appeared in [4].i are the model parameters for video object i.31) … u1 (31.31) u1 (2.31) … u2 (31.31) u2(31.i × MADi qi 2 (3.2) (a) Player 2 1 1 Player 1 2 … 31 u2(1.1) u1 (31.1) u2(2.2) … … … … … … 31 u2 (1.3 The utility matrices (a) matrix of player 1 (b) matrix of player 2 68 .1) u2 (31.1) … 2 u2 (1.8) Ri = X 1. The rate quantization function is shown in (3.2) u2 (2.i × MADi qi + X 2.1) … 2 u1 (1. Player 2 1 1 Player 1 2 … 31 u1(1.2) (b) Figure 3.2) u1 (2.i and X2.31) u1(31.8) where X1.

3. the expect utility of each player is depended on both players’ mixed strategies. the players have some probability distributions over the set of strategies.. Assume A is the matrix of player 1’s utilities and B is player 2’s.6) can be interpreted as follows: the utility of a player is the number of bits required to encode it with a chosen quantization level. u2>.9) . Each entry of the matrix contains a tuple <u1. xi≥0 and ∑x i i = 1 ) is the mixed strategy of player 1 in the game of Figure 3. for any other mixed strategies. We can also use two matrices: one represents the utilities of player 1 and the other represents player 2. Assume the players’ choices are nondeterministic.3. Therefore. The expected utilities of player 1 and player 2 are xAy T and x T By . B) is said to be in equilibrium if. x1 is the probability of player 1 choosing 1 and x2 is the probability of player 1 choosing 2. x and y: x * Ay *T ≥ xAy *T and x * By *T ≥ x * By T 69 (3.The utilities defined (3. i.e. where x is the mixed strategy of player 1 and y is the mixed strategy of player 2. if the total number of bits requested by both players is exceeding the total available bits for both players.5) and (3. The strategic game for VOP level bit allocation can be described as in figure 3. Assuming all players are rational. y*) for a bi-matrix game (A. Obviously. the game is expected to converge to equilibrium. if x (x= (x1 x 2 L x31 ) . For example. both players get nothing. it is also called a bi-matrix game. A pair of mixed strategies (x*. The probability distribution is called a mixed strategy of a player.

The expected utilities of the equilibrium are allocated to player 1 and player 2 respectively.e. we derive a simple process that simplifies the computation of expected utility. the remaining number of bits is updated by taking out the number of bits allocated and the bi-matrix is updated accordingly. In other words. i.e. After each play.j) for all j. Assume we have constructed matrix A and B based on the rate-quantization function and the available bits. y*) is called mixed strategy Nash equilibrium. The game will terminate when all the remaining bits are allocated. in matrix B. since it is the optimal strategy for a player in the “Nash” sense. Similarly. i is the dominated strategy and k is the dominant strategy. (3.j)≤ u1(k. LCP can be solved by a complementary pivot algorithm due to Lemke [37]. the players will choose the mixed strategy Nash equilibrium. column j is the dominated by column k.. such that u2(i.(x*. x * Ay *T and x * By *T are the expected utilities. According to the characteristics of this game. row i is the dominated by row k. such that u1(i. we know that if a strategy of a player is dominated by other strategies of the same player. In each play of the game. The game is played iteratively.9) can be converted to the form of linear complementary problem (LCP). there is no 70 . then action i of player 1 is said to be dominated by action k. a mixed strategy is said to be Nash equilibrium if no player has any positive reason for changing his mixed strategy. i.. if there exists if there exists some k.j)≤ u2(i. In matrix A.k) for all i. From the definition of Nash equilibrium. then strategy j of player 2 is said to be dominated by strategy k. if there exists some k. assuming none of the other players will change their mixed strategy.

4. matrix A and B will be simplified to a form as shown in Figure 3. after eliminating the dominated row and column.j)=u1(i. With matrices in the above form. u1(i. mixed strategy (x. A pair of mixed strategy (x. Thus.j) for any i and n. the following relations are held: 1. Proof: xAyT = x’AyT =k for any x’ and xByT = xBy’T =r for any y’. If the total bits in an entry do not exceed the available bits.j)≥u2(n. (x. and u2(i.j)≥u1(m. according to the definition of mixed strategy Nash equilibrium in (3.positive reason the player will play this strategy since there is always a better choice. we can eliminate the dominated row and column in the matrices without interfere the solution of game. since a object’s bits is only related to its own quantization level.j) for i≤m.y) with such property can be find by solving equations: a1 y n = a 2 ( y n −1 + y n ) a 2 ( y n −1 + y n ) = a3 ( y n − 2 + y n −1 + y n ) L a n −1 ( y 2 + y 3 + L + y n ) = a n ( y1 + y 2 + L + y n ) 71 . u1(i.j) for j≤n. the probability of the player carrying out this strategy is zero. 2.y) is a Nash equilibrium.y) is a Nash equilibrium. if AyT=kenT and xB=ren. Therefore. Therefore.m) for any j and m.j)=u2(n.9). and u2(i. since the rate-quantization function is monotonously decreasing.

4 The simplified matrices by eliminating the dominated rows and columns. where ai>aj>0 and bi> bj>0. for i<j 72 .b1 x n = b2 ( x n −1 + x n ) b2 ( x n −1 + x n ) = b3 ( x n − 2 + x n −1 + x n ) L bn −1 ( x 2 + x3 + L + x n ) = bn ( x1 + x 2 + L + x n ) Player 2 q1 p1 Player 1 p2 … pn-1 pn 0 0 … 0 an q2 0 0 … an-1 an … … … … … … qn-1 0 a2 … an-1 an qn a1 a2 … an-1 an Player 2 q1 p1 Player 1 p2 … pn-1 pn 0 0 … 0 b1 q2 0 0 … b2 b2 … … … … … … qn-1 0 bn-1 … bn-1 bn-1 qn bn bn … bn bn Figure 3.

y n = n and y j = ⋅ for j ≠ 1 and j ≠ n a n −1 a1 an− j a n− j (3.11) The expected utility of (x. with the estimated number of bits for VOPi the quantization parameter QP for the VOPi can be calculated using (3. we have: x1 = bn −1 − bn b b − bn −i +1 bn .5 Quantization Level Calculation and Frame-skipping Control Given the model parameter X1. The following procedure distributes the bits to the VOPs: 1.y) is (an. x n = n and xi = n −i ⋅ for i ≠ 1 and i ≠ n bn −1 b1 bn −i bn −i (3. 3.Solving these equations. Conforming to the standard the QP is limited to vary between 1 and 31. Subtract the distributed bits from the remaining bits.i and X2. the change of QP is restricted to 25% of the VOP in the previous time instance. Eliminate the dominated rows and columns.8). bn). 4.i and MADi. To avoid buffer overflow. the encoder will skip encoding and transmitting one or more frames when the buffer occupancy is too high. The same method as VM8 [74] 73 . go to step 1. Construct matrix A and B. To maintain a smooth change of visual quality. based on the objects’ rate-quantization functions and the remaining bits for texture coding. If there are bits available. 2.10) y1 = a n − j − a n − j +1 a n a n −1 − a n a . 3. Distribute the bits according to the expected utility of the mixed strategy Nash equilibrium to the VOPs.

1. with the bit allocation among video objects based on the noncooperative strategic game. Five common test sequences are used in the experiments. “Children” and “container”. if the buffer occupancy is higher than 80% of the buffer size. The bit rate accuracy maintains above 98%. In the post-encoding stage.1. the bit rate accuracy is evaluated. Frame-skipping controls are performed in both pre-encoding stage and post-encoding stage. As reported in Table 2. Peak-signal-to-noise-ratio (PSNR) is widely used to evaluate the reconstructed signal quality. we encode the first 150 frames and the frame rate is 30Hz.is employed for frame-skipping control. In pre-encoding stage. We measure the PSNR of each VOP and average the PSNR values of all the VOP in a video object to obtain the reconstructed PSNR of a video object. They are the first two objects of “news”. frames will be skipped. As shown in Table 2. Different target bit rate are tested. Rate control has strong impacts on the visual quality of the reconstructed video signal. Compared with VM8 JMVO rate control algorithm. 3. from 128Kbps to 384Kbps with a step of 64Kbps. our algorithm achieves a similar accuracy level in bit rate control. our rate control algorithm can properly distribute the bits to 74 . “coastguard”. For all sequences. First. the results verified that our algorithms can accurately control the bit rate to the desired value. if the VOP header coding consumes too many bits such that there are no bits for the texture coding. “bream”.0). frames will be skipped to avoid buffer overflow.6 Experiments and Results We implement the proposed algorithm on the Momusys MPEG-4 software verification model (VM 8.

in the coding of “news” sequence at 30Hz and 128kbps. in the coding of “news” sequence at 30Hz and 256kbps. Comparing with VM8 JMVO rate control algorithm. The detail frame by frame buffer occupancy curve shows the fluctuation of buffer fullness. the algorithm is much easier to overflow or underflow the buffer. The distribution is closely related to the utility function of each video object. video object 1 is more aggressive to obtain bits than the video object 2. For example. which is embodied by their utility functions. the foreground object (video object 1) obtains a larger portion of bit budget through the competition with the background object (video object 2). From the buffer occupancy figure. the buffer occupancy with VM8 JMVO rate control algorithm fluctuates around 30%-70%. For example. Therefore. the buffer occupancy varies form 10%60%. Thus. while the buffer occupancy with our rate control algorithm is around 40%-65%. The buffer occupancy is examined and plotted in Figure 3. and our algorithm produces a smoother change of the buffer fullness level and the variation is about 15%-20% around the middle of the buffer size. the buffer occupancy of proposed algorithm is smoother and has fewer fluctuations. video object 1 wins more bits than video object 2.video object. Thanks to the PID 75 . The buffer occupancy is maintained around 40%-60% of the buffer size. Another example is the coding of “container” sequence at 30Hz and 256kbps. it can be observed that our rate control algorithm efficiently maintains the buffer fullness around the middle level of the buffer. If the fluctuation level is large. video object 1 consumes more bits than video object 2.5-3. By taking the same action. The coding complexity of video object 1 is higher than video object 2. With VM8 rate control algorithm.9.

Table 2. the frame to frame PSNR fluctuation is also a measure of video quality. our rate control algorithm produces a quick response to the buffer level change. In this comparison. in the coding of “coastguard” at 30Hz and 256kbps as shown in figure 3.1 also shows the number of skipped frames. For instance. in the coding of “children” sequence. 76 . our algorithm skipped few frames compared with VM8 JMVO rate control. Figure 3.control. fewer frames are skipped with our algorithm compared with VM8 JMVO rate control.10-3. Large PSNR fluctuation between consecutive encoding frames will degrade the visual quality. our algorithm much quickly responds to the buffer error than the VM8 MVO rate control algorithm. For example. Besides the average PSNR of each video object. the proposed algorithm and VM8 algorithm have similar level of fluctuation.7.19 shows the frame to frame PSNR curves. Due to the smooth buffer control and proper bit allocation.

14 37.22 33.9 40.5% 99.1 381.93 44.9% 99.1% 99.1% 99.8 28.22 31.38 32.4% 99.1% 99.2 382.98 43.8% 99.12 37.8 386.4 253.1 The performance comparison between game theory algorithm and the VM8 reference algorithm Target Sequence Alg.89 32.8% 99.Table 3.18 42.7% 99.5 56.3% 99.10 29.5 127.3 Total 382.59 34.93 45.3% 99.6% 99.2 139.2 117.7 145.6 137.92 33.0 316.5% 99.1% 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PSNR VO1 41.2 263.2 117.2 255.4 51.4 54.7 319.33 43.9% 99.9 63.1 52.0 98.90 38.6% 99.8 191.8 28.88 39.91 77 .22 VO2 41.6 72.1 126.1 97.7 172. VM8 GT VM8 GT news VM8 GT VM8 GT VM8 GT VM8 GT VM8 GT bream VM8 GT VM8 GT VM8 GT Bits Rate 384 384 320 320 256 256 192 192 128 128 384 384 320 320 256 256 192 192 128 128 VO1 205.78 34.05 42.34 29.51 44.8 190.0 61.7 92.26 42.0 319.1 242.70 33.69 33.07 40.56 36.7 127.6 Actual Bit Rate VO2 177.0% 99.8 75.71 44.4% 99.91 40.1 114.0 258.10 39.6 255.3 191.02 26.33 35.74 31.8 99.44 40.9 39.1 200.8 126.30 43.9 165.04 42.16 37.2 89.0 202.0 284.8% 99.4 151.1 98.20 38.8% 99.7 297.4 317.8 255.2 203.43 27.8 63.9 150.8 190.42 39.8 skipped Accuracy frames 99.4 88.

6% 98.6% 98.8 252.2 118.1 51.1% 99.9 79.23 36.8 107.88 28.4 171.14 23.47 25.68 38.71 28.9 106.7 138.1 147.6 188.27 27.46 29.0 102.9% 98.79 31.51 30.8 149.4 87.1 75.0 153.6 50.8 316.65 23.0 68.6 190.4 78.62 33.0% 98.8 110.5 316.1 380.16 34.76 31.7 183.7 190.0% 99.9 317.8 83.40 40.0 126.0 190.8 125.1% 98.1 .22 38.97 28.0% 99.0 315.7 139.94 30.4% 98.79 30.9 381.5 189.36 31.40 27.74 25.50 25.2% 98.08 27.Table 3.42 37.3 58.33 36.2 177.7 105.0 379.69 78 .8% 98.85 30.2 102.32 21.91 29.3 145.90 29.21 36.89 34.0 254.9 215.2 239.25 34.70 23.6 207.8 135.4 72.1 252.2 165.1% 99.79 34.Continued VM8 GT VM8 GT coastguard VM8 GT VM8 GT VM8 GT VM8 GT VM8 GT children VM8 GT VM8 GT VM8 GT 384 384 320 320 256 256 192 192 128 128 384 384 320 320 256 256 192 192 128 128 192.7% 98.6 99.87 39.8 163.95 39.0 253.6 126.3% 99.3 126.3 131.7% 98.8% 99.2% 99.86 37.05 39.2% 99.9% 0 0 0 0 0 0 0 0 0 0 10 5 6 4 7 5 9 6 6 3 39.6 112.5 54.6 190.7 75.8 127.9% 99.0 378.

0.8

0.7

0.6 Buffer Occupancy

0.5

0.4

0.3

0.2 VM8 GT 0 0 50 Fram es 100

0.1

Figure 3.5 Buffer occupancy in the coding of “news” at 30Hz and 256kbps

79

0.8

0.7

0.6 Buffer Occupancy

0.5

0.4

0.3

0.2

0.1

VM8 GT

0 0 50 Fram es 100

Figure 3.6 Buffer occupancy in the coding of “bream” at 30Hz and 256kbps

80

0.8

0.7

0.6 Buffer Occupancy

0.5

0.4

0.3

0.2

0.1

VM8 GT

0 0 50 Fram es 100

Figure 3.7 Buffer occupancy in the coding of “coastguard” at 30Hz and 256kbps

81

8 0.7 0.8 Buffer occupancy in the coding of “children” at 30Hz and 256kbps 82 .1 0 0 50 Fram es 100 GT Buffer Occupancy Figure 3.5 0.0.6 0.4 0.2 VM8 0.9 0.3 0.

1 VM8 GT 0 0 50 Fram es 100 Figure 3.4 0.3 0.2 0.7 0.9 Buffer occupancy in the coding of “container” at 30Hz and 256kbps 83 .5 Buffer Occupancy 0.6 0.0.

10 PSNR of “news” using VM8 at 256kbps 84 .50 45 40 35 Buffer Occupancy 30 25 20 15 10 5 0 0 50 Frames 100 Object 1 Object 2 Figure 3.

11 PSNR of “news” using the proposed rate control at 256kbps 85 .50 45 40 35 Buffer Occupancy 30 25 20 15 10 Object 1 5 0 0 50 Frames 100 Object 2 Figure 3.

50 45 40 35 Buffer Occupancy 30 25 20 15 10 Object 1 5 0 0 50 Frames 100 Object 2 Figure 3.12 PSNR of “bream” using VM8 rate control at 256kbps 86 .

50 45 40 35 Buffer Occupancy 30 25 20 15 10 Object 1 5 0 0 50 Frames 100 Object 2 Figure 3.13 PSNR of “bream” using the proposed rate control at 256kbps 87 .

60 50 40 Buffer Occupancy 30 20 10 Object 1 Object 2 0 0 50 Frames 100 Figure 3.14 PSNR of “coastguard” using VM8 rate control at 256kbps 88 .

15 PSNR of “coastguard” using the proposed rate control at 256kbps 89 .60 50 40 Buffer Occupancy 30 20 10 Object 1 Object 2 0 0 50 Frames 100 Figure 3.

40 35 30 Buffer Occupancy 25 20 15 10 Object 1 5 Object 2 0 0 50 Frames 100 Figure 3.16 PSNR of “children” using VM8 rate control at 256kbps 90 .

40 35 30 Buffer Occupancy 25 20 15 10 Object 1 5 Object 2 0 0 50 Frames 100 Figure 3.17 PSNR of “children” using the proposed rate control at 256kbps 91 .

60 50 40 Buffer Occupancy 30 20 10 Object 1 Object 2 0 0 50 Frames 100 Figure 3.18 PSNR of “container” using VM8 rate control at 256kbps 92 .

60 50 40 Buffer Occupancy 30 20 10 Object 1 Object 2 0 0 50 Frames 100 Figure 3.19 PSNR of “container” using the proposed rate control at 256kbps 93 .

a simple approach computing the mixed strategy Nash equilibrium is derived. 94 . The mixed strategy Nash equilibrium solution is used to compute the bit allocation. The experiment results show that the proposed game theory algorithm achieves accurate bit rate control and smooth buffer occupancy to avoid buffer overflow and underflow. The proposed algorithm can automatically and properly allocate bits among video objects based on the non-cooperative game.3. Based on the characteristics of the game. The utility functions are defined based on the rate-quantization relations of video objects. The game is non-deterministic.7 Summary In this chapter. A bi-matrix strategic game is employed to model the behavior of VOPs. a JMVO rate control algorithm based on non-cooperative game theory is proposed.

Due to the non-uniform distortion visibility of human eyes. To our knowledge. Two algorithms have been proposed in our study. Towards the goal of optimizing the current existing video compression algorithms. It has strong impact on the visual quality and the bandwidth utility. we formulated the MB level rate control problem as a multiple objective optimization problem. In the first algorithm. we have presented and discussed several motion estimation and rate control algorithms. 4. We converted the problem to a bargaining game and derived an optimal and fair bit allocation strategy from the Nash Bargaining Solution. based on cooperative and non-cooperative game theory respectively. the perceptual 95 . As a conclusion.1 Summary of Contributions Realizing the power of game theory in solving multiple objective optimization problems with multiple decision makers. we designed cooperative and non-cooperative games for rate control in video coding. our work is the first attempt to game theory to solve the video compression problems.CHAPTER 4 SUMMARY AND FUTURE WORKS Rate control plays an important role in the video compression and communication. we will summarize the major contributions followed by the discussion of future research.

which is the expected number of bit obtained. cooperative game allows communications and binding commitments among players. The proposed algorithm successively achieves accurate bit rate control. we have presented an object level rate control algorithm that employs non-cooperative game theory to model the behaviors of the VOPs and solves the bit allocation problem using a bi-matrix strategic game. such as MPEG-4. support the coding and transmission of independent video objects. Object-based video coding. smooth buffer fullness and promising visual quality. Our JMVO rate control algorithm achieves accurate bit rate control and smooth buffer occupancy to avoid buffer overflow and underflow. The game repeats until all the bits are distributed. 4. a steady status that both players will converge to.2. Players play the game by non-cooperative choosing a proper quantization strategy to optimize its expected utility. In a non-cooperative game. The players play 96 . JMVO rate control is a efficient rate control scheme for MVO coding. It leads to a bit allocation corresponding to the mixed strategy Nash equilibrium. The attempt of using non-cooperative matrix game for object level bit allocation is made. there is no communication and binding commitment allowed. Individual players play selfishly to maximizing their own utilities. We extended our work to cope with the MVO coding.redundancy can be further explored by incorporating the perceptual sensitivity of human vision system into the cooperative game theoretical rate control framework. Each VOP is regarded as a player.1 Joint Multiple Video Object Rate Control using Cooperative Game In chapter 3.2 Future Works 4. On the contrary.

The axiom of symmetry in the Nash bargaining solution can be relaxed in a generalized bargaining problem such that the players can have different powers in the bargaining. we plan to explore cooperative game theory models for the object level bit allocation in MVO coding. Refer to the cooperative game theory framework for MB level rate control in chapter 3. video objects may have different importance to the observer.cooperatively. The bit allocation problem among multiple objects can be mapped into a generalized bargaining problem with asymmetric players. cooperation in the video object coding may benefit the visual quality. The human vision system perceives a scene composition instead the individual objects. By taking advantage of this property. Nash bargaining solution can be used to solve the object level bit allocation problem in MVO coding.2 Improved Utility Function in Game Theoretical Models A virtue of game theoretical models and algorithms for rate control is the joint optimization of multiple utility functions by multiple decision makers. In the future research. The utilities are optimized for a group of players. By translating the importance into the bargaining power. we can apply cooperative game in object level rate control. which optimizes the perceptual quality under the constraint of fair bit allocation among MBs in a frame. video objects are encoded separately but composite together into a scene. Therefore. But the current 97 . in chapter 2 we have proposed the utility functions that are associated with MB’s perceptual redundancies of human vision system. Due to the disparity of the object characteristics and the user’s recognition. we proposed a cooperative game for MB level rate control. 4. In chapter 2. In MVO coding. such as MPEG-4.2.

The new criteria should account for the human visual sensitivity and perceptual property for video contents. the video contents have both spatial and temporal dimension. motions and changes in luminance. As one of our potential researches. Redundancies are added to the data streams before transmission such that channel error can be detected and corrected to some extent. While images have only the spatial dimension. etc.) as well as temporal dimension (for instance. 4. we will explore proper criteria for subjective quality of video contents. etc. In the game theoretical models. luminance and position in the scene composition. Channel coding is commonly used in error prone environment. size and position. different utility functions in can be defined for different players. Most of the existing rate control algorithms are designed based on this channel model. The perceptual property in temporal dimension has not been exploited in the utility functions in the game theory framework. The more bits are 98 .2. size. which the current rate-distortion is not apt for.criterion for quantitative perceptual redundancy only takes into account the spatial redundancy which is originally developed for the image instead of video. The video data transmission is strongly interfered by the condition of the wireless channel. this assumption is no longer true when the data is transmitting through the error prone wireless channels. Efficient mechanism is required to adapt the rate control algorithm to the channel condition. based on their individual subjective quality.). in spatial dimension (for instance. texture complexity. However.3 Game Theory Applications on Joint Source-Channel Rate Control The video communication over wired network presumes that there is zero channel error during the transmission.

the lower error rate occurs in the receiver end. who bids for bandwidth in an auction. The objective is to optimize the overall quality at the receiver end under the constraint of channel condition. To optimize the over all quality. 99 .devoted to the channel coding. proper bandwidth allocation between source coding and channel coding is necessary. But due to the constrained data rate. thus better reconstruction visual quality can be achieved. by jointly controlling the bit rate consumption of source encoders and channel encoders. Each video encoder or channel encoder can be regarded as a player. this implies a lower bit rate for the video encoder. which leads to the higher compression ratio and lower visual quality. Potential research can be done on using the game theory models for the bandwidth allocation among source coding and channel coding.

Chiang.” Applications of Digital Image Processing XIX. and Y. July 1997. Englewood Cliffs: Prentice-Hall. pp. [2] J. Q. “Macroblock Layer Rate Control. 1997. 1996. F. vol. J. and H. 1. “A Perceptually Tuned Subband Image Codec Based on the Measure of Just-Noticeable-Distortion Profile. vol. Proceedings of SPIE. Lee. Chou. Hang. 1988. and Y. pp. no. Design of Control System. 100 . “Rate Control of MPEG Video Coding and Recording by Rate-Quantization Modeling. 5. 467-476. 12-20. C.REFERENCES [1] E. pp. vol.” IEEE Transactions on Circuits and Systems for Video Technology. vol. and Y. 2847. [6] W. 2. pp. 6. [3] T. Q.” in Proceeding of 1995 International Conference on Image Processing. 1995. [4] T. Abdou. Cheng.” ISO/IEC JTC/SC29/WG11 MPEG97/M2555. 7. [5] C. 6. H. 1995.” IEEE Transactions on Circuits and Systems for Video Technology. [7] A.” IEEE Transactions on Circuits and Systems for Video Technology. “Adaptive Piecewise Linear Bits Estimation Model for MPEG Based Video Coding. “Image Restoration of Multiple Frame Sequences. Dec. 551-554. H. Feb. Liu. no. Zhang. “A New Rate Control Scheme using Quadratic Rate Distortion Model. D’Souza. Feb. Zhang. 246-250. pp. Ding and B. Li. 192-202. Chiang. 1996.

” IEEE Transactions on Circuits and Systems for Video Technology. 10. Mitra. Vector Quantization and Signal Compression. Y. Ma. 2001. 2002. K. Oct. M. 928-940. [10] Z. 970-982. 840-849. Mitra. He. Singapore. vol. vol. pp. Evolutionary Games and Population Dynamics. Hofbauer and K. and S. 8. [11] Z. 1992.” Second International Conference on Information. and S. 12. Nov.K. 12. no. U.” IEEE Transactions on Circuits and Systems for Video Technology. Protocols.I. no. Sigmund. K. “Low-Delay Rate Control for DCT Video Coding via ρ-domain Source Modeling.[8] Gersho and R. [9] R. 1999 101 .K. Schroder.” IEEE Transactions on Circuits and Systems for Video Technology. pp. Applications. no. ATM nectworks: Concepts. [12] Z. Mitra. He. pp. 2002. Kluwer Academic. K. vol. [14] P. and S. Communications and Signal Processing (ICICS ’99). Aug. 1998. Cambridge University Press. “Optimum Bit Allocation and Accurate Rate Control for Video Coding via ρ-domain Source Modeling. Addison-Wesley. 7-10 Dec.N. and S. Hosur and K. K. He. “Motion Vector Field Adaptive Fast Motion Estimation.. 11. Kin. Huber. 11. Handel. “A Linear Source Model and a Unified Rate Control Algorithm for DCT Video Coding. [13] J. Gray.

Geneva. [23] ITU-T Rec. [20] ITU-T. ISO/IEC JTC1/SC29/WG11. 1990. [21] ITU-T. R. 1997. Jain. 1995. 1995. [24] J. Video Codec for Audiovisual Services at p x 64 kbits/s.261. 1993.5 Mbits/s: Video. Stockholm. Recommendation H. 2002. 1981. 1999. vol. COM29.” IEEE Transactions on Communication. Oct. K. Video Codec Test Model for the Near-term 5 (TMN5). Test Model 5 (TM5). Draft. Video Codec for Low Bit-Rate Communication. Sweden.263. Coding of Moving Picture and Associated Audio. “Displacement Measurement and Its Application in Interframe Image Coding.0. ISO/IEC 14496-2. Advance Video Coding. MPEG-4 Video VM Version 8. 1993. Jain and A. [18] [19] ISO/IEC JTC1/SC29/WG11. 102 . Jul. Dec. Recommendation H. MPEG97/W1796. pp. [22] ITU-T SG 15 Experts Group for Very Low Bitrate Visual Telephony. [16] ISO/IEC CD 13818/2 (MPEG-2 Video). Part 2: Visual. H. 1799-1808.[15] ISO/IEC CD 11172/2 (MPEG-1 Video). 1995. Information technology----Coding of Moving Pictures and Associated Audio for Digital Storage Media at Up to About 1. [17] ISO/IEC CD 14496/2 (MPEG-4 Video).. Information technology----Coding of Natural/Visual Objects. Document N400. Dec.264 | ISO/IEC 14496-10 AVC. Information technology----Generic Coding of Moving Pictures and Associated Audio Information: Video.

Ishiguro. W. vol. ISO/IEC JTC1/SC29/WG11 N2725 Seoul. 2003. San Jose. Kluwer Academic Publishers.1-C9. “Complexity Analysis of the Emerging MPEG-4 Standard as a Basis for VLSI Implementation.” Proceedings of the IEEE. Krishna. K. Hirano. 5. Safranek. Iijima. no. 8. [28] R. pp. Kelly. Telecomm.. “Adaptive Rate Control using Nonlinear Regression. “Overview of the MPEG-4 standard.6. no. A. vol. R. Kuhn. New Orleans. Kim.. pp.” in Proceeding of SPIE Visual Communications and Image Processing. Ilinuma. 81. [26] F.5. C9. Jan. Auction Theory.” in Proceeding of 1981 National Telecommunication Conference (NTC81). Ed. Complexity Analysis and VLSI Architectures for MPEG4 Motion Estimation. [29] T.” Doc. Boston. Jayant. Oct. pp.[25] N. Y. 498-509.6. -Focus on Elastic Service over ATM Networks. San Diego. 1. Trans. Stechele. “Signal Compression Based on Models of Human Perception. U. 33-37. 1999. Johnston. Koenen. Koga. P.” IEEE Transactions on Circuits and System for Video Technology. 1998. 2002.S. Kuhn.” Eur. M. pp. 13. 1993. J. “Charging and Rate Control for Elastic Traffic. 103 . pp.A. no. Nov. [27] H. 10. Academic Press. 1385-1422. and T. “Motion Compensated Interframe Coding for Video Conferencing. [32] P. vol.1999. [30] [31] V. Korea. Mar. Jan. Algorithms. 1981. 432-439. 1997.

Y. [34] H. 2. and Y. T. 3. vol.E. 104 . Shie. 1998. vol.” IEEE Transactions on Circuits and Systems for Video Technology.” IEEE Transactions on Circuits and Systems for Video Technology. 878-894. “Scalable Rate Control for Very Low Bit Rate (VLBR) Video. 4. Lemke and J. 1964 [38] R. pp. W. 768-771. Zhang. 413-423. [39] L. pp. “A New Three-Step Search Algorithm for Block Motion Estimation. pp. Zeng. vol. pp. 2003. Lee. no. Q. B.” IEEE Transactions on Circuits and Systems for Video Technology. no. W. Chiang. “Bit-rate Control using Piecewise Approximated Ratedistortion Characteristics. Li. 6. 446-459. Y. F. [36] L. vol. [37] C. 1997. A. A. 2000. 488-502. 8. J. J. “Bit Allocation for MPEG-4 Video Coding With Spatial-Temporal Tradeoffs. J. Sept.. Lee. and M. Wang. and Y. pp. Aug. 85-87. Lee. 13. vol. pp. Jun. vol. and Y. Aug. Feb. Lee. Wang. 10. 1994. Ho. 438-442. “Scalable Rate Control for MPEG-4 Video. pp. 1. J. Zhang.” IEEE Transaction on Circuits and Systems for Video Technology. Lin.[33] H.” IEEE Transactions on Circuit and Systems for Video Technology. Liou. Lee. “Dynamic Search-Window Adjustment and Interlaced Search Block-matching Algorithm. Ortega. no. [35] J.T. Vetro. Howson. 6. Chiang. “Equilibrium Points of Bimatrix Games. Oct. D. Q. 1993. T. no. 4. J. S. no. 4.” in Proceeding of 1997 International Conference on Image Processing.” SIAM Journal of Applied Mathematics 12.

” IEEE Transactions on Circuits and Systems on Video Technology. [45] J. no. Nash.” Communications of the ACM. 100. Game Theory and Economics. D. 29. 2003. 54. pp. 18. 155-162. Sept. pp. pp. vol. vol. M. “Andrew: A Distributed Personal Computing Environment. Jan. pp. pp. 1996. 1928. 21.” Annals of Mathematics.” Econometrica. Palgrave. Serra. 3. and O. Smith. 1947. 1950. Apr. “The Bargaining problem. vol. 184-201. Mar. vol. J. vol.. 1953. Conner. Morris. [46] J. “Two-Person Cooperative Games. [43] J. Nash.[40] L. Rosenthal and F. Princeton (Princeton University Press). 36. “Zur Theorie der Gesellschaftsspiele. 48-49. Jan. [48] J. Aug. Howard. Feig. [44] J. pp. “Non-Cooperative Games. vol. von Neumann. 105 . 6. 1986. Theory of Games and Economic Behaviors. 4. pp.” Econometrica." in Proceeding of the National Academy of Sciences. 1944. Von Neumann. Satyanarayanan. Morganstern. vol. "Equilibrium Points in N-person Games. K. 128140. 295-320. Nash. Nash. 286295. [41] [42] Montet and D.” Mathematische Annalen. Liu. [47] J. 1951. no. 419-422. E. J. “A Block-based Gradient Descent Search Algorithm for Block-based Motion Estimation in Video Coding. M. 1950.

T. Pirsch. “VLSI Architectures for Video Compression. L. 1996 [54] C.[49] K. U. 385-393. [50] K. 2. vol. Lim. Webb. “A Study of MPEG-4 Rate Control Scheme and Its Improvements. Massachusetts. 106 .” Keynote speech at Picture Coding Symposium’ 96. 83. Ngan. Basic Feedback Control Systems. vol. Englewood Cliffs: Prentice-Hall. N.” IEEE Transactions on Circuits and Systems for Video Technology. Gehrke. Chen. Po and W. and Z. Demassieux. 1996. Harbor. Li.” in Proceeding of IEEE International Conference on Image Processing. vol. 13.. C. N. 6. Osborne and A. Feb. May 2003. Meier. Feng. M. “Macroblock Quantizer Selection for H. 3. no. MIT Press.S. pp. pp. May 2003.” IEEE Transactions on Circuits and Systems for Video Technology.L. Oct. 5. vol.365–368.” Proceedings of the IEEE. and G. Jun. [51] M. 1991.A Survey. Phillips and R. no. [53] Fernando Pereira. 2nd Edition. J. A Course in Game Theory. Z.A. pp. no. W. [52] F. K. Pan. 220246. “MPEG-4: a New Challenge for the Representation of Audio-Visual Information.263 Video Coding. 1995. 5. vol. 1994. 440-446. 1997. Rubistein. “Improved Single Video Object Rate Control MPEG-4.D. pp. 313-317.” IEEE Transactions on Circuits and Systems for Video Technology. Ma. “A Novel Four-Step Search Algorithm for Fast Block matching. Oehler and J. no. 1. 13. pp. [56] L. [55] P.

1999. of ACM STOC. 1991. “Best Wavelet Packet Bases in a RateDistortion Sense. vol. Sept. Vetterli. pp. 11.” IEEE Transactions on Circuit and System for Video Technology. Jaureguizar. vol. Roughgarden. “Bit Allocation for Dependent Quantization with Applications to Multiresolution and MPEG Video Coders. 1. F. pp. 104-113.” IEEE Transactions on Circuits and Systems for Video Technology. A. Ortega. [62] J. 1993. 4. 1. and N.[57] Puri and R. pp. 2. Vol. 3.” in Proc. vol. Ramchandran and M. pp. “Motion-Compensated Video Coding with Adaptive Perceptual Quantization. 5. I. pp. no. Garcia.” IEEE Transactions on Circuits and Systems for Video Technology. Ramchandran. 9. 882-889. Aravind. Feb. vol. “Rate Control in DCT Video Coding for LowDelay Communication. Ronda.” IEEE Transactions on Circuits and Systems for Video Technology.” IEEE Transactions on Image Processing. 9. 351-361. [59] K. Apr. Reed and Frederic Dufux. 1991. pp. Eckert. no. Lei. 1243-1258. pp. 2001. “Stackelberg Scheduling Strategies. [60] Eric C. No. [63] J. Jul. Dec. 533-545. Dec. 7. “Rate Control and Bit Allocation for MPEG-4.” IEEE Transactions on Image Processing. 1999. Ribas-Corbera and S. [61] T. vol. M. 160175. no. 6. and M. [58] K. 172-185. “Constrained Bit-Rate Control for Very Low Bit-Rate Streaming-Video Applications. no. 1994. Vetterli. 107 .

Dickinson. 593-598.” IEEE Transactions on Acoust.” in Proceeding of the 2nd International Conference on Multiagent Systems. [68] Tao. “A Rate-Quantization Model for MPEG Encoders. W. Speech. 1984. “Adaptive Model-Driven Bit Allocation for MPEG Video Coding.L. A. A First Course in Optimization Theory. Peterson. vol. 1996. W. [67] R. pp. and B. Oct. Perterson. Tao. Signal Processing. A. “Efficient Bit Allocation for an Arbitrary Set of Quantizers. 1996. H. 9. [65] Y. and H.” Rev. pp. Liou. pp. pp. Au.M. Noordwijkerhout. Shoham and A. B. K. "Fast Block-Matching Motion Estimation using Predictive Motion Vector Field Adaptive Search Technique (PMVFAST)". Stefanescu.C. “Limitations of Vickery Auction in Computational Multiagent Systems. 10. ISO/IEC JTC1/SC29/WG11 MPEG99/m5866. Sundaram. the Netherland. Sandholm. Dickinson.[64] T. “The Arbitrated Solution for Multiobjective Convex Programming. pp. [69] B. O. 2000. vol. Math. Gersho. 338-341. 108 . Mar. 299-306.” IEEE Transactions on Circuits and Systems for Video Technology. 29. 36. and M. Cambridge University Press. 1988. Sept. 1445-1453. [70] A. 1997.” in Proceeding of 1997 International Conference on Image Processing. no. vol. Pure Applicat. Roum. Feb.W. [66] Stefanescu and M. 2000. 147-157. Tourapis.

” IEEE Transactions on Circuits and Systems for Video Technology. 58 –72. and M. Proc. no. Jun. 1. Vetro and H. 1991. 2002. 186-199. [76] Viscito and C. Vetro. SPIE 4671. [74] A. [72] J.” in Visual Communications and Image Processing. Gonzales. “A Video Compression Algorithm with Adaptive Bit Allocation and Quantization. Issue: 10. vol. Sun. O." IEEE Transactions on Circuits and Systems for Video Technology. 109 . 6. and C. 698-709. 2004.” in SPIE Visual Commun. Wang. “Modified TMN8 Rate Control for Low-Delay Video Communication. and Y. Au. H. Spain. 1999. “A New Rate Control Algorithm for MPEG-4 Video Coding. 864-868. Nov.” IEEE Transactions on Circuits and Systems for Video Technology.” ISO/IEC JTC1/SC29/WG11 Coding of Moving Pictures and Associated Audio MPEG 97/M1631. pp. Sun. Liou. Volume: 12. pp. pp.C. [75] Sun Yu and Ishfaq Ahmad. CA. Oct. 2002. H. Feb. pp. “Joint Rate Control for Multiple Video Objects Based on Quadratic Rate-Distortion Model. [73] A. San Jose. "Highly efficient predictive zonal algorithms for fast block-matching motion estimation. 1997. 9. vol.L. vol.M. Image Processing. 1605-1. 14. C. Tourapis. Sevilla. no. pp 934 – 947. Jan. “MPEG-4 Rate Control for Multiple Video Objects.[71] A. Hsieh. Feb. Tsai.

263-compatibal Codecs and Its Application to Quantizer Selection. Jacquin. "Adaptive Motion Search with Elastic Diamond for Mpeg-4 Video Coding. 1997. Zhu and K. pp. “Rate-constrained Optimal Block-Adaptive Coding for Digital Tape Recording of HDTV. 110 . 1 pp.[77] S. vol. 2. 9. vol. 100-112.” IEEE Transactions on Circuits and Systems for Video Technology.” IEEE Transactions on Image Processing. A.377-380. Liou. Ma. 287-290. 1. I. 1991. W. Yang. Zheng. “A New Diamond Search Algorithm for Fast BlockMatching Motion Estimation. Gersho." in Proceeding of 2001 International Conference on Image Processing (ICIP2001). S.” in Proceeding of 1997 International Conference on Image Processing. [79] W. Ahmad and M. no. Feb. K. 41-44. vol. no. “A Normalized Rate-Distortion Model for H. Oct. and N. [78] K. H. pp.L. Jayant. 2000. Mar. Wu and A. 1. pp. 2001 [80] S.

111 . degrees from the Department Electrical Engineering at Zhongshan University.S. From 2000 to 2001. he was a Ph.D. He transferred to the University of Texas at Arlington in spring 2002. video compression. China. in 1997 and 2000 respectively. video processing. His research interests include multimedia systems. student at the Computer Science Department of Hong Kong University of Science and Technology.Eng and M. video surveillance. He received his doctorate in Computer Science and Engineering in August 2005. video over wireless device.BIOGRAPHICAL INFORMATION Jiancong Luo received the B. 3D video. 2D/3D image for remote sensing and medical application. pervasive audio/video collaboration. game theory application in video compression.

- 2013Comparison of Compression Performance of 10-bit vs. 8-bit Depth, under H.264 Hi422 Profile-Comparison of Compression Performance of 10-Bit vs. 8-Bit Depth, Under H.264 Hi422 Profileby rathode494
- A Guide to MPEG Fundamentals and Protocol Analysisby Ziad T. Ahmed
- V2I1068by editor_ijarcsse
- Mpeg Video Compressionby Nirav Darji

- Adaptive Residual Interpolation: A Tool for Efficient Spatial Scalability in Digital Video Codingby Wesley De Neve
- Multi-View Video Coding Algorithms/Techniques: A Comprehensive Studyby ijeraeditor
- videocompressionbasics-mpeg2-091031123547-phpapp02by marit1950
- A Modern Approach to Video Compressionby PrometheusUV

- FRACTAL AND MULTIFRACTAL ANALYSES OF COMPRESSED VIDEO SEQUENCES
- Gall - MPEG
- Video Compression 1 H 261
- Mpeg-2 Video Compression Technique Report
- data compress
- intoPIX - Reliable Video Quality from Production to Archiving with JPEG 2000 - White Paper IABM Sourcebook
- A guide to MPEG
- H264 Video Compression Standard
- 2013Comparison of Compression Performance of 10-bit vs. 8-bit Depth, under H.264 Hi422 Profile-Comparison of Compression Performance of 10-Bit vs. 8-Bit Depth, Under H.264 Hi422 Profile
- A Guide to MPEG Fundamentals and Protocol Analysis
- V2I1068
- Mpeg Video Compression
- mpec-2
- The H. 264-AVC Video Coding Standard for the Next Generation Multimedia Communication
- Bock09 - Video Compression Systems
- 10 Bit High Quality MPEG-4 AVC Video Compression
- A Guide to MPEG Fundamental and Protocol Analysis
- Final Project 112
- JPEG
- Video_wm_H264_4
- MPEG Compression Standards
- PhD Research Proposal of jan Paul
- Introduction to Video Compression
- mpeg
- Adaptive Residual Interpolation
- Multi-View Video Coding Algorithms/Techniques
- videocompressionbasics-mpeg2-091031123547-phpapp02
- A Modern Approach to Video Compression
- nch4
- Rahat Mpeg Ieee Format
- umi-uta-1031

Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

We've moved you to where you read on your other device.

Get the full title to continue

Get the full title to continue reading from where you left off, or restart the preview.

scribd