This action might not be possible to undo. Are you sure you want to continue?

Welcome to Scribd! Start your free trial and access books, documents and more.Find out more

Vida Movahedi

October 2006

Contents

A simple example Definitions Huffman Coding Algorithm Image Compression

A simple example Suppose we have a message consisting of 5 symbols. e. [ ] How can we code this message using 0/1 so the coded message will have minimum length (for transmission or saving!) 5 symbols à at least 3 bits For a simple encoding. length of code is 10*3=30 bits .g.

Intuition: Those symbols that are more frequent should have smaller codes. length of encoded message will be =3*2 +3*2+2*2+3+3=24bits .A simple example cont. there must be a way of distinguishing each code For Huffman code. yet since their length is not the same.

p2.. a2. pi>0. 10. . pI} where P(x)=P(x=ai)=pi.0007 h(pi) 4.. å pi = 1 Shannon information content of x h(x) = log2(1/P(x)) i 1 2 ai a b c z pi . Px) x: value of a random variable Ax: set of possible values for x .Definitions An ensemble X is a triple (x.0575 . Ax={a1.0263 . .1 6. log P( x) xÎAx 3 . Px={p1.0128 .3 5.4 Entropy of x 1 H ( x) = å P( x). .. Ax. 26 . aI} Px: probability for each value .2 .

satisfies L(C. H(X)+1) The Huffman coding algorithm produces optimal symbol codes .X).X)Î[H(X).Source Coding Theorem There exists a variable-length encoding C of an ensemble X such that the average length of an encoded symbol. L(C.

1}+={0.010.001.000. l(x): length of codeword .10.01.111} {0.1}3={000.1}+ c(x): codeword for x.11.00.1.001.Symbol Codes Notations: AN: all strings of length N A+: all strings of finite length {0. } A symbol code C for an ensemble X is a mapping from Ax (range of x values) to {0. .

b . 1/4 . c .Example Ensemble X: Ax= { a . d } Px= {1/2 . 1/8} c(a)= 1000 c+(acd)= 100000100001 (called the extended code) C0: ai a b c d c(ai) 1000 0100 0010 0001 li 4 4 4 4 . 1/8 .

e. under the extended code C+. no two distinct strings have the same encoding. i. y Î AX . + "x. x ¹ y Þ c + ( x) ¹ c + ( y ) .Any encoded string must have a unique decoding A code C(X) is uniquely decodable if.

The symbol code must be easy to decode If possible to identify end of a codeword as soon as it arrives àno codeword can be a prefix of another codeword A symbol code is called a prefix code if no code word is a prefix of any other codeword (also called prefix-free code. instantaneous code or self-punctuating code) .

X) of symbol code C for X is L(C .The code should achieve as much compression as possible The expected length L(C. X ) = xÎ Ax å P( x)l ( x) = å p l i =1 | Ax | i i .

Example Ensemble X: Ax= { a . d } Px= {1/2 . b . 1/8} c+(acd)= 0110111 (9 bits compared with 12) C1: ai a b c c(ai) 0 10 110 111 li 1 2 3 3 prefix code? d . 1/4 . 1/8 . c .

David Huffman and his MIT information theory classmates given the choice of a term paper or a final exam Huffman hit upon the idea of using a frequency-sorted binary tree and quickly proved this method the most efficient.History In 1951. the student outdid his professor. In doing so. who had worked with information theory inventor Claude Shannon to develop a similar code. Huffman built the tree from the bottom up instead of from the top down .The Huffman Coding algorithm.

Combine these two symbols into a single symbol. .Huffman Coding Algorithm 1. and repeat. equal length. differing in last digit) 2. Take the two least probable symbols in the alphabet (longest codewords.

c .0 0.2 11 0 d 0.15 011 .25 10 1.3 1 e 0. 0. 0.15} 0 0.45 1 c 0.25. 0. 0.25.25 00 b 0.15 010 0.15.55 1 1 0 0 a 0.2. e } Px={0. d . b .Example Ax={ a .

Statements Lower bound on expected length is H(X) There is no better symbol code for a source than the Huffman code Constructing a binary tree top-down is suboptimal .

Disadvantages of the Huffman Code Changing ensemble If the ensemble changesà the frequencies and probabilities change à the optimal coding changes e. in text compression symbol frequencies vary with context Re-computing the Huffman code by running through the entire file in advance?! Saving/ transmitting the code too?! Does not consider blocks of symbols strings_of_ch àthe next nine symbols are predictable aracters_ . but bits are used without conveying any new information .g.

. problems e. max [wi+length(ci)] . 1. .1}) Adaptive Huffman coding Calculates frequencies dynamically based on recent actual frequencies Huffman template algorithm Generalizing probabilities à any weight Combining methods (addition) à any function Can solve other min.g. n-1} (not just {0..Variations n-ary Huffman coding Uses {0.

Lossless JPEG: . or some linear predicting function à Decorrelate the raw image data 2.version 1: DPCM with arithmetic coding . such as Huffman coding. A linear predictor such as DPCM. A standard coding technique. arithmetic coding.version 2: DPCM with Huffman coding .Image Compression 2-stage Coding technique 1.

DPCM Differential Pulse Code Modulation DPCM is an efficient way to encode highly correlated analog signals into binary form suitable for digital transmission. storage. or input to a digital computer Patent by Cutler (1952) .

DPCM .

Encode the Huffman tree and save the Huffman tree with the coded value.Huffman Coding Algorithm for Image Compression Step 1. Step 2. Step 3. Encode the residual image. Build a Huffman tree by sorting the histogram and successively combine the two bins of the lowest value until only one bin remains. .

Compute the symmetry histogram S S(y)= H(y) + H(-y). Find the range threshold R for N: # of pixels . y>0 3.Huffman Coding of the most-likely magnitude MLM Method 1. P: desired proportion of most-likely magnitudes å S ( j) £ P ´ N < å S ( j) j =0 j =0 R -1 R . Compute the residual histogram H H(x)= # of pixels having residual magnitude x 2.

Information Theory. 2003. Inference. (2) Wikipedia. C. Cambridge University Press.References (1) MacKay. D. (4) O Neal .J.C.. http://en. Y.wikipedia. and Learning Algorithms. .C. A new losseless compression scheme based on Huffman coding scheme for image compression .C. and Chang.org/wiki/Huffman_coding (3) Hu.

- Huffman Coding
- Mutual Information LLR
- A FEC Decoding in LTE and WiMAX Systems
- Computation
- 2008_Rouillard_ICCGI
- Implementing rate compatible turbo codes
- Introduction to Video Compression
- Liu Detection
- 1.Lossy Compression and Iterative Reconstruction for Encrypted Image
- 25.Partial.full
- Code & Life Safety Signs
- huffman
- Video Technology
- Improving the security of images transmission
- description
- codes-and-cryptography
- Analysis of an Alternative Interleaving Scheme in IDMA (WIP)
- IS-54 dan IS-136
- 28845645 Tele Graffiti
- History
- BICM Paper
- IJETTCS-2015-01-30-52
- GEOPAK Terrain and Subsurface Modeling v8 1
- 14-Digit Signaling Point Code
- LD04_ch14.pdf
- HD TV Encoding and Decoding
- Adaptive Trilateral Filter
- Anatomy of a Printable Coupon
- Logic User Guide
- 08 Computer Memory2

Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

We've moved you to where you read on your other device.

Get the full title to continue

Get the full title to continue reading from where you left off, or restart the preview.

scribd