76 views

Uploaded by nassr_ismail

Huffman Coding

save

You are on page 1of 33

HUFFMAN CODING

1

Overview

In this chapter, we describe a very popular coding algorithm called the Huffman coding algorithm Present a procedure for building Huffman codes when the probability model for the source is known A procedure for building codes when the source statistics are unknown Describe a new technique for code design that are in some sense similar to the Huffman coding approach

2

1

data compression basics

Huffman Coding Algorithm

3

Huffman Coding Algorithm

4

2

data compression basics

Huffman Coding Algorithm

5

Huffman Coding Algorithm

6

3

data compression basics Huffman Coding Algorithm 7 Huffman Coding Algorithm 8 4 .

data compression basics Huffman Coding Algorithm 9 Huffman Coding Algorithm 10 5 .

data compression basics Minimum Variance Huffman Codes 11 Minimum Variance Huffman Codes 12 6 .

data compression basics Minimum Variance Huffman Codes 13 Minimum Variance Huffman Codes 14 7 .

data compression basics Minimum Variance Huffman Codes 15 Huffman Coding (using binary tree) Algorithm in 5 steps: 1. Combine the smallest two by addition 4. generate code by alternating assignment of 0 and 1 16 8 . GOTO step 2. Find the grey-level probabilities for the image by finding the histogram 2. until only two probabilities are left 5. Order the input probabilities (histogram magnitudes) from smallest to largest 3. By working backward along the tree.

18 9 . In step 1 we find the histogram for the image. giving 4 possible gray levels. The image is 10 rows by 10 columns.data compression basics Huffman Coding (using binary tree) Coding Procedures for an N-symbol source Source reduction List all probabilities in a descending order Merge the two symbols with smallest probabilities into a new compound symbol Repeat the above two steps for N-2 steps Codeword assignment Start from the smallest source and work back to the original source Each merging point corresponds to a node in binary codeword tree 17 Example 1 We have an image with 2 bits/pixel.

20 10 . the probabilities are ordered. Step 1: Histogram 19 Example 1 Step 2.data compression basics Example 1 Converted into probabilities by normalizing to the total number of pixels Gray level 0 has 20 pixels Gray level 1 has 30 pixels Gray level 2 has 10 pixels Gray level 3 has 40 pixels a.

data compression basics Example 1 Step 3. 21 Example 1 Step 4 repeats steps 2 and 3. combine the smallest two by addition. Step 4: Reorder and add until only two values remain. d. where reorder (if necessary) and add the two smallest probabilities. 22 11 .

4 branch 23 Example 1 The assigned 0 & 1 are brought back along the tree & wherever a branch occurs the code is put on both branches 24 12 .6 branch & 1 to 0. actual code assignment is made. Start on the right-hand side of the tree and assign 0’s & 1’s 0 is assigned to 0.data compression basics Example 1 Step 5.

data compression basics Example 1 Assign the 0 & 1 to the branches labeled 0. appending to the existing code. 25 Example 1 Finally.1 & 0.3.2 branch) 26 13 . the codes are brought back one more level. & where the branch splits another assignment 0 & 1 occurs (at 0.

is the most likely to occur (40% of the time) & thus has least information in the information theoretic sense.data compression basics Example 1 Now we have Huffman code for this image 2 gray levels have 3 bits to represent & 1 gray level has 1 bit assigned Gray level represented by 1 bit. 27 Exercise Using the example 1. EE465: Introduction to Digital Image Processing 28 14 . g3. find a Huffman code using the minimum variance procedure.

5 (NEW) compound symbols 29 0.5 0.25 1 0.5 0.125 0.125 1 30 15 .25 0 0.5 0.25 0.5 codeword 0 0 1 10 110 111 0.5 0.data compression basics Example 2 Step 1: Source reduction symbol x S N E W p(x) 0.25 (EW) Example 2 Step 2: Codeword assignment symbol x NEW S 0 N 1 0 S EW 10 E N 1 0 W 111 110 W E 1 0 p(x) 0.5 0.25 0.5 0.125 0 0.125 0.25 0.

2 0.1 0.4 0.1 0. 31 Example 2 Step 1: Source reduction symbol x e a i o u p(x) 0. at each merging point (node).2 (ou) 0.4 0.data compression basics Example 2 0 1 NEW 0 1 0 S EW 10 N 1 0 110 W E 1 0 NEW 1 0 1 S or EW 01 N 1 0 000 001 W E The codeword assignment is not unique.6 (aiou) 0.2 0.4 0.4 (iou) 0. we can arbitrarily assign “0” and “1” to the two branches (average code length is the same). In fact.2 compound symbols 32 16 .4 0.2 0.2 0.

1 0.data compression basics Example 2 Step 2: Codeword assignment symbol x e a p(x) 0.2 0.6 0 (aiou) 0.1 0.4 0.4 0.4 0.4 (iou) 0.2 0.2 0.2 0.2 (ou) 000 0010 0011 compound symbols 33 Example 2 0 1 (aiou) e 00 01 (iou) a 000 001 (ou) i 0010 0011 o u binary codeword tree representation 34 17 .4 1 codeword 1 01 i o u 0.2 0.

4 0.122bps r l H ( X ) 0.2bps 5 H(X ) i 1 pi log 2 pi 2.878bps 35 Example 3 Step 1: Source reduction compound symbol 36 18 .122=0. we have to spend three bits per sample.1 4 2.1 4 0.1 codeword length 1 1 01 2 3 000 0010 4 0011 4 5 l i 1 pi li 0.4 1 0.2 0.078bps If we use fixed-length codes.2 0. which gives code redundancy of 3-2.2 2 0.2 3 0.1 0.data compression basics Example 2 symbol x e a i o u p(x) 0.

data compression basics Example 3 Step 2: Codeword assignment compound symbol 37 Adaptive Huffman Coding 38 19 .

data compression basics Adaptive Huffman Coding 39 Update Procedure 40 20 .

data compression basics Update Procedure 41 Update Procedure 42 21 .

data compression basics Update Procedure 43 Update Procedure 44 22 .

data compression basics Update Procedure 45 Dynamic Huffman Coding 46 23 .

t(1) 48 24 .t(1) * r represents the root * 0 represents the null node * t(1) denotes the occurrence of T with a frequency of 1 47 TE Stage 2 (First occurrence of e) r / \ 1 t(1) / \ 0 e(1) Order: 0.data compression basics T Stage 1 (First occurrence of t ) r /\ 0 t(1) Order: 0.1.e(1).

t(1).t(1) : Misfit 49 Reorder: TEN r / \ t(1) 2 / \ 1 e(1) / \ 0 n(1) Order: 0.1.n(1).1.2.e(1).data compression basics TEN Stage 3 (First occurrence of n ) r / \ 2 t(1) / \ 1 e(1) / \ 0 n(1) Order: 0.n(1).e(1).2 50 25 .

data compression basics TENN Stage 4 ( Repetition of n ) r / \ t(1) 3 / \ 2 e(1) / \ 0 n(2) Order: 0.2.t(1).e(1).t(1).3 : Misfit 51 Reorder: TENN r / \ n(2) 2 / \ 1 e(1) / \ 0 t(1) Order: 0.n(2).n(2) are swapped 52 26 .1.e(1).2 t(1).n(2).

1.n(2).t(1).n(2).2.3 53 TENNES Stage 6 (First occurrence of s) r / \ n(2) 4 / \ 2 e(2) / \ 1 t(1) / \ 0 s(1) Order: 0.4 54 27 .1.s(1).data compression basics TENNE Stage 5 (Repetition of e ) r / \ n(2) 3 / \ 1 e(2) / \ 0 t(1) Order: 0.e(2).t(1).e(2).

2.1.e(2).data compression basics TENNESS Stage 7 (Repetition of s) r / \ n(2) 5 / \ 3 e(2) / \ 2 t(1) / \ 0 s(2) Order: 0.t(1).3.s(2).3.n(2).t(1).n(2).5 s(2) and t(1) are swapped 56 28 .5 : Misfit 55 Reorder: TENNESS r / \ n(2) 5 / \ 3 e(2) / \ 1 s (2) / \ 0 t(1) Order : 0.e(2).s(2).

1.5 N(2) and e(3) are swapped 58 29 .data compression basics TENNESSE Stage 8 (Second repetition of e ) r / \ n(2) 6 / \ 3 e(3) / \ 1 s(2) / \ 0 t(1) Order : 0.e(3).e(3).1.n(2).t(1).3.s(2).6 : Misfit 57 Reorder: TENNESSE r / \ e(3) 5 / \ 3 n(2) / \ 1 s(2) / \ 0 t(1) Order : 1.s(2).t(1).n(2).3.

data compression basics TENNESSEE Stage 9 (Second repetition of e ) r / \1 e(4) 5 0/ \1 3 n(2) 0/ \1 1 s(2) 0/ \1 0 t(1) 0 Order : 1.3.1.s(2).5 59 ENCODING The letters can be encoded as follows: e:0 n : 11 s : 101 t : 1001 60 30 .e(4).t(1).n(2).

n frequency = { 1(4) + 2(2) + 3(2) + 1(4) } / (4+2+2+1) = 18 / 9 = 2 61 ENTROPY Entropy = i=1.( 0.11 ) = .22 + 0.(0.44 + 0.22 + 0.11 * log20.22 * log20.11) / log2 = 1.n (length*frequency)/ i=0.22 + 0.22 * log20.11 * log0.22 * log0.44 * log0.44 + 2(0.8367 62 31 .n (pi log2 pi) = .data compression basics Average Code Length Average code length = i=0.44 * log20.

The problem with static coding is that the tree has to be constructed in the transmitter and sent to the receiver. 64 32 . Considering this. the average code length will improve if the transmitted text is bigger. it need not be sent. actually the performance of dynamic coding is better. piece of code etc. Dynamic coding is better.in this exercise. Also. Since the tree in dynamic coding is constructed on the receiver as well. But.data compression basics Ordinary Huffman Coding TENNESSE 9 0/ \1 5 e(4) 0/ \1 s(2) 3 0/ \1 t(1) n(2) ENCODING E:1 S : 00 T : 010 N : 011 Average code length = (1*4 + 2*2 + 2*3 + 3*1) / 9 = 1.89 63 SUMMARY The average code length of ordinary Huffman coding seems to be better than the Dynamic version. The tree may change because the frequency distribution of the English letters may change in plain text technical paper.

data compression basics Summary of Huffman Coding Algorithm Achieve minimal redundancy subject to the constraint that the source symbols be coded one at a time Sorting symbols in descending probabilities is the key in the step of source reduction The codeword assignment is not unique. it does not know where to start) 65 33 . Exchange the labeling of “0” and “1” at any node of binary codeword tree would produce another solution that equally works well Only works for a source with finite number of symbols (otherwise.

- Sh Ann OnUploaded byChavalam Srigiri
- 2063Uploaded byDang Duy
- Binary Search TreeUploaded byMax Wong
- s442012_klUploaded bydodoman666
- 20121120 Quiz 8a SolutionsUploaded byPaul Jones
- Tree.pdfUploaded byfsshidiq
- Assignment 6Uploaded bypiyank
- Low Power Digital Signal Processor Architecture For Wireless Sensor Nodes By Using Parallel Prefix TechniqueUploaded byInternational Organization of Scientific Research (IOSR)
- commqaUploaded byhakkı_a
- Data Structures and Algorithms (2)Uploaded byKeith Tanaka Magaka
- lec18Uploaded byPushparaj Karu
- FSP-PaperUploaded byYutyu Yuiyui
- ict12-lec5Uploaded byFaheem Babar Chattha
- Minimizing Navigation Cost Using an Efficient Cost Based Ranking and Navigation TechniqueUploaded byseventhsensegroup
- ps3solUploaded byJayesh Gorasia
- Data11Uploaded byJuan Hansen
- An overview of distributed MST algorithmsUploaded byArkanath Pathak
- breadth mapUploaded byMinni Narayanan
- 5.1 Mining Data StreamsUploaded byRaj Endran
- Concept of FeaturesUploaded byNalin Reddi
- Fitted Elmore Delay- A Simple and Accurate Interconnect Delay ModelUploaded bynukkrith
- Cs 301 Final Term Solved Paper 1Uploaded byRizwanIlyas
- CS2251 Design and Analysis of Algorithms1QBUploaded byVjay Narain
- P10 - C++Uploaded byRohit Kshirsagar
- K-means-EM-Cobweb-WEKA.pdfUploaded byGabriel Gregorius Rusfian
- A Simplifed and Dynamic Unifed StructureUploaded byΑλέξανδρος Γεωργίου
- Analyzing Characters (TNT and Mesquite)Uploaded bySerenaChen
- Data StructureUploaded byRiddhimaa Gupta
- Cau truc Du Lieu & Giai ThuatUploaded byTACOMPUTER
- Red Black TreesUploaded byKavya Rastogi

- HSDPA Plus SwitchUploaded bynassr_ismail
- 256 QAM FeatureUploaded bynassr_ismail
- Network Capacity OverviewUploaded bynassr_ismail
- 64 QAM FeatureUploaded bynassr_ismail
- Iub Port Available Bandwidth Utilizing Ratio.pdfUploaded bynassr_ismail
- Add Intra-Frequency Neighboring CellUploaded bynassr_ismail
- Location Tracking of Mobiles (Smart Filtering)Uploaded bynassr_ismail
- Important Parameters HuaweiUploaded bynassr_ismail
- Carrier Aggregation Adaptive PCC AnchoringUploaded bynassr_ismail
- Kalman Filter Handouts.pdfUploaded bynassr_ismail
- UL-Interference-UL-RSSI-Ericsson.pdfUploaded bynassr_ismail
- UL RSSIUploaded bynassr_ismail
- Dropctrlerrindseqerr ParameterUploaded bynassr_ismail
- 2 DriveTest Analysis Ver1.pdfUploaded bynassr_ismail
- Uplink Pdtch-pacch UsageUploaded bynassr_ismail
- white-paper-understanding-mobile-terminated-call-failures.pdfUploaded bynassr_ismail
- Type1 Frame StructUploaded bynassr_ismail
- CME_Poster_Feature_OM_Management(01).pdfUploaded bynassr_ismail
- 9780857296344-c2.pdfUploaded bynassr_ismail
- Power BoostingUploaded bynassr_ismail
- Netact Tutorial.pdfUploaded bynassr_ismail
- MDT1Uploaded bynassr_ismail
- 3GPP_CORBA_Bulk_CM_Northbound.pdfUploaded bynassr_ismail
- DL Power AllocationUploaded bynassr_ismail
- PDCCH ParametersUploaded bynassr_ismail
- NSN- HW Parameter MappingUploaded bynassr_ismail
- R-Criterion ParametersUploaded bynassr_ismail
- Nokia Consistency CheckUploaded bynassr_ismail
- Pa-pb Power DistUploaded bynassr_ismail
- Cn 1859721 AUploaded bynassr_ismail

- Single SchedUploaded bysagarbolisetti
- Sol to hw15,16Uploaded byDiponegoro Muhammad Khan
- Problem Set 9Uploaded byAlex Wang
- Thesis - Heuristic and exact algorithms for vehicle routing problemsUploaded byJhonatan Ríchard Raphael
- Data Structure Questions BankUploaded bynooti
- Persistent Placement Papers(Technical C and C++)Uploaded byiamsweetsam
- Numerical AnalysisUploaded byAndrew Do
- Orden Amien to sUploaded byMario Sosa
- Modelos I. Seccion Dorada (1)Uploaded byMichael Reynolds
- Simulated Anne a LingUploaded byraj
- Datastructures HandoutUploaded byAman Sahay
- Estructura de Datos_ Unidad 1 Analisis de AlgoritmosUploaded byAngel Hernández
- Chain Matrix Multiplication and 0/1 KnapsackUploaded byMuhammad Umair
- Método de Gauss-SeidelUploaded byEduardo Cruz Ruiz
- Duality 1Uploaded bySayantan Pal
- Interpolacao PolinomialUploaded byLeinylson Fontinele
- A New Algorithm for the Open-Pit Mine Production Scheduling ProblemUploaded byGonzalo Sepúlveda Ramírez
- Matematicas Discretas Camino mas corto y a la AnchoUploaded byOmar Bustamante
- métaheuristique.docxUploaded bysayda ben sghaier
- Introdução a AlgoritmosUploaded byIsaque Costa
- Sudoku ReportUploaded bywdusln
- ExercisesUploaded byJuan Romero-Requejo Gesto
- Lecture 210 CtUploaded bymagictopsuper
- Acm Cheat SheetUploaded byIsrael Marino
- 03-1-Programacion DinamicaUploaded byMichael Saldaña
- CCP Lab ManualUploaded byajayhschinnu
- DFS and BFSUploaded byranadip das
- A Hash Based Frequent Itemset Mining Using RehashingUploaded byEditor IJRITCC
- Quiz2 3510 Cheat-sheetUploaded byKatherine Pham
- LinkedUploaded byrsrinisha215