0 views

Uploaded by Imran Basha

z

- Lec-10 Trees - BST
- Mca
- DIP 2 marks
- Ugc Net Model Paper
- The Blocker Tag-Selective Blocking of RFID Tags for Consumer Privacy
- Decision Trees
- rpart
- Data Structures-sayed hassan beary
- An Introduction to Image Compression
- Evochess Final
- Lecture 6
- Lecture 6
- tikzpgfmanual
- BMC Impact Solutions 7.3 - Event Management Guide.pdf
- Representations
- Influence Diagrams.pdf
- Alphanumeric Codes
- Advanced Course in Machine Learning
- Final Exam DataComm (1)
- Kemp Tenenbaum 2008 Structural Form

You are on page 1of 31

com 1

Contents

Prefix Code

Representing prefix Codes Using Binary Tree

Binary Tree Terminology

Decoding a Prefix Code

Example

Huffman Coding

Cost of Huffman Tree

Optimality

Prefix code

If every word in the code has the same length, the code is called a

fixed-length code, or a block code.

distinguished by its possession of the "prefix property", which requires that

there is no code word in the system that is a prefix (initial segment) of any

other code word in the system. For example, a code with code words

{9, 59, 55} has the prefix property; a code consisting of {9, 5, 59, 55} does

not, because "5" is a prefix of "59" and also of "55".

word without requiring a special marker between words.

Prefix codes

Suppose we have two binary code words a and b, where a is k bits long, b

is n bits long, and k < n. If the first k bits of b are identical to a, then a is

called a prefix of b. The last n k bits of b are called the dangling suffix.

For example, if

Representing Prefix Codes using Binary Trees

A prefix code is most easily represented by a binary tree in which the external nodes

are labeled with single characters that are combined to form the message. The

encoding for a character is determined by following the path down from the root of

the tree to the external node that holds that character: a 0 bit identifies a left branch

in the path, and a 1 bit identifies a right branch.

In order for this encoding scheme to reduce the number of bits in a message, we use

short encodings for frequently used characters, and long encodings for infrequent ones.

A fundamental property of prefix codes is that messages can be formed

by simply stringing together the code bits from left to right. For example,

the bit-string.

0111110010110101001111100100

encodes the message "abracadabra!". The first 0 must encode 'a', then

the next three 1's must encode 'b', then 110 must encode r, and so on as

follows:

|0|111|110|0|1011|0|1010|0|111|110|0|100

a b r a c a d a b r a !

Binary Tree Terminology

Decoding a Prefix Code

11000111100

Example

counts so that each symbols relative frequency of occurrence is known.

we assume the following frequency counts: A : 15, B : 7, C : 6, D : 6, E : 5)

2. Sort the lists of symbols according to frequency, with the most frequently occurring

symbols at the left and the least common at the right.

3. Divide the list into two parts, with the total frequency counts of the left half being as

close to the total of the right as possible.

4. The left half of the list is assigned the binary digit 0, and the right half is assigned the

digit 1. This means that the codes for the symbols in the first half will all start with 0, and

the codes in the second half will all start with 1.

Example

subdividing groups and adding bits to the codes until each symbol has

become a corresponding code leaf on the tree.

Huffman Coding

Huffman (1951)

Uses frequencies of symbols in a string to build a variable rate prefix

code.

Each symbol is mapped to a binary string.

More frequent symbols have shorter codes.

No code is a prefix of another.

Example:

a 0

b 100

c 101

d 11

Huffman Coding

Q. Given a text that uses 32 symbols (26 different letters, space, and some

punctuation characters), how can we encode this text in bits?

Huffman Coding

Q. Given a text that uses 32 symbols (26 different letters, space, and some

punctuation characters), how can we encode this text in bits?

Huffman Coding

Q. Some symbols (e, t, a, o, i, n) are used far more often than others.

How can we use this to reduce our encoding?

A. Encode these characters with fewer bits, and the others with more bits.

Huffman Coding

Huffman Coding

A. Use a separation symbol (like the pause in Morse), or make sure that there is no

ambiguity by ensuring that no code is a prefix of another one.

c(b) = 010

c(e) = 1

Cost of a Huffman Tree

Let p1, p2, ... , pm be the probabilities for the symbols a1, a2, ... ,am,

respectively.

Define the cost of the Huffman tree T to be

C(T) is the expected length of the code of a symbol coded by the tree T.

C(T) is the bit rate of the code.

Cost of a Huffman Tree

Input: Probabilities p1, p2, ... , pm for symbols a1, a2, ... ,am, respectively.

Output: We would like to find a prefix code that is has the lowest possible

average bits per symbol.

Example of Cost

a b c d

Ex. c(a) = 11

c(e) = 01

c(k) = 001

c(l) = 10

c(u) = 000

is a prefix of the path of y.

Optimality

Principle 1

In a Huffman tree a lowest probability symbol has maximum distance from

the root.

exchanging a lowest probability symbol with one at maximum distance will

lower the cost.

C(T) - C(T) = (h-k)(q-p) >= 0

Optimality

Principle 2

The second lowest probability is a sibling of the smallest in some

Huffman tree.

If we can move it there not raising the cost.

Optimality

Principle 3

Assuming we have a Huffman tree T whose two lowest probability symbols

are siblings at maximum depth, they can be replaced by a new symbol whose

probability is the sum of their probabilities.

The resulting tree is optimal for the new symbol set.

Optimality

1. If there is just one symbol, a tree with one node is optimal. Otherwise

2. Find the two lowest probability symbols with probabilities p and q

respectively.

3. Replace these with a new symbol with probability p + q.

4. Solve the problem recursively for new symbols.

5. Replace the leaf with the new symbol with an internal node with two

children with the old symbols.

Optimality

Principle 3 (cont)

If T were not optimal then we could find a lower cost tree T. This will

lead to a lower cost tree T for the original alphabet.

Optimality

111010001111101000 ?

Optimality

111010001111101000 ?

A. simpel

Optimality

111010001111101000 ?

A. simpel

A. Change encoding of p and s to a shorter one.

This tree is now full.

Optimality

Definition. A tree is full if every node that is not a leaf has two children.

Claim. The binary tree corresponding to the optimal prefix code is full.

with a high frequency?

Optimality

Definition. A tree is full if every node that is not a leaf has two children.

Claim. The binary tree corresponding to the optimal prefix code is full.

with a high frequency?

October 15, 2015 comp_dep_educ@yahoo.com 31

- Lec-10 Trees - BSTUploaded byTaqi Shah
- McaUploaded byamritanshuu
- DIP 2 marksUploaded bypoongodi
- Ugc Net Model PaperUploaded bymkumbhcar
- The Blocker Tag-Selective Blocking of RFID Tags for Consumer PrivacyUploaded by87291472BG
- Decision TreesUploaded bySwapnil Joshi
- rpartUploaded byRobert Robinson
- Data Structures-sayed hassan bearyUploaded bysayedhassan
- An Introduction to Image CompressionUploaded byRajiv Kumar
- Evochess FinalUploaded byzorrin
- Lecture 6Uploaded byShubham
- Lecture 6Uploaded byYesu Babu A
- tikzpgfmanualUploaded byKaThEXD
- BMC Impact Solutions 7.3 - Event Management Guide.pdfUploaded bygits
- RepresentationsUploaded byLaurentiu Suciu
- Influence Diagrams.pdfUploaded bysrinivasan2081
- Alphanumeric CodesUploaded byJabru
- Advanced Course in Machine LearningUploaded bysafu_117
- Final Exam DataComm (1)Uploaded bySandeep Kumar
- Kemp Tenenbaum 2008 Structural FormUploaded byM Smiller
- Lecture3 NotesUploaded bynguyen293
- 8. Tree - BasicsUploaded byMads
- 10.1.1.5.9657Uploaded bynitin_unlucky
- Programs AlgoUploaded byAjay Raj Srivastava
- 1-s2.0-S0304397510006122-mainUploaded byIván Aveiga
- Adsa RecordUploaded bypenchalaiah550
- Chapter 7 Solid Modeling Real SyllabusUploaded bykannanviknesh
- IJA new tree data structure to extract frequent pattern from transactional databaseERTV1IS3081Uploaded byhamecky
- 10.1.1.97Uploaded byDurgamanasa
- 10.1.1.25.483Uploaded byJiang Long

- MakingThings ElectronicsUploaded byShashank Pundir
- Crafting Literature Review .pptUploaded byImran Basha
- 10.1016@j.eswa.2018.03.064Uploaded byImran Basha
- naeem2013.pdfUploaded byImran Basha
- Review of ProbabilityUploaded bychanphat01001
- QCCE.pdfUploaded byImran Basha
- Han 2015Uploaded byImran Basha
- Introduction to ScilabUploaded byImran Basha
- Cmc the BestUploaded byImran Basha
- C2D34C38d01Uploaded byMudit Mittal
- WIMAX_VIVUploaded byImran Basha
- Wireless Access Tech_VIV.pptUploaded byImran Basha
- seminar silent sound technology.pptxUploaded byImran Basha
- Basic ElectronicsUploaded byHari
- Electronic vs Electrical DevicesUploaded byImran Basha
- Image Enhancement in the fd.pptUploaded byImran Basha
- coding_standards_pt1.pdfUploaded byImran Basha
- 12_chapter4(DISTANCE MEASURES).pdfUploaded byImran Basha
- Chapter10.pptUploaded byImran Basha
- Compression Lecture 1AUploaded byImran Basha
- Image Compression Coding SchemesUploaded byresmi_ng
- 13imagecompression-120321055027-phpapp02.pptxUploaded byTripathi Vina
- UNIT III.pptUploaded byImran Basha
- C#.NET 2005 BasicsUploaded bysanoj
- NotificationUploaded byssankar451

- Huffman Coding using MATLAB (PoojaS)Uploaded byPooja S
- Two Marks AnswersUploaded byAnonymous lt2LFZH
- Range CoderUploaded byftwilliam
- Shayeghi AlaUploaded byAlok Thakkar
- CS1002Uploaded bybalabasker
- 16 GreedyUploaded byvimal47089
- String CompressionUploaded byvatarpl
- Information Theory and EntropyUploaded bychintar2
- THEORY OF CODESUploaded byWayan Bentar
- MIT6_02F12_chap03Uploaded bygautruc408
- huffman codingUploaded byZohreh Pashazadeh
- Chapter Three New (2)Uploaded byayadman
- Compression & DecompressionUploaded byAnurag Patsariya
- Information TheoryUploaded byHarshal Ambatkar
- EntropyUploaded byanialrakita
- Source CodesUploaded byVikashKumarGupta
- Ada Note HkyrsUploaded byPratik Petkar
- CompressionUploaded byPriyanka Anand
- 4-LosslessCompressionUploaded byAnonymous PDGLsjN1Q5
- 10.1.1.83.5317Uploaded byEvalboite Eval Boite
- DAAUploaded bySrini Vas
- Lecture_7_fUploaded byhab
- Defination of CodesUploaded byblue
- Lempel_ - HammingUploaded bydavidkhoi
- Fixed Code GoodUploaded byNelson Raja
- CdngCryptgrphyUploaded byfifster
- Ch_03 Huffman & Extended Huffman (PPT)Uploaded byHermandeep Kaur
- Ec1009 Digital Image ProcessingUploaded byanon-694871
- Image Compression by Ezw Combining Huffman and Arithmetic EncoderUploaded byIAEME Publication
- Rfc 7932Uploaded bydarezzo.guido