- Quick Start Guide de En
- SAP INT
- 37361131_APC_AddOns_en_2
- Class-3
- TimeGlider Tutorial
- link3
- Suresh Document1
- Visual LISP™ Function Chart
- Tutorial Queue
- 1_coding and Decoding
- Aprendiendo-Unity-Guía-de-Programación-Unicode-Encoding-Conflict
- Add a Field to New Condition Table in Pricing
- UT Dallas Syllabus for cs2336.001 05s taught by George Steinhorst (csteinh)
- VR
- g.txt
- checkingin
- TURNITIN test for university student.pdf
- Traffic object detection literature survey
- PERT Networks Test
- What is a DMZ and How to Configure DMZ Host
- README.txt
- E Sanotorium Abstract
- 372SSL20V3 Service Master Components.pdf
- Prashant Thesis (Autosaved)
- Fanuc Encoder CalibrationEnglish
- Average With Indirects
- Linked Lists 02
- 100528
- AP_Linked-Check-3000-6000-001
- Master Pass
- mooc_mmse_lecture_2.pdf
- [B.P. Lathi] Solutions Manual for Modern Digital a(BookZZ.org)
- Multipath Channel Model Using DSP Techniques
- mooc_mmse_lecture_3.pdf
- Design of a Baseband Section for LTE-A Mobile Communication
- Shen
- 10_04_678_688
- RCL-3
- 2016-2017_DC_lecture_plan_objectives_outcomes.docx
- mooc_mmse_lecture_1o1.pdf
- S-88 2111 Guidelines
- mooc_mmse_lecture_5.pdf
- 2006-05-12-Ling43 CMOS Differential Amplifiers IIA
- mooc_mmse_lecture_4.pdf
- Sgn 2206 Lecture New 5
- Ark Neeraj A
- fdg
- Subcarrier Diversity Matlab
- Verilog LAB Introduction
- matlabgb
- EDC MAUALd
- plcc
- Fourierdc
- EDC UNIT 2
- Continuous Wave Modulationxxx
- matlazxc
- Ss Instruction Plan-1
- Verilog Lzdvbab 2
- AP Eng College List
- 01378070

Lecture 3: Source Coding Theory

Jörgen Ahlberg Div. of Sensor Technology Swedish Defence Research Agency (FOI)

Code trees and tree codes 3. Coding & codes 2. Optimal codes The source coding theorem .Outline 1.

Types of codes: – FIFO: Fixed input (i. # source symbols. fixed output. variable output. – FIVO: Fixed input. Should be comma-free. – VIVO: Variable input.e. . variable output.. – VIFO: Variable input.Part 1: Coding & codes Coding: To each sorce symbol (or group of symbols) we assign a codeword from an extended binary alphabet. fixed output (i.e. FIVO and VIVO are called variable length codes (VLC).. # code symbols).

a4} probabilities P(a1) = ½ P(a2) = ¼ P(a3) = P(a4) = 1/8. …. a1 a2 a3 a4 FIFO: 00 01 10 11 FIVO: 0 01 110 111 .Example Assume a memoryless source with alphabet A = {a1.

Four different classes Singular Non-singular Uniqely decodable Instantaneous a1 a2 a3 a4 0 0 0 0 0 010 01 10 10 0 Decoding probem: 010 could 00 10 mean a1a4 or a2 or a3a1. but the first symbol (a3 or a4) cannot be decoded until the third ’1’ arrives (Compare 11010 and 110010). 11 110 110 111 All codes Non-singular codes Decoding probem: 1100000000000000001… is uniqely decodable. Uniqely decodable Instantaneous .

. – Don’t care whether it’s a3 or a4.Data Compression Efficient codes utilize the following properties: – Uneven symbol probability – Inter-symbol dependence (memory source) – Acceptable distortion Examples: – The FIVO example – ”There’s always an a3 after a1”.

0 a1 0 1 1 1 a4 a2 0 a3 Tree codes are comma-free and instantaneous. …. our favourite example code {a1. a4} = {0. 10.Part 2: Code Trees and Tree Codes Consider. 111}. The codewords are the leaves in a code tree. 110. again. No codeword is a prefix of another! .

. it is possible to construct a code tree. if this is valid for a given set of codeword lengths.4) Conversely.Kraft’s Inequality For a uniqely decodable code with codeword lengths li we have (Proof: Sayood 2.

Kraft’s Inequality and Tree Codes If KI is valid for a set of codeword lengths. – The tree then has 2lmax leaves. – Place the codewords. – Let’s illustrate. . there is a tree code with such lengths. Proof: Create a maximal tree with the size from the longest codeword length lmax. cut the tree. and use KI to prove that there is enough leaves.

.. Possible whenever KI is valid. 3.. Place l2 in the tree.e.. i. 2lmax (1 – 2-l1) leaves remain. 2.and 2lmax – 2lmax – l1 = 2lmax (1 – 2-l1) leaves remain. Then 2lmax – l1 leaves disappear. 3} and {li} = {1. 2. . 2-l1 · 1. 3} ! Cannot be used: cut! Place l1 in the tree. . Then 2lmax (1 – 2-l1 – 2-l2) leaves remain. After placing N codeword lengths.lmax = 3 leads to this tree: l1 l2 l4 l3 Try with {li} = {1. 2.

Part 3: Optimal Codes Average codeword length [bits/codeword] Kraft’s Inequality Optimal codeword lengths ) the entropy limit is reached! But what about the integer constraints? li = – log pi is not always an integer! .

.The Source Coding Theorem Assume that the source X is memory-free. We have: We can come arbitrarily close to the entropy! . blocks of n symbols. and create the tree code for the extended source.e. i.

e. 1952) to create an optimal tree code! . the Huffman algorithm (D. p2 = 0.25.75 e = 1 Instead..75 ) l1 = log 4 = 2 l2 = d – log 0. use.Huffman.g.In Practice Two practical problems need to be solved: – Bit-assignment – The integer contraint Theoretically: Chose li = – log pi – Rounding up not always the best! – Example: Binary source p1 = 0.

. Kraft’s Inequality.Summary Coding: Assigning binary codewords to (blocks of) source symbols. The Source Coding Theorem. Tree code . Variable-length codes (VLC) and fixed-length codes. Instantaneous codes ½ Uniqely decodable codes ½ Non-singular codes ½ All codes Tree codes are instantaneous.

- Quick Start Guide de EnUploaded byJos Angel
- SAP INTUploaded byMani Kandan
- 37361131_APC_AddOns_en_2Uploaded byMateo Quispe
- Class-3Uploaded byPavan Kumar
- TimeGlider TutorialUploaded byDidina Ratiu
- link3Uploaded byraza887
- Suresh Document1Uploaded bySureshreddy Medapati
- Visual LISP™ Function ChartUploaded byCarlos Valle
- Tutorial QueueUploaded byAdila Azni
- 1_coding and DecodingUploaded byPratik Pasiyawala
- Aprendiendo-Unity-Guía-de-Programación-Unicode-Encoding-ConflictUploaded byMaloni Alcantara Jimenez
- Add a Field to New Condition Table in PricingUploaded byTan Jason
- UT Dallas Syllabus for cs2336.001 05s taught by George Steinhorst (csteinh)Uploaded byUT Dallas Provost's Technology Group
- VRUploaded bybenjamin_alwes
- g.txtUploaded byIvan Kokorin
- checkinginUploaded byapi-350941923
- TURNITIN test for university student.pdfUploaded byMohd Hairil Haikal
- Traffic object detection literature surveyUploaded byperumalvignesh
- PERT Networks TestUploaded by4gen_3
- What is a DMZ and How to Configure DMZ HostUploaded byzatiri
- README.txtUploaded byrenebart4459
- E Sanotorium AbstractUploaded byNatheem Safin
- 372SSL20V3 Service Master Components.pdfUploaded byAnthony Pozo
- Prashant Thesis (Autosaved)Uploaded byAnyun Sharma
- Fanuc Encoder CalibrationEnglishUploaded byLeonardo Borges
- Average With IndirectsUploaded byJuan Mendoza
- Linked Lists 02Uploaded byAnup Konnur
- 100528Uploaded byvol2no5
- AP_Linked-Check-3000-6000-001Uploaded byHHaui63
- Master PassUploaded byNikhil Gupta

- mooc_mmse_lecture_2.pdfUploaded bysrinivaskaredla
- [B.P. Lathi] Solutions Manual for Modern Digital a(BookZZ.org)Uploaded bysrinivaskaredla
- Multipath Channel Model Using DSP TechniquesUploaded bysrinivaskaredla
- mooc_mmse_lecture_3.pdfUploaded bysrinivaskaredla
- Design of a Baseband Section for LTE-A Mobile CommunicationUploaded bysrinivaskaredla
- ShenUploaded bysrinivaskaredla
- 10_04_678_688Uploaded bysrinivaskaredla
- RCL-3Uploaded bysrinivaskaredla
- 2016-2017_DC_lecture_plan_objectives_outcomes.docxUploaded bysrinivaskaredla
- mooc_mmse_lecture_1o1.pdfUploaded bysrinivaskaredla
- S-88 2111 GuidelinesUploaded bysrinivaskaredla
- mooc_mmse_lecture_5.pdfUploaded bysrinivaskaredla
- 2006-05-12-Ling43 CMOS Differential Amplifiers IIAUploaded bysrinivaskaredla
- mooc_mmse_lecture_4.pdfUploaded bysrinivaskaredla
- Sgn 2206 Lecture New 5Uploaded bysrinivaskaredla
- Ark Neeraj AUploaded bysrinivaskaredla
- fdgUploaded bysrinivaskaredla
- Subcarrier Diversity MatlabUploaded bysrinivaskaredla
- Verilog LAB IntroductionUploaded bysrinivaskaredla
- matlabgbUploaded bysrinivaskaredla
- EDC MAUALdUploaded bysrinivaskaredla
- plccUploaded bysrinivaskaredla
- FourierdcUploaded bysrinivaskaredla
- EDC UNIT 2Uploaded bysrinivaskaredla
- Continuous Wave ModulationxxxUploaded bysrinivaskaredla
- matlazxcUploaded bysrinivaskaredla
- Ss Instruction Plan-1Uploaded bysrinivaskaredla
- Verilog Lzdvbab 2Uploaded bysrinivaskaredla
- AP Eng College ListUploaded bysrinivaskaredla
- 01378070Uploaded bysrinivaskaredla