Scribd Logo
Upload

Welcome to Scribd!

  • Itc Notes Compress

    Uploaded by

    Aniket Mane
    13 views20 pages

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document

    Copyright:

    © All Rights Reserved
    Download as PDF or read online from Scribd
    Flag for inappropriate content
    13 views20 pages

    Itc Notes Compress

    Uploaded by

    Aniket Mane

    Copyright:

    © All Rights Reserved
    Download as PDF or read online from Scribd
    Flag for inappropriate content
    of 20
    -NEORMATION THEORY AND CODING __INFORMA TON THEOHY ___10£C55_ sy Tp receiver Knows the message boeing Lransettted ,the cuncut: ‘of inforresbion Lone 12 Zero. | thD] DT, ts the talormation coursed roessage £1, ‘ind ig! a jar veasried seep orvauck af folevenin coxred to &, and gy te "I,+7,". as ; there cure. Ha aN equally Like yy messages , Leah "Eoeonatin Louried by each message asill bo N bite, Ddthe bnory Sepmbele 0’ And “L' axe Hronavitted tutlhy probeliies Va. on 7 Ca e Find the Corresperag Sell Informatemn: S42 self Information tn a0 == tee = bony yt = a bile Thus tk con toe basa} Rok mrte Information Js | corvied doy 0 Seas Lilaly message « | @) colomlecte ths Ormoucl ol Injormebion if bimasy digits (Limite ) cour with epral Akelibioed iy Lirasy Poms La AEE RA AK FU ERRINORT WT oF GLOGT ECKGRWT PRE 'G______INFORMATION THEORY __10RC3s & tle know thak in binary PCM , there cre -only tue levels, Fe 4 OF O. Sine they Occur comfy equal Aihelh their probabiLittes { occurrence sill be Rak ard Baa | Hence oxount of Information Corred Se given by , ve d jond diy ee eis BEER poe e Ma HE +1 =tye 5 T= tego 3.1 Bp Ad, leg 2 & C3 ere Cure. equally Likely cond Aideperckel tressages, Ui) oF i th ce lal buat ‘cowmied bby [te = 0 bis | Tienes | ‘SUNIL S HARAKANNANAVAR SG BALEKUNDRI INSTITUTE OF TECHNOLOGY, BELAGAVI PAGE 6 Lu THEORY _______+__10F C55. Prowe te following statement, "I receives. know Me poe Je being ae a moun ef information courted te" Jose”. obs Here et te ahaked that wecelver “known” Uke othe | Dre messoge ta transmitted . Here probobt will be Roel. ehormation courted doy tris ml means af oceunrente op th Dheropore , the panourk of of message js, Ik = Cr: | | = oq, + i ae-the ied by message} Ord wae . ; = ea a m, Ua prove arr + ce te Syotonet Bol Nepean awte mm & uh vb aebreNotessin. aleiee Leche, the crear, ifomabon Ce “| teehee a the bobviduol areounke: carved doy ene ay mam ty = bey and 1, = ley (+) ; “Informadian coured dua ts m, ond mf, @ 1 Se baer) | = tage (GC | SUNICS HARAKARNANAVAR 5G BALEKUNDRT INSTITUTE OF TECHNOLOGY, BELAGAVI PAGE INFORMATION THEORY AND CODING _ ___wrormanion tHtoRy > Ly =o, (4) t one) ; ee From) eq @), [Seca MUR awdunation se pachabilt BEES Rll pcan tp “ fea in studied ,is 0.4 . the prebadbileby bs n the examination filed - mu a Al gaume that the prchalrlelir 4 the ela A exam 2 0-6 beet Gabe Brnourt of ar “ge Yecoive. if ayou ose told teal the ue: mae _ exammation - Fusther, wiht Saget TT fe mt byou veceive ,i{, 44ou Oat told | a es ae: “| tics event tab shadeot | th Ther pC A/e) = ae tochioatet alin pases an cae given PAB = ee oe [ap ment ao CP. consaterG. dtecvde memoryless Sbusle' ctrl ovstpts ue odo Here. Dts Source ert t. tm, abr foto Kod B’ ae og condvilowk owe Ba Kevypose Atatelte two Soucy usthie The SOU C “are Peep bal tx APeciofomnelo) content of each Orlpylr from The Aomae c's SoS: Source A ond B ens one biteosh. Hence psokabaltier ap occureece co One bit at asl Loa, PCA) = ond P(B)=t Since pecd ik ie boy, pce) = POD x PCB) =x} = [Therepores, the foformat Content {som souree'c’ = mnt LCC) = hogy Fey = eat . ve face =a bie SUNIL SHARAKANNANAVAR = §G BALEKUNDRI INSTITUTE OF TECHNOLOGY, BELAGAVI PAGE q CODING __ INFORMATION THEORY — ©) Consider. a bt Source actuch fosces 2 fel.o9 wand Crdpatt ot’ tho head Oppo ond o Oy o-4eul poss that te the wee Conk ef ech fom Hh Louteg Now, thok the dow £4 4, What then te Hthe £0lf- information of an mbit dolock ee Mike outce + $3: There vox foo Outpt “A cund'O from Ue 45urce, sa} cue equally Likely, Hence pacbability efcach ontput i, | =L ard podab PO) => Me Bet Hee K=0,1 Hente, Infowostion Content of each -ovtpuit- iss, i - ~ & | ~]T =1 ee f of ine pe poe, block » tar pachailita of euch east POX) Eb xd xe Bs seers | the Lolornetion conterst -of Bk ei | » 160 =the +/Lcx) =m bie | * Tushi{ieatfen - fo Lecter te mieasuse 1c 4, Infoimeden, va Logosittren © -expresetion fs chosen fer ey es because fF al eloung ALALOMA + SUNILSHARAKANNANAVAR _ 5G BALEKUNDRI INSTITUTE OF TECHNOLOGY. BELAGAVT PAGE iO Yi The informatson content or 0h 20) ormalsn bf Onn : Cannot be negative « Each message mmast £ontotn cedatn aroount ef. information 2 | the Lowest posaible 2lf information i “zero” achich, leccuure for o dure event Bhiote PAu Everk) =| + eq: wero-tevel Tok” ; More information 9 cowed by oo ders Mikely message. 43, sthen. fire leat syrobolt Dre transottled the tobal self iawentrce) be eqvral to the dum of individual Boll fnformension « -G | To prove thir, fet us Consider that ten itdeperdent symbole "gc ard st” ore transmiled 07th probable Pr ord Pr respectively « dhen See data talf—trforednlson te qher by _ _ . 2 Pe othe (Seo 81) Tuy = ce PCsk). Pp C82) a oa : = beer dy => Pa Ey, Herte the proof. = | OF TECHNOLOGY, BELAGAVI PAGE | dt INEQRMATION THEORY AND CODING __ INFORMATION THEORY ECS € * Lomment en the mah Certeol! othe fellow on Ng Tererrow the. Steg Tite from eas oF ny Hoauill &now fn Bangalore dts wintes ht the plone weil) a in the vaxt One ous. So}: loformedion Lorvtenk of tke Ip ake. fret atntarned dow nat camy- omy nahormabion Bote Ur te ture Urol 2 rises A50m cout: See ata Seale eatatopy ake ib Kouries Lede mogligible Lalo ie | Dye a ae TY In the whiker g2aten enow fallin B sa ide pee thes evet tz Very Pate « So UP Lowriel Large omvoditasltol ! i> aha Arne atetanenk piel albedo stg te UC ce Anour itt doetmet--wWardron exact a wt gee aenateetoe ti Hone Lee moderate roprinalten NT OF SYMBOLS IN eee ms oo INDEPEN DENT SEQUENCES z AAR Petar Consider o ee neces Bourne p findepa oe ee AL Siyrel ALTE ae SEAS, G++, 54} ——— =e Rae es Papell SUNILS HARAKANNANAVAR 5G JALEKUNDRIINSTITUTEOF TECHNOLOGY -BETAGAVT PACE 1 pris, ne Sveeiancrer “yl Syprbels « dhs long sequence than Levtatns | Pe nawerbe ef message of Pipe Si, tea waives: ol Rebaagen ay Aare &, | ant ye ree eg ap tikt, the £ell Informebim aS, bys bite oe PL number ef: messages ey ep Lt ta of a re RL rvober Of Mesengex aL types Conley Geis ate dhe total ne ARR GH 4 Mere er sxpobals ts given by , T hotel eel opyeeia eT: op Py loge bie vo THolal SL Plog ke : ond Average Lill -inhormebon = Eich) | ein i kts pt x ir £ell—Informatony 42 also Sa ” of source's! dercled by +1CS) . » ne) = 3B Pg] t/a ene [_, MOBMATION THEORY AND CODING ___INFORMATION T11EOR} -I0ECSS_ Thus .H¢s) represent the” scala doky "on the | Toverage eumount-of Surpsise’ of. the Source . [ULustrocion 4% Racin ens et 4 concidey a binary source voit Louxe yer eal achastna es te! 322} |The, Entropy Hes) = = (Ploy aes se 256 4+ 255° am 256 : wo | HCD = o, 103% bile/msq sai Pe the om ie AMiCadcuby fe Ve 2 Ratrond!i. | "lich Vo ty ee those 8d Ret 6'=fs5,5q ) ath Pate) then, Enhropy Hs!) = 1 HES) = 9,994. boite/ meq &qpobel In Hoe Cone, Ht fe gy Yrs schalher 6 ws 84 tetrorind (titre tion 3 | OER ee5, cc} wth pS fh AT ety amy a Ith eas St ee obech S4mbd ie teasly ‘SUNIL S HARAKANNANAVAR SG sea INSTITUTE OF TECHNOLOGY, BELAGAVT PAGE ve TON THEORY. __108¢85__ SE veprecerte rast ee ee he ae fgom S= £8,539 sy Successive Symbols ous = ld tig te He Net raracdione al aae bahay ney Ba ipa ad Ma den bag lA Yero-Memony” Leuices, (NeeRneTion Raia Ret 9 Bhat tke 032 anvitled loeleds efor vl seine vi ee tre ioforn wy a * "| bite sec os BPS P| Consider 0 Source S=(8,) Sy, Ssh with P= is a Find @) 4elf Informatpn of each message (&) Brtvopy of, 2ource ‘Ss’, | Pg sep information 8) = T, = Loy = foqd = 1 bis | self-Infermon of S = t= leq. = logy = a bk | selL Informatn of Sy = = = leg = log = o& bile SUNIL'S HARAKANNANAVAR 5 G BALEKUNDHI INSTITUTE OF TECHNOLOGY, BETAGAVT PAGE 1S ‘ - INFORMATION THEORY AND CODING ___ INFORMATION THEORY __ —_ ——_l0EGSs. B). Average Information Content oy Entropy: -f given by, | Hes) = SRT; = Pry Paty + Pedy int =(HO+tLO) tse @. | © | HOS) = 6s brle/meq 2ymbel the collector voltage 4h 0. Rextrin crrouik is de die hoebisen | 6 ound —12voltz «dhe voltage cow take on nly-these valys -5 6,7, —Aycl, -12 volte -ansth ps v4 uth= ve fe Vecored tra iteupt bot thes oltage zaled sith retordes.. Detemme the cwerage +l tnfowntim s0.ss0c d 10) Le fecoid tn berms of bite / Leonel » 7 Gens q eons ple Hy eter}. Now , Enbeopy, BH Wenge i ae on +4¢8) Ee hilage fe Mat ten | eee es. er apt Rouse ome one four Go) &) $9 And 84 wilh et prcbobiltties dob awd tp vespeckvely edhe snccersve Haenboly of tke So1uue He by the Lousce cue gtoctictically, tardependent: Colonel SB She entropy-of ta source Lh ofven by SUNIL SHARAKANNANAVAR $C BATEKUNDRIINSTITUTE OF TECHNOLOGY, BELAGAVI PAGE 16 JON THEORY ECS 3 m= 2ZP glee es = Er bog = blog +R boy + Pade +B tog =Saovt pi ae ee = 0-5292-4/0-43088'+0-5 40-5 zi HGS) = 4.95914. bite /meq eymbel HA Source tothe a ee es apa he each. 2% le intenal . & symb ACMA Ast prob-tattes OL peor | helow: Find the amount of fnfarmaken gained by obser os en Hing each-of, Woah el oy the entropy of ung. SES the 2elf—lnformatm"T. t2 qrven by Ea =ley, = bite: @ Sioa we have four Lymbelt, Bo K=0,1,2,3 Me when K=0 , Ty «beg, 4 = log 3, = 1-392 bie Kal) , = bey, p= lop Yous = 797 bite k=2) Ty = log, Yp, = Neg, Yow = t1322 be K=3, 5 Sle, =te9, Yo.) = Baar bile SUNILS HARAKANNANAVAR §( BALEKUNDRI INSTITUTE OF TECHNOLOGY, BELAGAVI PAGE 477 ——INFORMATION THEORY ——IWECSS_ dhe eitsopy -of Mia Louice te aiven boy HX) =H, Pe teg t +t / msg. eyo =e Ly Dy eh Keo SRE tA tN +hIs =0-4-(1-322) 0-3 (073]) + 0.2(2:922)+04 (e202) ee pee Lowes 2M One OL &ik 9} PVE eA m-4ec « athe sepmbol, prebalilttic: ort ro. ek and | a respectively. Prod the Kousce Aoifopy end Inforwkgy rate | 3) 3 . Sum shor As given by HCs) = = Py deg & a sherpa de kino Infermedsony Rate Re is given hy , Re = ihe HCD here qiver that; T= 10% Bec bel 1.9395 bite Eng sped Rs =1 UTE OF TECHNOLOGY, BELAGAVI PAGE 1g | the De eect cs ), a a Veg and Lie At Yose* “Whe seme a scab cpaianlebiacs Masaniing Uke. Ose deseo erdlepenslertly, ed Ure average Informalion abe of ls Cour. soto: dhe ethepy di given ay, oie. =e Bete ’ # Bate rhe laa = Slope] con + ha" Gven_%_= 2000 my Syerbelal sec\ ' | ‘ ie eee gion by Rete He) wd * = 2000 x 69413 bile | cara ah | | etx 1 pace Hs Bea AE 5 a fe the ef of xX 2 Selft Since a re we cama Pete ce erat Dba tet ong Be eon Lowest nae tononw 8 x= n P q “bike, PeX=x) = ‘cpr qe te Event. SUNILSHARAKANNANAVAR § G BATEKUNDRIIN ;INEORMATION THEORY AND CODING ___ INFORMATION THEORY __ t fol coin te terred rz «bot A= fement of, B= nae oe Event & Comey rove Coen Anfecmetion compared 3 buadrentef 5 Haile a wheal y ls elimation ¢ te peay=PQ=a) = G @8) (08)"" = 0-305 POR) = LGB) = Ce way Ce: oe oaie7s « edad Pestle = = hog, PCA) SE insaee | ; + ~ t Hol elprforenbe Or = Ty ag E>. > O-2184S TT, =2-143 bets UTEOF TECHNOLOGY, BETAGAVT PAGE op oe Penida eet ise cae oh Oe i Pot acs the va Vvasi oft Dumber of heads | Dawmbes_ of tstoll Pal gg and wos pa 0:> {hay =fos, eas seal a | peer reemeratianee | | i seohalbility of occurence. Palenlate beeen i Pessina ' pete many il -dash code. dhe ee vedere ae a. clot laste for iort-

    You might also like