Hes) =f hog eJa) + @ oe. @) pace the £iqral is Ee a Sener ‘ag! ts treo Sy. Ts =Ltingx =2LX500 ¢s%y = 1000 symbols [acc ; Dformedion rode RB, =m HCs) = Love Sipthods x Lgua bite Sec Bret: @) Consider S$ =§s0,8) 783} vith dhatistice £03 peso y. colutteds 0} Enhopy oh Source, ety ¢ geoned onda. EAensin| aH dhe’ entropy +f Source Ls we by Hes) = a Pri dog 4 Ps +4¢s) = eae pols bogs +0. ees Jace) = elae bbe icq Spt I> dhe enbepy of Ue second aide, extensen 18 qoen by Hc) = LHCS) = ax beiae we | HCS) = 2.36056 bita/ma Sypolo| SUNILSHARAKANNANAVAR SG 5 \LEKUNDRIINSIITUTE OF TECHNOLOGY, BELAGAVI PAGE 46 (ssp afINFORMATION THEORY AND CODING INFORMATION THEORY. 10ECSS AA gowe amie one of Le four. probabls messages My; May Mg amd Mg auth probabilities of Te, Fe, ial respectively « Frnd the eotepy of the Source . Rat-oll Ucn fox the second extercion of this Lou. Hence ebow Hes')=aHt9) SOP + dhe enthopy of, the dovwrce Us given toy, 4 HC) = PCM) eS af i, PC) = ie tees tet tee fo [HOS = 1.7968 bre] rosy Symbol dhe decom} Ader (oxtenctin) ot of Lhe base Bouice will Inowe (4)* = +6 : : Symbot Probaleiltty A Paoboltbibe MyM, 49/256 Ma M, Te M, My 35] 256 MaMa Sipe ™, Ma 7/13 Mam We4 My M4. THe Ma Me gq MMi 25/258 vo Mehd The a Ne 25)asc MaMa! 5)ioe 1a. N15 B/12e M4 v2 Mg M4 Sfiae 2 me : ple la \64 the. entrepey of the acond eter sunsets st) =3,) Ste “ERGR) Ea B)@ +E) 25 + Mosiet Fea? +2 tS) O Hg lgedep inate 3.5924. Lott: msg & bo} See ‘OF TECHNOLOGY, BELAGAVI PAGE 47INFORMATION THEORY AND CODING __ INFORMATION THEORY soccss Ci) For the first order. Maslov Souice wot a gource alphabet | | S=$A,8.c} Shon inti» 1) Compu the probabilities tt} Bind Hes) and Hs"). P ool} s From the given Gate diagram, Lira £1ale equations One als PCA =p PCA) + pPCE) ———(1 Pce) =pPce) +p P(A) ————@) Polen P@ Be P(e) _—- Sie 2) Adding oll Ure egrottone we get PCAD+ PCB) HPCC) = ap [CPC +PCED+ PCO | “oop = => P= Now anbstitnts Pale in @ ase get , POA) SL PCAD +d5PCO P™ “ PCAS =i peed “ Peay ei i Pe) => PCA) =F) | Aa ausing Pally to @ , we get rap =$PCB) +5 PCAD - PCB) = PCA) ——_—— 8) Now veers Ke @ ond © wwe get Pca) =!PCB) = PO) But PCA)+PCB)+PCC) =I ® SUNILSHARAKANNANAVAR $G BALEKUNDIR INSTITUTE OF TECHNOLOGY, BELAGAVI PAGE AqINRURMAUN LHIBUIGY ANU COLING INFURMALIUN 1HTEAH0. ea PCA PCA -+ PCAD =) BPCAD =) PCA) = Vo Since Wt, PCA) = AB) =PCC) [era] st hag eal ety ole ies Ha “=n log Radoget Rel he = = Fe) Hig PCD Ha EPC He = (Jen +Usyen +H) 0) Aad, LenS =a HCs) a SUNILS HARAKANNANAVAR SS LERDNDRTTNSHIOTEOFTIC! GY BELAGAVI PAGE SpINFORMATION THEORY AND CODING INFORMATION THEORY Conetdan ine. ater Aiogsawn of Markey .£iurce .0f fqr i compile tha atale probolailities ayy Find entropy 4 cache stole 10> Cred tuber off lke coe ° AR f Fae the stale diagiom, Ura alalz equatms Con bee worttten. ou, PCA) =0-6 PCA) 40-5 PCO) —— 9) PCB) = 0-4 PCA) 40.5 PCD) ——@) Peep 7=0r6 PCO“ 0's PCB): ———-(3) PCD) =0.4 PCO + 0-5 PCB) ———y Faom 50), PCA) =0-6 PCA)-+0-5 PCD) PCA)—DEPCAD = 015 PCD) 0-4 PGS) = 0-5 PCD) pcs) =1.25 P(D) —=—6) Brbsttinte © in @ we get, bal d PLB) =©-4)¢r25) PCE) + 0-5/PCD), “F RCA = PoRD @ aunrbatitate © Jin ma get, O-4.PCC) = 05 PCD) “. Pec)? = 1-25 PCD) @) But PCA) +PCR) +P CCV+PCD) =1 ——&) Fuster gubetitrtrg 6) + @ Jin @ we gee 1-25 PCb) + PCD) 41.25 PCD) + PCD) =| a “495 PCD) = 1 PCD) 2 os eae =105) ate = PCA) = 1-25 xPCD) mes PCR) | Se) O.€ PAGE 5) SUNIL SHARAKANNANAVAR 5G BALEKUNDRI INSTITUTE OF TECHNOLOGY, BELAGAVIPr the entropy ef Me eouce Le ann 04 , ANFUKMALIUN THEUKY AND CODING: AINFUKMA LION LHEUKY. wees + PCB) =PCD) oe pce) = e+ PC) = 1.25 PCD) oP ae = S. Hi ej Pg Mt it u Ee es he ae Pacbops + Rnlepe Ha = 0.6 bog Yo.6 + 004. bug + O+O “+ [Ha =0.971 Steele ob ma , = Pex ton! +B y Oe Be Ma a ea, He =0O+0 1015 bog Yor +-0-5L0q Yo.8 Etats || ov mag defmabot | Ube, = ope 1 Ab He Rare Rena fe lags Rade = O+ OF 0-6 Log Yor. 0'4 Log Vong tt =0-47| bre alate Pr aasasoge Strat =Ru/ ! Hy = Rind +B glag he Foctag t+ Fandog = 0-5log1_ +05 ta to+0 « [FS Bie st} et Hc) =H= Pa Ps PCAD Ha + PCED Hp t+ PCO) He + PCD) He BNCMABIO +E (0-7) 40) ‘SUNIL SHARAKANNANAVAR §GB \CEKUNDRIINSTITUIE OF TECHNOLOGY, BELAGAVT PAGE 59. noiINFORMATION THEORY AND CODING INFORMATION THEORY. OECSS ae the first wider Maskell Medel ahoin in figs) find US Stade prcbabelebes , enbopy a cae ab od okey of U BOUL es aoe ie for etode & +! PCAD = O2PCA) +-0.2PCC) ———Q) fox etre B —>- PC&) =0.2PCey-+0-3Pcc) ———@) Role c < ee OE AEA) pene NO BRS) 05RD G) Enom Eq) we get, O-SPCA) = 0-2 RC) . PCA) = 1/4 PEC) Facn Eq @y we get- oe PCB) =0-3 PCO f * pea) = PCCD f © a bie howe cay 4 peep Rey =| LpeCY + BPE PCO = reese a “Fons | From EG 3@, we Free arose Sis, ee iFacm Ba ©) use ad, "pce =(S) PCO = = 3x s. “\Pce /, BELACAVI SUNILS HARAKANNANAVAR SG BALE Self From the Marko} meee ee buwemendiie suse ven boy FLAGAVI = PAGE 3,INFORMALION LHEUKY AND CULING ANEUKKMALIUN LHEUKY WEL pide nhepigy renehiietate de qiven by t=” Ps i Si qo! Bj for abide A, Ha = Pag re = etal Hee Toe be = Py ie = Foal ip =0 +00 leq. wo : B = Ornhue el ofy Rata He =, Fe beads =Peatog et Pee 4 Pec bet itr Oo Vaart OS bog Yous For atk 6, Fox str C:, dhe ed! Me Rource ia Ne by, — Pci) Hi . Ha +PCRDHe + PCO He = 2) (0-722) 1%) (0-792) 4G) a=) "SUNIL SHARAKANNANAVAR 5G B\TFKUNDRT OF TE VRELAGAVI PAGE 4.INFORMATION THEORY AND CODING INFORMATION THEORY @) For the frst ordes Maskeff elown din +90 hy Find tae atatonesy distribution i> Find enbopy ef, cock otale and srrdaary of OMe leet odjort &ouice and very HCOKHCE), GID: doen alate dingaam, Blake Sancbiont ow quran by TT fon. ashrthe A >. PCA) = 0:6 PCAD +0-3PCB) 0.2 Pee) o fer abate B —- PO) 0-4 PCA) -- O-5P(B) +-0-3 PCo) for alalc c—> PCC) = 0-4 PCA) 10-4 PB) +0-4-PCCD pce) —Peed =0-3PCR) —0.) POD 0-7 PCB) = 014. PED ; pce? =2 ip ae Menliphyting €95@ lay dae ges, @PCAD = LAPCA)+0-6PCBI+06POn ©) Muleplymng €9-@ oy 3 we SPO) = 0-6 PH O-GP CR) + tr CC).—_—6) Subtrading 2956 4300 Gg) ace, qe. 4 PCA) — BPC) =0-6 PCA) —0-6 POD 1.4-PCA) =a 4 ROD PCA) aie Pcc) ——— 2 oe Be PCA) + PCR)+PCO =I Babstbuding volmet, es PO+S PCOFPCC) =} Pc) UF +241) =) From Ey @, Pca) =4 xp) 29, x V4. 2. [Pcey = ae ‘SUNIL HARAKANNANAVAR — §G BALEKUNDRI INSTITUTE OF TECHNOLOGY, BELAGAVI PAGE 55From Eq +@, PCAD=!2 yb ea : [PCA = 3/y 1 The ae ma tale is eet hai Te ae Py Is fer stake A 5 Re D = Pa eg haley + A = o-6bog t +0-dbeg hy + 0-8 Log Yous Ha =1-371 bets /maq Symba] For Aires he Ragloy gb Pac gL =o. toy Yous 0-5 fog Yous 0-2 bog Yo..9 = ot eon 3 For stale Hes) =H = 8 pc Hy pcp + PCED Hy + PCC) He 37) +E Vos) (485) +4) (1-571) o ie 2s mace Sagmba| © aia Nelj @=HCs) = = PO) Og Yormey | = tOtey dont penet os st Pcoleg t. : 4295+ cmpsng 20, [IDs stance BD Hes) < HCE) Tea ‘SUNIL S HARAKANNANAVAR SGT \LEKUNDRI INSTITUTE OF TECHNOLOGY, BELAGAVI PAGE 5%j INFORMATION THEORY AND CODING INFORMATION THEORY. G} you ore aslerd to low om imfewnation splint Mie alee fer about 100 etudents putasith BE (E&CE) or Ceilinin Awiivertd Ver todadt xdagd Lean dean grove oa > Ge actscad fortisher stinclies rk Tete MEK oS Cut] dere iy dito foduubnes in India 7 C Bae the dot goran tales Geeiett e e gor the qe ir .BouLce sate ts Lie aiaage oe ee. ln sme pi ott erat fd une ie Ll =e eee dts rr From stale "ce stale 6 wore Fon Shoda A> PCA) = 4 AL pcm) +d PCB +4 KO —_—wH for state B —+ PCB) = i PCE +4 Pea — 8 Fer stile ¢—> PCCD = YaROI+ Ye PCAD == ———B) From EXO, peay =d[rew + PeeTt IO) = *[TPCA) = Yo | ‘SUNILSHARAKANNANAVAR —$G BALEKUNDRI INSTITUTE OF TECHNOLOGY, BELAGAVI PAGE SFPCB) = yy teer+d ('4) . From @ we ger, oe PO) +4 Ce) ina ia pec Ha = Pra log gi + Pan bey tat Ph HAP. = bega +14 bogey, bog Hy a5 be meg fyb} ot bls He = Pon Taq Vinx + Palog Ypge't Pac tog f +H = bloga +, beg2 + 0/ ond mpgs Rex leg Te ate te (io dogat +. 04a log 2 1 = 1 bit] meq -€ymbe/ et alate Re carra iy eee ~given fay, HS on Pte = PCR) thy PCB) Ha + PCO) He = (iD (5) HA) +I) ieee, SUNIL SHARAKANNANAVAR 5G BALEKUNDRI INSTITUTE OF TECHNOLOGY, DELAGAVI PAGE SGINFORMATION THEORY AND CODING INFORMATION oo red rng of 0- Cotleebsr Le want to design oO dylan fe repost” Ue Sy npatesarche hacked a ty beagle Se (2), duvet GENS: turning 7 CR) heads et 4 ee ert & fofor a be Fronted lids qr ba ee Sura aerdoien poe ittce Source us wen ye nq folewel, LEP Cons p> On the owenage, Patt vie Lome ry st ieee twexe Eurrng ight by Orctof ase cone Ua reported Ahead fat er Pested, 100 ro) (av 9) reported fob i pey “so Of them repork repork aia oe eae card 50 Her) 2 vight duis pene ee a stn 3H Pete ees ont of (00 Cay iileiy Ie + p pend , 60 ee pode the ne ee yrs BN ry os gt hie rs Snbosagrent epetting pestabe. ee sett: Flock atten OS Tren eteetn Dre qe =e y PORD= fy gy POD = yy “{5) a ee. Cty. POD Pode — Ka Fig: Source Model SUNIL SHARAKANNANAVAR §G BALFKUNDRI INSTITUTE OF TECHNOLOGY, BELAGAVI PONT Aeoded PAGE 59INFOKMALION BROKE AND CUUING IINEUKMALLUN LAEUKY the enkiopy 2, coach etek & , Hiei leas =F lope Pa log de + Paton =f lego + + 1092 +0 Hy = By bog he + he bog ts + Bobea, eer ts, hit Hy = bo ore / msg Seymbe} ps stale te = Ridge 1 Belo Bla. . oy = Og le dele Hy = 1 bit] mq &yrmbol or shaken |” Now, eS rs eae ne Nests SEP, Nt q = PH + PHat Pols = UQOt DOS) + Cy Hl = 1685 bile fata Syprbo} . Informedion rote 5 Ry Pied tte evbsepy H of Ate Bousce « ie S-Find. G2 Berd Ga On verity G.2Gn7y 7H aly ‘ae BA) z Seley Faem Uke Sobre Azogicn vee leo fig) ie alBlr equalow, ies hao usatthen wos, ‘ po = 2Pu) Hae ——€_14) POD = 3. Pon +k ro ———@ tO yay _ z bec = Poe og POD BADE ee: Ey & vomited A Pt PO) = Ay PC2) Hue, Pa) + Pee) =! Ba Ple) Ff PG2) =! ¢ Frew EGG | POV ES x [rome] 2 te = Zt bop fox gbnte 2 Seo + ay Fideg- + Fis, oq = ot bog +h Sogo “Tay = 041g bts /rnsq syrobel Hy = Rade fey, + hha boy Vhs fox elabew, yy ep eS we [Hi = Or en3 tote [rosy Sypnbo| [LSHARAKANNANAVAR SG BALEKUNDRI INSTITUTE OF elINFORMATION THEORY AND CODING INFORMATION THEORY IOECSS. ce Enhopy of the Bovies te grven buy Hae Ran SPO EPH, s 2 Co-a es) +4 (0-¢113) | ce = oats Arts [msg sugmloo| ty To Jind @,, Ga and to On, Lis reaprized bs Arcus e tree diagno « stdin cheno Xx Pea 4/63 Van Wig Vie “th By Slur ahi A163 2163 ‘leq . Yee Via alas fur Sle (a) YyVY at|ue Fg: 2 Tree fue igi sheng feast ts Wise ks for tte Masketf SUNILSHARAKANNANAVAR — § G BALEKUNDRI INSTITUTE OF TECHNOLOGY, BELAGAVI PAGE 62__ INFORMATION THEORY AND CODING INFORMATION THEORY __ Nov At tie ond of Aft pba Sotewal pias «tl S=keyz} 5 POD = “2 prose 5 °Pe apres Pye a a OR) ble Know Mot, 2 Qn =p = Eee “Foe fo N=I, Gu HH) = z pCret) og ie = Gi 4S) = 2 bog t+8 Sng t +2 oq E "1G, =H(S) = St bits /rnsq atgrobo| Letewel ts =f xx, xz, 22, 27/ZR ZY} salt ppebithes, P Go) = (aye Cay Cid = 415) PKA) = 6/7) (8) (V3) # 2/21 o, Pe (p08) A) + Gg) U8 Yaar aN. PGY) = (28) (24) = The P Gx =Gi) CHE) = ay PCy2) = 4) Cla) C14) Ss POM) GNC Gy = tbs The atopy ef the Our od thio. errd Of Recon Syrbd HOS) = Fedeg 3h (Spt O45 se logo es bah “pes A, deg 22. SAH?) = 2s SAFE Brba]rosg degre far Nea -5 G = HS) 2 a.ci7e 2 x : + [Gy = usoey brte/rneg Ryrmboof we SUNILSHARAKANNANAVAR 5G DALEKUNDRI INSITIUIE OF TECHNOLOGY, BELAGAVI FACE G3,INFORMATION THEORY AND CODING INFORMATION THEORY WBCSS. Wy » ae Lerncspordding prebabdrtes Are Colenloted ov, xXx > F163 xx ale? xx 4163 axe > 2168 ace Me Vze22 —> et Z2 t eer eb ee te Y AZB— plete yz alte ZyZ > Se Y¥22—> (os Zyy Ua yy2 Ffnr Sma 222 vepredts tusice go, wr, sb oe a medsfied 08 277.24, +34 aa = Ho )=Stee ete] ar “a pleat is Se HAs LS) © Hone tenlaslor ghPr 2 is «|Hce?)< 3.523 bde/maq Syreel_ A Now, Fa N=B joe dares. fae Be 3 2 3533 = a ij Ga = bt78 Ate [neq 2qmbo| St Lom be easily lsemed Urol, ‘SUNILS HARAKANNANAVAR SG BALEKUNDRI INSTITUTE OF TECHNOLOGY, BELAGAVI PAGE Z4Hq 4 Find INFORMATION THEORY AND CODING @) Pos the frat order Moskov Rouice shown in ow Ure tee diag sam a 30 sipabet Inderal « Assume: PCO =PO)=POdsL ind Gj, ond G2 Ory VA Sd}: even PCD=PQ)=PO) 3. dhe ee Spt onc A INFORMATION THEORY ot Mee end. qi742 74 = hese belsio Ek a Spl pst 4 $ ale @ rY shaea ‘stale ed thee} of Trlewal _Sersnd_ it ‘SUNIL 5 HARAKANNANAVAR 4 Indialobite 'S G DALERUNDRI INSTITUTE OF TECHNOLOGY, BELAGAVI PACE GSINFUKMALIUN J HEUIY ANU CODING AINPURMA 110 1 HBUKY aweLs> fer st atale, Fo 2 atale , ; De ee Es given tur, == Pa ae Je! i males Cig Hy = Uabege tebe: te toa Hy = 5 bef] aq symbol iy =bilap et Bo loge + Bole aa ee +. bog for Satie, Shan, Recon €4 Boater / Hes) qi by Wet, POxx) = PCYY)=PG2) =e Fam tree Arogsam P cry) = POD) = PCY) = PCya) = an) = Pox) =Y 2 PUNE 5 Gp = HC) = ez, Cond) bog Veer’) +, cennpormng vals tat, jue Lam conchnda tral ‘SUNIL SHARAKANNANAVAR §GB.\TERUNDRI INSTITUTE OF TECHNOLOGY, BELAGAVI PAGE ¢ “lg yi L Ne 4 Yo bog? HC) = = Pts =O Nasa” D)AEOD = rte 46x ae 16s = 3.085 bite /mag syria} i742 7H