oste08 Motus * 4 @ “INFORMATION THEORY GinYEe Gm) aw Measune ae © a, ec . fn iyermalin sowie gerne “Oh ier, olowkythb'empunt of ixpmakin angel Ay be Sart Glypuent: jor dyson syle, OF Omi. A mathenahiol mod con e'ppemaiatat te qpantiyy Wu amount of | . cmnatin omy by a signal, symbols - tote mente + ae - Gein, 7) T wid ey ca Ago pice. tne wor my il Ryn, dt il i ocomuence q saints) bub Nag Bi stoners eps He CGN og iatn omega by ans BM etotement woke monk iz 0 comma event, the and me The at stot Gonte egrde of gureprive'- Pee RO eal a a dans ak the nau of iypmaon creyed hy 2 ONO 3 fron 4 tho Ovawienct - a unly pr 1 ch he ibaa oak usp te ant pron xl ya of gies bY tke tds qe # ™ qe fe aye Pe fe te — “4 _PRopernies : . t 5 phe pala comcyelty "abeage conot be sa 8 Teor ies) i" Shs i lly ad yu watt GO defile tok De poe oj casa BAS out ts tpakin ceed will Je ot | Ret K > T= ly ° a ‘Scanned with CamScannerPhe patin y compote, clatenent,, ualich i udependent Ben by the sum of iuindaol ey, irfovmoticr LCM, my = Lm) +L D the eset lai ae te a eine tem 5 T+ ty, (1 ee oe: "i cote == tay (fe) ule YY y-2, ik bt | Hy vB sunt wale | Qh v= 10, wate Molly Cod pets, Proaems ‘ probolitin | Gidea 6 a ig 8 spoils 0, & vid, poops or and 0-4 + ope coneyer Oy, Bit Ocal bet. Seton lif ett ey, (qe) Te Jone Ce = 0-780 ie - ly, is Ye = 1-309 Dib J) Cuil’ 6 cue erithay Symbol, Ss and Sy iL coonspoti pees prshobelcn 3p, and H,. Fad Be 8 inpsotin of te, hib af ba (1) tb on (i Hoathy X° i _ bes ‘Scanned with CamScanner, Tae de (qin ow 4a) @ LED co Ay Ba 29,68), ys deuE) (RNY C 13 A) 1, 2) Cha) Is- lege (35, Ye +n 160 bilr tae ty ( Gee O] Hovey Ta = Me ( az) Hail = one ey Zs = bei +) Hasty = 0 600» Hoste q ne . ; / Fo. =; bog, (3), = sett tr Ty > doje ( Uis) “nob = +3863. node SYA pi of che ix alla sulla woutly y e,,outeome, | he et one i contidiad be X ot dud one OF Yi. 3 evunir ave itentsid (P= 6 Cmaps Ca¥) te exadly dil ty 3 (HD B= SCx,y) 2 Gey) tran ume cuier ke Dixy (my ao FJ. Find Oo probaly and Av jyormnabms —convyed by each one Mate $= 200 0,9 0,9 G8 (1d Ge a ee pre 8 Ca» $918), 64D, YB, QO Gd G2), (8,3, iW), GP G0 | C0 Gad, Cd, GD GD, CHO _ i Scanned with CamScanner(5.0, 69,63, 6Y (59 6, (60, G9: Gi) Cl. 19 Jal , O pe] aren, ance, 4.d,Ca OGD, 6.9, (5.9 (5p If. 104 fp +m = woes % mA Gey hei ha) Sih = eye (i) O= } OH) 013,09, G0 , Oo, 9,6, 0, 5), 4.2) 6,49) (4,6) 206.15, (613), (4.9), 626. Co 2] Fae" ag op Wy Sai i 6 8b Bs x qe = 1k “a> My Ga)” ae GD & {es a9, (3, » aay Gia) G0} aCe on aE Te dong (i) 2 ss) it ‘Scanned with CamScannerod}osisos @ sje, ERAGE FORMATION CONTENT CeNTROPY) oF HeBePE, SyMeoLs x J 3 , ain SGN HONG, INDEPENDENT SECUENCES CzeRo MEMORY ‘eaincey, (onider 0 Zee0 Memey Sree which, ericson of Une cant Sygtet t rot dependent en enivsabn 6 peuniou symbol ga Contes a. source entiig spmsca B= 4 8, sa, 4 : su} with respecbule proobelitier P= hehe ia ot hs Pa} tna meno centuhing L symbols; $) will -otun on an oven FL - Sidley Sq will opps RL tim and. 0 on Sere be Me sy a Thawte PL. tuber of gynbols axe prevent cnieh ose. oa Ang yore stat rppenabin conse iby symbcls type S wi? = fe *4a( F) | | Yee Ainley pr sy ee L dog () . | mS he a (oo) the Adal ipormabitn will be ST IY ag a bli) ‘Scanned with CamScanner« Nn py Sag Mol) ; Ine oxsoge infra coco ‘oj WON oti Le Aye Balad os ancyy denoted bf YES) MIM oa) Way= Tort 4 = ok a * N H(s) = Zr (Gr) 1 oyod Hla) oe dhe omg wii ot auuae orncunt ¢ eurpsie ite awinge iapmatiin sah — Ry fs dypudar LL Re = He HD) fit @) Bre war sty = Apotel, eal] baud rat — ym Gh dtexsh, ertow Loy souree enile 5 ambole dv odin - fhe symbol probaliduber axe { 0, nas, o1as 0.065, grtease fid Ge J vty ond, the. utoage ypten att te Sous ir N - he R(a)= a be bog Or Ails| wprtet i= b ; Hla) = a bo boa (- i \ Hay= 0-8 Sos a + O15 Mal 3) + wats by (oo) + 0068S Log 4 sa) 9+ 062 401 0696 L095 ( i 006d J i ‘Scanned with CamScanner !hie as OS + 0.3150 + 0.254025 0 ; nine bet 12160 is] ayotl | vata wall q he Source psoas suprmation ) hee Me H (8) a Sanh 2 S__ = abo mbt ate | il aon el ans | gscox H8150 fe oars delle a vg wb ti Re. ' o das “Hoag an inpemalini, Dune One tty 4 Jase ont maid a H te toh - . et eo 10,000 agrbiel) He « i Set yy. 2h OS) sae if y= 10,008 eyribels| ee oa 1a apices —— |] 256 Nelé0. 32. ayriets ——— "Tew i b H(g= 128 > e) ta ew ee 3 “Cd \t. Hie) = % Libs]aprteol wn fale of sOLitee - Rit)- Hy HLS) = 10,000 # Rin = 40,000 bb) ae ¢ 10 Kise Lec = “” | —_ & ‘Scanned with CamScanner(© A dich Sune omits ane 4 Be DE ge ee We Cytol palates om utp, Ye, hy , ha» Fd he aypmalin content of sare Hasty [cgnted & ele iM wipe Ze pr lope f- ; gk te) (*) Hordliy [aaprnbet H(t) a By thf = Mn) = he Fie “te ite + “ines AN pi he 7. Hh Og? el HH(s) = 6105 Karty Jaymlat \ Ci) nal cgrant ‘ s | Wo Ze prloge (f) | er ae ¢ = wifi WO Me ay Gg eS & it Sip, 6 he Cans oi 7; doje Che) er y[y2 +543? nol) rp: L (i) Cons ap we aymlds X,¥,2 wilh prada 0-59 ond espe Sand at Cpe cya by ah te syne ake ings wee Tx toy Gay Tye bad iby 5) aaa he ye boty (2 = L130 bik ‘Scanned with CamScannera aed Gee Tie Bette tz © | Tmt = 6:0589 des es é udalititar 08,023, O°2) j sprtale hah Jaw Hep y eae oh e a o.12, 01,006 ist leaitiNce | 19a by + onloga(se)t Hige 08 los (93, ane icc) . | + 006 Mia (sai) | 9-12 bia (Sa) + ol bis (5D) a 2 (( i) yr = 24088 bits laymbet B nin oy gn ii 2 ap Kon ¥ « atob Be probebdity O ie a xp Pat apo NCO vauus p- BE thee Rp Ger : : ba | Ht) = Ze: degs ( Fr) tet and 'Y oie Via one oh aymbels | pot a =P bs | nlo- Phy Gs) + (p> ts os ne ‘Scanned with CamScannerOe a | Ob 0-410 O14 OREN ot 0-104 of 046490 | 4 ya oye PK 4 ” 0-4 r og # x OT ot - a x Ok 03 02 on |) 001 01 0a ohos ob ov of of a ~—~COS =P ince tk & 0 Bcrory chomel the moxinun hain 2 we be Cartes hit, benno cleus sua of yal dlls ea dad bout ne of Ot CH if » ped hin mS dy ciowkeer visite o oak Yao Bat of a ln 0 (os eles E lathe C1) Saleubote the Ay Inprematicn of eee cow aug Jingo sondent ¥ aa wong dale 4 “anpormoli we tat hue* eae? Ag bat ‘Scanned with CamScanner\ | phentont fet = & ZY a2) . bub = 0:666% * Pk = oxgset w= Zh ee(E) ply = 0-668* Ly ere bop, (— "fs (Sean) Pe. Gaase) 3, bik agntel x fj na = og (oD Fan = Me (GA 3. Tt * in (oes 7 S Put ts (is) Rye He HO Gi ret ae Prlst.= a gyn pt ee je aedaynils __ \ | Ry = ami 7 : of 1 ho oe ea br be al = Uns te z fe = 3002, x ott + _!} aul bay as \ Boa SIRO ils poe r bas = 3493. 65 Bits Jace R v ee ‘Scanned with CamScanner@ Sune or pucedour Adauk ix spnosenbd by 3 ms and o dol by Lins, The puesbabilly y dad. 4 ba tat 4 dot . = Pat * a 2 dns = Lp Past + Le fast > | Hot~ 3 ~ Blas = . Tons eG) Lae = bay Ga 0.50 be a =o His)= es, br ony fies y= Mon S Mny (4) + bow WD = 9.8113 Indete a Asis deve demder = jong ff Sag + MO bales Ata lors ‘Scanned with CamScannerho we Ha) @ fy = BAe (ilo . a imu OF ENTROPY 3° i) grteopy a contoinus Jaren pusboility , thak te or pene . vail ba “anda. OO enkiopy pucker. Cer po sap pein @). ba 4 é funchen oy is angunenh hak, Enbiopy | ae ‘a pele ay Be onder th whtch he cyofele of ong gos fiat] ze ow wad dog, Dt Z hes. Hals) = z 409, 6 Libpymtol Hal) = \ see dak Hglé) = a + t logy Ut) ree 4o99 (4) wae 8 Jie pagel > Mal. Hela) yak - Upper fan fy [ eerremar pRoreRTY] & (onaiden ebechion sosults yt ble announced (ose) _g opualy ong eons sha acti) 4 confotnal ue vow stop ‘and the othes oud — expedin fp whe j Uae? yee moun ingpunctin , Gat @ erlery of aus a | HE mousuen. fr et-paiath ayats ee Proof : | - a ‘Scanned with CamScannervenie dogy N- Hid z 2 hyg H-Z bia Ge) gr P ~ Bia = ghey N ‘Scanned with CamScannerig egualion (8) Unig egaton bag n- HO= Eh Me 2) I fer ey, a N tog Zz h Me oe no dlo = = . an Mage = lyn to? bye z hi [In sl Sr 2 al L ic / t enn Me C2 i! ly \ 2 nA Figen (ee, - N z ee {2 sins) \ apne , ‘a }1 (2 mex an aoe pyuetty« a@ yeas ur Hol Npr the syibolscshould. Me sa pols y] Fe cbowee Born enkeopy F Zb0- Pegoty nly ee oo ee Doo a se Sh Ge Coneipmding propablihes p,, & ne eal © Liypese we Plit the Symtot sy Laks m symbole , such thal ‘Scanned with CamScannerSue Sup a Sway SN WHE prcbatctite fg Pia ide, Ta Thwane a ve A ' = DE Re A tiAh Se cst vcmn: Alig Zh )-O a \ HG@)= HP, P, CF, 2k, whey fay Prix Figs +» + 500 lb|ace i x / \ (@) A ouinile Fannin ofa ee ie, tie OMe spt" bx io! parte fom for a god weunsbuciois the unage ~ullead 8 agin a ave caseory « eau all ea ae oi Macy, pnd “the aussi walt proms a eaninatiad, zvemy 4 dec tw, Ho foal || ' Hestil ~ nx wl goes the ont omy = (8 sous H@imax. = Logg 4 Soratoly 2 op, (yo x10°) zich ir . wx! Mia 8 ; uaxi* © 212. “tla jeane= (8D fg = Wy pallte r @ Rye te HO = - e 2 Lm RRO if oe 3M Hiilec @ A Blak § while “W pic Conacith of 505. Linas L, piliwse dygsmabin « Pesume cour Jim Conte of Say pie Pr covert (jel and ta: ei ddenent Con have 5G ceginen Louels » Picture ase wopintid a} Hs pak of 40 Kono) gece Calcot die owecoge fle of ‘rman conueigd ‘Scanned with CamScannersnag xcas Mtg OD Maes = de0e xo fone | © 30 fone 7 ik a. He HS Ya0x that wowey A BookMarer Lebiver @ ey bifort herarees # hase, tie potewing pane a hoses TE oa | Ort | sesocod oa bye thee wncuuer o. merase that olue to inj wey one of aj te hate wo not poaticipabig i the vou. expla how 40 acter # al inpormaticn Unc oot g view , Uw ayormobient value a hore known ly erie De hove Brown « Cu joue Bot @ ‘nym uweel oP Knoucr, 0 could he AYB, 6, D@E - thoupre We cosresponsing ingosmodtam wow Le ‘Scanned with CamScanner18, Te, Do Te yupeetniely we Misti) In Me (as = e64a7 Rh Tye | at) pane bie [a= rose Me Je Mis ( San) 19918 = PA 216997 th | Te = 16897 ity Raley od nesar be [Te h bOiy 50.504 bik 230504 ite | Dea, 38h+ Te sae bi | > boy (sn) = 21h wi | Cote (i) Scie He Mowe Chat ge iyied 8 Pudkoam the only ip. a - o ‘che clay. Kor ZS boy cc) ae tn &3) | Hy = 0-04 by \ (hen O- + ea oe tia (+ \ ) | [sm yr LVRS 5 of YL EXTENSION OF ZERO MemoRY SCURCE ee = SS SS (onaides a x0 Memory Saree aniting a symbols Sond G with probolashts Fond fa epeelndly « Thayer the anita py ull ke ALD —_ ‘Scanned with CamScannera) =o ® pla= Pp, bial +)" a a(t ) 2 b 3 ie (onaties UE ind tenant nt ve , "pain 4 puile cor et 2 we ee SiSyy S181 SS) ySab~ wi pel re Phy RG ea epee” yo jo gre net Bee & foun wit A logy LLY le ths Sad & ay Bh °bF pe aie pe b(t)" ie Wid Re) +A ews). w= ah > bog (ae bein an) * — oe 2k” br (RY Hed = 2p, (*) Pe boy as + 2 hh Mia (Z\r he lta (4) wri= oh *e (A) [ no + ob tein (-F) [re | piers (har) [ oe, big GA) +8 fng-+)] Hs» =alht+h) ff boy, ter" toi] Pe ‘Scanned with CamScannerukr [Ath & Hl) > 2 [' bife)e Remy] [e@- a Hs) pr (e Conse Zeae Memow soursce, ening 2 Ayrnbole XL Z_ ulln respecluie probabutilier 0-67 0-25 LO-4 « (olulate CH tabeopy 0 Us sowsce (if) AL oe symbols & theie onvetponding pricuoldiius os cetond eetoatin , alto pnd the eatecpy (iii) Prove thot (s+) = 2 HL ® bun Bem tac A) = og be(2) $03 as) i ontlena ft ) | Hit) = 8455 ois [sy wa) P(AR) = 0:6 806 = O36 g. P (KY) = 0G Hod HOE PCKL) = 0°6 Od = 008 p(VR) = 03 €Ob = O18 7 PW) = 03"03 20-07 POD > OA xO} = 0103 P[D)= 0) 04 a0. of Z PLY) = 01 #95 = 9sp3 P(12) = 01% 041 = oo! ‘Scanned with CamScannerH [= Oo ble) + P(xy) 4a bea) + @ f (x2) la} ia) + p(yx) "2 Gm) + roles f) ' P(oNbigy/L) see (gat 0 fad a +P (22) boyy / (ga) ale) = 036 tia (sz) r out oy, (5) ov 0b logy i) + out boy iteyt 0.09 toa (es) + 0103 1h. (a) = 0b Wy [t) + ovo lta (sca) + oot bog, a) bat Oe 35+- Obysst 03027 t uff) = + 0-6306+ Om 010664 Ose + 02435 4 O-1518 [uw 7501 Bit) oyndal su) peat. w H(ay= 2 HOD Hla) = 429985 Biting! UU) = Qe5909 Dike /eymbol oe ]Hfat}= 8H top ces a | [3 cones each << ; Aasre eth Goraek X Drdnond) Heose ‘Scanned with CamScanner0 fn Signal (3 oll to GooHe, 8h damp at Hi 3 Neat Nati they Somplet. 6 a a int (D sevel , the aumtzakn dvb ee ul endtent , Z ate suite po tabled =he ja. ids Oe enforatsn GS sate. : wh? = oeip. ps Ho) Zhi -L fog, H()= 4 be od. logy fae i hag fie) +i bse) ils) = bens. Wile egy | Ri = HI) HE lB) = 2% 500 He = 1000H2 fe Nygutt Rate | es Rla)= (000 ¥ 18113 [Ho = en wat «27 bib) i 8 Pie EXTENSION! Source” Nix "Prove TAT wy H( 4%) = nttts) proor: + Grodin a, source wth gots $852, 0+ pocbabitdeht P= fF, fy. wilt hase "Symbol. oe ou Bh af oun, ee din es om f , , a Co) =fP Cs, eC) , . . teal Ne «ye Te 4 euiain ro iL 3 ‘Scanned with CamScannerZz (oi) = 2 s t=) PCS) plsia) PED... PCs) | N = Poe) = z els: ne Meo, plSis) — - Erte & t=) 2) yl wo ta led Ploiye a i Sa —0 We eatiepy R(s\= 2 Pei) lagy i Me — er) ae h(s =A eleiy * poms . s (sii Pst) PISigy. . PCSin) Ree & HED 2 ZB eed Bf Or ere to ereeers ee , . 4 -@® Z et) ens scaly ‘hi, "t_ Ss 5] * 2 ,) ‘ it X ] tZ hi “," Zr... * fobs. ‘Scanned with CamScanner{ fo mpot wnt Notes nw =A el ote deb dete = Hl) Senile - conuila he above welt i oe © H(s°)= bla Hien) > lay ‘Scanned with CamScannera Yael @S om Important Notes INERRGE SivFORMATION CONTENT IN LONG DeHEN DENT ; SEQUENCES: A dsputtonl pucbaliaithi: mudd ct wenpuad tp wodel sour tn whith fypntol temiaaion depanste on pests Ayer. « homoded which Or ured to swoposent Saurste wilh, memory a colle — Marrkoby (0%) Markov . Mi Jn genial Ww” oydex Markeg!s Seurrer y the emnittion 4% $ ot the uuwent civionk t dependent on ih prio Kn | Syl MARKOFF MODEL « * i tate A yt ith memony con be repitereniee mang & 8 i ‘ ible lates | diogom. A siste diogam capensis i oling wi fasion preobiltins how Poe We syrbl enitid by th carer coh one Ome shown ii the tab chaguasn « , te Tre dliagron con be consbinueol pom Ow. ane chidgroon th prebiir of sydd anit by Be Sousoe Can Ht . disamnuned. fyom ts Tiee diagram « . | fh pins age fg? cn did Consus 0. Tee-Diograr of *f' say: ‘ ENTROPY AND INFORMATION RATE ‘ Jub enbopy o| Be Stoke R! be denoted by . Hp «Jf con be bladud by conichening al Be ouFpnep probolbitis a ae bes n> Z; Pu ta Ge bib pnb | ‘Scanned with CamScannereer lt ye Nos ¢ doles (6) she oo the Sours 7 AVERAGE INFORMATION ALR SYMBOL * pers -gymntol wo. mewtoge of The can ‘ajrrmattn content fongh Ni given it Gee 2 & elm) ; ig a loge ne ( a) pila __ as at 4 emenge hae fenghs Ue + uf g") RCS Enreey 4 a SUSU + _ fia uci amount 4 saprmnabin, a. syten tn mesioges * foosuies ysis. o Crowne » pj ant will be leat equitte HI. G24. Gar Zula fin Gu = 4A en NO @ Pot si se ig a) eniapy 4 eath state 2 Tolat éntiepy . Ug Zz EK @ ; Wr) Ye 4s ‘Scanned with CamScannerRise | Pixs Zrayrt ray JH Lr) - 3 p(2)= 0 é 5 5 P= PS 1/0) = 32 0Q@) 4h BP). ERO + 3 PR)=0 PUY) + PQ 24 ‘= 43 = fee. Pare 5 SY a) To fb Hm BM 2 ~ Kos oe f Jo Ls \ > ge IAD, = (Ay) i) = aml) bape) £ bn B+ 4 ot uy ‘Scanned with CamScannerHe N = @ S fae loge (IL saat oe Pay te ( \ + Pa i = ala gr FG) eee M & rae z H= & Hh ; ke! He Rye Heo we oes 3. p+ otto l 5 ey hata STIPTE sy ze rtm In (as Te fied & ond Gy We nook to conruck the om» rR = A cyme0e2oymaor enGTED emrie8 ‘Scanned with CamScannereens ne, % 6 5 = ae Fa} " gia * al? + slp a Ir PUY) = Se 2 =o PO)= ha, P(2Y) = Sug raps fe a PO) = Aas POS) = ‘le, P(zi= “lee reas uo ptva= Sus ray)= : = Pom) Hepa (aaa) 4 Freon (2) Poni Es Pea fg 6, = s by (2 + = oy (4) 8 & bon (2) ee ee = LD Pw hogy C1 ae FORA) egy (ty te bay 3 VTL af) © 5a) sy G + PWD) deg, LN er toga 4 * ie ries | fly) Je, mS yt PLYY) gg! iN el ely! ‘Scanned with CamScannereal Gretta (ayy © 4 Gs ai bi #4) & leg, (i) +L boty uss + ©) boty (NS) + be ( se ba(UE)* big My mfg Mw CD * + hls By = 10594 Bete [aymil = nest hele Hore dx voojad G.rG.>h ® pasha > hod > OE a : Hen pred Gf te gin aa gram alae a ennep7 pre Dat Gy>G.>G37 # "a Oa "Yop bss. From thie stabi chug 2 Ben dapam PHD =} | p= arenas ho 1 Fiy= 3.POd YPC ray = eRe 3 of , rays 2 p(2) = = M ee a) Fad Hy BH In= Ze fee Mia he) 2 Hp= Pe hay f \ fei L Pr ) ace Pe MR ce Relay n bis ( nt Fi2 bea; fh) ‘Scanned with CamScanner2 be Bila Gt ptoay 0 W= 0°9183 BPS 2 Ny = Py bogg . \ tel Ree) ob te Gaye he ona) Hy= Lt tm, r : oF) ee logy Cay + & b bi (45 [ Wo = 0-8 rs | t= > kh BB b= Hy PO) + HaPcey = 0:93. i. he o-aiterd poeta “4 yok ee [He o-rsza ops} pe % EO ys oe ya —~@ 03 Ny OA hs 3) Ke #27 ‘lie EAR Wyo qn ib ® : Os o* ee, 42Nt we ‘Scanned with CamScanner‘Scanned with CamScanner_ (Ds ee Rae 8 y ag P s Ba 7 Poy) 4 te get 4 UF P( 2%) = Bx en 2a a _ 9 1 3° 3° 3 63 PUXSX2 = 2 ayn ae & 7 3 3 3 is (Ue~ dx Sx pw ts oe p(X2y= OL m hp Zzxyy= tL fae 4 ZEt\ = = oe HPN8 for aby Se P(zyD= an + mnj= 4 rc ) mt ire i. pC 2) & PCZAD = les (zen No3 P(42) = "he p (2e)= a ‘Scanned with CamScanner(We = Te &) pC Y= the 2 Gu = = ee P Cm) tog, ea) vi i G2 1 P& ha Gt yr POD beta [sky yel2) hig (t hi ve \ ty ¢ Mok, e ae ~uie leg, (im) a PCX® Hy fae Pee rie Ph oa Plax) Moja ei fe a ae 2 & bya Br Za) * | boys (E) ) + (yO bos, 7 + PCYY) loin 5p poy) t_ bg, (te = L{e ar eee” » abe GE TR Peay 2 HD+ Zs (9). Ji, = b-308 ele fymit P (x24 ws log, t+ P(xee) by, oi Peo bs ed pe * re can + £(t2x) beg ee + ‘Scanned with CamScanner+ pra boy + Pt, + Pel, plzz Playt) fey 4 pla) bog + Plaraylog, 1 _ + Peyr)lon F(axv) P(2x1) P20) + Plar2) logy - + Pa log, 1, + PlYexdbg, 1 lian ply 2 + (v2) bo P(g 1. TIO og, 9 p(Yz2) PO2> ron Zh We (EB) + be lah + 1 log, wt pg Ot Lb Cyt i = it )* ie sae Bh “dW Bye gb bg 07 he mall ! ba, (D+, os es +e by jt we 11716 Joye eoee4 Lek | ayn GaSe Gg >to ; 1116 pas6ty | 3087 > bdper> 0-957 2 Ae Ye ‘Scanned with CamScannerEl (2h Ww nee MODULE 2, —— source — ENCODING —— watch Ouse of an irposmalit Source nuatg & a proms by » where sce is convaded inl Y~ avvay sequence ny yu the umber of oené_symba vad Fe ge Ade yo" — yor —bo,t sia 24 ita 1 ean i , oe eo Soe? ey eu, pin, 0 a ENN Bounce encoder ts used to_oluse beltry compresston of ple \Compwstion caulit be lascless or ey, dapentig an the applicatit Tyns OF foes pid ents a bck of y pot ces Err dymbel will be mop i oe ted alphabet » Rieck oder con be ‘Scanned with CamScannerP (0) (NONSINGULAR CODES § SS aa Md tide words ore dittinet and eatly didtnguishall : Siqulre ‘fe cade fal oe|@a oi? te oO ° 4 ° Seeman] 2 extension e 515, CaS SiSs= O00) Sisy = Ot $,8, “@ogy | 551-0000 $3 ='0001 &Sy= 001) SS) = 010 $% =0100 3 = O10) S4= on Sy = 110 Same code word sh 2 aut ¢yptls 45 = 00 Singular %% = 1100 Sty = 1 Oy Gi Ser Mt The doe erfinnian ct Mon-shalay wfswawr Sock edonaba Ce Silas : : (D. UNIGUELY DECODABLE Coves + nak co i hy uy deca fo oan i th neon sialck. Jo ffs vou af.) by: (ie 2 ‘Scanned with CamScanner4A hb 7 A ey osstos ‘ INSTANTANEOUS coe decodatsle Cod itkouk sueatente awtlr to ag thee Ur ‘Ko weil de ahh cidontify tie endl of Cede ode I, S o Bt cet oe of & oy i cont w be otontomenes Y te word teeny secede pe aucced Symbols ” Ade prot 4 dexelng i &- || © vyexrived equianet to be OU!IC0 Conider the ofo]tt Cod B 3} Sy (de 8 a “vutinton (D OPIMAL CODE “Fpalaniarews da bb Wo caudh tobe (pital, q.té hot 0 o 0 op ti $, GdeA S| Ss wor OF O Zero [0] indinoder and. of, wie and rim? ‘Scanned with CamScannerasl fe oa | (Jinstantereaut wedi) ke Gpalontanenus Sapa De po te ne wage ambiquily (or) the rsa for clei i pnt duypccdinl pr Oe code fo be te: Symeot * cone woro — — B or | # " > 10 ‘Froneitid —yprrnalitn — ABCA oop Sequenee — elo} i ot Hew decadang— AaAcDD Theyre, decoding will nck he Suceupal i ony ste cool worde Slot with any the valid coc worelz . Toseore ha meosany tovdliten of an iualtatonous cody i> thats b cc ‘Scanned with CamScannerES qo thee word should be prgix a any thee ch @ prep tee AL prefoe pte prpety, wwKME ws : y ©44" roo — woh et wine TFA Abe ge = lag ¢ ease wod* Oe N= Nox of Cadewenttt wi toal « The above propoiy 8 call ar Relf Me 4K CKMT) HA poegi tern aly He above. property , foil alt cooler Chat Sok Ae ohove. propor, peel not be - pregen - Acbleme \ ; ©) petsamie which col O pay dacigaamn & 0 prepe cole. ; 4 act alo diour . Lhe decizeen. * 4 cochs C ceded. oo | ! 110 yu _ / t110 | 110 fer ool oO) _—— oll oo Sor Cocke Aengitt * Ss 0 amy G10 ab, 5 lo 3 Sy, NO A ky & Mtl 3 ds ‘Scanned with CamScannerr feted , Sp TIN wi page Sy = Iho . Hence 7 pot vigtonloncowr cocle/ prepic code. The Moje Medion vq iy ie quien ty) | Hee vet (Co or 4) ay Et ye pg, gS 210695 44 By ditpectinn jus joundl thot ibe not a prep cools whith ole been uarifeicl Hy KMZ , ait doe aot sahify Cs pep 5 8 fee | tage gs] } 1-4 a QR} ol ame Sy) id 3-k %| 10 deh & Oo oC al - by Sat ote Pree Of Sy — 10, Sa-H1! «tener by uigpecticn , tr not a. Progr Coc - Fron fxoyt Me Millon \¢ hequatily . Zaks re ‘Scanned with CamScanner| ei Hee 22 a aa. oe ef eae ge taint? = 13150 44 = does ak soy Ue KO prepely <1 Hena not a prope Cooks, G00 We a prep % Sy = 001.2. Te be nat a Pregin tae. Pron keroje pie Maton prepenty x* Sy fer v4 4. 3, 438 Qeea ta rer ey = ObeaT & a oe Gatepict) Thougs the gen tod gobsipes KMI ie nok a Prope cock. ‘Scanned with CamScanner® oth -p Nome | Gete-D Lengdh = 40 2-4 i 3k 2 ; 110 3-4 0! Uy ' OD | ards e992 : ty Tnspeutin, thine i no pipe ithe ues cools unr, STE a Opp cocks, Auk wykein moy be veuptel ke] papel , . Zt 2 j=l obs Prat taty vt = jé4 ‘ [Qiu ub Sobinfely the KM T propesty » Usa Propet Ceo . Codle - Tive Jor Code ~ D ‘Scanned with CamScannerOe wil_ ma eon Ao 44 Qo Aaa aol ail2 grio rol FH) source coma Teonem Ae eet 3, al bl? INFORMATION coke, Fw thoon Ge pipltt let St be lhe i matin Sousce ea nial pont fou « Tha eu 9 ate Oe jm ts the sone encoder ushpse oulpuk i the coded dapreman the gyro sei, Sera: The onovage code wcord. 7 ef L a th ein. of me foe ae, gen PY 4 ik me X 100 Le ahh Js te enbony gies the ownage bejornaltain a ae A AOE, No symbol enttid fom ue Scurste Com be “Epre! at lies ble thon tant of Ur gtbepy * he, LD > AC4) Linn = HCA) Thus ue Kequixe at Leart (3) mumbo oy bs to represent ‘Scanned with CamScanner(oxlen conumunt! , H(9_ x 100 <7 Ls shy ecdundoy of 008 je =!-%| CHANNon's FIRST ENCODING THEOREM 5 ENCODING PROCEDURE an . Cuter tre Sounte apholet S andl bie conetponsting Past vk P fo @ den thfowmabicin dounce + - STEP 2 = hog the pubabiiher Go Ha ron-incwaiung ovdes C deevensing owhy). Ay for the convespondling coke SHEP 32° Comput the word fo each Symbol Sp gue by ts wa AT Y ho b> a(a)Im a 2 ee he i Oe satin pele integer, cof satigien He. guien | cox Gree y : Aepine the plowing povamets fiom the poobally Y\=0 Hah Yeh = Fite = 448 YW = ft hrB= Yt B ‘Scanned with CamScanner —_ i“L . ~~ Sipe =! G whe NO Hie total qusmber of Soursee symbole tn the sours Ser 5 — Expond A tr binary Alt ti number of, plat ogfés the cima potne , ier § — The numbes ager” decimal ploces in faooy mpreerlalien rey one the cout word py Une conteponcling big Sis atest consider 0. scuret with sane alphabet : £4 with conecpnii prcvabicties $= (B.8 1619), ee P= (0:1, 0-2. 0-3; 0-4) fod, th code words for supnbols axing snannon's ALGORITHM , Noo ind the source yicienry ond yedundonay « . pssr ser @ P =(0-4 ,0-3,0-2,0-1) ‘ g2(P68, A) y. sep @ de S. fel) Wz ‘a>? 3d frou] fez) ! ~ Wy > tpg (= 47d? [ aes] ! | Aye “e(5)® 9384 [acon] ‘Scanned with CamScannerJare [ged Ayes | ost all tis tytn r Sep G70 : a= WAP) = 0-04 = 0-4 93° 4arh = 04409 = 0.7 7 Ygtly = 0-7402 =0-4 Yee With = Oro 4 j Be 4 = 0.00 Taea T= (0-4)e On x2 = 08 — Ilo oexe = b6 —afr | 2% ob x2 Ff2 —s4 a2X2 oH — xo be Ya 0-01), a <0. 3 (ov, 1 OTR@= py i ‘ Oh x2= 08 I) ‘a3 OP = rb sin) ss > (0 dy >} ‘Scanned with CamScanner, @ 4 20% ' = 4 8d EES [lsh por ee pax dels —>! Aik, 3.94 5 o6xe 212 —! ae pane =o —> 21 2: 1119) Ve= * gymacts | Peotneiliny Og | Cove worn { ) Dd ou 2 00 : . ¢ 03 2 01 B O-2 3 lol A on 4 | bite 1 = HD x10 Lb N, His) = SZ Pe ie (A-) fel =f “ia. + fa b4a. 6) +h) +h ref) = 0-4 Wigs) + 0-3 luc rea whut) + OJ logy GN HID = BAGH ors vk N oe =I : 4 ‘Scanned with CamScanner= Nh thls Bry FOUAL FO 3X14 2X8 $06) AY! by = W846 $0.6 +0+y! — b= ay ‘ = | = Hw 100 0909-8) oo ¢ L 1, FI BEY y 66 oh Ne 76-93" REDUNDANCY Rea l-Ns {[lo-6] k = &3-06). % Coie divalt penoryter soe wifi $= C%) 1) o conespoubirg pobabuilut p> (05,0-3,02) a) find He Code wordt fpr Ue symbole uty ‘SHanwon,'s / raconmim , obio yd the Sarete eppiney, feats — py): Consides. Oe deal exlension 6) the sburate gad fit the word epptany Redewndohuy. we. Ss Cxin2) — P= (0-8,03,0: a) Pep 2 eters] 2 - = y ‘Scanned with CamScannerfinwices — Fain ENCOLING + Tht i on inprevernerd over Shanmonis Ast Hprllim « k aye Bets ceding tyicleny composed, to StPunon's ALGORM « S as 144 of cpt Ly pd PROCEDURE St \ © hrasag te pebablly ih neo-ceatng ‘onde ( atabaieg ove) & ie te prciabiier tte exciity of atte uth Oat ge ‘un €4 Ho pachabitinin tn fin the Grtups Lr almart ogupk +, Asign. hit ‘o' to olf the elements 4 a gaoup and be ‘t tp qu the elemontt cad qreup . @ lipo pO By diviiig cach greqp is 2 su | Acipe alt No pedhar dluvcion “ ox pesible « 2 le (ondidew the fea sors S= hy 8, C,D,E,F and P= O-1, 0-15 0-25, 0-35 , O-08,0107 * >) ‘Scanned with CamScannerSymBoLs PROBABILITY Cope WORD com \ 035 "28 m2 D - ¢ 2 oar itt “ ¢ i | . 6 os | lt A ov (ye a 4 0-08 [ilo ; 0.0% youl ic te 2 HED ane Ala = = rn (4) 2 fils (yt Alo Et aM pth ety + emt) NCE) 3° 2 cee bes Le Bahk in 2 Pity rads FB Rt Pally tA th Ly = doe oh Wye Baas at 20h Re = Ite igeGie}. 9g. & |. ; , 2 Cp € @ Sea, 80,0, €,F P= Om, 02, 0-2, 01, 0108, 10% ‘Scanned with CamScannertrove word Le Mees — rroanen4ry 2 i oh oo 5 on ot “ c ond lo = D ery J to Le bes 111o F Oron itt? * 5 il Hi)= Ze ROR [a] = SelM brs a te = 23 = Pidy Par Bye Pele bbe + el Yw= BO. y 1007. L * as. 39), Rn = I -4e 2h 60% cs —_d ‘Scanned with CamScannerwy, of were Me MUAN TNEgueLTY [kno] i A Newsy and supidient condition ov ethane tole uit, co dang fy, faye N 3 . Sw ot wt \ whe + —D Numbes of opal symbols wel Hr tt codt alphabet. NP Tal number g, courne syebels chests word lengih Gib... ly axe, Cocoagedl in’ ostending ovdey ,Solfiot Me eEQe.., “hy - ° we howe to cheese one length tod words prot nN nurnbr tay sgt Anbying BO pro prety 5 we ca te 6 Oty vlan NSE. a - fo neler 4 Ai jue hat & apr combuialnin sof % syonbcs pe 2, (ode words © stk ny pon min age end i ce cn p bagr t thane pe tet ny SH” 4, , eau we mutt dove enceding (1%) pee res Ca ewes oh . any 608 7 symbols of the code. alphabet . pe e=8, WS (H-m De Ne 2 nye jos 3, ny 2)an 8 - nal ¥ ‘ an gr—nt nyt a=ny¥-npy) a el pe heap gee hg LED, = Mast 1 fae oe Be | ‘Scanned with CamScanneres the he a ot $2 $152 ed a Mu with probity P= Sly y sopedsi ha ened jonecodk fox Ue Sourmte S, ib Sod ‘eatrion & oA exlatsién « (atulate Me ¢ & S= % , So pe ah, th. ois, as 0. to Sy \ 0.25 {il Sa fp ony fos enh “ae. syngas | PROBABILITY cove word | LENGTH | S on | 02s | N Ha = Z he (5) WD = hina & Malic) Hlo = 9. gid BS _ N Le Bak ee ‘ 2 Phra lL hs = Hh loo = Ble 184 =—= Rly = WETS ‘Scanned with CamScannerOD dad extonresn Sy 0 565 4 bas i 61S Si} o-tsts | we Sz 0.0625 SS gsi SS, Sb 1 eyoon PROBABILITY | ConewoRD cau | i L_ $s 0-505 ° L | SS ov 1815 lo a Pee o | oe, ois | tro ay SS 0/082 ht a 1 be nfs Kio? Zeb Myf eal hk ) Wage) + ts) +B 1 bs Hly = I+ 68xG pes ss i ‘Scanned with CamScannerLe & Ph @ \ enn the Pepe "| 4 = , | Leos > HD Rol b= Bis. Thal 7 ang =—— Rin = I-Ie * fat iD Su: S)SS) ewe s a ee S552 by S828 oy s bee $3252. Yow S285; thy - | ovis SSiSa Slew | S51 fy | S8se My ; ‘Scanned with CamScanneroyorg } 0 (o-yzif orlyog [0 ole o14o6 |) 1. rob olyoc |! or tgoe | 0.0467 |" 0.0867 o046F |! 00467 oong9 |! 0+ 0469 D01Ke |) o0lse — prvbolitur | codtund hd ae o-/4oe oa) rob Joo or l4Oe, )o| 00464 {100 cones | L101 0 0"84 Pris oom | bid ry 2 Macey : Oo"; tear) +h bate bbw zy a “e() TAC) MT) + but otk) Aiiy> sae Bt wl Ls 2 pelt Relat hele BEAL pe mn perieet "Tele ‘Scanned with CamScanner_, 2 33 ve @\| em Hla 108]. ~ 93° 831 ye \ 21o Me) = GltT ay fe eels *OnANNON S FIRST THEOREM [romero THEOREM, CS NOISE-LESS CODING THEOREM) . . sie roll ot One Jagr AE pol te preety Vi ion by ~ es mt The alge equabin sapyier vet pr ill Le the Leng - , | nent) bis sya ibis. (Fa) Me (0) eo S 4 “1 (Rw bog 7 ' ope | alee lg , Zh I War nv EA 2h bis bm) = Brut ia Z ta rT 0g, ¥ on wo 2 . 2/44 = ©) a, 7 1g? fa ‘Scanned with CamScannerHy £L erty @ ae oe dom eqn @ gor th ezlondod ur yrds se fate, He) eye pe HED ing oj loga 2 n Wi) h He 21,6 it log, ¥ 13,8 | didi Ven goat D4 | Wy 2 ln 2 1+ Hie og, nf pe Npyeg fark we Rave fox the posic Suse ufo Le i he apeange fg 6 tobe wuorts bos Ss ho fut thot, L 2 Lo @ a called os Node ~ lew cadhig, Toei shamon's Flt Thoem, > s ne i states tha Maen a. code aiphabet wilh Noite -leus cooburg, Thearern ym nk cial 0 ge weg nf GO wcrc be made te fo Ha Co Ay insennig the ex toate. — ‘Scanned with CamScanner=y UFEMANN Cove ( CoMmRCT cove =~ / Le cove (crm cone) Gi eds thie mits cod word Len among alt othr cwding aliens, hence thy ayrcalkd as dompAtr cones. FRCEDURE : ; th bitin Apo © tempuli ar me juan oa, apeaticns which ae mete “Fy otal phen 1 “uid inbges not opts pe prneteg met veedeword th as foltoait no stapes Cn) sopeinad Pt ¥ be Bea i fabiily 2 mr, Af Auery nil. ZOO pre awl coleeays take sinlige, vale acts Pleat, rs bo is pi ot = penbine Be ok’ pO ype anrnsig up 2 cn ta eae PT pepe -poils ee Luton 01 fafa as petite TT a de ee on 1 emt dation te poeta AG en Fiona Aap pti 0), o con roe Find Lhe + probabilitis ere 4 lage th age hae um Pla ts fe meet, AG sy Be ‘saderuend, fo thr stsmne et Lite Pepe poe fee ee he lg i pps Bre foboilita tn Lip (put tot) fost dhge 2 mocked ‘Scanned with CamScannervehi Vansbuel ‘gmat S= Aye, ‘ iene ‘4; pv s)o08 lode 6B) Ey Fug, A tony ie doeinece precuces a Leyiuence Y tiprdeng probabletis at Sia 4 welmovig we sm of tes pate Neto Poe £ a-1 ease frags] Sage |p te = is él bla [os | 1 |e | hI | | oo |» | | Ys | 0 | V3 | ao-T7g] 00 |% 4 on | 7 fon {iq po (610 | al | Iq | 0100 | 14 Jov00 [Pee Ia Jou Yor oro he ae 4) 010 | ‘ey 1/10 n10106 a5 om {te | biov0r {_ ‘Scanned with CamScanneror lo10 69 ‘ g | tea oe — F | wis O}a1 00 ae Me ololol ——$ eee Hg = Zz bi lo, & + D> te, Ca) 2b eld Me log, 2) ten an fon CU delenit”) * bbe?" a + . : 2 2.2894 BE Hip= "2200 vor Le Zope yt tyee panot lla tO* ho ia! Ian Xb aah b= mmole | Nez HEe 100"), L Ny = re Ro=l-tee 4A ‘Scanned with CamScannersymaos | Prasaeumes SMET SiMEX | A A ‘Is o ‘ls | “0 'p | 0 7 y & fs ae oe ao go i ilo Ya) | 2 Fj T re fo E 4 1 lg da, | B ' Yoo iia | } = T T F794 Jia p Vy aa Need = 17 G lee al [ken 2 ° } \ 1 ao. 2 at 2 ado 3 adi 3 QQd 3 7 -4Guy ‘Scanned with CamScanneres Us) = 1 Ghia ble a : ® l. z HW x loo. gr ae be Ais ma AiG Ip yy BPS = wa xwel zoo): (ee | ‘bo Sse | Rss = I 2 Sh ~~ ck il to onorso1s R= 0 . Form be comp Go S= %, ft Or cobe jovths it 1 aack exon Ws eney = 3&2 I a wo 4 pockaliitien Spe t | 01 0 OF 6 x OF y oe lo % gt Der i # 7 Wom pally 2 + Fy tt aor tad se) ‘Scanned with CamScannery Exten DED HAUEEMAN CODING The procaine ce Bons a4 Hoymon cods uilit Courie erTérem | hy i con be der em psoblem number’ (B) 2(5) the cob (mow'g 0s low as poaible) itis Coot uordls axe Some tpuge ~ the probablitus ace difpunt . Me by lens ot agp: trol o [Hosplmonn Codh ux tot u unine Y there ie, amab vot hs he Prono ©) he th conde sean farm pnblam ware GQ) He Kou ad wl Be wage Bo ep Sprains, OP VEY OS ° wok Bee probabil eal Ue lngh the che a ie de probly Should be pegotwe pours efor the cooks fo he NI Que. ‘Scanned with CamScannerAg THNETC CODING ® The be ip cods con be ovestatu by unig (i) (gnircler a pos with S= fx 1% 24 wilh senpoclve 0-6,0.2,0°a) flick the ced word fr He patois = 4 mewage . xy. bus ae é he 70-3 : Y ong ‘ ah = X qd 0 ae on wath Value =ctiyernce x Prob + Aue value = o2* 0: 6406 2 0 F2 y 202 *0'8 TO = oY = 9.12 * 06406 = grees oe x zoe x(0- D+ Ob = 0 +6864 ‘Scanned with CamScannerBacode the patlousing ante daning Lz aly THIS I$—HIS- Hit THatSs—s es ‘ | tecalien | ) | 00/0 “(o000,T) (g000,H) . (0000, L) (coo9, 8) | ( 0000, —) oor, $s) Cotol, H) (0@L10, -) (06105 T) (ocol, =) »-Hy TS~, urge Diciionany | Fant te content | gt oT [ a 4 ©000H c oo0oo0L s 0000 $ o000— a oot''s aH olotH Is ollo— OoloL 000/— rdeunads is foe oy 7) Ceodawerd. ot Jee" 0,. 4), (cbeeword at Lei" 9, £) Ceodiusead at beta 4s) Ccediwoord, ot bec" 0, -) (Ccedenserd at Loe" 1S) Curdewsord a Inet. 1) Ceoduvocrd, at toc" rg, -) Codeword of Loc” 4,1) (codeword, at lec", _) J Aire + Ys hina de § oSpua ly Whi Hen, add a ‘Scanned with CamScannerp— Sncede win LZ a3 the Abnivg ua) 48 WLOPN WH NWA WPN_P folw Wy Py Ny WL, Mid HI, P,P [Dict dec | Oict content [Finest at death i code uso | cco! W 0000) 0000 .W Odo o0c0— c000,~— oct! P ooooP 0000, | Ol00 N OOOON 0000, NY | Brot we cocl— | 0001 ,— O | Ilo Nw 0100 W doo, oul : ColoW eed 3 ott N Oolo,W beat 0O11,N 3 eolo P colo , P The four Coding alg 4 nepine Malis obs would’ be” wnkneudrr . Also raat of the ing Jounces Hand Gn fash Hints poisen memohig steer Hype Sela 1 Hubbmann, codes n20v ts Will bow ocsmetas Sa Jw foil to cchivue opti ey pos Four pals NS: ek F. AM 4 : a a caw Le acne | dy f ge 4 pes a So bee Lx tin (ing xv) ionady based approach‘ Lx pa de hk a Bio od ol ‘Scanned with CamScanner tod2 steciuir dyfnding na the Maing be he seb. bn, tede word bell Le Lie forme f joan, eo pohane index the imdb of ter ditions boku : Lhe Up sequence the lengert malta for fp eq a“ forts - a ‘hy ods word fo the Ayebol futcadinng the Longuat mately duden O used if the Auyabol is Wrjbivetes aes the 14 Hinae § mot an tw dictionary the dictionary ws updated an accordance usith ita tp sig Yangon the abing te ered reer wll Le the “compwaion TS tenses @ DMS weith 9 Ayboals Apel ws, P= $ O65, 03, tt bad he cael ex the message Wy ae anim fel T20e Wah. wr y z Bx ok ott On OFBYE] otzar th OF? B0cH QO FOR y Wy fo dp ® y \ OF, Ost o-6157) 0881 To-t265T | 0-201 . \ . \ x x x ® x oh 0 5 065 alan O22 OF292 Hence the vedi vodtd | tog $% YWXKY ds OFSOl. ae a6] 4] i+ ait * Prob 7, + base value ‘Scanned with CamScannerate Module 3 a Anfermalin Channels 4 Communication channel, TO eae, : Bi. | (aputl bate << Thanmilfy Phutical channel 4 i ferns ga ' heesothes ei <—— Pal commelnication channel CBiseaste) g ’ 1 eS as 7 m | deabe i= 4 » Onsactous ation Asi Ce ia The coninaication, Agata ie ied, lp amity 9 & physical or frousmittion Go Recubur, gH transmitter consists an & G & medlusiuter bottle Hy recréucr contin of dunedulster, 4 A decoclr. The Cemmmuuncea yo tee , Abze. “Ob, channtl accepls a Acqudnee inva Bhs Channel B Chora cteni pitied Py pohone Pig ib ‘Scanned with CamScannerduiqnr Whe o Cenuimad twth Te aug Ae 4 Cagttach, quedles § dueder - mnuenidttion channel bit ph ad B sg aloe preenictes Me checlu'cal P aL Ib fears Her j neces, dhe ip 4 op axe anale licta’cal wateforrrs. his porticn of channel often talled as tontinusus 0} meditation Channel. laccal communication channels sud cx t Analeg Ele as worce fond f wohile bound filep heme. Aiphens, high fragt Aadio Aystone,¢ Leopstcnnnan, dystorn These channels due subi Verlous list iliances , Jom ave duc to arneplet ct § I p Fubpense, variations a th. Uannel usin Lt pos band. Oty distus Mase ’ in @ Chand Mocclif ging ‘lp Signed in a vi da caldition | tr chal con ale coud Ta tl atatis ki ues, te vanes types 4 additive, t Liphicative nelte. SL of tase intredute Bers in data Bomanission & Lint: Ha, mon pole ot wshich the data caw be “Areneheord | gue, be Oran. > Channel Models Fe dinuucty Communi ation channwl ww be roiled oy Bopratiding ame Day LG Up alphabets Ke9 dita eG ao Ae s a abphadstt Y= Lye Macs 4GS: No DM Channel i, completely deribed by a p fek OL tramsition prot PCY ia 7) where i dtuat es ip Aymbele & yi ofp tymbel . ‘Scanned with CamScannerxt) devotis Lu prolays Cire Wh diepe Si home ma “ _giusent thal i Channel Mabe | Prob Aransilden mola] yi ‘ ,) eb P( Yi plpfan) —P(Yefw) 0 e (Yifaz ye 2, pls) P0Yalas) - a , ( sey Pv fai) i | Je cerditional pole P (fila) is gepos Hae Channa Aamsibion jurolr. thse conditienal. a Cam Ise tn Bie ‘ Cran ‘ali i ynlyal saprasonted Orr WOU tu ¢ (nl ee kt. mata ts pono Pade iin mu (99) anil mae Wm ‘Scanned with CamScannerhet us umbide. a disoute me lis Sour 4. gala boating aun 4/P alphabet yx hy, Xa Kah owl gp alplaleel v= SV Ya Ya, 3 - he. clifporent drawsilion fuvobs ane marked in Hie hand dicgaam a sheoum above, p, is the paole Ural Y, will Le Hacbivech teliens Ls taausmilted . x ie Pur P(N) + at PL Yay) Fay) Pas ply, my ) Ye Ya Ye x te [Pow] oa] hr As Pow y Sy] Per Pon Ba Pot Pa % ay] Pat Per Pas Poy Aine cach ip te tie Channd pesulls nm Put Piet Pig + Pig 2d WkT p(Ae)= P(BIA) Pla) Pe mye) = PCWy;) Pex) How POX, Wp) ® He ea Probability Keg Vp Mh ae edd all the join’ Paoli Jv jiesd J Han we gee 3. P(X 5 Yi) = PY; et w my yr EP yi) & PCa) i) ‘Scanned with CamScannerAappase tha j* symbsel i puceieed , A Js tromsmatted then error eccurs. ‘the frok of erer da given doy p= PY) + PLY) + PLY) we BPO ye) ety ree 2 So Poe Fl 30 if Bun the Taint Prob aie P xy) > PUA) POR We kreon , thy PIM, . wy PROM POR) POT | eevale) PUbIa) Muy wnt bs with P(xé) [PC War POD oe = [Popa #0 (> re Wy Pome) | Pui fia)» lan ‘Scanned with CamScannerPlata Vere fPCe rhe PCL, Ya) P(>C, Ye)! 5 é Plata WD Pl te, Y2) Plz, th) Pla wv) Plast dre alton malic wu dhe Tem | Tene foinkgalith wait ) ae ch hos the jpllrtng Proper © Poy) - Ps Yj) ie le Ine eG alt the tri af ('S Glumn 4 JPM gine te panko the 4% culgat— Pra of alt ofp ptt (PUK = Aemmalion jer ea Nei ry3) NES ‘fp ww © Le obtained by i pe dements oh iH tow of TPM. @) 2 pees) =I g = f pom Ye] = ). Je Ue wi a Uaemewk of TPM = wily - => Binaiy Symmdinie: Canral 4 RM). ST ae PCN a9) mk AG: vt 4 ‘Scanned with CamScannerLak ws Conic abi info bohre Taster ore Lip el | 8 op dugrolpals 4 shy yy» thas Prols give hg oa Pla.) i a _ ane X WY.) 000) 4 voy [ Pca.) xd] PU %/xe) PCM fi : , Pl Ye/-2,) PC a be ni eet Ounnrel is imi | 20 i, we? py Ve tore) 100) f ¢ Se hhh | - a S. { 4 a —> Fa the channel aka Doron ool Me Say Oe svabacnce "i ‘Scanned with CamScannerrad a : moo Given Channel moatiin, Find: Ba ising outiies - y dav the Channel oli ‘ Sole Channel mature - P(y)=|O8 # 0% Wich obit: =". a (pitta (08 0 02” | Or2 O06 092 O2 08 095 Luput + x Aya otdpat zy (dopa env) Babul untishiy) he tp x G the fp Y are Aoures with bein | Seon onbiepins HOD HCY): a8 Sheen in figure Wwhire OX) = 3 pcy:) log - ( el P(x: my weye S ey Jeg, Le j PY; ) ‘ ‘Scanned with CamScannerWnt dunce ondrepus te H(X) § AOD stud te ov omount” of impo en One » wowing” out a the system fou Aymbel , } on ‘unciataly nity about Lhe PG op 4 Hod g HCY). The enlrop of te Yp Ayambels Biamtinitsion ib Knew 26 ap is The edna } Hw f Ay 3 fea tt, cats! Potoutal eal my ¥ & del an spi (o) Lorditi oan Extiepy 4 whith i dunoked te Wey , 1 G81) Seam This sapsesunbs Hee, uncentodmity absouk x ii re eae muh more info ont would gain by | Ms filed ciao ¥7 jue tinea i le »| AAG i : fi PCY) "G) - 4 PON Bay 4 ‘Scanned with CamScannerpny A} & ¢ ) ke " chai H (2) ie cquivocation. Apocifrees in. teeslueds oud vunerrtoinitt about es behon foci Pie tt Hie the a uncerboi ni ic Lt Specefres Ta de if ri the Chawwll, behin man audpaut ignored de 4 pasents the, amount 4 in lost clue te motte. [ dlr ol Stoning es). Wty th Xp ie bent mee ‘ cbak ¥ 3 258 exci seh Ue). ME) FEEL) ey dry obswusen Heying fo quies Leth Hef | sw Ownnel X Td ey Be ane gh = ebay: a Ey He eh y= SE P(K,¥7) tog, 1 pb Die 5 ha ra Problems Sy ——(D Proue that HCx,Y) = H(X)+ ay), |Aeln Hix,y) = SS PCL, ¥) dogg tj 4 FOR.) = 2S PWS o\ be a Vj Gy) epi) i ‘Scanned with CamScanneraps ahs [= xy) 2%], + j “ Ce ) aja PCy) +e hax Mp) Iga ae 4 KO) | = H (x) THY) psn, ‘): co P(A,B) = PCB) (8) = P(A): P(E P@ Fev bw Ty ee isan Find HEX), H( ve HEY) x9), BOAPY , WV) i] PCX,Y) = 10-05 0 of O05 —_ 2} 0 Ol o+] fo} 4 31 0 ° 0-2 of 4[0:05 oof 0 a+} * Avba PCY) = & oye oo i) e (Xe, Y;) Px) = B Pex: ys ) = Oey), c 4 ‘Scanned with CamScannerPUY )= OL + PCY = O15; PCY3)= 0-5. MY) 202 (Y)- Ort Aaja Lt 0°15 logy Le + Ons HCY)? bth a bps Pie 5 P(x;) Log, — We O93 begs L va toed Lo Ka: ae ey Vo ons #h Ore “9410 HOGY. +s Pony) tog, 1 PEE) © ite O°5 bag, 1 sont Wigs Q2r oe i © 3. loje ot 12a he OD wf) + Bey, A(#)= S182 — 1 FhOF = 13493., | Mf Wf) = S1@aq~ [ato = 151 bps ( HOR) = HOS at hiofr So Ad channel malice w giver as follows | Po viie) = fore ack oat ole. chime oa oe 02] pOD= fe % he] ord OR OF EEL ace), HOY), HOG HONK TG HOLY). ‘Scanned with CamScannero-2 on 34 ees CCV) = [4B Ie}[oe ow eee \ OFF Ong is “ALB 1s) 2 Poy) ake Pr =F P Cx LJ a ‘PCX,V) =f oa 0-066. 0-066 0066 O'2 0-066. 0006 0066 OL PCXt) = 0332 0-334. ea Pl ase [p-282 0-339 | W(x) = EF PCxz) tog, fe) Pox) H(x)? Bx 0-832 ta, > 19584 bps 0-332 m RLY) = g rey) toyy i POD, i HY)> 3x 0-33 ley, = 188i bphe | 0-838 ; Hex, We. Oa xg Syne Pe er eeatets HOG) =. 495 bps H( Wx) = HOLY) — Bue) = as 10h | ‘Scanned with CamScannerOlle eet H(*/y) = HOX, YH Ly) 5 Q95 - 1564 = (+366 bps - fy IPM malian given 5 PCA, main Find, HEX), HCY), HC) HOYKD, Hox, ¥D - Fev the 5 oO Oo 0°05 orl ° 02 0°15 0 0 of Olg 0 A Aol — PCXi) = eae » PCY)=[o-25 ovat ogs] Ons i 0-8 HO! Spey) dogg -I 2 OBE hog) 1 + OS log 1 + O-35lq | “Nes . “Onls Soa Feed doy Ly. ar 2 OBL €0-m) 0°53 + O-KG: e 192 bps WO): CYA) beg i”! 4) O85 yp a+ Oe a ie at ; Voss P hg Oss = 1948 bpa- ' ‘Scanned with CamScannerHUX, Y)> Rx 0015 lg, f. Axor! tap, Lay Ry 0-06 Loy 1 + Ow ‘i i ba fet = O82 + [e328 + Oehar- Org © 3-04. bp q HOI) = Heyyy) - HOO = 304 — [oq2 2 Je Jo. bps. H(x/y) 2 3.09-|-98 2 [06 bps OY 8a 63 Pox >={) yl. F | 0-3 0-2 06)? “f [e. %) i Gel One pes) HOD, ROY), Boe, Yd, Ml HOY/x).. Ce KY fob 03 08 [yo th 03 O02 105. : Ol Orb O°5 POLY =fo-2 o15 O46 gts O05 0125 OH Or! O15 PLXt) = [ek } POy)=[o-ang 0:3 ot 0°25 ‘Scanned with CamScannerAe & PEK) Mog gt lo 1 Ply;) 2.0°5 deg, 1 * 0-25 fo I! + OWS log, | I 0.5 ee “pe Oats > 05 405 +06 yp = ©6500 bps- HlY >. a OD sy - 0°306, iid i ; “hee ode0 ty 2 re Hea = |. 5AQ bps (0:32.x2 + 0° 33] H(x, ¥) = 3-08 bpav HOME) = Hox, y) - OX) = | 564 bps HOY) = Hox, - Hy) = l50 bps: Aalto) —> Mural aia an ‘hawainfermalien) kT HLX) = BPO) tog, ob (ly) = gs POX, Y) A 1 mi 1H) HOO. is the uae info a Bie haind, ou md eumewst ef info = 5, pion H (Xly)s De thannel due to the channol moire. goctiuedl egerteeie ath space be Yu oltvued ofp | Spal Matus, info yin Sy 04 )Las) = AGO HY Fixsy)= § POn) — 2% POD “Fi rN WkT & PO) = 1. SCH) =I i ¥ yf > Ix; y)- a4 P(x) HH) “Pray : | Ss reed z j ( Me i) = 83 pee yey Gok! 7 ? PLx "Dal my . = SS POX, ¥y) hy, a ig 1 (9 “ rey = SS Pry Cx : pO Ly vd > £8 FOX, Y4) dog, [se ys) ‘4 # Pox) Pv) fe we [£(X3 y) = $8 Pex: 1%) Me Px) : oa Wi 5 Yb AP PCRS i) ‘ r ee ‘Scanned with CamScanrjerblssy) = £0952 Lty;x) = wily) he Ye), TS Guphical Rysusentation a Mutual Inge LTex;y)= Hx) — n(x ) / 4 ie XA W) LGR y) caw be the ght. OL m) intersection 4 ‘etl corsusperdh bey te \xay)= lvl — [xl HOXYY) = HOO+ H( Yy) Wd Wty, x)= Hex, y= Hy)t Al (9) HO) + HOY) = Hoy) +H OXY) Nex) H(A) = HEY) He) LOGY = Ley), es al that thw tells exa sie Gs lp ale tb, ip Hix,y)= HOt HCA/y) ae Hy) = Hex, y) - HEY), bul CXsy)= HOO - HDy) xy) = HO) — HEX,Y) + HC) a amalags fe: |xay]= Ix] yl-|xuy) Jalations cams be repseses AL the aboue — dak seably . The” sounte HUx) & the ductenaction HUY) hase Leen § ate tipy HY a Ahan in 7 boy a eveopping ‘Scanned with CamScanrThe enenlap bit the tive wtcls AMtoss Which a Hie mutual ime - we HOO H(x,¥) — Propertics , er p. It Mutual @ info. a thannel etlubiG 4 te £CX3y) = LEY; oe F Qe The Mutual eee te wom negokia Ho) we LOK} “yy Ss Mutical ihe com dé asta 3 a fe ie ie = Lixsy)= Hy)- Hype HOO ~ HUI), 4. Mutuol info can be beribhen tw , Teint » LOGy)2 Hex+, OT #0 > PT LOGY) ES Passh 4 BGS ¥) =. sgh) co Fem. tae sell Wwe T “ad a: Lee Mi ‘Scanned with CamScannerLOGY) = deg e PK: Ye) jee yj) 7" Pe ) CY) Concioles eg, PUK 44) Z I- P¢x2) iw PCA) PCY) Plxi, 4) Mubtiplying doth the Aides bos ‘ é. P(X: 47) Ss Bae feb 7. POE, Yg) ‘4 Pe«DeCH) peep Pox meri ly a ige £8 PO, sy Fee ID z PLP toy, € pe PCXs aly) f b= POL PCY ee Tis ie % eee Rr) rex cH] aye ee [Py i) 7% loge ie PX ¥)— £ POR) £.1cy) 3 WKT ie Mi g 08) » Sty) j ie % tang @ [ba Huyzo ‘Scanned with CamScannera Ratge Sele Sepa WKT the tote © Tromumissiony ferro 4 tohure fy w the symbol tate - Tre tug. inf tale is decided, def hhtas ngh He Channel “is § evo much, imfe Cw, bef Accomebauctid at “Hi Aeeréues. : 4 ds sere amt- of inhe 1a Lott ih the ogi) Channel , dhe mek anit ef ange ad; the Reef is the ructual info TCX} ¥). = Hence thu. omg fetts of Info Bansher iy dy 3 Rey = EXSY) 5 bps Hh charinal J dio moiny » Ae agp happen, “hak Harel oh Ba channel, mag th Auediste col lurdpendlent eh the input. ie Kizy HOY). WAT Lexsy) = HOY) -H(%y) 0) =O ie all Hic Ine Bert indy dhe channel is be G me impo is Hawauiltied outer, The ch oy Hone’ info ot ty suctimey ¢ The hs. eed ts be uulen, 4 ‘Scanned with CamScannerola 8 13 Ty Rebaiine Eribtiopuy ( Kall back ~ tabs Qittawes) Lemicis a ditceta duluibulien Hat hosa pababiltt pow" C« s another di.cute dist fiat has A probe feunc” Gu + Shem we def® tie Aelabie entno} ef P wak Y gisen bo d= = Px l (3) hs Vk (Py) t bla a Mvobway aes nell Le egal te ee Problems AQ Find, ‘hy mutual info Shevon Jn the fy pine: ahout thy channel PCx) =°0 +6 , PLX3)= , oF. x G(X) = ? ha. Asha topos oa] parelore ove) | —~ fos otf Ply) =[0°6 O-4) | Poe = PCy) PCR = [o-gs on) Ola, 0°28 ; 4 1+ Ong bor u(%y) = ow Jog SOE, St g 5 F 0°36 los. 4 = O'S +0364 r0r36 Ft 0" SI. = [1456 bps ‘Scanned with CamScannerWty) = £ hx: » 14 7 «) “Orb fog, i 4+ 06 ice HO} 0-94 bps ress Hoo tuo) — #OO%) = 99494 Ongaye [| t2b = 0185 bps SRT EL Dee = Ge @ PO) = Plta)= 0-5, 5, Abeta a) yy Mite [os 0-5] % a POM = | Fhe to Mey o Vag a” Ka acs, i)= [aes O18 “% Ys ° a HO, Y) = 0-345 lege + Qyoras iy) O85 bof olas = |4056 bps: Hx) =ax0e5 1 =! bpa- Ia oh ° yey) 2 Pr BbLAHE bps. g Lx, ¥) 2 14561045 #19056 =06 s_ ‘Scanned with CamScannerR= L(%,Y) + Me Uy 2 0°65564% x |c000 ae 6556-48 bits | see TZeoilwe motty channels axe Cascated , "room = [Ys fs Se Ye Ys 6 ¥3 3 0 ya 0 Ya Pix1) = Pika) = Ye LUCK 2) = %, % a zy % @ z 1 a Ys a 43 PC) = Lal ade) Z BI5Q = N30 PCZ sly tel )%g) oe e zy betes @25 PR )aoL x3 +he aliedxg 55 _ = WMfts. PCB] x, “pete = 44/90 Peale yea 4 3 a5 ilo = (9/50 Plz3/ i). 241. / do8 1a = Feo ‘Scanned with CamScanner