E- 3 ÄBTEN D= 2

k bn 0
°
•

:
=
F-
↳
.

II
- -
-
-
- -
-
-
-

-
-

kraftrslnequality-r.ae D-
ary
-

Prefix -
Free -
code
ex ists iff
k
w here la is the
z 5h = , lith
↳, length of the
Code Word

Proof Consider a
prefix free -
- code
of depth N .

The full free of depth N has DN leaves .

i
-
- -

:
93
'
D

A code word of length ln has DN

-
l"
leaves
[ ! - - -

at depth N
Hemming from it .

Ya
Erik I ⑨
-

KEI
IN
←
%
Kraft } hat ""
Y

depth
-

④ Order the codewords so IEL Elz Elz

,
E- . .
Elk =
N

Consider the full lree of depth N .

• h =\
az
LEGE
Assis "
a. EX to a node at depth e,

• Prune Hee at this node There remain :

79
l
DN DN
Thistle
-

so leaves
'
-

Since
Elz find l, we can a node at depth lz to which we
but
can assign az .

Again prune
the free now from the new node .

lz
There remains DN DN li
DN 0
-
-

-
-
>

§ D-
lk
< | Existance of a
pmfix -

free code
-

4=1
Examplei.tl - 5 D= 2 ( l, , lz f) =
( 1,2 4,4 4)
, , ,
. .
.

, ,

Is there a
prefix free code with the se lenghts ?
-
-

§ ,
Den = ätzt } -

2-4=7+3 =

pat
empty leaf !
Code : C =
{ 0,10 ,
1100
, 1101,1110 }

1-1-19
iv. |
-
% To
1-•
.

↳
=
-

Rooted trees with probabilities

suppose an EX has Px ( au ) =
Pn

HIN bits
Define Heeaf % j Belog
= =
Du >
>
.

Smiuppose
th probability
free has branding
• the J nodes :

j -
I J
,
.
.

,
.

Pj .

tot
AH

0,5

iii.
¥ tz
•
0,5

0,2
0,2

! ¥ ::
Pz = 0,5
} =
Pz
F

distribution
Branding Qjl )

§
• :
.

( o) =
0,6

QZCD =

0¥ =
0,4
 D= 3
Ein : 1<=4 ,

: ie:::-. :*
"

0 0,4
① •

$1793
P, = I B -0,3
-

~
Q , C.) = [ 0,4 0,3
,
0,3 ] 0
•
92

/#
,
• oi
Qzl ) =
% 03 -20
/
.

j.IO#j--zHeeaf=HlXI=l,846bits=HCp
, ,

H ( Q) =

1,571 bits ECL] § ,

Ben

H ( Q) =
Hd 's ) =
Hz ( %) =
0,9183 bits =
0,4Mt d) taz talk)

ternary Symbols

Path leaves arooted

Length Lemma : The
average depth of the in
-
-

free
probabilities is the
with sum of the

branding node probabilities

ECL] =

Pj
,

Proof The depth lk is the humber of

branding nodes encountered when

moving from the root to leaf k .

Ein @ =

Einer :D: äu ) )
-
=

Einfiele :* !: : :D
-

=P
,
Leaf Entropy -

Lemma

EE
Eil
"
e. -

→ :
-
.

PJTD

m=jH%tz,...,
°
D=
Qj (m ) =

B- -

HCQ;)

=P, [ E
Qjlmllog Qjlm ) ]

÷÷y„m
-

,
-
- w
m
:B Qjlm) > 0
Pwypjg
-

Pmlpj
.

log >
Pm -

logzpj →

adding over all

j
leaves as with
( logz Pm logzß ]
=§q
Pm
÷ .ae
-
- .

.cm , > o "* .. .

cancels a
term from node j -
I

""
÷: →

÷
:
* 0¥
It
Ea] =

logz 3
1,3
484610923
z
 Bands HK )
Enlropy Heeaf
=
:

HCQ;) Elogz D with =

Iff Qjcm) =
¥ for all m

HCX) =
Pi HCQ;) by the Leaf Entropy
-
-

Lemma
; -

Elogzb
E- ECL] D
-

logz

ECL] Z HI with iff a) Qj g- I ] uniform

ORE
=
=
,
. .
-

are
, , ,
loggt distributions over { 0,1 ,
- .
. D- I }
,

Exampt-D-2.ie , compressihg to bit then

EEL] Z HCX)