DHANALAKSHMI COLLEGE OF ENGINEERING, CHENNAI

DEPARTMENT OF INFORMATION TECHNOLOGY

IT2302 - INFORMATION THEORY AND CODING
UNIT - I
Part A
1. r!t" t#" Kra$t-M% &!''!() !)"*+a'!t, $(r t#" !)-ta)ta)"(+- %(.". /M01-132
Kraft-Mc million inequality:
K-1
2
-lk
1
K=0
The codeword length of the refi! code "ati"fy a#o$e equation% &ere l
k
i" the
codeword length of k
th
"ym#ol "
k
% 'nd K i" the total num#er of "ym#ol" in the alha#et%
(et u" aly the a#o$e equation to the refi! code%
)
2
-lk
= 2
-l0
* 2
-l1
* 2
-l2
*2
-l)
K=0
+rom the a#o$e equation o#"er$e that l
0
= 1, l
1
= 2, l
2
= ) and l
)
=) digit" -#it".% /utting
the"e $alue" in the a#o$e equation,
)
2
-lk
= 2
-1
* 2
-2
* 2
-)
*2
-)
= 1

K=0
Thu" Kraft 0 McMillian inequality i" "ati"fied%
2. Stat" t#" 3r(3"rt!"- ($ ")tr(3,. /A0M-112
/roertie" of 1ntroy:
a% 1ntroy i" 2ero if the e$ent i" "ure or it i" imo""i#le
& = 0 if
k
= 0 or 1
#% 3hen
k
= 14M for all the M "ym#ol", then the "ym#ol" are equally likely for "uch
"ource entroy i" gi$en a" & = log
2
M
c% 5er #ound on entroy i" gi$en a",
&ma! = log
2
M
3. #at !- S#a))() '!&!t4 /N0D-132
The 6hannon0&artley theorem tell" the ma!imum rate at which information can
#e tran"mitted o$er a communication" channel of a "ecified #andwidth in the re"ence
of noi"e% 7t i" an alication of the noi"y channel coding theorem to the archetyal ca"e of
a continuou"-time analog communication" channel "u#8ect to 9au""ian noi"e%
5. Stat" t#" C#a))"' Ca3a%!t, T#"(r"&. /N0D-122
The channel caacity of the di"crete memory le"" channel i" gi$en a" ma!imum
a$erage mutual information% The ma!imi2ation i" taken with re"ect to inut ro#a#ilitie"
/-!i.
: = ; log2-1*64<. #it"4"ec
&ere ; i" channel #andwidth%
6. #at !- t#" r"'at!()-#!3 7"t8"") +)%"rta!)!t, a). !)$(r&at!()4 /A0M-102
5ncertainity:
i% 7t i" ro#a#ility of occurrence of the e$ent"%
ii% 5ncertainity of e$ent decide" amount of information%
7nformation:
i% 7t i" the content recei$ed due to occurrence of e$ent"%
ii% 7nformation recei$ed in certain e$ent i" 2ero% ;ut information recei$ed in
rate e$en i" ma!imum%
9. #at !- a H+$$&a) C(.!):4
In Huffman coding a separate codeword is used for each character. The code
words produced by Huffman coding gives an optimum value.
;. Ca'%+'at" t#" ")tr(3, ($ t#" -(+r%" "&!tt!): -,&7(' 8!t# 3r(7a7!'!t, < = 106, , = 102,
> = 103.
?. D"$!)" +)%"rta!)t,
5ncertainity:
i% 7t i" ro#a#ility of occurrence of the e$ent"%
ii% 5ncertainity of e$ent decide" amount of information%
@. D!$$"r")t!at"A U)%"rta!)t,, I)$(r&at!() a). E)tr(3,. /N0D-102
S. N( U)%"rta!)!t, I)$(r&at!() E)tr(3,
1%
2%
7t i" ro#a#ility of
occurrence of the e$ent"%
5ncertainity of e$ent
decide" amount of
information%
7t i" the content recei$ed
due to occurrence of
e$ent"%
7nformation recei$ed in
certain e$ent i" 2ero% ;ut
information recei$ed in
rate e$en i" ma!imum%
7t i" the a$erage
information recei$ed
due to occurrence of
multile e$ent"
1ntroy i" 2ero if the
e$ent i" "ure or
imo""i#le%
10. r!t" t#" 3r(3"rt!"- ($ &+t+a' !)$(r&at!().
i% The mutual information i" "ymmetric% 7-=>?. = 7-=:?.
ii% The mutual information i" alway" o"iti$e 7-=>?. @ 0
11. Ca'%+'at" t#" a&(+)t ($ !)$(r&at!() !$ P
B
=1054
'mount of information : 7
k
= log2 -1 4
k
.
= log
10
A 4 (og
10
2
= 2 #it"
12. D"$!)" S#a))() Fa)( C(." M"t#(.
Thi" i" a much "imler code than the &uffman code, and i" not u"ually u"ed,
#ecau"e it i" not a" efficient, generally, a" the &uffman code, howe$er, thi" i" generally
com#ined with the 6hannon Method -to roduce 6hannon - +ano code".% The main
difference, "uch that 7 ha$e found, i" that one "ort" the 6hannon ro#a#ilitie", though the
+ano code" are not "orted%
13. #at !- aC"ra:" !)$(r&at!()4
'$erage information or 1ntroy i" gi$en a",
&ere,
k
i" the ro#a#ility of k
th
me""age%
M i" the total num#er of memorie" generated #y the "ource%
Part D
1. E<3'a!) 7r!"$', t#" -(+r%" %(.!): t#"(r"&. /92/N0D-102
S(+r%" C(.!): T#"(r"&A
1% 6ource "ym#ol" encoded in #inary
2% The a$erage code length mu"t #e reduced
)% Bemo$e redundancy . reduce" #it-rate
:on"ider a di"crete memoryle"" "ource on the alha#et
6 = C"0, "1, D D D , "kE
(et the corre"onding ro#a#ilitie" #e
C0, 1, D D D , kE
and code length" #e
Cl0, l1, D D D , lkE%
Then, the a$erage code length-a$erage num#er of #it" er "ym#ol. of the "ource i"
defined a"
K-1
/
K
l
7
K=0
7f (
min
i" the minimum o""i#le $alue of F(, then the coding efficiency of the "ource i"
gi$en #y
(min
F(
2. E<3'a!) H+$$&a) ")%(.!): a':(r!t#&. /52/A0M-112
i% 'rrange the gi$en "ource "ym#ol" in de"cending order with re"ect to it"
ro#a#ility
ii% 7f it i" a #inary &uffman &uffman coding add the la"t "ource $alue" into a "ingle
unit and Gnce again arrange the "ource $alue" n decrea"ing order a" o#tained in
"te 2%
iii% :ontinue the roce"" until only 2 "ource "ym#ol" are left%
i$% 6tart a""igning code" -0, 1. in the #ackward direction toward" the initial "tage%
v. Hetermination of &-S. and '$erage codeword length%
$i% :heck the condition for $alidity #y u"ing "ource coding theorem% 7f the condition
"ati"fy then calculate coding efficiency and code redundancy%
3. A .!-%r"t" &"&(r, '"-- -(+r%" #a- a) a'3#a7"t ($ $!C" -,&7('- 8#(-" 3r(7a7!'!t!"-
($ (%%+rr")%" ar" a- ."-%r!7". #"r" A
S,&7('- S0 S1 S2 S3 S5
Pr(7a7!'!t, 0.5 0.2 0.2 0.1 0.1
C(&3+t" t#" H+$$&a) %(." $(r t#!- -(+r%", ")tr(3, a). t#" aC"ra:" %(."8(r.
'"):t# ($ t#" -(+r%" ")%(."r. /122/A-112

5. E<3'a!) t#" 3r(3"rt!"- ($ ")tr(3,. /92/M01-132
'$erage information i" rere"ented #y entroy% 7t i" rere"ented #y & k log 2 -1 4 k.
1 Pr(3"rt!"- ($ ")tr(3,A
i% 1ntroy i" 2ero if the e$ent i" "ure or it i" imo""i#le
& = 0 if k = 0 or 1
ii% 3hen k = 14M for all the IM "ym#ol", then the "ym#ol" are equally likely
for "uch "ource entroy i" gi$en a" & = log2M
iii% 5er #ound on entroy i" gi$en a",
&ma! = log2M
6. E<3'a!) %#a))"' %a3a%!t, a). ."r!C" t#" %#a))"' %a3a%!t, $(r 7!)ar, -,&&"tr!%
%#a))"'. /?2/M01-132
:hannel caacity can #e e!re""ed in term" of mutual information% Mutual information i"
gi$en #y equation a",
n m
7-= > ?. = /-!
i
,y
8
. log
2
/-!
i
4 y
8
. 4 /-!
i
.
7 = 1 8 = 1
&ere /-!
i
,y
8
. can #e calculated from equation a",
/-!
i
,y
8
. = /-y
8
4 !
i
. 4 -y
i
.
+rom the a#o$e equation we ha$e,
/-!
i
4 y
8
. 4 /-!
i
. = /-y
8
4 !
i
. 4 -y
i
.
&ence we can write equation a",
n m
7-= > ?. = /-y
8
4 !
i
. /-!
i
. log
2
/-y
8
4 !
i
. 4 -y
i
.
7 = 1 8 = 1
n
/-y
i
. = /-y
8
4 !
i
. /-!
i
.
7 = 1
n m n
7-= > ?. = /-y
8
4 !
i
. /-!
i
. log
2
/-y
8
4 !
i
. 4 /-y
8
4 !
i
. /-!
i
.
7 = 1 8 = 1 7 = 1
The channel caacity of the di"crete memoryle"" channel i" gi$en a" ma!imum a$erage
mutual information% The ma!imi2ation i" taken with re"ect to inut ro#a#ilitie" /-!
i
.%
i%e%,
ma!
: = /-!
i
. 7-= > ?.
9. E<3'a!) &+t+a' !)$(r&at!() a). !t- 3r(3"rt!"-. /?2/M01-132
Mutual 7nformation:
Pr(3"rt!"- ($ M+t+a' I)$(r&at!()A
;. I) t#" &"--a:"-, "a%# '"tt"r (%%+r- t#" $(''(8!): 3"r%")ta:" ($ t!&"-A /122/N0D-132
L"tt"r A D C D E F
E ($
(%%+rr")%"
23 20 11 @ 16 22
-1. Ca'%+'at" t#" ")tr(3, ($ t#!- a'3#a7"t ($ -,&7('-.
-2. D"C!-" a %(."7((B +-!): H+$$&a) t"%#)!*+" a). $!). t#" aC"ra:"
%(."8(r. '"):t#.
-). D"C!-" a %(."7((B +-!): S#a))()-Fa)( t"%#)!*+" a). $!). t#" aC"ra:"
%(."8(r. '"):t#.
-A. C(&3ar" a). %(&&")t () t#" r"-+'t- ($ 7(t# t"%#)!*+"-.
?. A .!-%r"t" &"&(r,'"-- -(+r%" #a- $!C" -,&7('- F1, F2, F3, F5 a). F6 8!t#
3r(7a7!'!t!"- 0.5, 0.1@, 0.19, 0.16 a). 0.16 r"-3"%t!C"',. Ca'%+'at" a S#a))() Fa)(
%(." $(r t#" -(+r%" a). %(." "$$!%!")%,. /192/N0D-122
The 6hannon 0 +ano algorithm i" e!lained in ta#le%
M"--a:" Pr(7a7!'!t,
($
&"--a:"-
I II III C(."
8(r. $(r
&"--a:"-
N+&7"r
($ 7!t- 3"r
&"--a:"-
!." )
K
=1
=2
=)
=A
=J
0%A
0%1K
0%1L
0%1J
0%1J
0
1
1
1
1
0
0
1
1
0
1
0
1
0
100
101
110
111
1
)
)
)
)
The entroy -&. i" gi$en #y equation a",
M
& =
k
log
2
-1 4
k
.
K = 1
&ere M = J and utting the $alue" of ro#a#ilitie" in a#o$e equation,
& = 0%A log
2
-140%A. * 0%1K log
2
-14 0%1K. * 0%1L log
2
-140%1L. * 0%1J log
2
-140%1J. * 0%1J log
2
-140%1J.
= 2%22M1 #it" 4 me""age"
The a$erage num#er of #it" er me""age" < i" gi$en #y the equation a",

( 0 1
< =
k
n
k
k =0
&ere k i" the ro#a#ility of kth me""age" and nk are num#er of #it" a""igned to it%
/utting the $alue" in a#o$e equation,
< = 0%A-1. * 0%1K-). * 0%1L-). * 0%1J-). * 0%1J-).
= 2%)J
The code efficiency i" gi$en #y equation a",
:ode 1fficiency N = & 4 <
= 2%22M1 4 )%)J
= 0%KAM
@. r!t" -#(rt )(t"- ()A
/a2 D!)ar, C(&&+)!%at!() %#a))"'
The communication "y"tem con"i"t of tran"mitter, recei$er and channel%The
channel encoder and modulator form" a tran"mitter% 6imilarly demodulator and channel
decoder form" a recei$er% The modulation channel or the communication channel ha"
analog "ignal" through it%
7n other word", the "ignal" through communication channel are analog% The e!amle" of
"uch channel" are telehone channel", high frequency radio channel" and troo"catter
"y"tem"%The "ignal" through the channel" are corruted due to noi"e% &ence error" are
introduced in the data% Thu" nature of the channel and noi"e limit the ma!imum rate of
the data tran"fer%
;inary :ommunication :hannel
/72 D!)ar, S,&&"tr!% %#a))"'.
The #inary communication channel i" "aid to #e "ymmetric if /-y
0
4!
0
.=/-y
1
4!
1
.
;inary 6ymmetric :hannel
+or the a#o$e channel we can write the equation a",
DHANALAKSHMI COLLEGE OF ENGINEERING, CHENNAI
DEPARTMENT OF INFORMATION TECHNOLOGY
IT2302 - INFORMATION THEORY AND CODING
UNIT - II
Part A
1. Stat" t#" a.Ca)ta:"- ($ L"&3"'-G!C a':(r!t#& (C"r H+$$&a) %(.!):. /N0D-102

'd$antage" of (emel-Oi$ algorithm:
i% The codeword" are formed #y aending new character%
ii% <o "earate character "et i" "tored in the dictionary initially%
iii% The data character" a" well a" dictionary indice" are tran"mitted%
'd$antage" of &uffman coding:
i% :ode" for the character" are deri$ed%
ii% /reci"ion of the comuter doe" not affect coding%
iii% &uffman coding i" the "imle technique%
2. C(&3ar" LG 8!t# H+$$&a) %(.!):. /A0M-112
6%<o (O3 :oding &uffman :oding
1%
2%
:odeword" are new word" aearing
in the "tring%
:omlete character "et i" "tored in the
dictionary initially%
:ode" for the character are
deri$ed%
/reci"ion of the comuter doe"
not affect coding%
3. #at !- &"a)t 7, 3r"$!< 3r(3"rt,4 /A0M-112
i% 7n the refi! coding no codeword i" refi! of other codeword%
ii% The refi! code" are uniquely decoda#le%
iii% The refi! code" "ati"fy Kraft 0 McMillian inequality%
5. Stat" t#" &a!) a33'!%at!() ($ GUI. /A0M-112
The main alication of 957 are,
i% 957 facilitate" "mooth interface #etween the human u"er" and the
oerating "y"tem%
ii% /arameter" handling, control and monitoring i" done with the hel of 957%
J% D"$!)" Ha&&!): D!-ta)%" /A0M-112
The hamming di"tance #etween the two code $ector" i" equal to the num#er of
element" in which they differ% +or e!amle, let the two code word" #e,
= = -101. and ? = -110.
The"e two code word" differ in "econd and third #it"% Therefore the #anning di"tance
#etween = and ? i" two%
9. #at !- D('7, AC-14 /M01-132
Hol#y ':-1 i" u"ed for audio coding% 7t i" M/19 audio coding "tandard% 7t u"e"
"ychoacou"tic model at the encoder and ha" fi!ed #it allocation" to each "u##and%
;. C(&3ar" LG a). LG %(.!):. /N0D-132
S.N( LG C(.!): LG C(.!):
1.
2.
The codeword" are formed #y
aending new character%
<o "earate character "et i" "tored
in the dictionary initially%
:odeword" are new word" aearing in the
"tring%
:omlete character "et i" "tored in the
dictionary initially%
?. #at !- &"a)t 7, $r"*+")%, a). t"&3(ra' &a-B!):4 /N0D-132
3hen ear hear" the loud "ound, certain time ha" to #e a""ed #efore it hear"
quieter "ound% Thi" i" called temoral ma"king%
3hen multile "ignal" are re"ent a" in the ca"e with general audio a "trong "ignal
may reduce the le$el of "en"iti$ity of the ear to other "ignal", which are near to it in
frequency% Thi" effect i" known a" frequency ma"king%
@. #at !- 3"r%"3t+a' %(.!):4 /N0D-122
7n ercetual coding only ercetual feature of the "ound are "tored% Thi" gi$e"
high degree of comre""ion% &uman ear i" not "en"iti$e to all frequencie" equally%
6imilarly ma"king of weaker "ignal take" lace when louder "ignal to re"ent near#y%
The"e arameter" are u"ed in ercetual coding%
10. #at !- D('7, AC-34 /N0D-122
The audio "ignal i" not gi$en directly to #it allocation algorithm% Bather, a
"ectral en$elo of audio "ignal i" ro$ided to #it allocation algorithm a" well a" decoder%
The #it allocation i" not "ent to the decoder% ;ut the decoder recon"truct" the audio
"ignal" from rere"entation of "ectral en$eloe% &ence it doe"not need #it allocation
detail" directly%
11. #at !- -+7-7a). %(.!):4
7n "ignal roce""ing, 6u#-#and coding i" any form of tran"form coding that #reak"
a "ignal into a num#er of different frequency #and" and encode" each one indeendently%
Thi" decomo"ition i" often the fir"t "te in data comre""ion for audio and $ideo "ignal"%
12. Dra8 t#" 7'(%B .!a:ra& $(r 3+'-" %(." &(.+'at(r4
13. r!t" t#" .!$$"r")%"- 7"t8"") ."'ta &(.+'at!() a). DPCM4
HM encode" the inut "amle #y only one #it% 7t "end" the information a#out *P
or -P, ie% 6te ri"e or fall%
H/:M can ha$e more than one #it for encoding the "amle, it "end" the information
a#out difference #etween actual "amle $alue and redicted "amle $alue%
15. #at !- a.a3t!C" DPCM4
'dati$e differential ul"e-code modulation -'H/:M. i" a $ariant of differential
ul"e-code modulation -H/:M. that $arie" the "i2e of the quanti2ation "te, to allow
further reduction of the required #andwidth for a gi$en "ignal-to-noi"e ratio%
Part D
1. E<3'a!) t#" %()%"3t- ($ $r"*+")%, &a-B!): a). t"&3(ra' &a-B!):. H(8 t#", ar"
+-". !) 3"r%"3t+a' %(.!):4 /192/N0D-102
Fr"*+")%, &a-B!):A
i% The "trong "ignal reduce" le$el of "en"iti$ity of human ear to other "ignal" which
are near to it in frequency% Thi" effect i" called frequency ma"king%
ii% G#"er$e that "en"iti$ity i" $ery good #etween 2 to J K&2%
iii% Thi" "en"iti$ity of at A K&2 i" con"idered 0 d;% Gther "en"iti$ity are lotted
relati$ely%
i$% G#"er$e that "ignal ' i" a#out A0 d;% 7t i" well a#o$e the "en"iti$ity of human ear%
$% The "ame "ignal ' of A0 d; and one more "ignal ; at 200 &2% The "ignal ; i"
"trong-a#out Q0 d;. and it i" $ery clo"e to "ignal '% Hue to re"ence of "ignal #
the "en"iti$ity a" er modified "en"iti$ity cur$e% &ence "ignal ' will not #e
"en"ed% 3hile coding the two "ignal" ' and ; the "ignal ' can #e droed% Gnly
"ignal ; can #e coded% Thi" reduce" the #it rate and increa"e" comre""ion ratio%
T"&3(ra' &a-B!):A
3hen ear hear" the loud "ound, certain time ha" to #e a""ed #efore it hear"
quieter "ound% Thi" i" called temoral ma"king%
2. E<3'a!) !) ."ta!', A.a3t!C" H+$$&a) %(.!):, 8!t# t#" #"'3 ($ a) "<a&3'".
A.a3t!C" H+$$&a) %(.!):A /192/N0D-102
Pr!)%!3'"A
7n thi" method, the tran"mitter and recei$er de$elo the &uffman coding tree
dynamically deending uon the character in the data "tream% 7f the character i" not
re"ent in the data "tream, then it i" tran"mitted in it" uncomre""ed form% The recei$er
identifie" thi" character and add to the coding tree% 7f the character i" re"ent in the
coding tree, then it" codeword i" tran"mitted% Heending uon the codeword, the recei$er
identifie" the character%
S!7'!): 3r(3"rt,
i% ' tree i" "aid to e!hi#it "i#ling roerty if the node" can #e li"ted in order of
increa"ing weight and if e$ery node aear" ad8acent to it" "i#ling in the li"t
ii% ' #inary tree i" a &uffman tree if and only if it o#ey" the "i#ling roerty
iii% Thu", maintain the "i#ling roerty during the udate a""ure that we ha$e a
&uffman tree #efore and after the count" are udated
i$% 7f the newly incremented node now ha" a weight of w*1, the ne!t higher node will
ha$e a weight of w when udating i" needed
$% There may #e more node" after the ne!t higher one that ha$e a $alue of w a" well%
The udate -"wa. rocedure mo$e" u the node li"t till it find" the la"t node with
a weight of w
#1 A(1
#! "(!
## $(1%
#& '(!
#( )(!
#*((
#+(&
#,(,
#-(1,
#1 '(!
#! "(!
## $(1%
#& A((
#( )(!
#*( &
#+(*
#,( -
#-( 1 -
$i% That node i" "waed with the node with weight w*1
$ii% 7n thi" e!amle, H i" "waed with '
$iii% The ne!t node to #e incremented will #e the new arent of the incremented node%
7n thi" e!amle, it i" node RL
i!% '" each node i" incremented, a check i" erformed for correct ordering% ' "wa i"
erformed if nece""ary
<ot a &uffman Tree,
6till not a &uffman tree,
#1 '(!
#! "(!
## $(1%
#& A(&
#( )(!
#*( &
#+(+
#,( 1 %
#-( ! %
#1 '(!
#! "(!
## $(1%
#& A(*
#( )(!
#*( &
#+(,
#,( 1 1
#-( ! 1
#1 '(!
#! "(!
## $(1%
#&(&
#( )(!
#* A( *
#+(+
#,( 11
#-( !1

<ow it i" a &uffman tree again,
a% <ote that the code length for ' change" from ) to 2
#% 'nd the code length for ; and H change" from ) to A
I)!t!a'!>at!()
i% :reate a &uffman tree with two "ym#ol 1G+ and 16:'/1 with weight
#eing1
ii% (et : #e the incoming character
iii% 7f : i" in the &uffman tree, then tran"mit the &uffman code for : and udate
the &uffman tree, el"e tran"mit the &uffman code for 16:'/1 and the lain
M-#it character for :, and then udate the &uffman tree for 16:'/1 and :
T#" (C"r$'(8 3r(7'"&
i$% the weight of the root e!ceed" the caacity a" the rogram rogre""e" long
enough
$% The longe"t o""i#le &uffman code e!ceed" the range of the integer u"ed a" a
"tack
#1 '(!
#! "(!
##(11
#&(&
#( )(!
#* A( *
#+(+
#, $( 1%
#-( !1
$i% 7t i" needed #ecau"e the encoding and decoding ha"e" treat the &uffman tree
ath in re$er"e order
S('+t!()
$ii% Becalling the weight #y a factor of 142
$iii% 7t can o""i#ly re"hae the tree
3. A--+&" t#at t#" %#ara%t"r -"t a). 3r(7a7!'!t!"- ar" " =0.3, ) = 0.3, t=0.2, 8=0.1, .=
0.1. D"r!C" t#" %(."8(r. Ca'+" $(r -tr!): H)"8tI. E<3'a!) #(8 t#" ."%(."r
."t"r&!)"- (r!:!)a' -tr!): $r(& t#" r"%"!C". %(."8(r. Ca'+". /192/A0M-112
5. E<3'a!) a.a3t!C" H+$$&a) %(.!): $(r t#" &"--a:" HMa'a,a'a&I. /?2/N0D-132
The #a"ic &uffman algorithm ha" #een e!tended, for the following rea"on":
-a. The re$iou" algorithm" require the "tati"tical knowledge which i" often not a$aila#le
-e%g%, li$e audio, $ideo.%
-#. 1$en when it i" a$aila#le, it could #e a hea$y o$erhead e"ecially when many ta#le"
had to #e "ent when a non-order0 model i" u"ed, i%e% taking into account the imact of the
re$iou" "ym#ol to the ro#a#ility of the current "ym#ol -e%g%, SquS often come
together, %%%.%
'dati$e &uffman coding wa" fir"t concei$ed indeendently #y +aller and
9allager T+aller 1KQ)> 9allager 1KQMU% Knuth contri#uted imro$ement" to the original
algorithm TKnuth 1KMJU and the re"ulting algorithm i" referred to a" algorithm +9K% '
more recent $er"ion of adati$e &uffman coding i" de"cri#ed #y Vitter TVitter 1KMQU% 'll
of the"e method" are defined-word "cheme" which determine the maing from "ource
me""age" to codeword" #a"ed uon a running e"timate of the "ource me""age
ro#a#ilitie"% The code i" adati$e, changing "o a" to remain otimal for the current
e"timate"% 7n thi" way, the adati$e &uffman code" re"ond to locality% 7n e""ence, the
encoder i" SlearningS the characteri"tic" of the "ource% The decoder mu"t learn along with
the encoder #y continually udating the &uffman tree "o a" to "tay in "ynchroni2ation
with the encoder%
'nother ad$antage of the"e "y"tem" i" that they require only one a"" o$er the data% Gf
cour"e, one-a"" method" are not $ery intere"ting if the num#er of #it" they tran"mit i"
"ignificantly greater than that of the two-a"" "cheme% 7ntere"tingly, the erformance of
the"e method", in term" of num#er of #it" tran"mitted, can #e #etter than that of "tatic
&uffman coding% Thi" doe" not contradict the otimality of the "tatic method a" the "tatic
method i" otimal only o$er all method" which a""ume a time-in$ariant maing%
7n the adati$e ca"e the only o$erhead i" the n lg n #it" needed to rere"ent each of
the n different "ource me""age" when they aear for the fir"t time% -Thi" i" in fact
con"er$ati$e> rather than tran"mitting a unique code for each of the n "ource me""age",
the "ender could tran"mit the me""ageW" o"ition in the li"t of remaining me""age" and
"a$e a few #it" in the a$erage ca"e%. 7n the "tatic ca"e, the "ource me""age" need to #e
"ent a" doe" the "hae of the code tree%
6. E<3'a!) t#" Car!(+- C"r-!()- ($ D('7, AC- -tat!): !t- &"r!t- a). ."&"r!t-, !t# )"at
!''+-trat!()-. /?2 /N0D-132
Ver"ion" of Hol#y ':":
i% Hol#y ': 0 1
iii% Hol#y ': 0 2
i$% Hol#y ': 0 )
D('7, AC-1 wa" the fir"t digital coding technology introduced in 1KMQ with the
#eginning de$eloment of what would #ecome &HTV% (ater in 1KMK, Hol#y introduced
AC-2,which had a #etter audio quality and a reduced #it-rate% +inally, in 1KK1, Hol#y
introduced AC-3, which include" J%1 channel" and oerate" at )20 k#"%
':-), al"o known a" Hol#y Higital , i" a ercetual digital audio coding technique
that reduce" the amount of data needed to roduce high-quality "ound% /ercetual digital
audio coding take" ad$antage of the fact that the human ear "creen" out a certain amount of
"ound that i" ercei$ed a" noi"e% Beducing, eliminating, or ma"king thi" noi"e "ignificantly
reduce" the amount of data that need" to #e ro$ided% Hol#y (a#oratorie" de$eloed two
other ercetual coding "y"tem", ':-1 and ':-2% ;uilding uon Hol#yW" two re$iou"
coding "y"tem", ':-) wa" the fir"t coding "y"tem de"igned "ecifically for multichannel
digital audio% ':-) i" the "ound format for digital tele$i"ion - HTV ., digital $er"atile di"c"
- HVH "., high definition tele$i"ion - &HTV ., and digital ca#le and "atellite tran"mi""ion"%
':-) i" a J%1 format, which mean" that it ro$ide" fi$e full-#andwidth channel", front
left, front right, center, "urround left, and "urround right% ' low-frequency effect -(+1.
channel i" included for the "ound needed for "ecial effect" and action "equence" in mo$ie"%
The (+1 channel i" one-tenth of the #andwidth of the other channel" and i" "ometime"
erroneou"ly called the "u#woofer channel% ':-) al"o ha" a downmi!ing feature that en"ure"
comati#ility with de$ice" that do not "uort the J%1 format%
9. E<3'a!) L!)"ar Pr".!%t!C" C(.!):, !t# )"at !''+-trat!()-. /?2 /N0D-132
The "eech "ignal i" filtered to no more than one half the "y"tem "amling frequency
and then '4H con$er"ion i" erformed% The "eech i" roce""ed on a frame #y frame #a"i"
where the analy"i" frame length can #e $aria#le% +or each frame a itch eriod e"timation i"
made along with a $oicing deci"ion% ' linear redicti$e coefficient analy"i" i" erformed to
o#tain an in$er"e model of the "eech "ectrum ' -2.% 7n addition a gain arameter 9,
rere"enting "ome function of the "eech energy i" comuted% 'n encoding rocedure i" then
alied for tran"forming the analy2ed arameter" into an efficient "et of tran"mi""ion
arameter" with the goal of minimi2ing the degradation in the "ynthe"i2ed "eech for a
"ecified num#er of #it"% Knowing the tran"mi""ion frame rate and the num#er of #it" u"ed
for each tran"mi""ion arameter", one can comute a noi"e-free channel tran"mi""ion #it rate%
't the recei$er, the tran"mitted arameter" are decoded into quanti2ed $er"ion" of the
coeifficent analy"i" and itch e"timation arameter"% 'n e!citation "ignal for "ynthe"i" i"
then con"tructed from the tran"mitted itch and $oicing arameter"% The e!citation "ignal
then dri$e" a "ynthe"i" filter 14' -2. corre"onding to the analy"i" model ' -2.% The digital
"amle" "X-n. are then a""ed through an H4' con$erter and low a"" filtered to generate the
"ynthetic "eech "-t.% 1ither #efore or after "ynthe"i", the gain i" u"ed to match the "ynthetic
"eech energy to the actual "eech energy% The digital "amle" are the con$erted to an analog
"ignal and a""ed through a filter "imilar to the one at the inut of the "y"tem%
;. r!t" a ."ta!'". )(t" () A
/12 Ar!t#&"t!% %(."
i% &uffman coding ha" #een ro$en the #e"t fi!ed length coding method a$aila#le
ii% ?et, "ince &uffman code" ha$e to #e an integral num#er of #it" long, while the entroy
$alue of a "ym#ol may -a" a matter of fact, almo"t alway" "o. #e a faction num#er,
theoretical o""i#le comre""ed me""age cannot #e achie$ed
iii% +or e!amle, if a "tati"tical method a""ign K0Y ro#a#ility to a gi$en character, the
otimal code "i2e would #e 0%1J #it"
i$% The &uffman coding "y"tem would ro#a#ly a""ign a 1-#it code to the "ym#ol, which i"
"i! time" longer than nece""ary
$% 'rithmetic coding #ya""e" the idea of relacing an inut "ym#ol with a "ecific code% 7t
relace" a "tream of inut "ym#ol" with a "ingle floating-oint outut num#er
$i% To encode the fir"t character ; roerly, the final coded me""age ha" to #e a num#er
greater than or equal to 0%20 and le"" than 0%)0
$ii% 'fter the fir"t character i" encoded, the low end for the range i" changed from 0%00 to
0%20 and the high end for the range i" changed from 1%00 to 0%)0
$iii% The ne!t character to #e encoded, the letter 7, own" the range 0%J0 to 0%L0 in the
new "u#range of 0%20 to 0%)0
i!% 6o, the new encoded num#er will fall "omewhere in the J0th to L0th ercentile of the
currently e"ta#li"hed%
!% Thu", thi" num#er i" further re"tricted to 0%2J to 0%2L
/22 LG a':(r!t#& /192 /N0D-122
i% 7f the me""age to #e encoded con"i"t" of only one character, (O3 outut" the code for
thi" character> otherwi"e it in"ert" two- or multi-character, o$erlaingZ,di"tinct attern"
of the me""age to #e encoded in a Hictionary% ZThe la"t character of a attern i" the fir"t
character of the ne!t attern%
ii% The attern" are of the form: :
0
:
1
% % % :
n-1
:
n
% The refi! of a attern con"i"t" of all the
attern character" e!cet the la"t: :
0
:
1
% % % :
n-1
LG (+t3+t !$ t#" &"--a:" %()-!-t- ($ &(r" t#a) ()" %#ara%t"rA
i% 7f the attern i" not the la"t one> outut: The code for it" refi!%
ii% 7f the attern i" the la"t one:
[ if the la"t attern e!i"t" in the Hictionary> outut: The code for the
attern%
[ 7f the la"t attern doe" not e!i"t in the Hictionary> outut: code-la"t/refi!.
the outut:code-la"t:haracter.
?. E<3'a!) !) ."ta!', 1PEG !&a:" %(&3r"--!() t"%#)!*+"-. /192 /N0D-122
\/19 i" an image comre""ion "tandard which wa" acceted a" an international "tandard
in 1KK2%
i% He$eloed #y the \oint /hotograhic 1!ert 9rou of the 76G471:
ii% +or coding and comre""ion of color4gray "cale image"
iii% ?ield" acceta#le comre""ion in the 10:1 range \/19 i" a lo""y comre""ion
technique
i$% ;a"ed on the H:T
$% \/19 i" a general image comre""ion technique indeendent of
7mage re"olution
7mage and i!el a"ect ratio
:olor "y"tem
7mage come!ity
$i% ' "cheme for $ideo comre""ion #a"ed on \/19 called Motion \/19 -M\/19.
e!i"t" \/19 i" effecti$e #ecau"e of the following three o#"er$ation"%
$ii% 7mage data u"ually change" "lowly acro"" an image, e"ecially within an M!M
#lock1!eriment" indicate that human" are not $ery "en"iti$e to the high
frequency data image"
Therefore we can remo$e much of thi" data u"ing tran"form coding
The image i" di$ided u into M!M #lock"
2H H:T i" erformed on each #lock
The H:T i" erformed indeendently for each #lock
Thi" i" why, when a high degree of comre""ion i" reque"ted, \/19 gi$e" a
]#locky^ image re"ult
DHANALAKSHMI COLLEGE OF ENGINEERING, CHENNAI
DEPARTMENT OF INFORMATION TECHNOLOGY
IT2302 - INFORMATION THEORY AND CODING
UNIT - III
Part A
1. D!-t!):+!-# 7"t8"") :'(7a' %('(r ta7'" a). '(%a' %('(r ta7'" !) GIF. /N0D-102
The 97+ format i" u"ed mainly with internet to rere"ent and comre"" grahical
image"% 97+ image" can #e tran"mitted and "tored o$er the network in interlaced mode,
thi" i" $ery u"eful when image" are tran"mitted o$er low #it rate channel"%
2. #at !- t#" &!)!&+& $ra&"0-"% r"*+!r". !) MPEG4 /M01-132
M/19-Motion /icture" 1!ert 9rou -M/19. M/19 wa" formed #y the 76G to
formulate a "et of "tandard" relating to a rangeof multimedia alication" that in$ol$e"
the u"e of $ideo with "ound
3. Stat" t#" &a!) .!$$"r")%" 7"t8"") MPEG C!."( %(&3r"--!() a':(r!t#& a). H.291.
M/19 &%2L1
M/19 "tand" for Motion /icture" 1!ert
9rou-M/19.% 7t wa" formed #y 76G%
M/19 ha" de$eloed the "tandard" for
comre""ion of $ideo with audio%
&%2L1 $ideo comre""ion "tandard ha"
#een defined #y the 7T5-T for the
ro$i"ion of $ideo telehony and $ideo
conferencing o$er an 76H<%
5. #at !- TIFF4 /N0D-132
Tagged 7mage +ile +ormat-T7++. i" u"ed for the tran"fer of image" a" well a"
digiti2ed document"% 7t "uort" AM #it" of i!el" re"olution"% 1L #it" are u"ed for each of
B, 9, and # colour"% The code num#er indicate" articular format in T7++%
6. #at !- MPEG4 /A0M-102
M/19 "tand" for Motion /icture" 1!ert 9rou-M/19.% 7t wa" formed #y 76G%
M/19 ha" de$eloed the "tandard" for comre""ion of $ideo with audio% M/19 audio
coder" are u"ed for comre""ion of audio% Thi" comre""ion mainly u"e" ercetual
coding%
9. #at !- Gra3#!%- I)t"r%#a):" F(r&at/GIF24 /A0M-102
9rahic" 7nterchange +ormat-97+. i" u"ed for rere"entation and comre""ion of
grahical image"% 1ach i!el" of 2A-#it colour image i" rere"ented #y M-#it" for each of
B, 9 and ; colour"% 6uch image ha" 2
2A
colour"% The 97+ u"e" only 2JL colour" from the
"et of 2
2A
colour"%
;. D"$!)" C(." E$$!%!")%,
The code efficiency i" the ratio of me""age" #it" in a #lock to the tran"mitted #it"
for that #lock #y the encoder i%e%,
:ode efficiency = Me""age #it" 4 Tran"mitted #it"
= k 4 n
?. #at ar" t#" "rr(r ."t"%t!() a). %(rr"%t!() %a3a7!'!t!"- ($ #a&&!): %(."4
The minimum di"tance -dmin. of &amming code" i" )% &ence it can #e u"ed to
detect dou#le error" or correct "ingle error"% &amming code" are #a"ically linear #lock
code" with dmin = )
@. #at ar" Ha&&!): %(."-4
&amming code" are -n,k. liner #lock code" with following condition":
i% <um#er of check #it" q @ )
ii% ;lock length n = 2q0 1
iii% <um#er of me""age #it" k = n 0 q
i$% Minimum di"tance dmin = )
10. D!$$"r")t!at" %()C('+t!()a' %(." $r(& 7'(%B %(."-.
;lock code" take Ik num#er of me""age #it "imultaneou"ly and form In #it
code $ector i" al"o called #lock% :on$olutional code take" one me""age #it at a time and
generate" two or more encoded #it"% Thu" con$outional code" generate a "tring%
11. #at !- %()-tra!)t '"):t# $(r %()C('+t!()a' ")%(."r-4
:on"traint length i" the num#er of "hift" o$er which the "ingle me""age #it can
influence the encoder outut% 7t i" e!re""ed in term" of me""age #it"%
12. #at !- t#" +-" ($ -,).r(&"-4
6yndrome gi$e" an indication of error" re"ent in recei$ed $ector I?% 7f ?&T
= 0, then there are no error" in I? and it i" $alid code $ector% The non 2ero $alue of
?&T i" called I"yndrome% 7t" non 2ero $alue indicate" that I? i" not a $alid code
$ector and it contain" error"%
Part D
1. E<3'a!) t#" 8(rB!): ($ 1PEG ")%(."r a). ."%(."r, 8!t# a 7'(%B .!a:ra&. /N0D-102
There are two tye" of \/19 algorithm"%
Da-"'!)" 1PEGA Huring decoding, thi" algorithm draw" line until comlete image
i" "hown%
Pr(:r"--!C" 1PEGA Huring decoding, thi" \/19 algorithm draw" the whole
image at once, #ut in $ery oor quality% Then another layer of data i" added o$er the
re$iou" image to imro$e it" quality% /rogre""i$e \/19 i" u"ed for image" on the
we#%the u"ed can make out the image #efore it i" fully downloaded%
2. E<3'a!) t#" MPEG a':(r!t#& $(r C!."( ")%(.!):, 8!t# a 7'(%B .!a:ra&./192/N0D-
102
MPEG = M(t!() P!%t+r" E<3"rt- Gr(+3
M/19 i" a family of encoding "tandard" for digital multimedia information
M/19-1: a "tandard for "torage and retrie$al of mo$ing icture" and audio on "torage
media -e%g%, :H-BGM.%
i% (ayer 7
ii% (ayer 77
iii% (ayer 777 -aka M/).
i$% M/19-2: "tandard for digital tele$i"ion, including high-definition
tele$i"ion -&HTV., and for addre""ing multimedia alication"%
$% 'd$anced 'udio :oding -'':.
$i% M/19-A: a "tandard for multimedia alication", with $ery low #it-rate
audio-$i"ual comre""ion for tho"e channel" with $ery limited #andwidth"
-e%g%, wirele"" channel".%
$ii% M/19-Q: a content rere"entation "tandard for information "earch
3. E<3'a!) L!)"ar 3r".!%t!C" %(.!):. /92/A0M-112
L!)"ar 3r".!%t!C" %(.!): -LPC. i" a tool u"ed mo"tly in audio "ignal
roce""ing and "eech roce""ing for rere"enting the "ectral en$eloe of
a digital "ignal of "eech incomre""ed form, u"ing the information of a linear
redicti$e model%
T1U
7t i" one of the mo"t owerful "eech analy"i" technique", and one of the
mo"t u"eful method" for encoding good quality "eech at a low #it rate and ro$ide"
e!tremely accurate e"timate" of "eech arameter"%
(/: analy2e" the "eech "ignal #y e"timating the formant", remo$ing their effect" from the
"eech "ignal, and e"timating the inten"ity and frequency of the remaining #u22% The roce""
of remo$ing the formant" i" called in$er"e filtering, and the remaining "ignal after the
"u#traction of the filtered modeled "ignal i" called the re"idue%
The num#er" which de"cri#e the inten"ity and frequency of the #u22, the formant", and the
re"idue "ignal, can #e "tored or tran"mitted "omewhere el"e% (/: "ynthe"i2e" the "eech
"ignal #y re$er"ing the roce"": u"e the #u22 arameter" and the re"idue to create a "ource
"ignal, u"e the formant" to create a filter -which rere"ent" the tu#e., and run the "ource
through the filter, re"ulting in "eech%
;ecau"e "eech "ignal" $ary with time, thi" roce"" i" done on "hort chunk" of the "eech
"ignal, which are called frame"> generally )0 to J0 frame" er "econd gi$e intelligi#le "eech
with good comre""ion%
5. H(8 ar" P-$ra&" a). D-$ra&"- ")%(." a). ."%(."4 /102 /N0D-132
; frame"
The M/19 encoder al"o ha" the otion of u"ing forward4#ackward interolated
rediction% The"e frame" are commonly referred to a" #i-directional interolated rediction
frame", or ; frame" for "hort% '" an e!amle of the u"age of 7, /, and ; frame", con"ider a
grou of icture" that la"t" for L frame", and i" gi$en a" 7,;,/,;,/,;,7,;,/,;,/,;% '" in the
re$iou" 7 and / only e!amle, 7 frame" are coded "atially only and the / frame" are
forward redicted #a"ed on re$iou" 7 and / frame"% The ; frame" howe$er, are coded #a"ed
on a forward rediction from a re$iou" 7 or / frame, a" well a" a #ackward rediction from a
"ucceeding 7 or / frame% '" "uch, the e!amle "equence i" roce""ed #y the encoder "uch
that the fir"t ; frame i" redicted from the fir"t 7 frame and fir"t / frame, the "econd ; frame
i" redicted from the "econd and third / frame", and the third ; frame i" redicted from the
third / frame and the fir"t 7 frame of the ne!t grou of icture"% +rom thi" e!amle, it can #e
"een that #ackward rediction require" that the future frame" that are to #e u"ed for #ackward
rediction #e encoded and tran"mitted fir"t, out of order% Thi" roce"" i" "ummari2ed in
+igure Q%1L% There i" no defined limit to the num#er of con"ecuti$e ; frame" that may #e
u"ed in a grou of icture", and of cour"e the otimal num#er i" alication deendent% Mo"t
#roadca"t quality alication" howe$er, ha$e tended to u"e 2 con"ecuti$e ; frame"
-7,;,;,/,;,;,/. a" the ideal trade-off #etween comre""ion efficiency and $ideo quality%
;-+rame 1ncoding
The main ad$antage of the u"age of ; frame" i" coding efficiency% 7n mo"t ca"e", ; frame" will
re"ult in le"" #it" #eing coded o$erall% _uality can al"o #e imro$ed in the ca"e of mo$ing
o#8ect" that re$eal hidden area" within a $ideo "equence% ;ackward rediction in thi" ca"e allow"
the encoder to make more intelligent deci"ion" on how to encode the $ideo within the"e area"%
'l"o, "ince ; frame" are not u"ed to redict future frame", error" generated will not #e
roagated further within the "equence%
Gne di"ad$antage i" that the frame recon"truction memory #uffer" within the encoder and
decoder mu"t #e dou#led in "i2e to accommodate the 2 anchor frame"% Thi" i" almo"t ne$er an
i""ue for the relati$ely e!en"i$e encoder, and in the"e day" of ine!en"i$e HB'M it ha"
#ecome much le"" of an i""ue for the decoder a" well% 'nother di"ad$antage i" that there will
nece""arily #e a delay throughout the "y"tem a" the frame" are deli$ered out of order% Mo"t one-
way "y"tem" can tolerate the"e delay", a" they are more o#8ectiona#le in alication" "uch a"
$ideo conferencing "y"tem"%
/ -redicted. frame
i% :oded with motion comre""ion u"ing a"t 7 frame" or / frame"
ii% :an #e u"ed a" reference icture" for additional motion comen"ation
6. E<3'a!) t#" ")tr(3, ")%(.!): 7'(%B- ($ 1PEG -ta).ar.. /92 /N0D-132
\/19 i" an image comre""ion "tandard which wa" acceted a" an international "tandard
in 1KK2%
a% He$eloed #y the \oint /hotograhic 1!ert 9rou of the 76G471:
#% +or coding and comre""ion of color4gray "cale image"
c% ?ield" acceta#le comre""ion in the 10:1 range
\/19 i" a lo""y comre""ion technique
d% ;a"ed on the H:T
e% \/19 i" a general image comre""ion technique indeendent of
i% 7mage re"olution
ii% 7mage and i!el a"ect ratio
iii% :olor "y"tem
i$% 7mage come!ity
f% ' "cheme for $ideo comre""ion #a"ed on \/19 called Motion \/19 -M\/19.
e!i"t"
\/19 i" effecti$e #ecau"e of the following three o#"er$ation"
g% 7mage data u"ually change" "lowly acro"" an image, e"ecially within an M!M
#lock
i% Therefore image" contain much redundancy
h% 1!eriment" indicate that human" are not $ery "en"iti$e to the high frequency
data image"
i% Therefore we can remo$e much of thi" data u"ing tran"form coding
1PEG E)%(.!): OC"rC!"8
9. r!t" -#(rt )(t"- () H.291 C!."( %(&3r"--!() -ta).ar.. /?2 /N0D-132
H.291 C!."( %(&3r"--!() -ta).ar.A
'n 7T5-T &-"erie" "tandard alica#le to $ideohone or $ideo conferencing
1% Video coding algorithm i" de"igned for tran"ort u"ing the Beal-time Tran"ort
/rotocol -BT/.
2% Gerate" in real-time with limited delay
)% Tran"mi""ion #it rate i" at multile" of LAK#it4"
F"at+r"- ($ H.291 0 6ource format
;it rate
i% The target #it rate i" ` LAK#" to 1K20K#"
ii% :7+ -:ommon 7ntermediate +ormat. 0 <T6: a /'(
iii% _:7+ -_uarter :ommon 7ntermediate +ormat.
i$% 't 2K%KQ frame" er "econd with A:2:0 chrominance "u#-"amling -?::
;
::
B
.
$% /icture 0 coded a" luminance and two color difference comonent" -?, :
;
and
:
B
.
$i% 9rou of #lock" -9G;.
$ii% Macro ;lock -M;.
I."a
i% +ind "imilar art" in other image"
ii% 1ncode where it wa" found -i%e% motion $ector.
iii% /re$iou" decoded image 0 Beference image
i$% 7mage to code 0 Target 7mage
$% 1ncode the re"idual only
H.291 C(."r
T"%#)!*+"- +-".A
i% Two dimen"ional -2-H. M = M H:T to remo$e intra-frame correlation
ii% Oig-2ag order to "can the tran"form coefficient"
iii% Bun (ength coding for 2ero-$alued coefficient" after quanti2ation
i$% Motion e"timation i" alied to $ideo "equence to imro$e the rediction
#etween "ucce""i$e frame"
$% Tran"mi""ion rate" control in the range of = LA K#"
$i% 1rror re"ilience including "ynchroni2ation and concealment technique required
in tran"mi""ion code, to co$er u channel error"
$ii% :ommon 7ntermediate +ormat -:7+. and _uarter :7+ -_:7+. for a "ingle
"olution to different $ideo format" -<T6: 4 /'(.
DHANALAKSHMI COLLEGE OF ENGINEERING, CHENNAI
DEPARTMENT OF INFORMATION TECHNOLOGY
IT2302 - INFORMATION THEORY AND CODING
UNIT - IJ
Part A
1% Stat" t#" Car!(+- &"t#(.- +-". $(r t"<t %(&3r"--!(). /N0D-102
The $ariou" method" u"ed for te!t comre""ion%
1% (o""y comre""ion
'll original data can #e reco$ered when the file i" uncomre""ed%
2% (o""le"" comre""ion
3ith lo""le"" comre""ion, e$ery "ingle #it of data that wa" originally in the file
remain" after the file i" uncomre""ed
2% #at !- S,).r(&"4 /N0D-102
6yndrome gi$e" an indication of error" re"ent in recei$ed $ector b?c if ?&
T
= 0,
then there are no error" in b?c and it i" $alid code$ector% The non 2ero $alue of ?&
T
i"
called b"yndromec% 7tc" non 2ero $alue indicate" that b?c i" not a $alid code $ector and it
contain" error"%
)% #, ar" %,%'!% %(."- "<tr"&"', 8"'' -+!t". $(r "rr(r ."t"%t!()4 /N0D-102
i% They are ea"y to encode
ii% They ha$e well defined mathematical "tructure% Therefore efficient decoding
"cheme" are a$aila#le%
A% Stat" t#" -,).r(&" 3r(3"rt!"-. /A0M-112
The "yndrome roertie" are,
i% 6yndrome i" o#tained #y 6 = ?&
T
ii% 7f ? = = then 6 = 0 i%e%, no error in outut
iii% 7f ? = = then 6 d 0 i%e%, there i" an error in outut%
i$% 6yndrome deend" uon the error attern only, i%e%, 6 = 1&
T

J% Stat" t#" 3r(3"rt!"- ($ %,%'!% %(."-. /M01-132
The roertie" of cyclic code" are,
a. L!)"ar!t, Pr(3"rt,
Thi" roerty "tate" that "um of any two codeword" i" al"o a $alid codeword% +or
e!amle let =1 and =2 aretwo codeword"% Then,
=
)
= =
1
* =
2
&ere =) i" al"o a $alid codeword% Thi" roerty "how" that cycli" code i" al"o a linear
code%
7. C,%'!% Pr(3"rt,
1$ery cyclic "hift of the $alid code $ector roduce" another $alid code$ector% ;ecau"e of
thi" roerty, the name bcyclicc i" gi$en% :on"ider an n-#it code$ector a" "hown #elow:
= = C!
n-1
,!
n-2
,e%%,!
1
,!
0
E
L% r!t" "<a&3'" $(r r"3"t!t!() %(.". /N0D-132
' "ingle me""age #it i" encoded in to a #lock of bnc identical #it" roducing a -n, 1. #lock
code% There are only two code word" in the code%
i% all-2ero code word
ii% all-one code word
Q% r!t" t#" 3r(3"rt!"- ($ a -,).r(&". /N0D-132
The "yndrome deend" only on the error attern" and not on the tran"mitted code
word% 'll error attern" that differ #y a code word ha$e the "ame "yndrome%
M% #at ar" t#" t,3"- ($ 1PEG a':(r!t#&-4
There are two tye" of \/19 algorithm"%
Da-"'!)" 1PEGA Huring decoding, thi" algorithm draw" line until comlete image
i" "hown%
Pr(:r"--!C" 1PEGA Huring decoding, thi" \/19 algorithm draw" the whole
image at once, #ut in $ery oor quality% Then another layer of data i" added o$er the
re$iou" image to imro$e it" quality% /rogre""i$e \/19 i" u"ed for image" on the
we#%the u"ed can make out the image #efore it i" fully downloaded%
K% #at !- -3at!a' $r"*+")%,4
The rate of change of i!el magnitude along the "canning line i" called "atial frequency%
10% E<3'a!) R+)-'"):t# %(.!):4
The runlength encoding i" "ile"t lo""le"" encoding technique"% 7t i" mainly u"ed
to comre"" te!t or digiti2ed document"% ;inary data "tring" are #etter comre""ed #y
runlength encoding% :on"ider the #inary data "tring
1111110000011111eee%
7f we aly runlength coding to a#ou$e data "tring, we get,
Q,1> L,0> J,1> ),0ee
Thu" there are "e$en #inary 1", followed #y "i! #inary 0" followed #y fi$e #inary 1"
and "o on%
11% #at !- ar!t#&"t!% %(.!):4
The arithmetic coding offer" the oertunity to create a code that e!actly
rere"ent" the frequency of any character% ' "ingle character will #e defined #y #elonging
to a "ecific inter$al% +or more frequent character" larger inter$al" will #e u"ed contrary
to rare character"% The "i2e of tho"e inter$al" i" orortional to the frequency% The
arithmetic coding i" the mo"t efficient rocedure #ut it" u"age will #e re"tricted #y
atent"%
Part D
1. F(r a '!)"ar 7'(%B %(.", 3r(C" 8!t# "<a&3'" t#at
/!2T#" -,).r(&" ."3").- ()', () "rr(r 3att"r) a). )(t () tra)-&!tt". %(." 8(r..
/?2
/!!2 A'' "rr(r 3att"r)- t#at .!$$"r 7, a %(."8(r. #aC" t#" -a&" -,).r(&".
/?2 /N0D-102
i% the num#er of codeworde i" 2
k
"ince there are 2
k
di"tinct me""age"%
ii% The "et of $ector" Cg
i
E are linearly indeendent "ince we mu"t ha$e a "et of unique
codeword"%
iii% linearly indeendent $ector" mean that no $ector g
i
can #e e!re""ed a" a linear
com#ination of the other $ector"%
i$% The"e $ector" are called #ai"e" $ector" of the $ector "ace :%
$% The dimen"ion of thi" $ector "ace i" the num#er of the #a"i" $ector which are k%
$i% 9
i
f : the row" of 9 are all legal codeword"%
$ii% ' -n,k. #lock code , where ;lock : encoder accet" a #lock of me""age "ym#ol" and
generate" a #lock of codeword
$iii% (inear : addition of any two $alid codeword re"ult" in another $alid codeword
C(." rat"
i% :ode rate increa"e" , error correcting caa#ility decrea"e" more #andwidth
efficiency
ii% :ode rate decrea"e" , error correcting caa#ility increa"e" wa"te of #andwidth%
2. C()-!."r t#" :")"rat!() ($ a /;,52 %,%'!% %(." 7, t#" :")"rat(r 3(',)(&!a' g/x2 = 1
Kx K x
3
.
/!2 Ca'%+'at" t#" %(." 8(r. $(r t#" &"--a:" -"*+")%" L1001M a). %()-tr+%t
-,-t"&at!% :")"rat(r &atr!< G./?2
/!!2 Dra8 t#" .!a:ra& ($ ")%(."r a). -,).r(&" %a'%+'at(r :")"rat". 7, t#"
3(',)(&!a'. /?2 /A0M-112
G!C") .ata
9enerator /olynomial, 9-. = ) * * 1
Me""age "equence M = 1001
-i.% To determine length of the coded "equence:
To degree of the generator olynomial i" q = )% &ence there are ) check #it" in coded
"equence% (ength of me""age" "equence i" k = A% Therefore total num#er of #it" in the
coded "equence will #e,
< = k * q
= A * ) =Q
Thi" i" -Q, A. cyclic code%
-ii.% To determine tran"mitted codeword:
The check #it olynomial i" gi$en a",
:-. = rem T
q
M-. 4 9-. U
The me""age" "equence i", M = 1001
M-. =
)
* 0
2
* 0 * 1
6ince q = ),
q
M-. will #e,

q
M-/. =
)
* 0
2
* 0 * 1

q
M-/. = )-
)
* 0
2
* 0 * 1.
=
L
* 0
J
* 0
A
*
)
Thu" the remainder i", :-. =
2
*
: = -1 1 0.
The "y"tematic form of the codeword i" gi$en a",
= = -m
)
m
2
m
1
m
0
: c
2
c
1
c
0
.
= -1 0 0 1 : 1 1 0.
Thi" i" the required tran"mitted codeword%
3. E<3'a!) -,).r(&" a). !t- 3r(3"rt!"-. /92 /M01-132
i% &uge "torage memory -and "earching time. i" required #y "tandard array
decoding%
ii% Hefine the "yndrome
- = CH
T
= -% * ". H
T
= "H
T

iii% The "yndrome deend" only on the error attern and not on the tran"mitted
codeword%
i$% Therefore, each co"et in the array i" a""ociated with a unique "yndrome%
The "yndrome roertie" are,
i% 6yndrome i" o#tained #y 6 = ?&
T
ii% 7f ? = = then 6 = 0 i%e%, no error in outut
iii% 7f ? = = then 6 d 0 i%e%, there i" an error in outut%
i$% 6yndrome deend" uon the error attern only, i%e%, 6 = 1&
T

5. E<3'a!) -,).r(&" ."%(.!): !) '!)"ar 7'(%B %(."- 8!t# "<a&3'". /62 /M01-132
&amming :ode i" a (inear ;lock :ode% (inear ;lock :ode mean" that the
codeword i" generated #y multilying the me""age $ector with the generator matri!%
Minimum weight a" large a" o""i#le% 7f minimum weight i" 2t+1, caa#le of detecting 2t
error #it" and correcting t error #it"%
6yndrome decoding i" a more efficient method of decoding a linear code o$er a noi"y
channel% 6yndrome decoding i" minimum distance decoding u"ing a reduced looku
ta#le%
6. E<3'a!) t#" Ha&&!): %(."- 8!t# "<a&3'". /192 /N0D-122
&amming :ode" are "till widely u"ed in comuting, telecommunication, and other
alication"% &amming :ode" al"o alied in
i% Hata comre""ion
ii% 6ome "olution" to the oular u22le The &at 9ame
iii% ;lock Tur#o :ode"
' TQ,AU #inary &amming :ode
i% (et our codeword #e -!
1
!
2
e !
Q
. g +
2
Q
ii% !
)
, !
J
, !
L
, !
Q
are cho"en according to the me""age -erha" the me""age it"elf i"
-!
)
!
J
!
L
!
Q
..%
iii% !
A
:= !
J
* !
L
* !
Q
-mod 2.
i$% !
2
:= !
)
* !
L
* !
Q

$% !
1
:= !
)
* !
J
* !
Q
$i% 7f -y
1
y
2
e y
Q
. i" recei$ed and a#c d 000, then we a""ume the #it a#c i" in error
and "witch it% 7f a#c=000, we a""ume there were no error" -"o if there are three or
more error" we may reco$er the wrong codeword.%
$ii% +or an Tn, kU linear code, the generator matri! i" a khn matri! for which the row
"ace i" the gi$en code%
$iii% ' check matri! for an Tn, kU i" a generator matri! for the dual code% 7n
other word", an -n-k.hk matri! M for which M< = 0 for all ! in the code%
i!% +or a gi$en r, form an r h 2
r
-1 matri! M, the column" of which are the #inary
rere"entation" -r #it" long. of 1, e, 2
r
-1%
!% The linear code for which thi" i" the check matri! i" a T2
r
-1, 2
r
-1 0 rU #inary
&amming :ode = C<=-!
1
!
2
e !
n
. : M<
T
= 0E%
9. E<3'a!) !) ."ta!', C,%'!% %(."-. /192 /N0D-122
They o""e"" rich alge#raic "tructure that can #e utili2ed in a $ariety of way"%
They ha$e e!tremely conci"e "ecification"% They can #e efficiently imlemented u"ing
"imle "hift regi"ter"% Many ractically imortant code" are cyclic% :on$olution code"
allow to encode "tream" od data -#it".%
i. &amming code i" u"eful #ut there e!i"t code" that off /olynomial rere"entation
of cyclic code"%
C(x) = C
n-1
x
n-1
+ C
n-2
x
n-2
+ + C
1
x
1
+ C
0
x
0
,
where, in thi" cour"e, the coefficient" #elong to the #inary field {0,1}%
ii% That i", if the codeword i" (1010011) -c6 fir"t, c0 la"t., we write it a" x
6
+ x
4
+ x

+ 1
iii% 'ddition and "u#traction of olynomial" 0 Hone #y doing #inary addition or
"u#traction on each #it indi!idua""#, no carry and no #orrow%
i$% Hi$i"ion and multilication of olynomial"% Try di$ide x
$
+ x
2
+ x

+ 1 #y x

+ 1%
$% 6ame -if not larger. error control caa#ilitie" while can #e imlemented much
"imler%
$i% :yclic code i" a linear code that any cyclic "hift of a codeword i" "till a codeword%
$ii% Make" encoding4decoding much "imler, no need of matri! multilication%
$iii% ' (n,k) cyclic code can #e generated #y a olynomial g(x) which ha" degree n-k
and i" a factor of x
n
- 1%
:all it the generator olynomial%
i% 9i$en me""age #it", -m
k-1
m
1
m
0
), the code i" generated "imly a":
ii% 7n other word", C(x) can #e con"idered a" the roduct of m(x) and g(x)%
DHANALAKSHMI COLLEGE OF ENGINEERING, CHENNAI
DEPARTMENT OF INFORMATION TECHNOLOGY
IT2302 - INFORMATION THEORY AND CODING
UNIT - J
Part A
1% #at ar" %()C")t!()a' %(."-4 H(8 ar" t#", .!$$"r")t $r(& 7'(%B %(."-4 /N0D-102
:on$olution code" are error detecting code" u"ed to relia#ly tran"mit digital data
o$er unrelia#le communication channel "y"tem to channel noi"e%
2% Stat" t#" 3r!)%!3'" ($ T+r7( %(.!):. /N0D-102
The "ignificance of Tur#o coding i",
i. High weight code words
ii. 'ecoder generates estimates of codewords in two stages of decoding and
interleaving.deinter leaving.
iii. This is li/e circulation of air in turbo engine for better performance. Hence these
codes called turbo codes
)% #at ar" t#" r"a-()- t( +-" a) !)t"r '"aC"r !) a t+r7( %(."4 /A0M-112
'n interlea$er i" a de$ice that rearrange" the ordering of "equence of "ym#ol" in a
determini"tic manner% The two main i""ue" in the interlea$er "i2e and the interlea$er ma%
7nterlea$er i" u"ed to feed the encoder" with ermutation" "o that the generated
redundancy "equence" can #e a""umed indeendent%
A% D"$!)" %()-tra!)t '"):t# /A0M-112
The con"traint length -K. of a con$olutional code i" defined a" the num#er of
"hift" a "ingle me""age #it to enter the "hift regi"ter and finally come" out of the encoder
outut%
K= M * 1
J% #at ar" t#" .!$$"r")%"- 7"t8"") 7'(%B a). %()C('+t!() %(."-4 /M01-132
6%<o ;lock code" :on$olution code"
1%
2%
)%
The information #it" are followed #y
the arity #it"%
There i" no data deendency
#etween #lock"
5"eful for data communication"%
The information #it" are "read along
the "equence
Hata a""e" through con$olutional
code" in a continuou" "tream%
5"eful for low 0 latency
communication"%
L% D"$!)" C()-tra!)t L"):t# ($ a C()C('+t!()a' C(.". /N0D-132
:on"traint length i" the num#er of "hift" o$er which the "ingle me""age #it can
influence the encoder outut% 7t i" e!re""ed in term" of me""age #it"%
Q% #at ar" %()C('+t!()a' %(."-4 /N0D-122
A convolutional code in which parity bits are continuously interleaved by
information (or message bits.
M% D"$!)" T+r7( %(." /N0D-122
The /arallel :oncatenated :on$olutional :ode"-/:::., called Tur#o :ode", ha"
"ol$ed the dilemma of "tructure and randomne"" through concatenation and interlea$ing
re"ecti$ely% The introduction of tur#o code" ha" gi$en mo"t of the gain romi"ed #y the
channel 0 coding theorem%
Part D
1. E<3'a!) !) ."ta!', t#" J!t"r7! a':(r!t#& $(r ."%(.!): ($ %()C('+t!()a' %(."-, 8!t# a
-+!ta7'" "<a&3'". /192 /N0D-102
The Viter#i algorithm erform" Ma!imum likelihood decoding%7t find a ath through
trelli" with the large"t metric -minimum &amming di"tance4minimum 1uclidean
di"tance.%
i% 't each "te in the trelli", it comare" the metric of all ath" entering each "tate,
and kee" only the ath with the large"t metric -minimum &amming di"tance.
together with it" metric% The "elected ath i" known a" "ur$i$or ath%
ii% 7t roceed" in the trelli" #y eliminating the lea"t likely ath"%
' :on$olutional code i" "ecified #y three arameter" -n,k,K. or -k4n,K. where
i% Bc=k4n i" the rate efficiency, determining the num#er of data #it" er coded #it%
ii% % i" the "i2e of the "hift regi"ter%
iii% :on"traint length = nZK, i%e% the effect of each #it ha$e it" influence on nZK #it"%
Pr(7'"&- () J!t"r7! A':(r!t#&A
i% :omutational comle!ity increa"e" e!onentially with con"traint length%
ii% The u"ually u"ed &amming di"tance in V' i" "u#-otimum and therefore lo"e
"ome erformance%
iii% Viter#i algorithm i" a M( -otimum. algorithm if 1uclidean di"tance i" u"ed%
A33'!%at!() ($ J!t"r7! A'(:(r!t#!&A
i% :on$olutional decoding and channel trelli" decoding%
ii% 6eech and character recognition %
iii% Gtical character recognition%
i$% H<' "equence analy"i" %
2. C()-tr+%t a %()C('+t!() ")%(."r $(r t#" $(''(8!): -3"%!$!%at!()- A rat" "$$!%!")%,
102, %()-tra!)t '"):t# 3, t#" %())"%t!()- $r(& t#" -#!$t r":!-t"r t( &(.+'(-2 a.."r
ar" ."-%r!7". 7, t#" $(''(8!): "*+at!()-, g
1
/x2 = 1 K x K x
2
, g
2
/x2 = 1 K x
2
.
D"t"r&!)" t#" (+t3+t %(."8(r. $(r t#" &"--a:" L10011M. /192 /A0M-112
9i$en:
g
1
-x. = 1 * x * x
2

g
2
-x. = 1 * x
2
&ere g
1
-x. and g
2
-x. ha" only two inut"% &ence we mu"t interret the con"traint length
a" nM%
&ere rate efficiency = k 4 n = 1 4 2
n = 2
6ince con"traint length i" gi$en a",
nM = A
2 h M = A
M = A 6ince n = 2
Thu" there are two "torage location"%
g
-1.
-. = 1 * and g
-2.
-. =
m = -1 1 1 0.
m-. =
)
*
2
*
!
-1.
-. = g
-1.
-. m-.
= -1 * .-
)
*
2
* .
=
A
*
= 0 * * 0 * 0 *
A
=
i
-1. = C0 1 0 0 1E
'nd, !
-2.
-. = g
-2.
-. % m-.
= -
)
*
2
* .
=
A
*
)
*
2
= 0 * 0 *
2
*
)
*
A
=
i
-2. = C0 0 1 1 1E
&ence outut "equence after multile!ing !
i
-1. and !
i
-2. will #e
=
i
= C0 0 1 0 0 1 0 1 1 1E
3. E<3'a!) !) ."ta!', T+r7( ."%(.!):. /192 /A0M-112
D"$!)!t!()
i% &igh weight code word"
ii% Hecoder generate" e"timate" of codeword" in two "tage" of decoding and
interlea$ing-deinter lea$ing%
iii% Thi" i" like circulation of air in tur#o engine for #etter erformance% &ence
the"e code" called tur#o code"
T+r7( ")%(."r .!a:ra&
T+r7( ."%(."r
:riterion
0 +or n ro#a#ili"tic roce""or" working together to e"timate common "ym#ol", all
of them "hould agree on the "ym#ol" with the ro#a#ilitie" a" a "ingle decoder
could do
5. H(8 !- .
$r""
."t"r&!)" +-!): Tr"''!- a':(r!t#&4 E<3'a!). /52 /N0D-132
Tr"''!- R"3r"-")tat!() ($ C()C('+t!()a' C(."
6tate diagram i" ]unfolded^ a function of time
Time indicated #y mo$ement toward" right
:ontent" of "hift regi"ter" make u S"tateS of code:
i% Mo"t recent inut i" mo"t "ignificant #it of "tate%
ii% Glde"t inut i" lea"t "ignificant #it of "tate%
iii% 'llowa#le tran"ition" are denoted #y connect" #etween "tate"
i$% tran"ition" may #e la#eled with tran"mitted #it"
E<a&3'" ($ Tr"''!- D!a:ra&
k=1, n=2, %=$ con$olutional code
6. E<3'a!) !) ."ta!', %()C('+t!()a' %(." a). %(&3ar" !t 8!t# 7'(%B %(."-./?2/N0D-122
D'(%B J"r-+- C()C('+t!()a' C(."-
;lock code" take k inut #it" and roduce n outut #it", where k and n are large there i"
no data deendency #etween #lock" u"eful for data communication"%
:on$olutional code" take a "mall num#er of inut #it" and roduce a "mall num#er of
outut #it" each time eriod%
i% data a""e" through con$olutional code" in a continuou" "tream
ii% u"eful for low- latency communication"
iii% k #it" are inut, n #it" are outut
i$% <ow k a n are $ery "mall -u"ually k=1-), n=2-L.
$% 7nut deend" not only on current "et of k inut #it", #ut al"o on a"t inut% The
num#er of #it" which inut deend" on i" called the Scon"traint lengthS %
$i% +requently, we will "ee that k=1
9. E<3'a!) !) ."ta!', T+r7( %(."- a). t#"!r +-"-. /192 /N0D-122
T+r7( C(."-
7111 7nternational :omm conf 1KK) in 9ene$a;errou, 9la$ieu!% : b<ear 6hannon
(imit 1rror-:orrecting :oding: Tur#o code"c ro$ided $irtually error free communication
at data date4ower efficiencie" #eyond mo"t e!ert" though Tur#o code" )0 year" ago%
i% <on"y"tematic
ii% <onrecur"i$e com#ination of con$% 1ncoder" ;errou et al at 1KK)
iii% Becur"i$e 6y"tematic
i$% ;a"ed on "eudo random
$% 3ork" #etter for high rate" or high le$el of noi"e
$i% Beturn to 2ero "equence"
T+r7( E)%(."r
i% The k-#it #lock i" encoded < time" with different $er"ion" -order.
ii% /ro the "equence remain" BTO i" 142
<$
% Bandomne"" with 2 encoder"> error ro of
10
-J
%/ermutation" are to fi! d
min
% Becur"i$e 6y"tematic :oder% Beturn to 2ero
"equence% <on recur"i$e encoder "tate goe" to 2ero after $ b0c% B6: goe" to 2ero with
/= 142
$
if one want" to tran"form con$% into #lock code> it i" automatically #uilt in%
7nitial "tate i will reeat after encoding k %
C()C('+t!() E)%(."r-
i% Tur#o Hecoding
ii% Tur#o Hecoding
+or n ro#a#ili"tic roce""or" working together to e"timate common "ym#ol", all of them
"hould agree on the "ym#ol" with the ro#a#ilitie" a" a "ingle decoder could do
The inut" to the decoder" are the (og likelihood ratio -((B. for the indi$idual "ym#ol d%
((B $alue for the "ym#ol d i" defined - ;errou. a" Tur#o Hecoder%
The 676G decoder ree$aluate" the ((B utili2ing the local ?1 and ?2 redundancie" to
imro$e the confidence
Tur#o Hecoding
i% 5
i
: modulating #it C0,1E
ii% ?
i
: recei$ed #it, outut of a correlator% :an take any $alue -"oft.%
iii% Tur#o Hecoder inut i" the log likelihood ratio
i$% B-u
i
. = log T /-?
i
i5
i
=1.4-/-?
i
i5
i
=0.U
$% +or ;/6K, B-u
i
. =2 ?i4 -$ar.
2
$i% +or each data #it, calculate the ((B gi$en that a "equence of #it were "ent Tur#o
Hecoding
;. E<3'a!) %()C('+t!()a' ")%(."r 8!t# a) "<a&3'". /92 /A0M-102
:on$olutional :ode":
:on$olution code" are error detecting code" u"ed to relia#ly tran"mit digital data o$er
unrelia#le communication channel "y"tem to channel noi"e%
i% The information #it" are "read along the "equence
ii% Hata a""e" through con$olutional code" in a continuou" "tream
iii% 5"eful for low 0 latency communication"%
7nterlea$ing
3e ha$e a""umed "o far that #it error" are indeendent from one #it to the ne!t
7nterlea$ing
3e ha$e assumed "o far that #it error" are indeendent from one
#it to the ne!t
7n mo#ile radio, fading make" bursts of error likely%
7nterlea$ing i" u"ed to try to make the"e error" indeendent again
1rror :ontrol :oding , j ;rian H% 3oerner , reroduced #y: 1rhan '% 7<:1
Heth
Gf
7nterlea$ing
1
1
L
2
11
)
1L
A
21
J

)1
Q

2L
L
J
2K
10
)0
)0
)A
)J
)J
(ength
Grder
;it"
Tran"mitted
Grder
;it"
Becei$ed
i% ;oth code" con"tructed o$er 9+-2JL. -M-#it"4"ym#ol.
ii% Guter code i" a -2M,2A. "hortened B6 code
iii% 7nner code i" a -)2,2M. e!tended B6 code
i$% 7n #etween coder" i" a -2M,A. cro""-interlea$er
$% G$erall code rate i" r = 0%QJ
$i% Mo"t commercial :H layer" donct e!loit full ower of the error correction
coder
$ii% 7nner code rate i" k, con"traint length Q con$olutinal encoder
$iii% Guter :ode -2JJ,22). B6 code o$er 9+-2JL. 0 correct" any #ur"t error"
from con$olutional code"
i!% G$erall :ode Bate i" r= 0%A)Q
!% ' #lock interlea$er held 2B6 :ode word"
!i% Hee "ace channel i" "e$erely energy limited #ut not #andwidth limited