Souvce Cocdinq

flphabet 1t is the collectton of letters or Smbols

Symbols

Codewovd
Set ofbinavy seguences that are assgneel to alphebets
 Example
cocdewovel
Lettevs

Alphabet
a3
a4

Defintion
he conversion of the output of a ousCYete memovyless

Source (pms) into a seguence of binary (binare

symbols (binary
device that performs
cocdeworel is callel Source cocing. The

the source encocder.

thu's con uersion s callecl

binay4
Discvete Source
memoryless encooleY
SouYce Coms) Seouence

noal (o objectiue
DRecduce equived. banokwicdth

Recduce memory to store

bit Yate reguvec tor representation of

3) To minumu2e the avevGge

the reckundancy of the intormatton

the source velucinq
SOuYCe.

Redun dant informakion is unneecdec Cov) duplicakecd informab en

Reckundank bits ave extva binary ligits thak are generatecd and

movec coith a dala tvanser to ensuve thak no bits were lost

 oluving the dato transfer

Aim o souvce cocunq is to vecluce recduncdancy ok information

(ie compression of data) and hence minimize avevage bit rate .

Data so m
Source k3
Destinakion
Compresston

usinq di ffevenk ddaka compression algorithm

Shanhen Fano Algovi thm
Huffman cocling Plseti thm
*Souvce encockev is to olo laka compression

Ecample- u e coant to send soma ile (Aucliolvideo limage hiAe).

olata compres5 ton algovi thms on this s o me file to
we apply
ile Ivansmission of So m file is costlier
get K0 compressecl .

than transmission of lk0, We remove Some in formakion in

s0 me file such that no signikicant intormahion is lost Recdundant

removec from oriqína. tile to geb IKB compresscd
informadion is

Ale This is calleck vecdumolancy while doinq this representation

inlormation
we should not lase. any oviginal
Souvce coclinq the ovem

D ms X , coith entropy HCxthe avevage

t stats thak for a

is bouncdeol as L H(x)
codecworol lenqth L pev symbol

close to H(), thus wth Lman H(K)

further,L can be made as
 be covítten a s
the cocde eflicienmcy
can

H )

a measuve of the average informakie

kton
it is clelined
as
Entvopy

Con tent Source symbol.

per

of cocles (ov) roperties

of codes
Classificakion

x e d leneth coles:-

A xecd Aenqth coole is one cohose colecworol kenath

is fxcel
Example
code code 2

O0
J J is not a fiacecl
dength coole

(Vaviable Length
Coode

t has fixel length ceole cwith length 2

2 Vaviable Length co odes :

A vaviable length coole is ne whose cocdecword

length is not ice

3)Distinct cocdes:-
A code is dstinct each cocdewo vol is

coclecoovels
distinquishable from other
 Fzample:
code code 2 code 3
(Oo oo

OO

Dot a cistinct coode istinct cocdes

4 Pretix - tYee codes -

A sufficient condlition to assure thak a cocle is uniquel

decochealble is that no cocle word be the prefix of any other co ode

wovd. Codles thak satis fy this condition ave calleck pre fix -free

codes.
Sn a prefioc- fvee cocle, no coclecoorel is s preix f

another.

Example
code code 2 cole 3 code

oo

Dot a
ve fix Pre fic ot a

fvee cocAe free cocde PYe i ree colde

fvee cocde

Jeve oo Coceuoovc th
is Yepeakecd
 suth
reisc an e Dangling
cocles aned b
Consilev two
a

To check whether any codeuwo

of another cole cwovd. Heve
completelu present in the stavtin9

completely present in the

of the coclewove, it is
o is one

starting of the another coclesorcl. So here a

istheprei

of "
The remaininq term is calledl elana ling suff.

Heve
b: 01o

Dangling suffi (os)

Pre feac

Uniguely Decodable cocles:- CuDc):

sequence of cocles thak can be elecocledl

in only one cGY

Algorithm to check UDC:-

D Construct a list of all codecworcls

2)oe prefix Founol Je is uDC EnD

Prefx found AdeAdd dangling suffic to our list

JF DS- Some codeword Dot U DC

no more unaque 95 J E is UDC

 Example

Jn this cocewoals, prefix tound

no in any other codewovods
S0 t 1S UDC

Example
agO1 Find UDc (or) not ?
O Constvuct a list of all codeuwovcls

o,1,o
Check any prefx founel (ov) not.
Heve 'o is the prefic of oi. So cwe qo to

3r step
Adol dangling suffix to ourlist.

pre fioc dangling suffic

o,1o1,0
Heve dangling suffic (Ds) is same as the one of the coclecoov!

So it is not UDC

Example
0 2 Fincd uDc (or) not2
O Constvuct a list of all cocle worcls

10,1,
Check any preli found ov not
(
Heve is the prelix o So e go to
 3 step
suffix to ouv list ?
ndd dangling
1o, u,11,0J

pre fic dangling suffix

So we add the list of codeworols

Then again check.

acdl acan to
of 1 So we
i s the prefiac
.

the istof cocdeworos.

o,1, 111 OO

is Sqme as the one of cooleword So it is no t UDC

Here DS

6)Jnstantaneous Cocdes
The instantaneous codes, no
cocleuworcl is a prefix

For this veason, prefix free cooles are Some times

of another cocdeword.
-

known as ins tan taneous cocles

Test for instantoneous property

sufhcient concution tor a uniquely decodable coo!.

A necessavy and

to be instantaneous is that no complete coorod of a co de be a

anyother cocdetwordd
pre fix of

Example

O1
$3
 instantaneous. 3 c o d e doesnot
and 2set o codes ane

prefix of the codeword

beca use o"
is the
satisy prelix pvopevty
OI s the pre fix of the
.o is the prefix of o
O

not instantaneous
O1 Hence the 3 Code is
(odewovd

Optimal cocdes "

A cocde is saic to be optimal if it is instantaneou5

has mnimum avevaqe L for a

qiven source ith c quen
and

the 9ymbos
pvobabilitu assiqnmert for
source

Kraft Inegua lity

Let be a oms cwith alphabet x : (ia1,2,- m).

Assme that the length of the assigned binary Codecworol

covvesponcing to X is n

K 2

wich is known as Kraft ineguality

Problem
Consicdev a DmS x with symbols i , t : , 2,3,4 . Table

ist four possible binavy cocdes:

-

code A code G code c co de D

O O

12 O
 the cocdes except
cocde salisy the Kva f
)Show that all

ne guality
decocdable but codes
Shocw that codes A anol D ave uniguel

guely odecodable
and Cave not

Answev
Fov code A; n na 3 n4 2

K 2

2 2 - 2

K 2 2 +2 +2 +2

= O. 25 +0. 25 +0. 25 +0.25

K I
Sakis Kralt ineguality

)Fov code n , n2 2, ns 2 , n4 3

K:

- 2
n 3
K 2 2+2 2 2

25 O. 125
0 S t0 25 + O. +

K 125

Uot
Dot satis ty the Kvatt
ineguaity
fov c o c n22 3 43
 2 3
K
2+2 2 +2
O.5+ O 25+ O 125 t 0.125

Sakisy the Kraft ineguality

f o v code D; n ,n 2 3, 3 3 , n4 3

-3 - 3

K 2 22 2
0.5+ 0-125+o.125+O.125

K 0 3 75

Sakisfy the Kralt ineguality

Al codes except code sakisfy the Kvaft inequality
Codes A ond D ave prefix - fvee cocles. They are umguely

decocdable. Code G does not Satisfy the Kraft ineguality, ancl it

is not uniguely decoolable . Although cocle C does satisky the

Kraft inequality, but it is not uniguely odecoodable

ampltng
ntoduction
Adigital siqnal is supevior to an analog sig na

because stiS move robust to noise anol can

eastly be recoueved

Covrected and ampliiecl. Fov this reason, the

tencdency today
is to change an analoq Signal to ligital data

Changin9 ainalog sigynal to digital signal

Samplinq quanti zing Cocun

 analoq sig nal to a olagital signal
One c a n convert a n
Sampling anol uantizing (collectiuely calle Analoq to odigta

Conuerter ADC)

Samplina process
t is a basic opevation in the dqital commuunAc at ion. fA

ContinuouS time Siqnal is Conuevtecl to liscrete time s iqnal by

Sampling process.In this proceses, continuous time anakoq sig nal

in finite numbev fvalues) is sampleo by measuring tS

Amplitucle at a oiscrete instants

sianal is conuevtedl ento

So, the contimuous tme analoq

Corresponcinq seguence of samples thna are usually Spacec.

in time.Jt is necessary to choose the sampunq vate

uni omly

uniguely odebines the

properly, So the seguence of samples

oviginal analos signal

Sample
3tis a piece of data taken vom the cohole clata

wich is continuous in the time ekomain

Cwhen a SouYCe qeerates an analog siqnad and if that

hasto be igitizec, having s ancl Os, ie; high cov)low, the

Signal has to be iscvetizecl in time his iscvetization of

analog signel is calkecl sampling

 x(O Continuous time
Signal

Sampled
Sig na

Ts

Samplima Yate;
To sample a continuous time signal x(t) is to
To
repvesent at discrete numbe of points, t n Ts wheve
Ts
is the sampling perioc ano m integev
To dis CYetize the signals, the gap betcueen the

be termecl a
should be fixecd. Tha can as
qap
Samples
Samping peviodl T

Samplinqequenc t
Ts
s
(Sampling vate)

Sanplinq peviocl Ts
 rake tov the pevfect veconstvuction of the continuous time sia nal

from samples
when the sampling vae becomes exactly egual to 2w

samples per seconot fov a siqnal bandluoidth of W Hevtz, then

calleo nyguist rate.

it is

yguist rate 2 W (H2)

2x Highest reguency com po nent

sampling rate must be qveater than or

egual to
ny9ust
vake
s 2w

Samplina heovem: (nDy9,uist The orem )

he saFeicient number of samples of the signa

st be taken so that the original signal is Yepre sentect in

its Samples compketely. Also, it shoulo be possible to vecoue

struct the oviginal signal completely from its samples.

The number ok samples to be taken odepencls

on maXimun signal reguency presewk in the signal. Sampng

theoremgiues the complete icdea about the sampling of siqnal

A continuous time sigmal can be

completely vepvesentecd in its

Samples and vecouevecd back if the Sampling freguency fs22W

lerefs is
sampling fvequency ancl W is the maa1mum

reguency present in the signal.

Let us consicdev that siqnad (E is
stictly banddlimitecd, witl
no
veguency components igner than W Hevtz. Thal is fouviev

tansfovm x (f) of «() has the p ropevty that

fov Ifl w

he spectvum of Such siqnal is shown in fiqure

-w +W

We neeoa samplinq frequency

veguency at whuch there
Coould be no loss oF information , even oaltev samplinq Fov

this, we have the yguist vake that the samping freguency

should be two times the maximum (veguency. Jt is the

citica rate of sam plinq.

the siqnal x (t) is Samplec greater than (or)

egual to the Uygust rate, the oviqinmal signal can be vecouereo

ano if it is sampled beloco the nyguist rale, the Signal

Cannot be vecov ereo

)when s 2w
Wow we willconsiclev cuhen sampling fveguengy
veguend

eual to twice the maaimum signal freguency

 x(f)

fs 2 W

- fs -W O W

This is the spectrum of Samplecd vevsion of ( ) i t h

ts 2 W . Heve he infovmation is vepvoduceol coithout any loss

up amd hence recovery

is possible.
heve is no muCin

vate
Hence, this is a
gooo samplinq
2 When fs > 2W

is move
nDo we will consicler when samplinq freguency

than twice the maximum freguency of the siona.

fs 2W

W W

Cnaps no
chance of
aliasing
-

LV
Wfs

This is the spectvum of Sampled version of «(t) at fs > 2w

 Heve the infovmation Yeprocducech without any loss There is

no mx inq up anc hence Yecoue vy is possble. heve is

a ga
between the two neiqhbouying pectvums . herefove theve is no

muXnq possible an hence no aliasinq Thus ohen the sig nal

is to be veconstvucteodrom its samples, all freguency com ponent|

aYe veconstructeol faith fally since theve is no aliasinq

3) When ts < 2w

nNow coe coill be consider cwhen sampling eguency s

ess than twice the maximum sígnal fveguency

x f) fs < aw

fs
W

O +W
A.asing
Ts is the spectvum of sampled vevsio of «(t) cwith ts 2W

he spectvurn interfeve cwith each otheer s is callecd aliasing

infovmation,wich lcads to mixinq ano

heouevlappinq o up

oss of inovmatton here fove bcause of b aliasing the siqnal

Cith all its contents in ts samplecl

S not Yeprrsented prqpery
version
Eects of aliasing

Distovtion
be recouereod
lost ano it cannot
The data is

To avoid aliasing

Samplinq Yate must be ts 2

StvictlM bandlimit the siqnal to W

Prealias ilter shoulel be used. to linit band of

requencies of the sienal to w H2. Ay intrmation signal

consists of large ranqe of reguencies, There fove to decicle a

freguency is alwGus a problem.

Sampling
Theve fove a
signal is Srst passecl through a LPF

This ilte blocks all the wich

veguencies ove above W Hz

cohich ave unwantec. Tis callecl banellimitinq of the siqnal

). hen it becomes easy to odecicde sampling tveuency

Since maximum fveguency is ixeol at w Ha.

The LPF is then calleod prealias ilter Since t

is useo to prevent aliasing effect.

Reconstruction lterLow pass lter (ov Jnterpolation tev

The
The Loco pass lter is use. to e coue oviginal scna

fromits samples is is knoton as interpolation filtev. A LPF

is a filtev which passes only low
freguencies upto
upto aa spe
spechec
ci hec
cut off requency andd vejectS alother
fveguencies above cuk afC
veuency
Amplitucde

- w
W

Heve is
shavp change response ak
cut off equency, that is
amplitude responsebecomes sudcenly zevo ak cut off fveguency
which is not
possible practically. is means that an ideal LPF
is not physically
realizable. Jn place of coleal LPF, we use

practical iltev.
Ampitucle

-W W

Tis is the
fveguency vesponse of practical LPf.
Amplitude
vesponse decveases slowly to become 2evo.
o miimi 2e the
sstem samplinq rate,cwe
desive that the
iler
antialiasinq ilter have a small tranSitton
bandwide. Thck is from W to Cfs -w H2 a
tvrans it ion
banc is Yequired.
 AxCf)

-W W

-s-W -fs -fs+w |-W W s-W

-w W

Spectrum of oviginal signal

6) Spectvum of samplecl signal

)Amplitudle vesponse of pra.ctical LPF

Differen types of samplin technigues

Basically three types of sampling technig ues, namely,

)3leal Sampling
E is also knotn as instan taneous Sampling Cor) impulse

Samplinq. lvain of impulse is useol as a carrie signal for cdeal

3n this sampling technigue, the sampling fun ction

Sampling is a

tvain o impulses andl the pinciple useol is known as multiplication

 principle

x(t)
3(t)

This circit is known as the Switchinq samples. J we as5ume

ehar the closinq time of t o f the stoitch arproaches to zero, then

the owtpub q(t) of the civcut will contain only instan to neous

value of the inpuk signal (¬)

wicdth of the pulse approaches zero, the instantaneous

F of height egual to the

Samplig ves a train o
impulses
ualue of the inpu signal c(t) at the sampling
instantaneous

instan.
xlt) message Signa
input Siq nal
@signal to be
Samplecd

Sampling function

111111 t
Cov) impulse tvan

Sampleol signal
@)Resultant
Sgnal

2) DakuYal Sampling

Naural samplinq is a practical methocl of sampling

inwhich pulse have finite wicdth egual to

Sampling is done in accorolonce with the cavrier signal hich is

oliqital in natuve
C( n.

(0

Cwith the help of functional diaqram of a natural ampler,

samplecl signal g() is obtainec by multipicakion of mpling

function c() and the input signal(t)

Oken he switch s i s closecl.
cCE goes hiah, Therefove,
9 C ) ( ) , when c(0: A

when c(E) o

where A is the ampitude of c C0

9C) cc).«(0

c(t) is the troin of pulse

l a t Top mping
Flat top ike
samplinq is
naluvad sampling lat
Jn la
top sampling (OY) Yectanqulav pulse sampling the top of the samples
remauns constant a nd
anc is e2ucl to the instanta.neous value of the

base boanod siqnal () at the. of

oc stavt sampling

JE can be obser veol that only stavtinq ecdge of the pulse

value of the messaqe signal c(t).

vepresenk the ins tan taneous