KW

WW
K J
K J
K J
K J
K J

KW

J
J

--

J
Source Coding
J

EF

KEADCF

KEData ReductionFW
KEData CompressionFW

E Memoryless SourcesFW
K

Huffman Coding J

--

E FE David HuffmanF

Shannon F

KEBound

KERedundancyF

E Codeword LengthFK
KK

KESymbolsFM
K K

K10 K
KM-1

K
K10

K10 K

E Huffman Code Tree F K

K

L E Average or Expected LengthF K

EF

--

l P
i 1

(1-9)

PiiliW

K

J

K J

J

0.3
0.22
0.2
0.12
0.1
0.04

X1
X2
X3
X4
X5
X6

K JP(X)X K

K0.1 + 0.04 = 0.14X6X5 K

K0.14 + 0.12 = 0.26X4EX6X5F K

K0.2 + 0.22 = 0.44X3X2 K

EX3X2F K
K0.26 + 0.44 = 0.7

--

K0.7 + 0.3 = 1.0X1 K

K K

K
K101WX3

WE JF
L = 1 0.3 + 3( 0.22 + 0.2 + 0.12 ) + 4 ( 0.1 + 0.04 ) = 2.48 bits.

P(X)

X1

0.3

X2

0.22

X3

0.2

X4

Codeword
0
0
0.44

1.0

100
101

0.7
0

0.12

110

0.26
X5

X2

0.1
0.04

0
0.14

1110

1111

JE JF

Hamming Coding J

--

E Richard Hamming F

E Hamming DistanceF
K

KEF
KRAM

E Control BitsF

FEData BitsF
K E2

mE m , nF
E 8 , 7 F K n
k K 7 8

Wn

n 2k k 1

(2-9)

m=k+n

J
W
2, 3, 4, 5, 6, 10

W

J J
--

J
m

3
7
15
31
63
1023

1
4
11
26
57
1013

2
3
4
5

6
10

J
K100110111010101

K2

W
F2EParity BitsF K
KEKKK168421

KEKKKK7653F K

K
W

KE1,3,5,7,9,11,..FW1

KE2,3,6,7,10,11,14,15..F W2

KE4,5,6,7,12,13,14,15,..F W4
KE16-31,48-63,80-95,..F 1616W16
K

--

?1?0?1?1 K
K

7E7 , 4F
K43
KW

Kp1, p2, and p41, 2, 4W

d3, d5,d6 and d7E3,5,6,7F

K J

J
Bit position

Encoded data bits

Parity
bit

p1
p2

p1 p2 d1 p4 d2 d3 d4
X

X X

X
X X

X X X X

p4

p1 d 3 d 5 d 7
p2 d 3 d 6 d 7
p4 d 5 d 6 d 7

(3-9)

XORKE XORF

X1

X2 Output
--

0
0
1
1

0
1
0
1

0
1
1
0

K JE7F

K12

W1100

p1 d 3 d 5 d 7 1 1 0 0

p2 d 3 d 6 d 7 1 0 0 1

p4 d 5 d 6 d 7 1 0 0 1

W7

0111100

E1, 2 and 4F

KE3, 5, 6 and 7F

E7,4F J

Data Word

0 0 0 0
0001
0010
0011
0 1 0 0
0101
0110
0 11 1

Decimal Value

0
1
2
3
4
5
6
7
--

Code Word

0000000
1101001
0101010
1000011
1001100
0100101
1100110
0001111

1000
1 0 01
1010
1011
1100
11 0 1
1110
1111

8
9
10
11
12
13
14
15

1110000
0011001
1011010
0110011
0111100
1010101
0010110
1111111

K6

W0110

p1 d 3 d 5 d 7 0 1 0 1

p2 d 3 d 6 d 7 0 1 0 1
p4 d 5 d 6 d 7 1 1 0 0
W7
1 1 0 0 1 1 0

WE6F
1110110

p1 d 3 d 5 d 7 1 1 0 0

p2 d 3 d 6 d 7 1 0 0 1
p4 d 5 d 6 d 7 1 0 0 1

--

KE p3 p1F

E 1 + 2 =3F
K01

Gray Coding J

EFrank GrayF

E Reflected Binary CodeF

K
W

KEF J

KEADCF J
K J

K J

W
K

KEMost Significant BitF

--

E Modulo 2F

K
K

J
K1 0 0 1 0 0 1
1001001

1101

EF11EF

0000

1110

1101

0000

K
K

EF1110

1 1 0 1 10 1W

J
KK4

K J

--

WEF1101

W
1 + 1 = 0 0, 1 + 0 = 1 1, 0 + 1 = 1 1

1 0 1 1

EN = 4 BitsF J

Gray Code
0000
0001
0011
0010
0110
0111
0101
0100
1100
1101
1111
1110
1010
1011
1001
1000

Binary Value
0000
0001
0010
0011
0100
0101
0110
0111
1000
1001
1010
1011
1 10 0
1101
1110
1111

--

Decimal Value
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

W
K

E Modulo 2F

KEMost Significant BitF

K

J
K1 1 0 1 1 0 1

W
1 1 0 1 1 0 1 K

EF1 1EF K
0

011 K
000 K

1110

0011
0000

EF1110

1 0 0 1 0 0 1W

--

K
K
K

K J K
K

J

0.24
0.21
0.2
0.12
0.1
0.08
0.04
0.01

X1
X2
X3
X4
X5
X6
X7
X8

K K
K K
K K
W K

--

2
5

7
8

KK01011101110101010 K
W K

Data Word

Decimal Value

0 0 0

001

010

011

1 0 0

101

110

11 1

Code Word

K1 1 0 1 0 1 1 K

K1 0 0 0 1 0 1 K
W K

Gray Code

Binary Value
0 0 0
0 0 1
0 1 0
0 1 1
1 0 0
1 0 1
1 1 0
1 1 1

--

Decimal Value
0
1
2
3
4
5
6
7