ChannelCoding

Hafiz Malik
Dept. of Electrical & Computer Engineering
The University of Michigan-Dearborn
hafiz@umich.edu
http://www-perosnal-engin.umd.umich.edu/~hafiz
Channel Coding
class of signal transformations designed to
improve communication performance by
enabling the transmitted signals to better
withstand channel distortions such as noise,
interference, and fading.

Channel coding can be divided into two
major classes:
1. Waveform coding by signal design
2. Structured sequences by adding redundancy
Waveform Coding
deals with transforming waveform into better
waveform robust to channel distortion hence
improving detector performance.
Examples:
Antipodal signaling
Orthogonal signaling
Bi-orthogonal signaling
M-ary signaling
Trellis-coded modulation

Structured Sequences
deals with transforming sequences into better
sequences by adding structured redundancy (or
redundant bits). The redundant bits are used to detect and
correct errors hence improves overall performance of the
communication system.
Examples:
Linear codes
Hamming codes
BCH codes
Cyclic codes
Reed-Solomon codes
Non-Linear codes
Convolution codes
Turbo codes
Waveform Coding

M-ary Communications
Send multiple, M, waveforms
Choose between one of M symbols instead of 1 or 0.
Waveforms differ by phase, amplitude, and/or
frequency
Advantage: Send more information at a time
Disadvantage: Harder to tell the signals apart or
more bandwidth needed
4-ary Amplitude
Each symbol sends 2 bits
Deciding which level is correct
gets harder due to fading and
noise
RCV needs better SNR to achieve
accuracy
11
01
10
00
Orthogonal Signals
Definition

This means that the signals are perpendicular
to each other in M-dim space
For a correlative receiver, this means that each
incoming signal can be compared with a
model of the signal and the best match is the
symbol that was sent
( ) ( )
0
0
b
T
i j
C i j
p t p t dt
i j
=

=

=

}
Multi-Amplitude Shift Keying (MASK)
Send multiple amplitudes to denote different
signals
Typical signal configuration:
+/- p(t)
+/- 3 p(t)

+/- (M-1) p(t)

Multi-Phase
Binary Phase Shift Keying (BPSK)
1: |
1
(t)= p(t) cos(e
c
t)
0: |
0
(t)= p(t)cos(e
c
t+t)
M-ary PSK


Re
Im
x x
( ) ( )
2
cos
k c
p t p t t k
M
t
e
| |
= +
|
\ .
Re
Im
x x
x x
x
x
x
x
Quadrature Amplitude Modulation
(QAM)
Amplitude-phase shift keying (APK)

( ) ( ) ( ) ( ) ( )
( ) ( )
cos sin
cos
k k c k c
k c k
p t p t a t b t
p t r t
e e
e u
= +
= +
2 2
tan
k
k k k k
k
b
r a b
a
u
| |
= + =
|
\ .
r
i

u
i

Multitone Signaling (MFSK)
M symbols transmitted by M orthogonal
pulses of frequencies:


Receiver is a bank of mixers, one at each
frequency
Higher M means wider bandwidth
needed or tones are closer together
( )
2 /
k M
N k T e t = +
M-ary Comments
As M increases, it is harder to make good
decisions, more power is used
But, more information is packed into a symbol
so data rates can be increased
Generally, higher data rates require more
power (shorter distances, better SNR) to get
good results
How do we compare performance?
Symbols have different meanings, so what
does the probability of error, P
E
mean?
If a detection error is made, then more than
one bit is wrong
DCS can be faster at the price of being less
sensitive
15
Detect Error On Credit Card
16
Formula for detecting error
Let d2, d4, d6, d8, d10, d12, d14, d16 be all the even values
in the credit card number.
Let d1, d3, d5, d7, d9, d11, d13, d15 be all the odd values in
the credit card number.
Let n be the number of all the odd digits which have a value
that exceeds four
Credit card has an error if the following is true:
(d1 + d3 + d5 + d7 + d9 + d11 + d13 + d15) x 2 + n +
(d2 + d4 + d6 + d8 + d10 + d12 + d14 + d16)
0 mod(10) =
17
Detect Error On Credit Card
d1
d2 d3 d15 d16
n = 3
18
Now the test
(4 + 4 + 8 + 1 + 3 + 5 + 7 + 9) = 41
(5 + 2 + 1 + 0 + 3 + 4 + 6 + 8) x 2 + 3 = 61
41 + 61 = 102 mod (10) = 2
3
19
Credit Card Summary
The test performed on the credit card number is called a
parity check equation. The last digit is a function of the
other digits in the credit card. This is how credit card
numbers are generated by Visa and Mastercard. They start
with an account number that is 15 digits long and use the
parity check equation to find the value of the 16
th
digit.

This method allows computers to detect 100% of single-
position errors and about 98% of other common errors (For
All Practical Purposes p. 354).
Examples
ISBN (international standard book number)
0 20 1 36186 8
UPC (universal product codes)
12-digit sequence
0 16000 66610 8
21
What is a code?
A code is defined as an n-tuple of q elements. Where q is any
alphabet.

Ex. 1001 n=4, q={1,0}
Ex. 2389047298738904 n=16, q={0,1,2,3,4,5,6,7,8,9}
Ex. (a,b,c,d,e) n=5, q={a,b,c,d,e,,y,z}

The most common code is when q={1,0}. This is known as a
binary code.
22
The purpose
A message can become distorted through a
wide range of unpredictable errors.

Humans
Equipment failure
Lighting interference
Scratches in a magnetic tape

23
Why error-correcting code?
To add redundancy to a message so the
original message can be recovered if it has
been garbled.

e.g. message = 10
code = 1010101010
24
Send a message
Message Encoder Channel Decoder Message
10 101010 noise 001010 10
25
Encoding
Nave approach
Hamming codes
26
Take Nave approach
Append the same message multiple times.
Then take the value with the highest average.
Message:= 1001
Encode:= 1001100110011001
Channel:= 1001100100011001
Decode: = a1 = Average(1,1,0,1) = 1
a2 = Average(0,0,0,0) = 0 ... (a1,a2,a3,a4)
Message:= 1001
Nave Solution: Example
Single CheckSum -
Truth table:




General form:
Data=[1 1 1 1]
Message=[1 1 1 1 0]

Repeats
Data = [1 1 1 1]
Message=
[1 1 1 1]
[1 1 1 1]
[1 1 1 1]


A B X-OR
0 0 0
0 1 1
1 0 1
1 1 0
Why Nave Solution is Inefficient?
( )
ratio noise to signal the is S/N
Capacity Channel raw is W
Capacity Channel is C
/ 1 log
2
N S W C + s
Shannon Efficiency
Repeat 3 times:
This divide W by 3
It divides overall capacity by at least
a factor of 3x.

Single Checksum:
Allows an error to be detected but
requires the message to be discarded
and resent.
Each error reduces the channel
capacity by at least a factor of 2
because of the thrown away
message.


29
Hamming [7,4] Code
The seven is the number of digits that make the code.
E.g. 0100101
The four is the number of information digits in the code.
E.g. 0100101

30
Hamming [7,4] Encoding
Encoded with a generator matrix. All codes can be formed from
row operations on matrix. The code generator matrix for this
presentation is the following:


| |
k k n k k
P I G

=
(
(
(
(

= :
1 1 1 1 0 0 0
0 1 1 0 1 0 0
1 0 1 0 0 1 0
1 1 0 0 0 0 1
31
Hamming [7,4] Codes
16 2
4
=
1000011
0100101
0010110
0001111
1100110
1010101
1001100
0110011
0101010
0011001
1101001
1001010
1111111
0111100
0011001
0000000
128 2
7
=
Codes

Possible codes
Multi-dimensional Codes
Code Space:
2-dimensional
5 element states
Circle packing makes more
efficient use of the code-space
Cannon Balls
http://wikisource.org/wiki/Cannonball_stacking
http://mathworld.wolfram.com/SpherePacking.html
Efficient Circle packing is the same
as efficient 2-d code spacing
Efficient Sphere packing is the same
as efficient 3-d code spacing
Efficient n-dimensional sphere packing is the same as n-code spacing
34
Minimum Weight Theorem
Definitions
Proof of Theorem
35
Definitions
The weight of a code is the number of nonzero components it
contains.
e.g. wt(0010110) = 3

The minimum weight of Hamming codes is the weight of the
smallest nonzero vector in the code.

e.g. d(G)= 3
(
(
(
(

=
1 1 1 1 0 0 0
0 1 1 0 1 0 0
1 0 1 0 0 1 0
1 1 0 0 0 0 1
G
36
Definitions
The distance between two codes u and v is the number of
positions which differ
e.g. u=(1,0,0,0,0,1,1)
v=(0,1,0,0,1,0,1)
dist(u,v) = 4

Another definition of distance is wt(u v) = dist(u,v).
37
Definitions
) , ( ) , ( ) , ( w v dist v u dist w u dist + s
) , ( ) , ( u v dist v u dist =
0 ) , ( = u u dist
For any u, v, and w in a space V, the following three conditions
hold:
38
Definitions
The sphere of radius r about a vector u is defined as:

} ) , ( | { ) ( r v u dist V v u S
r
s e =
e.g. u=(1,0,0,0,0,1,1)

(0,0,0,0,0,1,1)
(1,1,0,0,0,1,1)
(1,0,0,0,0,0,1)
(1,0,0,0,0,0,1)
(1,0,1,0,0,1,1)
(1,0,0,1,0,1,1)
(1,0,0,0,1,1,1)
39
Minimum Weight Theorem
If d is the minimum weight of a code C, then C can correct
t = [(d 1)/2] or fewer errors, and conversely.

40
Proof
Want to prove that spheres of radius t = [(d 1)/2] about
codes are disjoint. Suppose for contradiction that they are
not. Let u and w be distinct vectors in C, and assume that
) ( ) ( w S u S v
t t
e
u w v
41
Proof
By triangle inequality
t w v dist v u dist w u dist 2 ) , ( ) , ( ) , ( s + s
u w
v
Since spheres of radius t = [(d 1)/2] so and this gives

But since

We have a contradiction. Showing the sphere of radius t about
codes are disjoint.
42
Proof
1 2 s d t
1 2 s s + s d t dist(u,w) dist(u,v) dist(u,w)
d w u wt w u dist > = ) ( ) , (
43
Result of Theorem
Since d(G) = 3 then for t = [(3 1)/2] = 1 or fewer errors, the
received code is in a disjoint sphere about a unique code
word.
(
(
(
(

=
1 1 1 1 0 0 0
0 1 1 0 1 0 0
1 0 1 0 0 1 0
1 1 0 0 0 0 1
G
Parity-Check Matrix
The parity check matrix is found by solving the generator matrix for

For each generator matrix G, there exists an (n k) x n matrix H, such that
the rows of G are orthogonal to the rows of G; i.e.,

where H
T
is the transpose of H, and 0 is an k x (n k) all zeros matrix .
The matrix H is called the parity-check matrix, that can be used to decode
the received code words.
0 =
T
GH
| |
| |
3 3 4 3
3 3
:
:
1 0 0 1 0 1 1
0 1 0 1 1 0 1
0 0 1 1 1 1 0
x x
T
I P
I P
H
=
=
(
(
(

0 =
T
GH
45
Channel Decoding
1. Brut force Approach: list all possible
messages
2. Syndrome Decoding
46
Brute Force Approach:
List all messages
This is done by generating a list of all the possible messages.
For something small like the Hamming [7,4] codes the task is
feasible, but for codes of greater length it is not. An example
of a list is as follows:
Code words 1000011 0100101 0010110
0000011 0000101 0000110
Other 1000001 0100111 0010100
Received 0010011 0001101 1010110
Words 1100011 1100101 0110110

47
Brute Force Approach:
List all messages
For example, if the received code was 0001101 then it would
be decoded to 0100101 from the list.
Code words 1000011 0100101 0010110
0000011 0000101 0000110
Other 1000001 0100111 0010100
Received 0010011 0001101 1010110
Words 1100011 1100101 0110110

48
Syndrome Decoding
Consider a transmitted code c
m
and y is the received sequence,
y can be expressed as,

where e denotes binary error vector.
The decoder calculate product




(n k)dimensional vector s is called the syndrome of the error
pattern. Or in other words, s contains the pattern of failure in
parity checks.
e c y
m
+ =
t
yH
( )
t
t t
m
t
m
t
eH
eH H c
H e c
yH s
=
+ =
+ =
=
Standard Array
Let us arrange the 2
n
n-tuples that represent all possible received
sequences in an array, such that first row contains all the
codewords (2
k
) and first column contains all correctable error
patterns. The standard array format of (k,n) code is as follows:
k n k k n k n k n
k n k k n k n k n
k
k
k
e c e c e c e
e c e c e c e
e c e c e c e
e c e c e c e
c c c c
i
i
i
i
i


+ + +
+ + +
+ + +
+ + +

2 2 2 2
1
2
1 2 2 1 2 1 2
1
1 2
2
2
2 2 1 2
1
2
1 1 1 1
2
1 0






Standard Array
Note that codeword c0 is trivial codeword, that is, all-
zeros codeword. In addition, it is also error pattern of
all-zeros.
Each row in the standard array is called a coset.
An error pattern in the first column, called coset leader.
A coset leader is the vector with the minimum weight in
the coset.
Each coset consists of 2
k
n-tuples
there are 2
n k
cosets.
51
Syndrome Decoding
The first step is to create a list of syndromes corresponding the
coset leaders. The syndrome of each vector y is found by
T
Hy y syn = ) (
When a code is received, the syndrome is computed and
compared to the list of syndromes. Let the coset leader to
the syndrome by e. Finally the code is decoded to x = y e.
52
Syndrome example
(
(
(
(

=
1 1 1 1 0 0 0
0 1 1 0 1 0 0
1 0 1 0 0 1 0
1 1 0 0 0 0 1
G
(
(
(

=
1 0 0 1 0 1 1
0 1 0 1 1 0 1
0 0 1 1 1 1 0
H
Note that G=(I |P) and H = ( | I).
T
P
53
Syndrome example
Let x:= 1001100 be the original message
Message Encoder Channel Decoder Message
1001 1001100 noise 1000100 ?
Compute the syndrome of the received code
T
Hy y syn = ) (
(
(
(

=
1 0 0 1 0 1 1
0 1 0 1 1 0 1
0 0 1 1 1 1 0
H
(
(
(
(
(
(
(
(
(

1
0
1
0
0
0
1
(
(
(

=
1
1
1
54
Conclusion
A code of minimum weight d is called perfect if all the
vectors in V are contained in the sphere of radius t = [(d
1)/2] about the code-word.

The Hamming [7,4] code has eight vectors of sphere of
radius one about each code-word, times sixteen unique
codes. Therefore, the Hamming [7,4] code with minimum
weight 3 is perfect since all the vectors (128) are contained
in the sphere of radius 1.