ECE 141 Lecture 14:

Convolutional Codes
Tree,Trellis, and State Diagrams,Transfer Function of a Convolutional Code, Decoding
Convolutional Codes,Viterbi Algorithm

ECE 141: DIGITAL COMMUNICATIONS 1

Convolutional Codes
❑ Can be described as block codes with memory.
❑ Uses memory to generate codeword, in effect, the generator
matrix also contains some form of memory.
❑ If the message vector is 𝒎, and state vector is 𝑺, the
codeword 𝒄 can be expressed as:
𝒄 = 𝑓𝑐 𝒎, 𝑺
❑ Shift registers are used for memory.
❑ An 𝑛, 𝑘, 𝑚 convolutional code has 𝑛 output bits, 𝑘 input bits,
and memory 𝑚.

ECE 141: DIGITAL COMMUNICATION I 2

Implementing Convolutional
Codes
❑ Example: 𝑘 = 1, 𝑛 = 3.
1
𝑐
2
𝑐

𝑏 𝑧 −1 𝑧 −1

3
𝑐
❑ The memory of this example is 𝑚 = 2 which is the
maximum amount of shift registers for a single bit input.
❑ A convolutional code 𝒄 = 𝑐 1 𝑐 2 𝑐 3 𝑇 is formed from
the three outputs by multiplexing them.

ECE 141: DIGITAL COMMUNICATION I 3

Implementing Convolutional
Codes
❑ Example: 𝑘 = 2, 𝑛 = 3.
𝑏 1 𝑧 −1 𝑐 1

2
𝑐

𝑏 2 𝑧 −1 𝑐 3

❑ The memory of this example is 𝑚 = 1.

❑ The two-bit input is split into two parallel streams.

ECE 141: DIGITAL COMMUNICATION I 4

Generator Matrix
❑ Since convolutional codes have memory, defining a generator
matrix requires partitioning the encoder.
1
𝑐
2
𝑐

𝑏 (1) 𝑧 −1 𝑧 −1

3
𝑐

❑ If the message is an impulse function, i.e., 1 followed by 0s

infinitely, the corresponding outputs are after 𝐾 + 1 cycles:
𝑐 1 = 100 𝑐 2 = 110 𝑐 3 = 111
❑ These are impulse responses per output.
ECE 141: DIGITAL COMMUNICATION I 5
Generator Matrix
❑ The impulse response of an output defines the generator
matrix from input to output.
1
𝑐
2
𝑐

𝑏 (1) 𝑧 −1 𝑧 −1

3
𝑐

❑ The generator matrices for this example are:

𝒈𝟏 = 100 𝒈𝟐 = 110 𝒈𝟑 = 111
❑ For a given message sequence, 𝒎, the outputs are:
𝒄 𝟏 = 𝒎 ∗ 𝒈𝟏 𝒄 𝟐 = 𝒎 ∗ 𝒈𝟐 𝒄 𝟑 = 𝒎 ∗ 𝒈𝟑
ECE 141: DIGITAL COMMUNICATION I 6
Example
❑ 3,2,1 convolutional code
𝑏 1 𝑧 −1 𝑐 1

2
𝑐

𝑏 2 𝑧 −1 𝑐 3

❑ The generator matrices are:

0 1 1 1 0 0
𝑔1 = 𝑔2 = 𝑔3 =
1 1 1 0 1 1

ECE 141: DIGITAL COMMUNICATION I 7

State Diagram and Trellis
Diagram
❑ As a code with memory, a convolutional code forms a finite
state machine.
❑ RECALL: State diagrams visually portrays the transition
behavior of an FSM. Trellis diagrams visualizes the time
evolution of an FSM.
❑ The maximum number of states of an 𝑛, 𝑘, 𝑚 convolutional
code is 𝑆 = 2𝑘𝑚 but the actual number of states depend on
the number of shift registers in the encoder.
❑ For 𝐿 shift registers, the number of states are 𝑆 = 2𝐿 .

ECE 141: DIGITAL COMMUNICATION I 8

Example
❑ Draw the state diagram of the 2,1,2 convolutional code
defined the by the encoder below.
1
𝑐
2
𝑐

𝑏 (1) 𝑧 −1 𝑧 −1

3
𝑐

❑ Solution: use a truth table!

1 1 2 1 1 3 1 1 1
𝑐𝑛 = 𝑏𝑛 𝑐𝑛 = 𝑏𝑛 + 𝑏𝑛−1 𝑐𝑛 = 𝑏𝑛 + 𝑏𝑛−1 + 𝑏𝑛−2

ECE 141: DIGITAL COMMUNICATION I 9

Example
1 1 1 1 2 3 𝑆𝑛−1 𝑆𝑛
𝑏𝑛 𝑏𝑛−1 𝑏𝑛−2 𝑐𝑛 𝑐𝑛 𝑐𝑛
0 0 0 0 0 0 00 00
0 0 1 0 0 1 01 00
0 1 0 0 1 1 10 01
0 1 1 0 1 0 11 01
1 0 0 1 1 1 00 10
1 0 1 1 1 0 01 10
1 1 0 1 0 0 10 11
1 1 1 1 0 1 11 11

1 1 2 1 1 3 1 1 1
𝑐𝑛 = 𝑏𝑛 𝑐𝑛 = 𝑏𝑛 + 𝑏𝑛−1 𝑐𝑛 = 𝑏𝑛 + 𝑏𝑛−1 + 𝑏𝑛−2

ECE 141: DIGITAL COMMUNICATION I 10

State Diagram
1/111

0/000 00 10

0/011
0/001 1/100
1/110

01 11 1/101

0/010

ECE 141: DIGITAL COMMUNICATION I 11

 Trellis Diagram
0/000 0/000 0/000 0/000 0/000 0/000 0/000 0/000
00
1/111 1/111 1/111 1/111 1/111 1/111 1/111 1/111

01

10

11
1/101 1/101 1/101 1/101 1/101 1/101

0 𝑇 2𝑇 3𝑇 4𝑇 5𝑇 6𝑇 7𝑇
❑ The diagram shows two terminal states at 00 and 11.
❑ After two transitions, the trellis enters steady-state.

ECE 141: DIGITAL COMMUNICATION I 12

Notes
❑ Convolutional codes enable transmissions to approach the
channel capacity.
❑ In general, convolutional codes has two terminal states; the
all zero state and all one state.
❑ In practice, the convolutional code should always reset to the
all zero state after transmission. This ensures that any new
transmission starts at the all zero state.
❑ This is done by appending 𝑚𝑘 zeros to the input sequence.
❑ By doing this practice, decoding becomes easier since the
initial state is known.

ECE 141: DIGITAL COMMUNICATION I 13

 Example
❑ Trace the trellis path of the input sequence 𝑏 = 10011 in a
2,1,2 convolutional code defined by the encoder and state
diagram shown below.
1/111

0/000 00 10
1
𝑐
2
𝑐
0/011
0/001 1/100
1/110

𝑏 (1) 𝑧 −1 𝑧 −1

01 11 1/101

3 0/010
𝑐

ECE 141: DIGITAL COMMUNICATION I 14

 Example
00
1/111 1/111

01

10

11

❑ The message sequence 𝑏 = 10011 and it will be appended

by 00 .
❑ The resulting codeword sequence is:
𝑐 = 111 011 001 111 100 010 001

ECE 141: DIGITAL COMMUNICATION I 15

The Transfer Function
❑ The information of how many paths are possible to arrive in a
certain codeword sequence with given weight can be found in
the transfer function.
❑ Specifically, for a convolutional code, we need to know hoe
many possible paths transition from the initial state (all zero)
to the final state (also all zero).
❑ The minimum distance between different paths can be
computed using this – useful for determining bit error rate.

ECE 141: DIGITAL COMMUNICATION I 16

Computing the Transfer
Function
1. Split the state diagram at the all zero state.
2. Label the initial all zero state 𝐼 and the final all zero state 𝐹.
3. Eliminate the self-loop around the zero state.
4. Label every branch as 𝑋 𝛼 𝑌𝛽 𝑍.
▪ 𝛼 – weight of the codeword output, 𝛽 – weight of the codeword
input, 𝑍 represents the branch.
5. At each state, compute for 𝑊 which is the sum of all the
weight of the branches going into the state.
6. Solve for the transfer function:
𝑊𝐹
𝑇 𝑋, 𝑌, 𝑍 =
𝑊𝐼

ECE 141: DIGITAL COMMUNICATION I 17

Example
❑ Solve for the transfer function of the convolutional encoder
with the state diagram shown below.
1/111

0/000 00 10

0/011
0/001 1/100
1/110

01 11 1/101

0/010
ECE 141: DIGITAL COMMUNICATION I 18
 Example
❑ The modified state diagram is:
1/101 𝑋 2 𝑌𝑍

𝑋𝑌𝑍 𝑋𝑍
11
1/100 0/010

𝑊3
1/111 0/001
0/011 𝑋 2𝑍
00 10 01 00
𝑋 3 𝑌𝑍 𝑋𝑍
𝑊2 𝑋 2 𝑌𝑍 1/110
𝑊𝐼 𝑊1 𝑊𝐹

❑ The path 00 10 11 01 00 has a weight:

𝑋 3 𝑌𝑍 𝑋𝑌𝑍 𝑋𝑍 𝑋𝑍 = 𝑋 5 𝑌 2 𝑍 4

ECE 141: DIGITAL COMMUNICATION I 19

 Example 1/101 𝑋 2 𝑌𝑍

𝑋𝑌𝑍 𝑋𝑍
11
1/100 0/010

𝑊3
1/111 0/001
0/011 𝑋 2𝑍
00 10 01 00
3 𝑋𝑍
𝑋 𝑌𝑍
𝑊2 𝑋 2 𝑌𝑍 1/110
𝑊𝐼 𝑊1 𝑊𝐹
❑ At each state, the 𝑊 functions are:
𝑊1 = 𝑋 2 𝑍𝑊2 + 𝑋 2 𝑌𝑍𝑊3 𝑊2 = 𝑋 2 𝑌𝑍𝑊1 + 𝑋 3 𝑌𝑍𝑊𝐼
𝑊3 = 𝑋𝑌𝑍𝑊2 + 𝑋 2 𝑌𝑍𝑊3 𝑊𝐹 = 𝑋𝑍𝑊1
❑ Goal: solve for 𝑊𝐹 /𝑊𝐼 .

ECE 141: DIGITAL COMMUNICATION I 20

Example
𝑊1 = 𝑋 2 𝑍𝑊2 + 𝑋 2 𝑌𝑍𝑊3 1
𝑊2 = 𝑋 2 𝑌𝑍𝑊1 + 𝑋 3 𝑌𝑍𝑊𝐼 2
𝑊3 = 𝑋𝑌𝑍𝑊2 + 𝑋 2 𝑌𝑍𝑊3 3
𝑊𝐹 = 𝑋𝑍𝑊1 4
❑ Using 2 and 3 :
𝑋𝑌𝑍
𝑊3 = 2
𝑊2 5
1 − 𝑋 𝑌𝑍
❑ Using 1 and 5 :
1 − 𝑋 2 𝑌𝑍
𝑊2 = 2 𝑊 6
𝑋 𝑍 1 − 𝑋 2 𝑌𝑍 + 𝑋𝑌 2 𝑍 1
❑ Using 2 and 6 :
𝑋 5 𝑌𝑍 2 1 − 𝑋 2 𝑌𝑍 + 𝑋𝑌 2 𝑍
𝑊1 = 𝑊 7
1 − 𝑋 2 𝑌𝑍 − 𝑋 4 𝑌𝑍 2 1 − 𝑋 2 𝑌𝑍 + 𝑋𝑌 2 𝑍 𝐼
ECE 141: DIGITAL COMMUNICATION I 21
Example
❑ Using 4 and 7 :
𝑋 6 𝑌𝑍 3 1 − 𝑋 2 𝑌𝑍 + 𝑋𝑌 2 𝑍
𝑊𝐹 = 2 4 2 2 2
𝑊𝐼 8
1 − 𝑋 𝑌𝑍 − 𝑋 𝑌𝑍 1 − 𝑋 𝑌𝑍 + 𝑋𝑌 𝑍
❑ The transfer function is:
𝑋 6 𝑌𝑍 3 1 − 𝑋 2 𝑌𝑍 + 𝑋𝑌 2 𝑍
𝑇 𝑋, 𝑌, 𝑍 =
1 − 𝑋 2 𝑌𝑍 − 𝑋 4 𝑌𝑍 2 1 − 𝑋 2 𝑌𝑍 + 𝑋𝑌 2 𝑍

ECE 141: DIGITAL COMMUNICATION I 22

Transfer Function
❑ Generally the transfer function is of the form
𝐴 𝑋, 𝑌, 𝑍
𝑇 𝑋, 𝑌, 𝑍 =
1 − 𝐵 𝑋, 𝑌, 𝑍
❑ Using the Taylor Series expansion:
1
= 1 + 𝜖 + 𝜖2 + 𝜖3 + ⋯
1−𝜖
❑ We can get the general form of the transfer function:
𝑇 𝑋, 𝑌, 𝑍 = ෍ 𝐴𝑑𝑗𝑙 𝑋 𝑑 𝑌𝑗 𝑍 𝑙

❑ 𝐴𝑑𝑗𝑙 is the number of paths of length 𝑙, weight 𝑑 for the

codeword, and weight 𝑗 for the message words.
❑ 𝑗 and 𝑙 are functions of 𝑑.
ECE 141: DIGITAL COMMUNICATION I 23
Minimum Distance
❑ The minimum distance can be determined. Take the transfer
function and substitute 𝑌 = 1 and 𝑍 = 1.
𝑇 𝑋, 𝑌 = 1, 𝑍 = 1 = 𝑇 𝑋 = ෍ 𝐴𝑑 𝑋 𝑑

❑ Minimum value of 𝑑 is the minimum distance between any

valid codeword sequence with the all zero path.

ECE 141: DIGITAL COMMUNICATION I 24

Decoding Convolutional
Codes
❑ For an 𝑛, 𝑘, 𝑚 convolutional code:
❑ Let 𝑐 = 𝒄 𝟎 … 𝒄 𝑳−𝟏 be the codeword sequence and
𝑟 = 𝒓 𝟎 … 𝒓 𝑳−𝟏 is the received sequence.
❑ Note:
𝒄 𝒍
= 𝑐1𝑙 … 𝑐𝑛
𝑙
𝒓 𝒍
= 𝑟1 𝑙 … 𝑟𝑛
𝑙

❑ In a channel with hard decision:

(𝑙) (𝑙) (𝑙) (𝑙)
𝑃 𝑟𝑚 ≠ 𝑐𝑚 = 𝑃 𝑃 𝑟𝑚 = 𝑐𝑚 = 1 − 𝑃
❑ The optimal decoder maximizes the likelihood function. The
decoded codeword 𝑐Ƹ selects the codeword sequence that is
closest to 𝑟:
𝑐Ƹ = argmax 𝑃 𝑟 𝑐
𝑐

ECE 141: DIGITAL COMMUNICATION I 25

Minimum Distance Problem
❑ The likelihood between the received codeword and the
transmitted codeword is defined by the distance between
them.
❑ The problem reduces to a minimum distance problem.
𝐿−1 𝐿−1

𝑑 = ෍𝑑 𝒄 𝒊 ,𝒓 𝒊
=෍ 𝒄 𝒊
+𝒓 𝒊

𝑖=0 𝑖=0
❑ Note: the operation inside the parenthesis is modulo-2
addition and the summation is a normal summation.
❑ The codeword selected should minimize the distance – must
compare 2𝐿 possible codeword sequences.
❑ REMARK: 2𝐿 is a lot of codewords.

ECE 141: DIGITAL COMMUNICATION I 26

Viterbi Algorithm
❑ Recall the Viterbi Algorithm.
❑ This algorithm reduces the number of comparisons in deciding
the transmitted codeword from 2𝐿 to 𝐿 ∙ 2𝑁 .
❑ Review: minimum distance between cities given multiple
possible paths.
1 30 4

100
2 5
19 6
𝐼 𝐹

3 6
5

ECE 141: DIGITAL COMMUNICATION I 27

Distance Between Two Cities
Problem
❑ First step is to get to the nearest cities from 𝐼. Consider all
the different paths from 𝐼 to cities 1, 2, and 3.
❑ Now, consider the paths going towards cities 4, 5, and 6. First,
look at 4:
1 30 𝑑 = 120 4

100
2
𝐼

ECE 141: DIGITAL COMMUNICATION I 28

 Distance Between Two Cities
Problem
❑ Looking at cities 5 and 6:
1

100
2 𝒅 = 𝟏𝟏𝟗 5
19
1 𝐼

3
100
2
𝐼

3 𝒅 = 𝟏𝟏𝟓 6
5

ECE 141: DIGITAL COMMUNICATION I 29

Distance Between Two Cities
Problem
❑ Remove the eliminated paths and label the accumulated
distances on each path, we simplify the diagram shown below.
❑ Now, evaluate the paths to city 𝐹. The shortest distance will
be the best path from 𝐼 to 𝐹.
1 4

100
2 𝑑 = 119 5
19 6
𝐼 𝑑 = 125
𝐹

3 𝑑 = 115 6
5

ECE 141: DIGITAL COMMUNICATION I 30

Example
❑ 2,1,2 convolutional code defined by the state diagram
below. Solve for the transmitted sequence if the received is:
𝑟 = 101 011 001 110 100 010 001
1/111

0/000 00 10

0/011
0/001 1/100
1/110

01 11 1/101

0/010

ECE 141: DIGITAL COMMUNICATION I 31

 Example
❑ Determining the surviving path.
0/000 2 4 5 3 4 5 6
00
1/111 1 7 8 7 2

01 1 4 6 5 6 7
6 6 7 2 7
1 3 6 2 5 6 7
4 5
10 7 6 5

4 6 5 2 7 8
11 5 7 6 5 6
1/101

𝑟 101 011 001 110 100 010 001

ECE 141: DIGITAL COMMUNICATION I 32

 Example
❑ Surviving paths.
0/000 3 4 5
00
1/111 1 2

01 1
2 7
1 2 7

10

11 5 6
1/101

𝑟 101 011 001 110 100 010 001

𝑚/
ෝ 𝑐Ƹ 1/111 0/011 0/001 1/111 1/100 0/010 0/001
𝑚/𝑐 1/111 0/011 0/001 1/111 1/100 0/010 0/001

ECE 141: DIGITAL COMMUNICATION I 33

Example
❑ Using the same 2,1,2 convolutional code, solve for the
transmitted signal if the received signal is:
𝑟 = 101 100 001 110 100 011 101
1/111

0/000 00 10

0/011
0/001 1/100
1/110

01 11 1/101

0/010

ECE 141: DIGITAL COMMUNICATION I 34

 Example
❑ Determining the surviving path.
0/000 2 3 4 6 7 7 9
00
1/111 4 6 5 7 5

01 4 5 7 6 4 8
3 3 6 4 7
1 4 5 5 8 6 8
7 3
10 4 8 6

1 6 6 3 7 7
11 2 4 5 5 5
1/101

𝑟 101 100 001 110 100 011 101

ECE 141: DIGITAL COMMUNICATION I 35

 Example
❑ Surviving paths.
0/000
00
1/111
5

01
3 4 7
1
3
10 6

1 3

11 5 5
1/101

𝑟 101 100 001 110 100 011 101

𝑚/
ෝ 𝑐Ƹ 1/111 1/100 0/010 1/110 1/100 0/010 0/001
𝑚/𝑐 1/111 0/011 0/001 1/111 1/100 0/010 0/001

ECE 141: DIGITAL COMMUNICATION I 36