Professional Documents
Culture Documents
Turbo Codes
and
Principles and applications
Catherine Douillard
ENST Bretagne
CNRS TAMCIC UMR 2872
Catherine.Douillard@enst-bretagne.fr
• Turbo decoding
• Performance
• Conclusion
1
Concatenated Codes
2
Concatenated Codes :
Performance Improvements
Performance of concatenated codes suffers from the fact that inner
decoder only provides “hard” (0 or 1) decisions to outer decoder
π
xi
d^i
Decoder 1
π ^^
-1 Decoder 2
y1i (SISO) (SISO) di
y2i
xi = (2Xi-1) + ni (Xi=di)
y1i = (2Y1i-1) + n1i
y2i = (2Y2i-1) + n2i
3
Iterative (“Turbo”) Decoding of
Concatenated Codes: principle (II)
- Based on a Soft Input Soft Output (SISO) elementary decoder
- Iterative decoding process (turbo effect)
First iteration
xi
π
-
π
-1 1
Decoder 1 Decoder 2 + Zi
y1i (SISO) (SISO)
y2i
Z: extrinsic information
Iteration p
p-1
Zi
π
-
+ Decoder 1
- p
π Zi
-1 Decoder 2 +
xi
+ +
(SISO) (SISO)
y1i
y2i π ^
di
4
Parallel Concatenated Codes
k bits
rate : k/r1
k bits
Data Code 1 r1 bits
Global code rate:
rate : k/r2 k/(k+r1+r2)
π Code 2 r2 bits
5
Parallel Concatenation vs Serial Concatenation:
Performance Comparison under Iterative Decoding
BER
Typical serial
5 concatenation
10-1 behaviour 1 E
5
uncoded erfc ( b )
2 N0
10-2 with
5 Typical parallel ∞
2
10-3
concatenation erfc ( x) = ∫ exp(−t 2 )dt
π t=x
5 behaviour
10-4 "Waterfall" region
5
10-5
5
10-6
5
"Error floor" region
10-7
5
10-8
Convergence
0 1 2 3 4 5 6 7 8 9 10 11 12
abscissa
Ga = 10 log( Rd min )
6
Turbo Codes Encoder
Xi
Data
Code 1
di Y1i
Redundancy
Y2i
π Code 2
natural coding rate : 1/3
Recursive Systematic
Convolutional (RSC) Codes
7
Systematic Convolutional (SC) codes
Xi Xi=di
di di D D
D D
Yi Yi
NSC SC
(Non Systematic Convolutional) (Systematic Convolutional)
(Elias code)
di D D di D D
NSC Yi SC Yi
00 00
00 00
11 11 df = 5 df = 4 11 01
01 00 01 10
01 01
10 10 Xi Yi 10 00
di =0
10 10
01 Xi Yi 01
11 di =1 11
8
SC codes Bit Error Rate
BER
1,E+00
1,E-01
1,E-02
1,E-03
Uncoded
1,E-04 SC
NSC
1,E-05
1,E-06
1,E-07
1,E-08 Eb/N0
0 2 4 6 8 10 12
????
9
Recursive Systematic Convolutional (RSC)
codes Xi
Xi
di di D D
D D
[1,(1+D+D2)/(1+D2)] Yi
NSC Yi
[(1+D2),(1+D+D2)]
Xi
di
D D
[1,(1+D2)/(1+D+D2)] Yi
Xi Xi
di
di D D D D
RSC Yi
NSC Yi
00 00
00 00
11 11 11 11
df = 5 df = 5
01 01 00
00
01 01
10 10 Xi Yi 10 10
di =0
10 10
01 Xi Yi 01
11 di =1 11
10
RSC Codes Bit Error Rate
BER
1,E+00
1,E-01
1,E-02
1,E-03 Uncoded
SC
1,E-04
NSC
1,E-05 RSC
1,E-06
1,E-07
1,E-08 Eb/N0
0 2 4 6 8 10 12
The permutation
11
Turbo Codes Permutation
The 1st function of Π : to compensate for the
vulnerability of a decoder to errors occuring in bursts
xi
Xi Decoder 1
Data y1i
Code 1
di Y1i
Y2i π
π Code 2
y2i
Decoder 2
xi = (2Xi-1) + ni (Xi=di)
y1i = (2Y1i-1) + n1i
y2i = (2Y2i-1) + n2i
Xi
Data
Code 1
di Y1i
Y2i
π Code 2
12
Turbo Codes Permutation
Xi
Data
Code 1
di Y1i
Y2i
π Code 2
Permutation Regular
Condition : k = M.N
13
Permutation Regular
Another representation
i 0 1 2 … k-2 k-1
Data in natural order
i = Π(j) j 0 1 2 … k-2 k-1
Data in interleaved order
Π(0) = 0
for j = 0L k − 1 k-1 0
Π(1) = P
i = Π ( j ) = Pj mod k
Π(2) = 2P
Π(2) = 3P
Permutation Regular
M columns
0100000010...0000000
period : 7 ...................................
0000000000...0000000 N rows
0000000000...0000000
period : 7 0000000000...0000000
k → ∞ ⇒ d(w=2) → ∞
14
Permutation Regular
M columns
0110100000...0000000
period : 7 ...................................
0000000000...0000000 N rows
0000000000...0000000
period : 7 0000000000...0000000
k → ∞ ⇒ d(w=3) → ∞
Permutation Regular
M columns
0100000010...0000000
0000000000...0000000
0000000000...0000000
period : 7 0000000000...0000000
0000000000...0000000 N rows
0000000000...0000000
0000000000...0000000
period : 7
0100000010...0000000
.....................................
k → ∞ ⇒ d(w=4) is limited
15
Permutation Regular
M columns
0110100000...0000000
0110100000...0000000
0000000000...0000000
period : 7 0110100000...0000000
0000000000...0000000 N rows
0000000000...0000000
0000000000...0000000
period : 7
0000000000...0000000
.....................................
16
Permutation pseudo-random (= controlled disorder)
Everyone has his own tricks
For instance, the turbo code permutation
DVB-RCS/RCT (ETSI EN 301 790 and 301 958)
Built from a regular permutation N-1 0 Π(0) = Q
for j = 0L N − 1
i = Π ( j ) = Pj + Q mod N Π(1) = P+Q
P and N are relatively prime integers
j mod 4 = 3 ⇒ Q = P4
Permutation
In conclusion :
17
Turbo Decoding
Turbo Decoding
18
Turbo decoding SISO algorithms
19
Turbo decoding SISO algorithms: MAP
20
Turbo decoding SISO algorithms: MAP
Example: 0 0
0 1 1
* λ i (2) = γ 0 ( Ri ,1,2)α i −1 (1)βi (2)
1 2 2
* λ i (2) = γ1 ( Ri ,0,2)αi −1 (0)βi (2)
3 3
i-1 i
di = 0
di = 1
γ1 ( Ri ,1,0)α i −1 (1)βi (0) + γ1 ( Ri ,2,1)α i −1 (2)βi (1)
Λ (d i ) = ln L
γ 0 ( Ri ,0,0)α i −1 (0)βi (0) + γ 0 ( Ri ,3,1)α i −1 (3)βi (1)
+ γ1 ( Ri ,0,2)α i −1 (0)βi (2) + γ1 ( Ri ,3,3)α i −1 ( 3)βi (3)
L
+ γ 0 ( Ri ,1,2)α i −1 (1)βi ( 2) + γ 0 ( Ri ,2,3)α i −1 (2)βi (3)
21
Turbo decoding SISO algorithms: MAP
1 R −C 2
γ j ( Ri , m′, m) = Pr{d i = j} × exp− i i
2πσ 2 2σ 2
Where Ci is the expected symbol along the branch from state m'
to state m and Ri is the received symbol .
22
Turbo decoding SISO algorithms: LOG-MAP
and Max-Log-MAP
The MAP algorithm in the logarithmic domain (MAX-LOG-MAP)
[12][13]
Solution 2: MAX-LOG-MAP
ln(e a + eb ) ≈ max(a, b)
• In the Log domain, one can show that Λ(di) can be written as:
xi
Decoder Λ (d i ) = xi + Z i
yi
23
Turbo decoding The turbo principle
The turbo decoder structure
π +
-
x1
xi
Decoder 1 ^
Z2 di
y1 i (SISO)
y2 i
Z1 Decoder 2
π x2 (SISO)
π
-1 +
Turbo codes
Performance
24
Turbo Codes Performance
BER Gaussian channel, code rate=1/2
Gaussian channel
1.0e-01
QPSK modulation
MAP decoding
Xi
uncoded
di
1.0e-02
RSC
S1 S2 S3 S4 37,21
1.0e-03 Y1i
Interleaving Yi
256X256 Y2i
1.0e-04
Iteration #1 RSC
S1 S2 S3 S4 37,21
#6 #2
1.0e-05
#18 #3
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 ICC’93
Eb/N0 (dB)
Recent improvements in
turbo codes
25
Circular Recursive Systematic
Convolutional (CRSC) Codes
26
Improvements Circular encoding
di
D D D
Yi
000
001
010
011
100
101
110
111 0 1 1 0 1 1 0 1
di
D D D
Yi
000
001
010
011
100
101
110
111
27
Improvements Circular encoding
28
Decoding CRSC codes
29
Improvements Double binary codes
binary double-binary
data (bits) data (couples) A
X
B
C1 Y1 C1 Y1
Π Π
C2 Y2 C2 Y2
10-1 1 E
5
uncoded erfc ( b )
2 N0
10-2 with
5 ∞
2
10-3 erfc( x) = ∫ exp(−t 2 )dt
π t=x
5
Bad convergence,
10-4 high asymptotic gain
5
10-5
5
10-6
5
Good convergence,
10-7 low asymptotic gain
5
10-8
0 1 2 3 4 5 6 7 8 9 10 11 12
Ga = 10 log (R.dmin)
QPSK, AWGN channel, target BER : 10-8
30
Improvements Double binary codes
Contribution to the convergence property of TCs
k
erroneous
path
locked
patterns
double-binary :
C1 decreases the correlation C1
when decoding
binary
Π k Π
k /2
C2 k /2
C2
k
Intra-symbol permutation
2 0 1 1 3 1 3
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
(A,B) becomes (B,A) 0 0 0 0 0 0
0 0 0 0 3 0 0 0 0 0 0 3 0 0
before vertical encoding 2 0 1 1 3 0 0 0 0
0 0 0 0
once out of two 3 0 0 0 0 0 0 3 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
2 0 0 0 0 0 0 2 3 0 0 0 0 0 0 3 0 0
1 3
31
Improvements Double binary codes
Other avantages of double-binary codes
-1 iteration
10 #1
• QPSK modulation
-2
10 #5 • R=1/2
(binary-input channel)
• AWGN channel
Shannon limit
-3
10
• Duo-binary 8-state turbo code
-4
10
• Large block size
-5
10 #10
• MAP algorithm
#20
-6 • No quantization
10
#15
E b /N0
0.1 0.2 0.3 0.4 0.5 0.6 0.7
0.35 dB
32
Conclusion
33
Convolutional Turbo Codes Example of performance
FER
1.0e+00
1.0e-01
1.0e-06 • 8 iterations
1.0e-07
FER
5
10-1 QPSK 8-PSK 16-QAM 64-QAM
5
10-2
5
10-3
5 0,8 dB
0,8 dB
10-4
5 0,6 dB
0,85 dB 0,9 dB
10-5
5
10-6 8-state
5
1,7 dB
10-7 16-state
5 0,8 dB 0,9 dB
10-8
2 3 4 E / N (dB) 5 6 7 8 9
b 0
34
Turbo codes : hot topics
Turbo Communications
• Turbo demodulation
• Turbo equalisation
• Turbo detection
• Turbo synchronisation
• Source turbo decoding
turbo processor
local extrinsic
intrinsic 1 information
information
4
2
probabilistic
processor
3
shared
intrinsic
information
35
Turbo codes References
References
[9] C. Weiss, C. Bettstetter and S. Riedel, “Code construction and decoding of parallel
concatenated tail-biting codes,” IEEE Trans. Inform. Theory., vol. 47, n° 1, pp. 366-
386, Jan. 2001.
[10] C. Berrou, C. Douillard and M. Jézéquel, “Multiple parallel concatenation of circular
recursive convolutional (CRSC) codes,” Annals of Telecommun., vol. 54, n° 3-4, pp.
166-172, 1989.
[11] C. Berrou, P. Adde, E. Angui, and S. Faudeil, “A low complexity soft-output Viterbi
decoder architecture”, Proc. ICC’93, Geneva, Switzerland, May 1993, pp.737-740.
[12] P. Robertson, E. Villebrun, and P. Hoeher, “ A comparison of optimal and sub-
optimal MAP decoding algorithms operating in the log domain”, Proc. ICC’95,
Seattle, WA, 1995, pp.1009-1013.
[13] A. J. Viterbi, “An intuitive justification and a simplified implementation of the MAP
decoder for convolutional codes”, IEEE Journal on Selected Areas in Comm., Vol. 16,
N°2, Feb. 1998.
[14] D. Divsalar, S. Dolinar and F. Pollara, “Iterative turbo decoder analysisbased on
density evolution”, IEEE JSAC, vol. 16, n°5, pp.891-907, May 2001.
[15] S. ten Brink, “Convergence behavior of iteratively decoded parallel concatenated
codes”, IEEE Trans. Commun., vol. 49, pp.1727-37, Oct. 2001.
36
General information about Turbo Codes
• C. Heegard, S. B. Wicker, Turbo Coding, Kluwer Academic Publishers, 1999
• Branka Vucetic, Jinhong Yuan, Turbo Codes, Principles and Applications, Kluwer
Academic Publishers, 2000
• C. Schlegel, Trellis coding, IEEE Press, 1997
• B.J. Frey, Graphical Models for Machine Learning and Digital Communication, MIT
Press, 1998
• R. Johannesson , K. Sh. Zignagirov, Fundamentals of Convolutional Coding, IEEE
Press, 1999
• Hanzo, Lajos, Adaptative wireless transceivers, John Wiley & sons, 2002
• Hanzo, Lajos, Turbo coding, turbo equalisation and space-time coding for transmission
over fading channels, John Wiley & sons, 2002
• IEEE Communications Magazine, vol. 41, n°8, August 2003
Capacity approaching codes, iterative decoding, and their applications
• WEB sites : http://www-turbo.enst-bretagne.fr/2emesymposium/presenta/turbosit.htm
is a good starting point
37