Learning Objectives

Coding Theory
CSM25 Secure Information Hiding

Get an overview of basic concepts in coding theory Get an idea of how matrix coding can be implemented Suggested Reading

Dr Hans Georg Schaathun

University of Surrey

Raymond Hill: A rst course in coding theory Oxford Applied Mathematics and Computing Science Series. 1986 Suggested Reading Fridrich et al: Writing on Wet Paper IEEE Trans. Signal Proc. Oct 2005, pp. 3923-,

Spring 2007

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

1 / 31

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

2 / 31

Coding Theory

Overview

Coding Theory

Overview

Overview
Matrix Coding and Error-Control Coding

Basic denitions

Duality between
Error-Control Coding Matrix Coding (e.g. F5)

A eld F means (essentially) that

Multiplication/division and addition/subtraction are dened on F a, b F , we have a + b F and a b F .

Today: a brief study of error-control

this will enable us to look further at matrix coding

The binary eld F2 :

Addition is logical XOR Multiplication is logical AND

A linear [n, k ]q code C is

linear set (vector space) of q k codewords each codeword is a vector (c1 , c2 , . . . , cn ) over an alphabet (eld) F of q element

A vector space (linear set) means that

a, b C , we have a + b C . a F , b C , we have ab C .

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

4 / 31

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

5 / 31

Coding Theory

Overview

Coding Theory

Overview

Example
The [7, 4]2 Hamming code. Generator matrix 1 0 G= 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 1 0 1 1 0 1 1 0 1 1 (1)

Hamming Matrix Coding

Embedded message

p3 11p3 110

 

s3 s3 = 010s3 1

4 Take any message m = (m1 , m2 , m3 , m4 ) F2

m2 1001m2 1 m1 0m1 1 m4 0m4 1 s1 s1 = 010s1 0 p1 00p1 0 m3 1m3 0 p2 11p2 0 s2 s2 = 00s2 0

Encoding: c = m G. Any such c is a codeword

The code is C = {c : m, c = m G}.

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

6 / 31

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

7 / 31

Coding Theory

Hamming Distance

Coding Theory

Hamming Distance

Hamming Distance

The Noisy Channel

Closest Neighbour Decoding

Two vectors a = (a1 , a2 , . . . , an ), b = (b1 , b2 , . . . , bn ). Hamming distance

d (a, b) = #{i : ai = bi } Number of positions where the vectors differ

Alice sends c Bob receives r Assumption: few errors more likely than many errors
i.e. probably d (c, r) is small

if r C , look for c
minimising d (c, r). This is called closest neighbour decoding

The Hamming distance is a metric (distance) in mathematical terms

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

9 / 31

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

10 / 31

Coding Theory

Hamming Distance

Coding Theory

Hamming Distance

Minimum distance

Covering radius

[n, k , d ]q code C Minimum distance d is

smallest distance between two distinct codewords d := mina,bC d (a, b) where a = b

Denition The covering radius is the largest distance between a word a C and the closest codeword c C . Draw spheres of radius around each codeword c C These spheres covers the entire space Smaller spheres would not cover the space

Let e =

d 1 2

spheres of radius e around each c C non-overlapping spheres closest neighbour decoding returns centre of sphere

Up to e errors are uniquely decodable.

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

11 / 31

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

12 / 31

Coding Theory

Hamming Distance

Coding Theory

Hamming Distance

Perfect codes

Exercise
Coding parameters

Denition If = (d 1)/2 (= e), we say that the code is perfect. The decoding spheres of radius e covers the space It is impossible to add another codeword without
reducing the error-correction capacity.

Let a code be given by generator matrix 11111 G = 11100 00111 What are the parameters [n, k , d ]q ?
Find n Find k Find d Find q

(2)

Hint: you may want to list all possible codewords rst. What is the covering radius ?
Dr Hans Georg Schaathun Coding Theory Spring 2007 13 / 31 Dr Hans Georg Schaathun Coding Theory Spring 2007 14 / 31

Coding Theory

The syndrome

Coding Theory

The syndrome

Parity check matrix

Hamming code

[n, k ] code C Described by k n generator matrix G Also, described by (n k ) n parity check matrix H
where G H T = 0

Note, both G and H have full rank

that is, no linearly dependent rows

1 0 G= 0 0 1 H= 1 1 Systematic form:

1 0 , 1 1 1 1 0 1 0 0 1 0 1 0 1 0 0 1 1 0 0 1 0 1 0 0 0 0 1 0 0 0 0 1 1 1 1 0 1 1 0 1

(3)

(4)

C = {c : H cT = 0} G = [I |P ] H = [P T |I ] where I is an identity matrix

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

16 / 31

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

17 / 31

Coding Theory

The syndrome

Coding Theory

The syndrome

The Syndrome

Exercise
Syndrome Decoding

Given a received word r,

the syndrome is s = rH T

Each syndrome can be mapped to an error vector e F n

Syndrome decoding Error patterns uniquely assigned to a syndrome can be corrected

Denition Let c be the transmitted codeword and r the received word. The vector e = c r is called the error pattern. Make a table of all possible syndromes for the Hamming code
list the most likely error pattern for each syndrome i.e. the error pattern of lowest Hamming weight

s = 0 corresponds to no error

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

18 / 31

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

19 / 31

Coding Theory

Design issues

Coding Theory

Design issues

The decoding problem

Block length

Compare two codes Encoding is simple matrix-vector multiplication Decoding of arbitrary codes is NP-complete
Small codes can easily be decoded
Exhaustive search Syndrome lookup table

[400, 200] code C1 able to correct 60 errors [100, 50] code C2 able to correct 15 errors

We can use one codeword of C1 or four of C2

C1 can correct any 60 errors C2 can correct 60 errors if they are evenly distributed over the four blocks

Intractible for larger codes.

Some classes of codes are known with fast decoding algorithms

Since errors are random, they are rarely evenly distributed In general, larger block lengths give better robustness.

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

21 / 31

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

22 / 31

Matrix and wet paper coding

Matrix coding

Matrix and wet paper coding

Matrix coding

Matrix coding

The main problem

[n, k ] matrix code C with low covering radius. k -bit message transmitted as the syndrome s. n-bit host signal corresponds to codeword/received word Encoder: change as few bits as possible, such that
the resulting syndrome match the message

Coding theory has focused on

maximising minimum distance correct many errors

Matrix coding requires

minimising covering radius minimise distortion

Decoder: multiply host by H to get syndrome s.

Hence, less theory available for matrix coding

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

24 / 31

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

25 / 31

Matrix and wet paper coding

Matrix coding

Matrix and wet paper coding

Wet paper codes

Implementation issues

Matrix code vs. wet paper code

The hard job is at the Sender. Embedding requires solving the decoding problem
which is NP-complete

Matrix coding
forbids more than P changes in the host allows changes in arbitrary positions

large codes with good decoding algorithms

few (if any) known with low covering radius

Wet paper coding

allows changes in specied positions in the host forbids changes in remaining positions

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

26 / 31

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

28 / 31

Matrix and wet paper coding

Wet paper codes

Matrix and wet paper coding

Wet paper codes

Random codes

The encoding problem

The hard job is at the encoder Slightly different problem from matrix coding Has to solve the equation

Random codes generally perform well

Only problem is complex decoding

Hc = m

Fridrich et al proposed random codes for wet paper.

where c is the cover m is the message

Some entries of c are locked,

remaining entries are unknowns in the equation

Solvable by Gaussian elimination

Polynomial O (n3 )

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

29 / 31

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

30 / 31

Matrix and wet paper coding

Wet paper codes

Simplication

Divide host signal into smaller blocks

embed independently in each block

Reduce problem size at decoder (O (n3 ))

Faster decoding

Shorter blocks
Less optimal, i.e. slightly increased distortion

Dr Hans Georg Schaathun

Coding Theory

Spring 2007

31 / 31