Tutorial of Orthogonal Matching Pursuit

A tutorial for Beginners and Dummies

Koredianto Usman
Sekolah Teknik Elektro dan Informatika

8 Mei 2014

Koredianto Usman

Tutorial of Orthogonal Matching Pursuit

Outline

Problem Statement

The Orthogonal Matching Pursuit

Koherensi Basis

2 / 13

1. Problem Statement
Consider the following very simple example: Given the following
sparse signals

1.2
x = 1
0
The following is the measurement matrix A (2 x 3):


0.707 0.8 0
A=
0.707 0.6 1
Therefore y = A x gives:


1.65
0.25




1.65
0.707 0.8 0
Now, Given that : y =
and A =
0.25
0.707 0.6 1
y=

How to find original x?

3 / 13

2. Orthogonal Matching Pursuit



1.65
BASIS Previous example: Given : y =
and
0.25


0.707 0.8 0
A=
0.707 0.6 1
Columns in matrix A are called BASIS (CHEN and DONOHO :
ATOMS). In the example, we have the following atoms:
b1 =



 
 
0.707
0.8
0
b2 =
b3 =
0.707
0.6
1

4 / 13

2. Orthogonal Matching Pursuit

Interpretation of equation Ax = y
Since A = [b1 b2 b3 ]; and if we let : x = [a b c], then
A x = a b1 + b b2 + c b3 A x is the linear combination of b1 ,
b2 , b3 .



1.2
0.707 0.8 0
Ax =
1
0.707 0.6 1
0




 


0.707
0.8
0
1.65
= 1.2
+1
+0
=y =
0.707
0.6
1
0.25

5 / 13

2. Orthogonal Matching Pursuit

1.2
0.707 0.8 0
Ax =
0
0.707 0.6 1
1




 


0.707
0.8
0
1.65
= 1.2
+1
+0
=y =
0.707
0.6
1
0.25


From the equation above, it is clear that atom b1 contribute the

biggest influence in y, next is b2 , dan last is b3 . ORTHOGONAL
MATCHING PURSUIT works reversely: we start finding which of
b1 , b2 , b3 that will influence the STRONGEST to y . Then the
SECOND STRONGEST from the residual, and so on.

6 / 13

2. Orthogonal Matching Pursuit

STRONGEST influence is measured using DOT PRODUCT /
INNER PRODUCT OMP Algorithm:
1 find atom with has the biggest inner product with y
pi = maxj < bj , y >
calculate the residue ri = pi pi < pi , y >
find atom with has the biggest inner product with ri
4 repeat step 2 and 3 until residue achieve a certain threshold
Geometrically:
2
3

Y
b1
(-0.707 , 0.707)

b2
(0.8 , 0.6)

y
(-1.65 , 0.25)

b3
(0 , -1)

7 / 13

2. Orthogonal Matching Pursuit

Here the dot product of y to any of b1, b2, b3:
< y , b1 >= 1.34
< y , b2 >= 1.17
and
< y , b3 >= 0.25
Taking the absolute value, we see b1 gives the biggest inner
product. Then, b1 is chosen as the atom in first step, DOT
PRODUCT 1.34. We next count the residue:




1.65
0.707
r1 = y b1 < y , b1 > =
(1.34)
0.25
0.707

8 / 13

2. Orthogonal Matching Pursuit

Y

Residue 1

b1

(-0.707 , 0.707)

y
(-1.65 , 0.25)

b2
(0.8 , 0.6)

b3
(0 , -1)

Next we count the DOT PRODUCT of this residue to b2 and b3

(no need to count with b1 , since this residue must perpendicular to
b1 ).
< r 1, b1 >= 1
< r 1, b3 >= 0.7
Taking the absolute value, we get b2 as the next strongest
influence.

9 / 13

2. Orthogonal Matching Pursuit

Next we count again the residue:



 
0.7
0.8
r 2 = r 1 < r 1, b2 > b2 =
(1)
0.7
0.6


0.099
=
0.099
From residue r2, we finally count the final DOT PRODUCT,
between r2 with the last b3 :
< r 2, b3 >= 0.099

10 / 13

2. Orthogonal Matching Pursuit

The following base is in STRONGEST influence in order: b1 , b2 ,

b3 , with influence measure from dot product is : 1.34, 1, dan
0.099

1.34
Therefore, the reconstructed x is 1
0.099

1.2
Original was : 1
0

11 / 13

3. Base Coherency
Base Coherency indicate how close one base to the others.
b2

b2
A3

b2
A3

A3
A2

A2

A2

A1
b1

b1

b1

A1
(A)

(B)

A1

(C)

Figure : Ilustrasi Koherensi Basis. (A). Koherensi tinggi. (B). Koherensi

Sedang. (C). Koherensi Rendah

12 / 13

3. Base Coherency

A simple measure for base coherency is INNER PRODUCT.

= max(i,j),i6=j (< Ai , Aj >)
Range of Coherency Value is : 0 || 1
0 : least coherency
1 : full coherent
We expect least coherency in our base in order to success in MP.

13 / 13