Tutorial of Orthogonal Matching Pursuit

A tutorial for Beginners and Dummies
Koredianto Usman
Sekolah Teknik Elektro dan Informatika

8 Mei 2014

Koredianto Usman

Tutorial of Orthogonal Matching Pursuit

Outline 1 Problem Statement 2 The Orthogonal Matching Pursuit 3 Koherensi Basis 2 / 13 .

65 −0.1.8 0 Now.6 −1 Therefore y = A · x gives:   −1.65 −0. Given that : y = and A = 0. Problem Statement Consider the following very simple example: Given the following sparse signals   −1.8 0 A= 0.707 0.25 0.707 0.6 −1 y= How to find original x? 3 / 13 .707 0.25     −1.2 x = 1  0 The following is the measurement matrix A (2 x 3):   −0.707 0.

8 0 b2 = b3 = 0.8 0 A= 0.6 −1 Columns in matrix A are called BASIS (CHEN and DONOHO : ATOMS). Orthogonal Matching Pursuit   −1.707 0.6 −1 4 / 13 .707 0. In the example.65 BASIS Previous example: Given : y = and 0.2. we have the following atoms: b1 =       −0.25   −0.707 0.707 0.

Orthogonal Matching Pursuit Interpretation of equation Ax = y Since A = [b1 b2 b3 ]. and if we let : x = [a b c].707 0.65 = −1.8 0 −1.707 −0. then A · x = a · b1 + b · b2 + c · b3 A · x is the linear combination of b1 .707 0.2.2 −0.8 0 A·x = · 1  0.707 0. b3 .6 −1 0.6 −1 0         −0. b2 .25 5 / 13 .2 · +1· +0· =y = 0.     −1.

2 −0.6 −1 0. b3 that will influence the STRONGEST to y .707 0.707 −0. dan last is b3 .25   From the equation above.707 0.707 0. 6 / 13 .2. ORTHOGONAL MATCHING PURSUIT works reversely: we start finding which of b1 .8 0 −1.8 0 A·x = · 0  0. Orthogonal Matching Pursuit   −1. it is clear that atom b1 contribute the biggest influence in y.65 = −1.2 · +1· +0· =y = 0. and so on.6 −1 1         −0. next is b2 . Then the SECOND STRONGEST from the residual. b2 .

y > calculate the residue ri = pi − pi · < pi .707) b2 (0.707 .25) X b3 (0 .2. Orthogonal Matching Pursuit STRONGEST influence is measured using DOT PRODUCT / INNER PRODUCT OMP Algorithm: 1 find atom with has the biggest inner product with y pi = maxj < bj . -1) 7 / 13 . 0. 0. 0.8 .65 .6) y (-1. y > find atom with has the biggest inner product with ri 4 repeat step 2 and 3 until residue achieve a certain threshold Geometrically: 2 3 Y b1 (-0.

17 and < y . Then. We next count the residue:     −1. we see b1 gives the biggest inner product.34 < y .34) · 0. b3: < y .34. Orthogonal Matching Pursuit Here the dot product of y to any of b1.25 Taking the absolute value. b1 > = − (−1. b1 >= −1.25 0. b2 >= 1.707 8 / 13 . b1 is chosen as the atom in first step.65 0.707 r1 = y − b1 · < y .2. b2. b3 >= 0. DOT PRODUCT −1.

-1) Next we count the DOT PRODUCT of this residue to b2 and b3 (no need to count with b1 . 0. b3 >= −0.2. we get b2 as the next strongest influence.25) b2 (0. Orthogonal Matching Pursuit Y Residue 1 b1 (-0.6) X b3 (0 . 9 / 13 . < r 1. 0.8 .707 . b1 >= 1 < r 1.7 Taking the absolute value.65 .707) y (-1. since this residue must perpendicular to b1 ). 0.

b3 >= −0.7 0.099 10 / 13 .7 0.6   −0. we finally count the final DOT PRODUCT.2.8 r 2 = r 1− < r 1. between r2 with the last b3 : < r 2. Orthogonal Matching Pursuit Next we count again the residue:     −0.099 From residue r2.099 = 0. b2 > b2 = − (1) · 0.

34. Orthogonal Matching Pursuit The following base is in STRONGEST influence in order: b1 . with influence measure from dot product is : −1.34 Therefore. 1. dan −0.2. b3 . the reconstructed x is  1  −0.099   −1. b2 .099   −1.2 Original was : 1  0 11 / 13 .

(C). Koherensi Rendah 12 / 13 . Koherensi Sedang. b2 b2 A3 b2 A3 A3 A2 A2 A2 A1 b1 b1 b1 A1 (A) (B) A1 (C) Figure : Ilustrasi Koherensi Basis. (B).3. Koherensi tinggi. (A). Base Coherency Base Coherency indicate how close one base to the others.

Base Coherency A simple measure for base coherency is INNER PRODUCT. Aj >) Range of Coherency Value µ is : 0 ≤ |µ| ≤ 1 0 : least coherency 1 : full coherent We expect least coherency in our base in order to success in MP. 13 / 13 .j). µ = max(i.i6=j (< Ai .3.