Compressive Sensing: An Overview

Presented by Vinaya L Shrestha

Nyquist-Shannon Sampling theorem

the sampling rate must be greater than twice the maximum frequency present in the signal
f s 2 f max

No more!

Sub-Nyquist sample and recover the signal almost exactly

Compressive Sensing (CS)

Recover certain signals from far fewer samples or measurements than traditional methods Relies on two principles: sparsity and incoherence

Sparsity

information rate of a continuous time signal may be much smaller than suggested by its bandwidth Permits efficient fundamental signal processing as accurate estimation, data compression, etc. Many natural signals are sparse i.e have concise representations when expressed in proper basis

Sparse Signals

In strict sense, sparse signals have at most S non-zero coefficients out of N, S << N, called S-sparse In practice, most coefficients are small, and the relatively few large coefficients capture most of the information Keep S-largest coefficients and discard the smaller ones without much perceptual loss

[]
0.8059878 0 0 0.6023637 0 2.7767681 0 0 0 0 0 0.3256375 1.0992125 0 0 0 1.1510231

Sparse vector of size N

Coherence

Quantity that measures similarity between two bases CS mainly concerned with low coherence pair Low coherence facilitates sub-sampling Coherence between bases and given by
, = n max k , k 1k , jn

In the range [1, n] i.e. , [1, n ]

Incoherence sampling theorem

A signal x is S-sparse in basis Select m measurements at random in such that

m , . S.log n

Example

Signal length n = 256 Sparsity S = 15 With bases with coherence 1, we can get m 36

Collecting incoherent measurements

Measurements are obtained by projecting the signal x on a set of measurement vectors Simply said, multiply sparse signal with a random matrix

Can we use any random matrix??

Restricted Isometry Property (RIP)

For each integer S = 1, 2, ... , define the isometry constant s of a matrix A, for all S-sparse vectors x, as the smallest number such that
1 s xl Axl 1 s xl
2 2

2
2

Matrix A obeys RIP of order S if s is sufficiently less than one Recovery of compressed samples possible only when the sensing matrix obeys RIP

Reconstruction

Exact if x is S-sparse Otherwise reconstructs the S largest entries of x Powerful if S is close to m Compressed signal can be exactly recovered from condensed data set by minimizing a convex functional

Reconstruction Algorithms

1 minimization: minimize xl 1 subject to A x = y

Practical, and often gives the correct answers.

Orthogonal Matching Pursuit (OMP) and Basis Pursuit (BP) widely used

Applications

Single-pixel imaging via Compressive Sampling Analog To Information (A2I) Converter Under-sampled Radial MR Acquisition and Highly Constrained Back Projection (HYPR) Reconstruction: Potential Medical Imaging Applications in the Post-Nyquist Era

MatLab Implementation

Compressed sensing and reconstruction of audio signal Audio Signal sampled at 11025 samples/second No. of samples 7444

[x, Fs, nbits] = wavread('be_quiet.wav');

Original Audio Signal

Signal Sparsity

FFT of the signal Plot the signal to see sparsity

x_f = fft(x); n = N/2; amp_spectrum = abs(x_f)/n; freq = (0:size(amp_spectrum, 1) - 1)/(2*n*dt); plot(freq, amp_spectrum(1:size(amp_spectrum)));

Signal in Fourier basis

Frequency Modulation

Perform Frequency Modulation and then FFT of the modulated signal

Fc = Fs; Fs = 10*Fc; freqdev = 2500; y = fmmod(x, Fc, Fs, freqdev); y_f = fft(y);

Sparse signal

Compressed Sensing

Perform Frequency Modulation and then FFT of the modulated signal Multiply the FFT vector with m x N measurement matrix to get compressed data No. of samples in original signal N = 7444 No. of measurements m = N/4 = 1861 Measurement Matrix msr_mat is 1861 x 7444 matrix Compressed signal y_c is 1861x1 vector
N = size(x, 1); m = round(N/4); msr_mat = qr(randn(N, m), 0); msr_mat = msr_mat'; y_c = msr_mat * y_f;

Compressed Sensing contd...

MatLab function randn generates random numbers from a standard normal distribution Sample random matrix
msr_mat = randn(m, N);

0.8059878 0.1314612 1.9843065 0.4976344 0.1487596 1.2165274 2.0907806 0.5084372 0.6023637 0.0401471 0.2108979 1.4054541 0.1617595 0.8849847 1.2449164 0.2244090 2.7767681 2.1029460 0.4369804 0.1712085 1.0028284 1.4280802 1.4800975 1.0265408 0.3256375 0.2671195 0.5748158 0.4690296 1.2405621 0.6259612 2.5191012 0.3181616 1.4288252 1.4892005 0.3021101 0.9439985 0.8536997 0.1832918 0.6251744 2.1554166 1.0992125 0.3366817 0.0689835 0.6061389 0.8067391 1.1437888 0.8110577 1.1510231

What about RIP?

Random matrices generally obey RIP if

mC .S log n /S

where C is some constant depending on each instance

Reconstruction

The compressed sensing vector y_c has only m = N/4 elements M = 1861 y_r is the reconstructed signal y_r is in frequency domain

[y_r, r, g, info] = spg_bp(msr_mat, y_c, options); [y_r, iters, activationHist] = SolveOMP(msr_mat, y_c, N);

Orthogonal Matching Pursuit

measurement matrix x s-sparse signal y = compressed data vector y = x implies

Each row in y is a linear combination of s-columns from with x

0.805 0.132 1.216 2.090 0.508 2.102 1.428 1.480 1.026 y= 2.776 0.325 0.267 0.625 2.519 0.318 1.099 0.336 1.143 0.811 1.151

To reconstruct the signal x, we need to determine which columns of participate in y

][ ] [

A 0.805A1.216B 0 2.776A1.428B B= 0.325A0.625B 0 1.099A1.143B 0

Orthogonal Matching Pursuit contd...

Begin an iterative process

Pick a column in that is most strongly correlated with the remaining

part of y Subtract off its contribution to y and iterate on the residual Iterate until the correct set of columns have been identified

Amplitude Spectrum of reconstructed signal

Inverse Fourier Transform

Frequency Modulated time domain signal obtained by performing Inverse Fourier Transform of the recovered signal FM demodulation is then performed

[y_r, r, g, info] = spg_bp(msr_mat, y_c, options); y_if = ifft(y_r); y_dm = demod(y_if, Fc, Fs, 'fm');

Reconstructed signal in time domain

How good is the Reconstruction?

Original Signal

Reconstructed Signal

References

E. J. Candes and Michael B. Walkin, An Introduction to Compressive Sampling J. A. Tropp and A. C. Gilbert, Signal Recovery from Random Measurements via Orthogonal Matching Pursuit