53 views

Uploaded by christina90

- Image Segmentation using clustering (Texture with PCA)
- pca
- High Frequency Trading Model Mit
- PCA in Java
- Distributed Event-Triggered Consensus Tracking of First-Order Multi-Agent Systems with a Virtual Leader
- ssexam4
- Online Handwritten Signature Verification Using Neural Network Classifier Based on Principle Component Analysis
- KEY DETERMINANTS FOR SUCCESSFUL PROJECT IMPLEMENTATION IN THE INDUSTRIAL AREA OF WEST BENGAL
- Ken Kel 1986
- 12A Comprehensive Methodological Analysis of Eigen Face Recognition and Reconstruction
- Multivariable Feedback Design
- Matrix Decomp
- CB3A2C3Cd01
- Differential Equations Exam 3 Review
- arya
- Improved BSP Clustering Algorithm for Social Network Analysis
- HaslerS03
- Brandt2011 Errata
- Sohrab Poor 2015
- PCA

You are on page 1of 64

edu

Spring 2007

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Harvard-MIT Division of Health Sciences and Technology HST.582J: Biomedical Signal and Image Processing, Spring 2007 Course Director: Dr. Julie Greenberg

HST.582J/6.555J/16.456J Gari D. Clifford

http://www.mit.edu/~gari

G. D. Clifford 2005-2008

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

What is BSS?

Assume an observation (signal) is a linear mix of >1 unknown independent source signals The mixing (not the signals) is stationary We have as many observations as unknown sources To find sources in observations - need to define a suitable measure of independence

For example - the cocktail party problem (sources are speakers and background noise):

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

z1 z2 A xN XT=AZT zN ZT

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

x1 x2

XT

N independent sources Zmn ( M xN ) linear square mixing Ann ( N xN ) (#sources=#sensors) produces a set of observations Xmn ( M xN ) .. XT = AZT

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

demix observations XT ( N xM ) into YT = WXT YT ( N xM ) ZT W ( N xN ) A-1

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

POLICE

POLICE

Signal source

Noise sources

Observed mixtures

ZT

XT=AZT

YT=WXT

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

XT

T

ZT

T

YT

XT

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Observation Sx(f ) x[n] - observation Noise Power Sd(f ) y[n] - ideal signal d[n] - noise component Ideal Signal Sy(f ) Filtered signal: Sfilt (f ) = Sx(f ).H(f ) f

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

BSS is a transform?

Like Fourier, we decompose into components by transforming the observations into another vector space which maximises the separation between interesting (signal) and unwanted (noise). Unlike Fourier, separation is not based on frequencyIts based on independence Sources can have the same frequency content No assumptions about the signals (other than they are independent and linearly mixed)

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Second order decorrelation = independence Find a set of orthogonal axes in the data (independence metric = variance) Project data onto these axes to decorrelate Independence is forced onto the data through the orthogonality of axes Conventional noise / signal separation technique

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Decompose observation X=AZ into. X=USVT S is a diagonal matrix of singular values with elements arranged in descending order of magnitude (the singular spectrum) The columns of V are the eigenvectors of C=XTX (the orthogonal subspace dot(vi,vj)=0 ) they demix or rotate the data U is the matrix of projections of X onto the eigenvectors of C the source estimates

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Decompose observation X=AZ into. X=USVT

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Eigenspectrum of decomposition

S = singular matrix zeros except on the leading diagonal Sij (i=j) are the eigenvalues Placed in order of descending magnitude Correspond to the magnitude of projected data along each eigenvector Eigenvectors are the axes of maximal variation in the data

[stem(diag(S).^2)]

Variance = power (analogous to Fourier components in power spectra)

To perform SVD filtering of a signal, use a truncated SVD decomposition (using the first p eigenvectors)

Y=USpVT

[Reduce the dimensionality of the data by discarding noise projections Snoise=0 Then reconstruct the data with just the signal subsapce]

Most of the signal is contained in the first few principal components. Discarding these and projecting back into the original observation space effects a noise-filtering or a noise/signal separation

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Real data

S2

n

Xp =USpVT

Xp

p=2

Xp

p=4

As in PCA, we are looking for N different vectors onto which we can project our observations to give a set of N maximally independent signals (sources) output data (discovered sources) dimensionality = dimensionality of observations Instead of using variance as our independence measure (i.e. decorrelating) as we do in PCA, we use a measure of how statistically independent the sources are.

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Assume underlying source signals (Z ) are independent. Assume a linear mixing matrix (A ) XT=AZT in order to find Y (Z ), find W, (A-1 ) ... YT=WXT How? Initialise W & iteratively update W to minimise or maximise a cost function that measures the (statistical) independence between the columns of the YT.

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

From the Central Limit Theorem, - add enough independent signals together, Sources, Z P(Z) (subGaussian) Mixtures (XT=AZT) P(X) (Gaussian)

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Gaussian PDF

x

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

SubGaussian

SuperGaussian

Central Limit Theorem: add enough independent signals together, Gaussian PDF ( =3) make data components non-Gaussian to find independent sources Sources, Z P(Z) ( <3 (1.8) ) Mixtures (XT=AZT) P(X) ( =3.4)

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Assume underlying source signals (Z ) are independent. Assume a linear mixing matrix (A ) XT=AZT in order to find Y (Z ), find W, (A-1 ) ... YT=WXT Initialise W & iteratively update W with gradient descent to maximise kurtosis.

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Given a cost function, , we update each element of W (wij ) at each step, ,

and recalculate cost function ( is the learning rate (~ 0.1), and speeds up convergence.)

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

10 5

-2 -1]

wij

Iterations

Gradient descent

Gradient Descent

Cost function, , can be maximum or minimum 1/

Imagine a 2-channel ECG, comprised of two sources;

Cardiac Noise

and SNR=1

X1

X2

Y1

Y2

Y1

Y2

Y1

Y2

Y1

Y2

Y1

Y2

Y1

Y2

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Y1

Y2

Y1

Y2

Y1

Y2

In general we require a measure of statistical independence which we maximise between each of the N components. Non-Gaussianity is one approximation, but sensitive to small changes in the distribution tail. Other measures include:

(Note: all are related to )

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Kurtosis is highly sensitive to small changes in distribution tails. A more robust measures of Gaussianity is based on differential entropy H(y), negentropy: where ygauss is a Gaussian variable with the same covariance matrix as y. J(y) can be estimated from kurtosis

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Mutual information (MI) between two vectors x and y : I = Hx + Hy - Hxy always non-negative and zero if variables are independent therefore we want to minimise MI. MI can be re-written in terms of negentropy

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Generative latent variable modelling N observables, X ... from N sources, zi through a linear mapping W=wij Latent variables assumed to be independently distributed Find elements of W by gradient ascent - iterative update by where is some learning rate (const) and is our objective cost function, the log likelihood

some real examples using ICA

POLICE

Observations

Separation of mixed observations into source estimates is excellent apart from:

Signals have been scaled

Why? In XT=AZT, insert a permutation matrix B XT=ABB-1ZT B-1ZT = sources with different col. order. sources change by a scaling A

AB ICA solutions are order and scale independent because is dimensionless

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

=3 =11

=0 =3

=1 =5

=0 =1.5

reflective of the actual source generators - no gold standard De-mixing might alter the clinical relevance of the ECG features Solution: Identify unwanted sources, set corresponding (p) columns in W-1 to zero (Wp-1 ), then multiply back through to remove noise sources and transform back into original observation space.

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Real data

X

Y=WX Z

Xfilt = -1 Wp Y

Courtesy of Springer Science + Business Media. Used with permission. Source: He, Clifford, and Tarassenko. Neural Computing & Applications 15 no. 2 (April 2006): 105-116. doi:10.1007/s00521-005-0013-y.

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Y=WX Z

Xfilt = -1 Wp Y

Courtesy of Springer Science + Business Media. Used with permission. Source: He, Clifford, and Tarassenko. Neural Computing & Applications 15 no. 2 (April 2006): 105-116. doi:10.1007/s00521-005-0013-y.

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Summary

PCA is good for Gaussian noise separation

ICA is good for non-Gaussian noise separation PCs have obvious meaning - highest energy components ICA - derived sources : arbitrary scaling/inversion & ordering

. need energy-independent heuristic to identify signals / noise - PC space only changes if SNR changes.

ICA assumes linear mixing matrix ICA assumes stationary mixing De-mixing performance is function of lead position ICA requires as many sensors (ECG leads) as sources Filtering - discard certain dimensions then invert transformation In-band noise can be removed - unlike Fourier!

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Maternal QRS

Richter M., T. Schreiber, and D. T. Kaplan. "Fetal EEG Extraction with Nonlinear Phase Space Projections." IEEE Trans Bio Med Eng 45 (January 1998): 133137. Copyright 1998 IEEE. Used with permission.

Fetal Q RS

Maternal ECG is much larger in amplitude Maternal and fetal ECG overlap in time domain Maternal features are broader, but Fetal ECG is in-band of maternal ECG

(they overlap in freq domain) 5 second window Maternal HR=72 bpm / Fetal HR = 156bpm

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Fetal QRS power region

ECG Envelope Movement Artifact QRS Complex P & T Waves Muscle Noise Baseline Wander

Adapted from W. J. Tompkins (ed.) Biomedical Digital Signal Processing: C Language Examples and Laboratory Experiments for the IBM PC. Englewood Cliffs, NJ: Prentice Hall, 1993.

Maternal

Noise Fetal

Richter M., T. Schreiber, and D. T. Kaplan. "Fetal EEG Extraction with Nonlinear Phase Space Projections." IEEE Trans Bio Med Eng 45 (January 1998): 133137. Copyright 1998 IEEE. Used with permission.

Cite as: Gari Clifford. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

- Image Segmentation using clustering (Texture with PCA)Uploaded byP Prakash Prakash
- pcaUploaded bytoprakalem99
- High Frequency Trading Model MitUploaded byprabinseth
- PCA in JavaUploaded bymohammad niudanri
- ssexam4Uploaded byAtif Imam
- Distributed Event-Triggered Consensus Tracking of First-Order Multi-Agent Systems with a Virtual LeaderUploaded byAnonymous vQrJlEN
- Online Handwritten Signature Verification Using Neural Network Classifier Based on Principle Component AnalysisUploaded byscience2222
- KEY DETERMINANTS FOR SUCCESSFUL PROJECT IMPLEMENTATION IN THE INDUSTRIAL AREA OF WEST BENGALUploaded byInternational Jpurnal Of Technical Research And Applications
- Ken Kel 1986Uploaded byAlejandra Arana
- 12A Comprehensive Methodological Analysis of Eigen Face Recognition and ReconstructionUploaded byBhanu Shree Gowda
- Multivariable Feedback DesignUploaded bynatalaric
- Matrix DecompUploaded byRamesh Kumar
- CB3A2C3Cd01Uploaded byItalo Chiarella
- Differential Equations Exam 3 ReviewUploaded byModernZen
- aryaUploaded byRavinder Ranga
- Improved BSP Clustering Algorithm for Social Network AnalysisUploaded byBONFRING
- HaslerS03Uploaded byHarish Kumar Tiwari
- Brandt2011 ErrataUploaded byviluk
- Sohrab Poor 2015Uploaded byDevarakonda Kondayya
- PCAUploaded byVALMICK GUHA
- 34157Uploaded byLuis Alberto Fuentes
- V2I1052Uploaded byAjay Gupta
- Eigenbased Multispectral Palmprint RecognitionUploaded byIJARP Publications
- 10.1.1.36Uploaded byehab_essa
- 4-Carassale-MarrxBrunenghiUploaded byItalo Adotti
- A Central Challenge in Analyzing Any Genetic Dataset is to Explore Whether There is Any Evidence That the Samples in the Data Are From a Population That is StructuredUploaded byYellapragada Akarsh
- physicaUploaded bypostscript
- LectureUploaded byJaved Hussain
- 1893839-AQA-MFP4-QP-JUN12.pdfUploaded byastargroup
- 8-principle-components-analysis.pdfUploaded byJulio César Chávez Galarza

- Fault Surfaces and Fault Throws From 3D Seismic ImagesUploaded byMahmoud Said
- Collocated Versus Non-collocated ControlUploaded byKarthikeyanMohan
- slepcUploaded byVan Dung Nguyen
- SAER-5711Uploaded byrjsquebrar
- Three Phase Induction Motor Incipient Rotor’s FaultsUploaded bybendiazz3430
- 07425250+++++Uploaded byCristian Daniel Antilao Pizarro
- Delia Schwartz-Perlov and Alexander Vilenkin- Measures for a Transdimensional MultiverseUploaded byDex30KM
- All sub CO & CLO IMPUploaded byanil1216kumar
- Ullmann's Encyclopedia of Industrial ChemistryUploaded byMaykkkow
- s Vd ProblemsUploaded byabhishek
- Complex ModesUploaded bybsitler
- 090601.ImplicitNotesUploaded bykasreedhar
- Optics NotesUploaded byAnant Saxena
- Group Theory - The Application to Quantum Mechanics [Meijer-Bauer]Uploaded bylinziyi
- 20. B.E. ECE.docUploaded bySampath Kumar
- Selection of Paramaters of Ecommerce Websites Using AHPUploaded byarcherselevators
- Practica 1Uploaded byMauricio De Stefani
- Syllabus 2012Uploaded bylokesh
- SolutionManual Pde PinchoverUploaded bycakil87as
- D. Belitz, T. R. Kirkpatrick and A. Rosch- Theory of helimagnons in itinerant quantum systemsUploaded bySaqqer
- Mathematica Cheat SheetUploaded byte123mp123
- STI12 Cyclic Symmetry at 6.0Uploaded byM. Ceyhun Sözbir
- May17.pdfUploaded byRadhe
- Signals and Systems Using MatlabUploaded byaxeans
- urca in rUploaded byalexa_sherpy
- Brake Analysis and NVH Optimization Using MSC.nastRANUploaded byMihir Shah
- Ibt SyllabusUploaded byyuvanus
- Lesson 7_ Principal Components Analysis (PCA)Uploaded bymekeller
- Arxiv Lecture Notes in Quantum MechanicsUploaded byHan Don
- Calculation of Electrical Parameters for Short and Long Polyphase Transmission LinesUploaded byyourou1000