You are on page 1of 34


By Guided By Ms.Arathi, Anila Mohan Asst.Professor, Muhammed Aslam. N Dept ECE, Pamela M.E.S.C.E Mariyadas Paulose Pratheek.T.K

 Introduction  Types of input images for fusion
   

Multifocus images PAN-Multispectral images MRI-CT images IR-Visual images

 Principal Component Analysis  Background Mathematics Supporting PCA  Steps in PCA based image fusion  Conclusion  References

• PCA stands for Principal Component Analysis

• Using this technique, we are going to fuse multi-

focus images to get an image of better visual quality. • Often due to variations in the screen depth , it is not possible to capture an image where the entire scene appears sharp. • Only scene areas that are at the focus plane appears sharp. Areas in front or behind the focus plane appears blurred

Types of input images for the fusion process
 Multifocus images
 PAN – Multispectral images  MRI-CT images  IR-Visual images

Multi-Focus Images

Multi-Focus Images

Panchromatic Image
 It is a 2D light intensity function, f(x,y) where x

and y are spacial coordinates and value of ‘f’ at (x,y) is proportional to the brightness of the scene at that point  In panchromatic image , only the total intensity of radiation falling on each pixel is recorded

Panchromatic Image

Multi Spectral Image
 Each component of f(x,y) indicates the brightness

of the scene at point (x,y) at the corresponding spectral band  A Multi spectral image is one that captures image data at specific frequencies across the electromagnetic spectrum  These type of images are captured in Remote Sensing Radiometers
So from all the explanations , we can represent an image as a matrix

Multi spectral Image

Medical Images –CT & MRI & Fused Image

Satellite Images captured in different sub-bands and the Fused Image

Visual & IR and Fused Images

Existing Image Fusion Techniques
 Like multi-focus image fusion, there is multi-

sensor image fusion which is the process of combining relevant information from two or more images into a single image  Image fusion methods 1.High pass filtering techniques 2.IHS transform based image fusion 3.PCA based image fusion 4.Wavelet transform image fusion 5.Curvelet transform based image fusion  This presentation illustrates PCA based image fusion

Background Mathematics
 Before getting into PCA we have to get some

elementary background mathematical skills

Standard Deviation
• To explain this we need a data set

• X=[1 2 3 4 5] , let it be a set. Here X1=1,X 2=2, etc
• There are number of things that can be

calculated about a data set • Mean elements

n= no of

Background Mathematics
 Consider two sets given below

A=[0 8 12 20] B=[8 9 11 12] Both have the same mean(10) . But the difference between these sets is that the spread of data • Standard Deviation of a data set is a measure of how spread out the data is • Standard Deviation is the average distance from the mean of the data set to a point • Standard Deviation is given by

Background Mathematics
• Standard Deviation of A=8.3266, Standard

Deviation of B=1.8257. As expected Standard Deviation of A is greater.

• It is the another measure of the spread of data in

a data set and is given by

Background Mathematics
Covariance • Standard Deviation and variance only operate on 1Dimentional set • Covariance is always measured between 2Dimentions • It gives how much the dimensions vary from the mean with respect to each other and it is given by

Background Mathematics
 If covariance is positive, it means both the

dimensions increases together  If covariance is negetive , it means one dimension increases and other decreases  If covariance is zero , it means two dimensions are independent of each other

Covariance Matrix
• Covariance matrix (C) for a set of data with ‘n’

dimensions, is a (n x n) matrix • If n=2 , then C= cov(x,x) cov(x,y) cov(y,x) cov(y,y)

Background Mathematics
Matrix Algebra • Consider the example for matrix multiplication given below

Background Mathematics
 In the first example , the resulting vector is not an

integer multiple of the original vector, but in the second example, the resulting vector is 4 times the original vector  3 2 =Eigenvector

1 3

=Non Eigenvector

• And the matrix is known as transformation matrix

Background Mathematics
 Eigenvector can only be found for square

matrices  And , given an (n x n) matrix that does have eigenvectors, then there are n of them

Principal Component Analysis
 There are several techniques used for image

fusion, among these techniques Principal Component Analysis is one of the easiest and simplest technique.  Principal Component Analysis (PCA) is employed to transform original image to its eigenspace.  By retaining the principal components with influencing eigenvalues, PCA keeps the key features in the original image and reduces noise level.

Principal Component Analysis
 The central idea of Principal Component

Analysis(PCA) is to reduce the dimensionality of a data set consisting of large no of interrelated variables , while retaining as much as possible of the variations present in the data set  This is achieved by transforming into a new set of random variables (Principal Components), which are uncorrelated and which are ordered so that the first few retains most of the variations present in all of the original variables  Let X be the data set having ‘p’ no of random variables X={(X1,Y1,Z1,…..) , (X2,Y2,Z2……) . . . . .} then by PCA we get dimensionally reduced

Principal Component Analysis

Principal Component Analysis
 These Principal Components are the

eigenvectors of covariance matrix of X.  Then for X we will get a (p x p) covariance matrix and corresponding ‘p’ no of eigenvectors  From these ‘p’ no of eigenvectors we select the first few retains most of the variations present in all of the original variables  These few is selected such that those having random values of maximum value

PCA based image fusion
 There are several steps while fusing multi-focus  

• •

images. They are given below Let I1 and I2 be the two multi-focus images I1 and I2 have same size. Let those be (n x n) matrices 2 Step 1: Consider a empty (n x 2)matrix A. Load all elements of matrix I1 into the first column of A and load all elements of matrix I2 into the second column of A . Now we got a 2D matrix A (data). Step 2: Then we create a matrix B such that, B=A’ x A Now we got a (2 x 2) matrix B Step 3: Now we will find the eigenvalues and corresponding eigenvectors of B

PCA based image fusion
Step 4: • Largest eigenvalue is found • Corresponding eigenvector for larger eigenvalue is found • This eigen vector is normalised and let its coefficient be a and b Step 5: • Fused image=(a x I1) +(b x I2)

MatLab Program
 I1=double(rgb2gray(imread('Time1.bmp')));  I2=double(rgb2gray(imread('Time2.bmp')));  A=[ ];  A=I1(:);

 b=I2(:);
 A=[A,b];  B=A'*A;  [v,d]=eig(B)  if(d(1,1)>d(2,2))

else a=v(:,2) end n1=abs(a(1,1)) n2=abs(a(2,1)) fusedimage = (n1*I1) + (n2*I2); imshow(uint8(I1));figure, imshow(uint8(I2)); figure,imshow(uint8(fusedimage));

 

  


Fused Image

Simulink Block Diagram

 PCA based fusion aims at combining two images

to get a single image which has the good information from both the images.  It is the simplest fusion technique, where the data is transformed to another set of co-ordinates, which better represents the data.  It calculates a weight for the input images based on their eigen values and eigen vectors, with which each image is multiplied and added to get the fused image.  It is not the best method of image fusion. Transform domain methods have shown to give much more promising results.

Questions?? ??