Principal Component Analysis

 Principal Component Analysis is a well-known dimension reduction technique.

 It transforms the variables into a new set of variables called as principal components.
 These principal components are linear combination of original variables and are orthogonal.
 The first principal component accounts for most of the possible variation of original data.
 The second principal component does its best to capture the variance in the data.
 There can be only two principal components for a two-dimensional data set.

PCA Algorithm-
The steps involved in PCA Algorithm are as follows-
Step-01: Get dataset.
Step-02: Compute the Mean value for the variables.
Step-03: Calculate the covariance matrix
Step-3.1: .Find the ordered pair
Step-3.2: .Compute covariance for all ordered pair
Step-04: Calculate the Eigen Value and Eigen Vector of the covariance matrix.
Step-05: Normalize Eigen Vector
Step-06: Deriving the new data set.
Example:

Consider the two dimensional patterns

Feature Example 1 Example 2 Example 3 Example 4

X 4 8 13 7

Y 11 4 5 14

Compute the principal component using PCA Algorithm.

Solution-
Step-01: Get dataset

Feature Example 1 Example 2 Example 3 Example 4

 X 4 8 13 7

Y 11 4 5 14

Number of features, n=2

Number of samples, N=4

Step-02: Compute the Mean value for the variables.

Step-03: Calculate the covariance matrix

Step-3.1: Find the ordered pair
(X,X),(Y,Y),(X,Y),(Y,X)

Step-3.2: Compute covariance for all ordered pair

Cov(x, y) = Cov(y, x) -11

Covariance Matrix
Step-04: Calculate the Eigen Value and Eigen Vector of the covariance matrix.
Eigen Value:

Eigen Vector of 1
Where t=1

U1= 11

U2 = 14 - 1

Eigen Vector u1 of 1=
 1=

Step-05: Normalize Eigen Vector

e1 =

e1=

Step-06: Deriving the new data set.

First PCA Example1 Example 2 Example 3 Example 4

P11 P12 P13 P14

P11=eT1

P11= [0.5574 – 0.8303]

P11 = -4.3.52

P12=3.7361

P13=5.6928

P14=-5.1238