- All IT Syllabus
- Computer Science
- kq1224-en.pdf
- Slides Lecture7 Ext
- Burgers Equation_ Stability Analysis
- PES Paper Revised
- Peche
- thesis
- Applied Linear Algebra by Boor
- Strurm Louville_Robin Boundary Condition
- Eigenvalue Analysis of Compatible and Incompatible Rectangular Four-node Quadrilateral Elements
- What is Blind Signal Separation
- Lyapunov Maple
- Holmes the French Way
- 64130-mt----linear algebra
- A Brief History of Linear Algebra - Jeff Christensen
- Physics Candidacy Exam 2008
- gharbi_techreport_2012
- ERDAS Spatial Modeler Language Reference Manual
- Analysis of the Behaviour of Reduced and Compressed Data with Various Learning Algorithms
- Sergei Winitzki-Linear Algebra via Exterior Products-lulu.com (2010)
- Chapter 3
- lecture 9 positive deﬁnite matrices
- Analysis of High Fidelity Quantum Driving
- Non-Iterative Method to Determine Static Stability Boundaries
- A general approach to tune the vibration properties of the mounting system in the high-speed and heavy engine
- s
- Linear Algebra Complete Book
- Parametrization of General Catmull-Clark Subdivision Surfaces and its Appli.pdf
- rref
- About Stacks.pdf
- Joyal Cayle y
- Chart of Entity Comparison.pdf
- Survey F14 Copy
- Entity Comparison
- Practice+Problems2_4031_F14
- ISYE+4301
- Practice Test A
- 1301_tg_horstmann_2012-3
- Roboethics a Navigating Overview - Spyros
- How-Much-Does-Brick-Cost.pdf
- EDi-IG
- Interest Factors
- Exercise07-step2
- Intermediate Accouting testbank ch13
- Affidavit General
- ECN301 Syllabus Fall II 2010
- Non-Resident Student Waiver US_11!3!2011
- Untitled
- Transfer-In GPA PDF
- San Thuong
- How to Mind Map
- GA Perimeter College to GA Tech Engineering
- Exercise 07
- ch13s

Principal Component

Analysis

2.1

Eigenvectors of the Covariance Matrix Derivation

2.1.1

Principal Component Feature Extraction

Consider a random vector x ∈ ℜ

n

for which a basis u is to be determined such that an approxi-

mation of x can be obtained by a linear combination of m orthogonal basis vectors:

xˆ = y 1 u 1 + y 2 u 2 + … + y m u m

[154]

**where m < n and y j is the weighting coefficient of basis vector u j formed by taking the inner
**

product of x with u j :

T

yj = x uj .

[155]

**By forming a k × n observation matrix X , the rows of which are the observations x k , we can
**

express Equation 154 and Equation 155 respectively in the following form:

T

ˆ = YU T

X

m

[156]

Y m = XU m

[157]

where U m is an n × m matrix whose columns are an uncorrelated basis for X , and Y m is a

k × m matrix whose columns are the coefficients for each column vector in U m . By the orthog157

then. We refer to Equation 156 as the PCAfeature re-synthesis equation and Equation 157 as the PCA-feature projection equation where the columns of Y m correspond to the estimated features and the columns of U m are the projection vectors which perform the linear transformation of the input to the new uncorrelated basis. U j = 1 . The problem.e. is defined as the element-wise operation: ∞ E[ψ(x)] = ∫ ψ ( x ) px ( x ) dx [161] –∞ where p x ( x ) is the probability density function of the random variable x . for deriving a PCA is to obtain the matrix U m such that the residual error in approximating x with xˆ is minimized: n ε = x – xˆ = ∑ y ju j [159] j = m+1 that is. say ψ ( x ) . A suitable criteria is the minimization of the expectation of the mean-square residual error: 2 2 ξ = E [ ε ] = E [ x – xˆ ] [160] where the expectation of an arbitrary function of x . i. the expansion of all the unused features results in a minimal signal which is the residual error ε . Since the expectation operator is linear and due to the condition that the columns of U m are orthonormal it follows that: 158 . it follows that: T Um Um = Im [158] where I m is a m × m diagonal matrix with unit entries.Eigenvectors of the Covariance Matrix Derivation onality of U m and by the imposition of an additional constraint that the columns of U m have unit norm.

[166] Thus. n ∂u j .Eigenvectors of the Covariance Matrix Derivation n n T ζ = E [ ε ε ] = E ∑ y i u i ∑ y j u j i = m + 1 j = m + 1 T n ∑ = 2 E[y j ] [162] j = m+1 from which it follows that: 2 E[y j ] T = E [ ( u j x ) ( x u j ) ] = u j E [ xx ]u j = u j Ru j T T T T [163] where R is the correlation matrix for x . [165] It is well known that the solutions to this equation constitute the eigenvectors of the correlation matrix R . …. we can re-express the residual error as the sum of the eigenvalues of the unused portion of the basis: n ζ = ∑ λj [167] j = m+1 159 . j = m + 1. This is a necessary condition for the minimum and gives the characteristic equation: ∂ ζ' = 2 ( Ru j – λju j ) = 0. the problem is equivalent to finding the eigenvectors of the covariance matrix Q x . It is also worth noting that the correlation matrix is related to the covariance matrix by the following expression: T Q x = R x – mm . for zero-mean or centered data. Since the columns of U are now determined to be the eigenvectors. Now by substitution of Equation 163 into Equation 162 we arrive at the quantity to be minimized: n ∑ ζ = T u j Ru j [164] j = m+1 We can express the minimization as a multi-variate differential equation by using a set of Lagrangian multipliers λ j and setting the derivative of the expectation of the mean-square error with respect to the basis components u j to zero.

Eigenvectors of the Covariance Matrix Derivation and the solution to the minimization reduces to ordering the basis vectors u j such that the columns with the smallest eigenvalues occur in the unused portion of the basis which also implies that the m columns of U m which are used for reconstruction should comprise the eigenvectors with the m largest eigenvalues. Since the eigenvalues form the diagonal elements of the covariance of the transformed data with all other elements equal to zero we can express the solution to the eigenvalue decomposition as a diagonalization of the input covariance matrix Q x . 160 . Now that we have arrived at the form of the solution for optimal orthonormal basis reconstruction (in the square-error sense) we must find a general form for representing the solution.

- All IT SyllabusUploaded byArijit Maji
- Computer ScienceUploaded byAnurag Garg
- kq1224-en.pdfUploaded byDAVID
- Slides Lecture7 ExtUploaded byMuhammad Zeeshan
- Burgers Equation_ Stability AnalysisUploaded bysiva_ksr
- PES Paper RevisedUploaded bymayalasan1
- PecheUploaded byMorglodos Morglodos
- thesisUploaded byАлексей Вахрушев
- Applied Linear Algebra by BoorUploaded byVikram Singh
- Strurm Louville_Robin Boundary ConditionUploaded byAnonymous ya6gBBwHJF
- Eigenvalue Analysis of Compatible and Incompatible Rectangular Four-node Quadrilateral ElementsUploaded byWei Zhang
- What is Blind Signal SeparationUploaded bysubhashissahoo
- Lyapunov MapleUploaded byMarluci Galindo
- Holmes the French WayUploaded byHéctor Mora
- 64130-mt----linear algebraUploaded bySRINIVASA RAO GANTA
- A Brief History of Linear Algebra - Jeff ChristensenUploaded byJoão Júnior
- Physics Candidacy Exam 2008Uploaded byWilliam Rawlinson
- gharbi_techreport_2012Uploaded byKarthik Sheshadri
- ERDAS Spatial Modeler Language Reference ManualUploaded byRolando Salas Lopez
- Analysis of the Behaviour of Reduced and Compressed Data with Various Learning AlgorithmsUploaded byIJIRST
- Sergei Winitzki-Linear Algebra via Exterior Products-lulu.com (2010)Uploaded byValdir Mendes
- Chapter 3Uploaded byKan Samuel
- lecture 9 positive deﬁnite matricesUploaded bykientrungle2001
- Analysis of High Fidelity Quantum DrivingUploaded byAnonymous s8fvgec
- Non-Iterative Method to Determine Static Stability BoundariesUploaded bymike151982
- A general approach to tune the vibration properties of the mounting system in the high-speed and heavy engineUploaded bybeinou
- sUploaded byWilliams Danilo Clemente Huanquis
- Linear Algebra Complete BookUploaded bySyed Muhamad Jahanzeb
- Parametrization of General Catmull-Clark Subdivision Surfaces and its Appli.pdfUploaded byRosaBarreto
- rrefUploaded byDivij

- About Stacks.pdfUploaded bycthunder_1
- Joyal Cayle yUploaded bycthunder_1
- Chart of Entity Comparison.pdfUploaded bycthunder_1
- Survey F14 CopyUploaded bycthunder_1
- Entity ComparisonUploaded bycthunder_1
- Practice+Problems2_4031_F14Uploaded bycthunder_1
- ISYE+4301Uploaded bycthunder_1
- Practice Test AUploaded bycthunder_1
- 1301_tg_horstmann_2012-3Uploaded bycthunder_1
- Roboethics a Navigating Overview - SpyrosUploaded bycthunder_1
- How-Much-Does-Brick-Cost.pdfUploaded bycthunder_1
- EDi-IGUploaded bycthunder_1
- Interest FactorsUploaded bycthunder_1
- Exercise07-step2Uploaded bycthunder_1
- Intermediate Accouting testbank ch13Uploaded bycthunder_1
- Affidavit GeneralUploaded bycthunder_1
- ECN301 Syllabus Fall II 2010Uploaded bycthunder_1
- Non-Resident Student Waiver US_11!3!2011Uploaded bycthunder_1
- UntitledUploaded bycthunder_1
- Transfer-In GPA PDFUploaded bycthunder_1
- San ThuongUploaded bycthunder_1
- How to Mind MapUploaded bycthunder_1
- GA Perimeter College to GA Tech EngineeringUploaded bycthunder_1
- Exercise 07Uploaded bycthunder_1
- ch13sUploaded bycthunder_1