Lagrange Multipliers in PCA

1. Principal Component Analysis

Principal Component Analysis (PCA) l mt k thut c dng :
Gim chiu d liu
Nn mt mt thng tin
Rt trch c trng
Trc quan ha d liu
tng: Chiu vung gc d liu ban u xung mt khng gian t chiu hn sao cho
phng sai ca d liu chiu l cc i.

2. Cc i ha phng sai d liu chiu
Cho trc d liu
1
N
n
x , vi mi x
n
c d chiu.
Mc ch: Chiu d liu trn xung khng gian m chiu, m<d sao cho cc i ha phng
sai d liu chiu.
Chng ta xem xt php chiu xung khng gian mt chiu (m=1).
nh ngha hng ca khng gian ny bng cch s dng vector u
1
vi d chiu
Mean ca d liu chiu l
1
T
u x vi x l mu trung bnh c tnh nh sau:
1
1
N
n
n
x x
N

Phng sai ca d liu chiu c cho nh sau:

Vi S l ma trn hip phng sai c tnh :

Cc i ha phng sai chiu u
1
T
Su
1
vi rng buc u
1
. ||u
1
|| khng tin n v hn, s
dng rng buc u
1
T
u
1
= 1 cho ta bi ton ti u:
Maximize u
1
T
Su
1
Subject to u
1
T
u
1
= 1
3. Lagrange Multipliers in PCA
gii bi ton ti u trn, ta s dng phng php nhn t Lagrange.
Xt th Lagrange sau (
1
l nhn t Lagrange)


T o hm

Cho ta kt qu (*)
R rng biu thc (*) trn cho thy, u
1
phi l vector ring ca S.
Nhn 2 v ca (*) bn tri vi u
1
T
v s dng rng buc u
1
T
u
1
= 1 ta c :

Theo nh thy th phng sai c cc i ha khi u
1
bng vi vector ring c tr ring ln nht l
1

Tm 2rd PC :
Gi s u
2
l vector ring th 2 (u
2
phi cng di v vung gc vi u
1
).
tm PC th 2 ta tr li vi bi ton ti u nh sau :

Li p dng PP nhn t Lagrange ta xt :

(
1
,
2
l cc nhn t Lagrange)
Tng t gii bi ton trn ta c :

Kt qu trn hm rng u
2
ng l mt vector ring ca S vi tr ring ln th hai
2
. Nhng chiu
khc c a ra theo kt qu gim dn ca gi tr ring.
Phng php nhn t Lagrange l mt trong nhng k thut mnh gii quyt bi ton ti u.
Lagrange kp ( Dual Lagrange) c cho l mnh hn c Quadratic Programming (QP). Tm quan
trng ca phng php nhn t Lagrange trong Khoa hc my tnh ni chung v trong CVR ni ring
l khng th ph nhn.
References :
https://www.sec.in.tum.de/assets/lehre/ws0910/ml/slideslecture4.pdf
http://spider.ipac.caltech.edu/staff/fmasci/home/statistics_refs/PrincipalComponentAnalysis.pdf