Dimensionality,

Reduc1on,
Mo1va1on,I:,,
Data,Compression,

Machine,Learning,
Data$Compression$
Reduce,data,from,,
(inches),

2D,to,1D,

(cm),

Andrew,Ng,
Data$Compression$
Reduce,data,from,,
(inches),

2D,to,1D,

(cm),

Andrew,Ng,

data compression makes our algorithm rum more faster

Data$Compression$
Reduce,data,from,3D,to,2D,

Andrew,Ng,
 Dimensionality,
Reduc1on,
Mo1va1on,II:,,
Data,Visualiza1on,

Machine,Learning,
 Data$Visualiza2on$
Mean,
Per,capita, Poverty, household,
GDP, GDP,, Human, Index, income,
(trillions,of, (thousands, DevelopQ Life, ,(Gini,as, (thousands,
Country, US$), of,intl.,$), ment,Index,expectancy,percentage), of,US$), …,
Canada, 1.577, 39.17, 0.908, 80.7, 32.6, 67.293, …,
China, 5.878, 7.54, 0.687, 73, 46.9, 10.22, …,
India, 1.632, 3.41, 0.547, 64.7, 36.8, 0.735, …,
Russia, 1.48, 19.84, 0.755, 65.5, 39.9, 0.72, …,
Singapore, 0.223, 56.69, 0.866, 80, 42.5, 67.1, …,
USA, 14.527, 46.86, 0.91, 78.3, 40.8, 84.3, …,
…, …, …, …, …, …, …,
[resources,from,en.wikipedia.org], Andrew,Ng,
Data$Visualiza2on$
Country,
Canada, 1.6, 1.2, plot these countries in
R2 and use that to try
China, 1.7, 0.3, to understand the space
of features of different
India, 1.6, 0.2, countries the better.
So, what you can do is

Russia, 1.4, 0.5,

Singapore, 0.5, 1.7, Reduce Data
from 50D to 2D
USA, 2, 1.5,
…, …, …,
Andrew,Ng,
 Data Visualization
Data$Visualiza2on$

Per. Person
GDP
(economic
activity)

Country size GDP

(Gross Domestic Product)

The axes Z1 and Z2 can help you to capture really what are the two main dimensions of
the variations among different countries. Andrew,Ng,
 Dimensionality,
Reduc1on,
Principal,Component,
Analysis,problem,
formula1on,
Machine,Learning,
 Let's say we have a data set like
this, and we want to reduce the
dimension of the data from two-
dimensional to one-dimensional.

In other words, find a line onto

which to project the data.
what PCA does formally is, it tries to find a lower
dimensional surface, onto which to project the data so that
the sum of squares of these little blue line segments is
minimized. The length of those blue line segments
sometimes also called the projection error.
 if we project the data onto this second line, the projection
errors will be huge. So these points have to move a huge
distance in order to get projected. And so that's why PCA will
choose the red line.
 In this example, hope that PCA will find the red
vector “u1”, so when projecting the data onto this
line we end up with pretty small reconstruction
errors. Whether PCA gives “u1” or “-u1”, it doesn't
matter.
PCA$is$not$linear$regression$

X —> Y

Andrew,Ng,
PCA$is$not$linear$regression$

Andrew,Ng,
 Dimensionality,
Reduc1on,
Principal,Component,
Analysis,algorithm,

Machine,Learning,
Data$preprocessing$
Training,set:,
Preprocessing,(feature,scaling/mean,normaliza1on):,

Replace,each,,,,,,,,with,,,,,,,,,,,,,,,.,
If,different,features,on,different,scales,(e.g.,,,,,,,,,,,,size,of,house,,,
,,,,,,,,,,number,of,bedrooms),,scale,features,to,have,comparable,
range,of,values.,

Andrew,Ng,
Principal$Component$Analysis$(PCA)$algorithm$
Reduce,data,from,,,Qdimensions,to,,,Qdimensions,
Compute,“covariance,matrix”:,
,
,
Compute,“eigenvectors”,of,matrix,,,,,:,
,[U,S,V] = svd(Sigma);,

Andrew,Ng,
Principal$Component$Analysis$(PCA)$algorithm$
From [U,S,V]
, , = svd(Sigma)
, ,
,,,,,,we,get:,,

Ureduce
Andrew,Ng,
Principal$Component$Analysis$(PCA)$algorithm$summary$
Ader,mean,normaliza1on,(ensure,every,feature,has,
zero,mean),and,op1onally,feature,scaling:,,
Sigma =

[U,S,V] = svd(Sigma);
Ureduce = U(:,1:k);
z = Ureduce’*x;,

Andrew,Ng,