Factor Analysis

Background

• What is factor analysis, anyone?

Background

• Factor analysis is a data reduction technique

• Used when variables are many
• And correlated
• No point when variables are uncorrelated
• Lots of correlated variables are there
• After FA, unmanageable no of variables gets reduced to a few manageable
factors
Background

• Common in what type of research – experiment, observation or

survey? Why?
• What is the DV in factor analysis?
• Why is it that the variables are correlated?
Background

• The DV is nothing i.e. no DV at all in FA

• The variables are correlated since there is a latent
factor that is driving these
• This is common between some variables, hence the inter-
correlation
• Plus this is not seen, unobservable and fuzzy
• This factor is indicated by the indicator variables
• Sometimes a priori e.g. OSL measure? What is OSL?
• Sometimes post-survey occurrence
 Steps in FA + Example

• First, collect data, enter them

• Subject them to FA using SPSS/SAS/Minitab
• Extract the factors
• Decide how many you want
• Name the factors
• Rotate the factors, if necessary
• Rename if necessary
• Validate using CFA or holdout sample
Examples

• Let us look at a couple of examples

FA – Issues & Key Terms

• Variance Extracted – of the 100% originally there, how much is

extracted?
• Can you extract 100%? Is it good?
• How many factors do you stop at then?
• How can you decide when to stop?
• There are essentially these criteria: Variance Extracted, Researcher
Discretion, Eigen Value & Scree Plot
FA – Issues & Key Terms

• Variance Extracted: stop at 75 or 80%; have norms

• Discretion of researcher: my data, will stop whenever
I want!!
• Scree Plot: stop where there is a sharp drop
• Eigen Value: The Variance explained by a factor
• If this is less than 1, drop the factor; why?
• What is the principle?
FA – Issues & Key Terms

• The principle is that the factor explains less variance than one
variable if Eigen value is < 1
• Why have a factor that explains less variance than just a single variable?
• Hence, drop it
• Communality: The variance of a particular variable explained by all
the factors
FA – Issues & Key Terms

• Rotation: Rotation is used to clarify and improve interpretation

• Basically rotate the axes and keep them perpendicular at a new place
• Factor scores: scores of respondents on factors, since variables no
longer there
FA – Issues & Key Terms

• "Rotation is quite analogous to taking a picture of the same object

from a different angle. For example, we may go up in a helicopter
and take an aerial photograph of the Grand Canyon, and we can
also take a shot from the floor of the canyon, looking through it
lengthwise, or from any other angle. There is no one "really
correct" view of the Grand Canyon. Each shot better highlights
some aspects more than others, and we gain a better impression
of the Grand Canyon from several viewpoints than from any single
one. Yet certain views will give a more informative overall picture
than others, depending on the particular viewer's interest.
FA – Issues & Key Terms

• "But no matter what the angle from which you photograph

the Grand Canyon, you cannot make it look like the rolling
hills of Devonshire, or Victoria Falls, or the Himalayas.
Changing the angle of viewing does not create something
that's not already there; it may merely expose it more
clearly, although at the expense of perhaps obscuring some
other feature."
• Arthur Jensen
FA – Some More Uses

• Can FZ be used in conjunction with MLR? How?

• Hint: Basically, what does FA do? What is one prerequisite for MLR?
FA – Some More Uses

• If lots of multicollinearity exists, can do FA, and use factor scores

instead of original IVs
• If there are 2 factors extracted thru Varimax, what will be the corr. between
the factor scores? Why?
FA – Some Problems

• What are some problems with FA?

FA – Some Problems

• Can be all data-driven, no theory, can just do lots of data mining

• May be sample specific
• Plus it is all so subjective
• Naming factors, when to stop extraction
• What is a high loading? Low one?
• No tests of significance for loadings
• Can end up with low communalities for some variables
FA - Summary

• All in all, FA is a very useful too

• Has plenty of uses in academics and industry
• Widely used in survey research
• Need to know this much at least
• Have not gone into the matrix stuff