Factor Analysis as a Tool for Survey Analysis

*Karan Khulbe, **Pradeep Kumar, #Prof. Yashwant Singh Thakur, #Course Coordinator, Shubham Dadariya,

*Department Of Business Management, Dr. Harisingh Gour Vishwavidyalaya, Sagar, MP,India.

Abstract Factor analysis is particularly suitable to extract few factors from the large number of related
variables to a more manageable number, prior to using them in other analysis such as multiple regression or
multivariate analysis of variance. It can be beneficial in developing of a questionnaire. Sometimes adding more
statements in the questionnaire fail to give clear understanding of the variables. With the help of factor analysis,
irrelevant questions can be removed from the final questionnaire. This study proposed a factor analysis to
identify the factors underlying the variables of a questionnaire to measure tourist satisfaction. In this study,
Kaiser-Meyer-Olkin measure of sampling adequacy and Bartlett’s test of Sphericity are used to assess the
factorability of the data. Determinant score is calculated to examine the multicollinearity among the variables.
To determine the number of factors to be extracted, Kaiser’s Criterion and Scree test are examined. Varimax
orthogonal factor rotation method is applied to minimize the number of variables that have high loadings on
each factor. The internal consistency is confirmed by calculating Cronbach’s alpha and composite reliability to
test the instrument accuracy. The convergent validity is established when average variance extracted is greater
than or equal to 0.5. The results have revealed that the factor analysis not only allows detecting irrelevant items
but will also allow extracting the valuable factors from the data set of a questionnaire survey. The application of
factor analysis for questionnaire evaluation provides very valuable inputs to the decision makers to focus on few
important factors rather than a large number of parameters.

KEYWORDS : factor analysis, types, method, steps, advantages, limitations.

1.Introduction
Factor Analysis is a multivariate statistical
technique applied to a single set of variables when
the investigator is interested in determining which
variables in the set form logical subsets that are
relatively independent of one another.[1] In other
words, factor analysis is particularly useful to
identify the factors underlying the variables by
means of clubbing related variables in the same
factor[2] . In this paper, the main focus is given on
the application of factor analysis to reduce huge
number of inter-correlated measures to a few
representative constructs or factors that can be used
for subsequent analysis[4]. The goal of the present
work is to examine the application of factor
analysis of a questionnaire item to measure tourist
satisfaction. Therefore, in order to identify the
factors, it is necessary to understand the concept
and steps to apply factor analysis for the
questionnaire survey. Factory analysis is based on
the assumption that all variables correlate to some
degree. The variables should be measured at least
at the ordinal level. The sample size for factor
analysis should be larger but the more acceptable
range would be a ten-to-one ratio[3,4]. There are
two main approaches to factor analysis: exploratory
factor analysis (EFA) and confirmatory factor
analysis (CFA). Exploratory factor analysis is used
for checking dimensionality and often used in the
early stages of research to gather information about
the interrelationships among a set of variables[5].
The confirmatory factor analysis is a more complex
and sophisticated set of techniques used in the
research process to test specific hypotheses or
theories concerning the structure underlying a set
of variables[6,7].
What is Factor Analysis?
It refers to a method that reduces a large variable into
a smaller variable factor. Furthermore, this technique
takes out maximum ordinary variance from all
the variables and put them in common score.
Moreover, it is a part of General Linear Model
(GLM) and it believes several theories that contain
no multicollinearity, linear relationship,
true correlation, and relevant variables into the
analysis among factors and variables.
Types of Factor Analysis
There are different methods that we use in factor
analysis from the data set:
1. Principal component analysis
It is the most common method which the researchers
use. Also, it extracts the maximum variance and put
them into the first factor. Subsequently, it removes
the variance explained by the first factor and extracts
the second factor. Moreover, it goes on until the last
factor.
Principal component analysis (PCA) is
a linear dimensionality reduction technique with
applications in data analysis, visualization and data
preprocessing. This is accomplished by linearly
transforming the data onto a new coordinate
system (principal components) such that the
directions capturing the largest variation in the
data can be easily identified. Many studies use the
first two principal components in order to plot the
data in two dimensions and to visually identify
clusters of closely related data points. Principal
component analysis has applications in many fields
such as population genetics, microbiome studies,
and atmospheric science.[8]
Applications Of PCA :
1.Intelligence
2.Resedential Differentiation.
3.Developement Indexes
4.Population Genetics
5.Market Research And Index Of Attitudes
6.Quantitative Finance
Limitations of PCA
PCA is a powerful and versatile technique, but it
also has some limitations that you should be
aware of. For example, PCA is sensitive to the
choice of variables and the order of observations,
meaning that different selections or arrangements
of your data may lead to different results.
Therefore, you should always justify your choice of
variables and ensure that your data is randomly
ordered before applying PCA. Another limitation is
that PCA is not robust to missing values, meaning
that if your data has gaps or errors, PCA may not
work properly or produce inaccurate results.
Therefore, you should always handle missing
values properly before applying PCA, such as by
imputing them, deleting them, or using methods
that can deal with them, such as probabilistic PCA
or multiple imputation.
A Closer Look at PCA
(Principal Component
Analysis)
Let’s examine the model to help us further
describe PCA. It assumes that we have a
high-dimensional representation of data that
is, in fact, embedded in a low-dimensional
space. We assume that L≈XV where L is
some low-rank matrix, X is the original data
and V is a projection operator.
Our original data has n features, and we wish to
reduce them to k features, where k << n. With this,
we need to project the data onto another vector
space [2].
PCA has many interesting representations, but
we’ll look at a formulation that we’ve found to be
the most natural:
 To find the Principal Components (PCs),
we wish to minimize the squared
reconstruction error between our original
data points and their projection onto
some k-dimensional vector space.
 Mathematically, we need to find a
projection matrix V that solves the
following optimization
problem:argminVTV=Ik‖X−XVVT‖F2
 X is our original data. XV is the projection
to the k-dimensional space, and XVV is
the reconstruction back to the original
space.
 Projection from a 2-dimensional space to
a 1-dimensional space can be illustrated
by the following image:
In the image above:
 The blue dots are our original data
vectors.
 The red dots are their projections.
 The blue lines are the reconstruction
errors.
 The dashed black line is the optimal
projection V that we found when solving
our optimization problem.
We have now successfully projected the data from
a 2-dimensional space (x,y axis) to a 1-dimensional
space (line).
2.Exploratory Factor
Analysis :
Exploratory factor analysis (EFA) is a classical
formal measurement model that is used when
both observed and latent variables are assumed to
be measured at the interval level. Characteristic of
EFA is that the observed variables are first
standardized (mean of zero and standard
deviation of 1). EFA is executed on the correlation
matrix between the items. In EFA, a latent variable
is called a factor and the associations between
latent and observed variables are called factor
loadings. Factor loadings are standardized
regression weights. Since EFA is an exploratory
technique, there is no expected distribution of
loadings; hence, it is not possible to test
statistically whether or not factor loadings are the
same across cultural groups. However, congruence
measures, such as Tucker's ϕ, have been
developed to indicate whether the pattern of
factor loadings across items on a factor is the same
across cultural groups. Sufficient congruence for
structural equivalence is usually taken to be found
if Tucker's ϕ exceeds 0.95. Values below 0.90 are
taken to indicate that one or more items show
deviant factor loadings and thus show
bias. Bootstrap procedures have been developed
to test the identity of factor loadings in EFA.
EFA is used to investigate structural equivalence.
However, since it works on standardized variables
(mean of zero and standard deviation of 1), this
model is not suited to detect nonuniform and
especially uniform item bias.
EFA is often used in the multidimensional situation
where more than one latent variable is measured at
the same time. Before evaluating congruence in
this case, the factor structures should be rotated
toward a target structure.
Exploratory factor analysis can be performed
by the following methods :
1.R- type factor analysis : when factors are
calculated from the correlation matrix, then it is
called R-type factor analysis.
2.Q-type factor analysis : when factors are
calculated from the individual respondent, then it is
said to be Q-type factor analysis.
Exploratory factor analysis has three basic
decision points: (1) decide the number of
factors, (2) choosing an extraction method,
(3) choosing a rotation method.
EXPLORATORY FACTOR ANALYSIS: DECIDING
THE NUMBER OF FACTORS
The most common approach to deciding the
number of factors is to generate a scree plot. The
scree plot is a two dimensional graph with factors
on the x-axis and eigenvalues on the y-axis.
Eigenvalues are produced by a process called
principal components analysis (PCA) and represent
the variance accounted for by each underlying
factor. They are not represented by percentages
but scores that total to the number of items. A 12-
item scale will theoretically have 12 possible
underlying factors, each factor will have an
eigenvalue that indicates the amount of variation
in the items accounted for by each factor. If a the
first factor has an eigenvalue of 3.0, it accounts for
25% of the variance (3/12=.25). The total of all the
eigenvalues will be 12 if there are 12 items, so
some factors will have smaller eigenvalues. They
are typically arranged in a scree plot in decending
order like the following:
From the scree plot you can see that the first
couple of factors account for most of the variance,
then the remaining factors all have small
eigenvalues. The term “scree” is taken from the
word for the rubble at the bottom of a mountain.
A researcher might Factors Eigenvalues Newsom 3
SEM Winter 2005 examine this plot and decide
there are 2 underlying factors and the remainder
of factors are just “scree” or error variation. So,
this approach to selecting the number of factors
involves a certain amount of subjective judgment.
Another approach is called the Kaiser-Guttman
rule and simply states that the number of factors
are equal to the number of factors with
eigenvalues greater than 1.0. I tend to recommend
the scree plot approach because the Kaiser-
Guttman approach seems to produce many
factors.
EXPLORATORY FACTOR ANALYSIS: FACTOR
EXTRACTION
Once the number of factors are decided the
researcher runs another factor analysis to get the
loadings for each of the factors. To do this, one has
to decided which mathematical solution to use to
find the loadings. There are about five basic
extraction methods (1) PCA, which is the default in
most packages. PCA assumes there is no
measurement error and is considered not to be a
true exploratory factor analysis; (2) maximum
likelihood (a.k.a. canonical factoring); (3) alpha
factoring, (4) image factoring, (5) principal axis
factoring with iterated communalities (a.k.a. least
squares). Without getting into the details of each
of these, I think the best evidence supports the
use of principal axis factoring and maximum
likelihood approaches. I typically use the former.
Gorsuch (1989) recommends the latter if only a
few iterations are performed (not really possible in
most packages). Snook and Gorsuch (1989) show
that PCA can give poor estimates of the population
loadings in small samples. With larger samples,
most approaches will have similar results. The
extraction method will produce factor loadings for
every item on every extracted factor. Researchers
hope their results will show what is called simple
structure, with most items having a large loading
on one factor but small loadings on other factors.
EXPLORATORY FACTOR ANALYSIS: ROTATION
Once an initial solution is obtained, the loadings
are rotated. Rotation is a way of maximizing high
loadings and minimizing low loadings so that the
simplest possible structure is achieved. There are
two basic types of rotation: orthogonal and
oblique. Orthogonal means the factors are
assumed to be uncorrelated with one another.
This is the default setting in all statistical packages
but is rarely a logical assumption about factors in
the social sciences. Not all researchers using EFA
realize that orthogonal rotations imply the
assumption that they probably would not
consciously make. Oblique rotation derives factor
loadings based on the assumption that the factors
are correlated, and this is probably most likely the
case for most measures. So, oblique rotation gives
the correlation between the factors in addition to
the loadings. Here are some common algorithms
for orthogonal and oblique rotation: Orthogonal
rotation: varimax, quartamax, equamax. Oblique
rotation: oblimin, promax, direct quartimin I am
not an expert on the advantages and
disadvantages of each of these rotation
algorithms, and they reportedly produce fairly
similar results under most circumstances (although
orthogonal and oblique rotations will be rather
different). I tend to use promax rotation because it
is known to be relatively efficient at achieving
simple oblique structure.
Advantages of EFA :
Exploratory factor analysis (EFA) is generally
used to discover the factor structure of a measure
and to examine its internal reliability. EFA is often
recommended when researchers have no
hypotheses about the nature of the underlying
factor structure of their measure.
Limitation of EFA :
The major limitation behind Exploratory Factor
Analysis is its simplicity. Hence, the researcher will
not get a reliable inference. Therefore, Exploratory
Factor Analysis is used less as compared to
Confirmatory Factor Analysis.
3.Confirmatory Factor
Analysis:
Understanding the relationships between different
variables is an important part of statistical analysis.
Confirmatory factor analysis is a procedure
researchers use to determine if their theories
about data relationships are accurate. If you're
interested in social research or statistics,
understanding how to apply this technique can
help you learn essential insights about your data.
What is confirmatory factor
analysis?
Confirmatory factor analysis (CFA) is a statistical
modeling method that assesses how accurately
different systems measure and evaluate a concept.
With this method, researchers use their
background knowledge to develop a hypothesis
about how to measure it, then apply CFA to test
the accuracy of their ideas. Researchers use
structural equation modeling software to conduct
confirmatory factor analysis because it requires
processing complex data sets with advanced
mathematical models and equations.
CFA is a popular research and data analysis
procedure in the social sciences, particularly
psychology, because it can address theoretical
models and concepts that are difficult to measure,
such as emotions and psychological symptoms. In
social sciences, these measurement systems are
usually survey questions, rating scales and other
inventories. For example, a researcher may use
CFA to determine how well each mental health
survey question shows anxiety disorder symptoms.
Key terms of CFA :
Here are some of the basic terms to know when
conducting a confirmatory factor analysis:[9]
Observed Variable
An observed variable is a factor that you use to
measure a concept. Observed variables include the
data you record during your research. Questions
on surveys often address different observed
variables.
For example, consider a mental health
professional using a survey to assess anxiety
symptoms. One survey question asks the
respondent to rate their stress level from one to
five. Because the respondent's stress levels may
show anxiety, and the survey provides a
quantifiable system to measure stress, it's an
observed variable.
Latent Variable
The latent variable, also known as the construct, is
the shared concept that different measurement
systems assess. Latent variables are difficult to
observe directly but can influence the outcome of
the observed variables in an experiment. For
example, the latent variable of anxiety may affect
the outcome of someone's reported stress levels.
Someone with anxiety may rate their stress at
level five, while someone without anxiety is likely
to choose a lower score. Although asking about
stress doesn't directly measure anxiety, it can still
provide researchers with insight into the
relationship between stress and anxiety.
CFA exists to assess the indirect relationship
between latent and observed variables.
Factor Loading
Factor loading is a number that describes how
closely an observed variable corresponds with a
latent variable. It is usually between zero and one,
although some data sets can produce factor
loadings higher than one when computing multiple
variables. Factor loadings with a higher value have
a stronger correlation with the latent variable.
For example, the data analysis on a survey for the
latent variable of anxiety produces a factor loading
score of 0.85 for question one and 0.33 for
question two. Because the factor loading in
question one is higher than in question two,
question one is likely better at identifying people
with anxiety than question two.
CFA VS EFA
Confirmatory factor analysis and exploratory
factor analysis are two complementary techniques
for reviewing research data. Exploratory factor
analysis identifies possible relationships between
variables, while confirmatory factor analysis tests
those relationships. Researchers with an extensive
background in a subject area often use
confirmatory factor analysis because they can
predict possible relationships in their data. They
use exploratory factor analysis to learn about new
patterns and identify innovative trends.
Steps of CFA :
1.Specify the latent variable :
Start by determining what concept you want to
analyze and establishing its theoretical definition.
Establishing a baseline to describe the latent
variable makes it possible for you to evaluate the
accuracy of the observed variables. You can define
the latent variable by listing characteristics or
collecting additional data. For example, if you
want to use CFA to determine if an intake survey is
a good assessment of self-esteem, start by
defining self-esteem. You may use your
professional knowledge to determine that self-
esteem involves traits like confidence, sociability,
adaptability and goals.
2.Determine measurement
methods :
Next, identify the measurement method you want
to test and which observed variables to include.
These variables are usually survey questions. You
may include multiple questions from the same
survey or choose questions from different surveys
depending on the type of analysis you want to
conduct.
Here are some example observed variables from a
survey assessing self-esteem:
Rank your confidence from one to five.
Agree or disagree: I’m uncomfortable accepting
complements from others.
Rate your adaptability from one to five.
3.Collect the data :
Gather the information you want to use in your
confirmatory factor analysis. Decide if you'll collect
initial responses from your own research or use
outside data from other sources. Try to secure a
large sample size of information to ensure an
accurate analysis. Once you have enough quality
information, input it into statistical modeling
software.
4.Establish consistent
parameters :
Using your statistical modeling software, establish
standardized parameters to evaluate the latent
and observed variables. Decide what
measurement system you want to use as the
standard and allow the software to convert all
other values to that measurement.

Example, if you want to use the rating system of

one to five, with one being "disagree" and five as
"agree," you first need to convert all other types of
questions to that format. For questions that ask
respondents to agree or disagree with a
statement, assign a value of one to "disagree" and
five to "agree." This allows the software to
Limitations of CFA
compute many data consistently.
Complexity: CFA requires a complex mathematical
model, which can be difficult to construct and
5.Compute the data : interpret, even for experienced researchers

Use your statistical modeling software to compute Assumptions: CFA assumes that the data collected
the factor loading for your data. Follow the is accurate and that the proposed model is correct.
prompts for your specific software interface to
produce your results. Most factor analysis
software puts this information in a table, although 1. One Factor Confirmatory Factor
some generate graphs and tables to express the Analysis
same information.

6.Interpretation :
Review the factor loading column of the factor
analysis table to determine how well each
observed variable relates to the latent variable.
Decide what factor loading value shows a
significant relationship, and use that to guide your
interpretation. For example, you may decide that
any variables with a factor loading of 0.75 are valid
for assessing self-esteem. If all the survey
questions have a factor loading over 0.75, you can
conclude that your survey is a good overall
measurement. 2. Two Factor Confirmatory Factor
Analysis
Advantage of CFA :
The main advantage of CFA lies in its ability to aid
researchers in bridging the often-observed gap
between theory and observation. For example, an
instrument might be developed by creating
multiple items for each of several specific
theoretical constructs.
 Review Matrix

S Author Year Organisation Key Factors of Factor Analysis

NO

1 M Tavakol 2020 Factor Analysis simplifies a matrix of correlations.

2 Syed Mohammad Ather 2009 Factor Analysis (FA) attempts to simplify complex and diverse
relationships that exist among a set of observed variables by
uncovering common.

3 Abbas 2019 Factor analysis (FA) is a multivariate technique that is used to

F.M. Alkarkhi, Wasin describe the relationships between different variables under
A.A. Alqaraghuli study (observable variables) with new variables called factors,
where the number of factors is less than the number of
original variables
4 Christof Schuster, Ke- 2005 The results from a factor analysis of a correlation matrix and
Hai Yuan the corresponding covariance matrix are not identical

5 Johnny R.J. Fontaine 2005 CFA offers a measurement model based on structural
equation modeling

6 S.E. Richards, E. Holmes 2015 Reviews different chemometrics methods for the analysis of
genomics, transcriptomics, proteomics, metabolomics,
and metagenomics datasets. It discusses a range of statistical
data integration techniques.

7 R. Wehrens 2009 FA is closely related to PCA. Rather than a mapping into

lower dimensions, or, equivalently, a rotation, that is PCA, FA
aims to fit an explicit model.

8 Michael C. Ashton 2013 Factor analysis allows the researcher to reduce many specific
traits into a few more general “factors” or groups of traits,
each of which includes several of the specific traits.

9 Nerea Martín- 2019 Factor analysis is a multivariable method that uses the
Calvo, Miguel observed data to define one or several vectors (one or several
Ángel Martínez- dietary patterns) grouping the food or food groups according
González to their degree of correlation.

10 Luisa Cutillo 2019 Factor Analysis (FA) is an independence technique, in which

there is no dependent variable

Objective of the study :

To find out different types of vector analysis in details.
Advantages of factor
analysis :
1. Both objective and subjective attributes can be
used.
2. It can be used to identify the hidden dimensions
or constraints which may or may not be apparent
from direct analysis.
3. It is not extremely difficult to do and at the
same time its inexpensive and gives accurate
results.
4. There is flexibility in naming and using
dimensions.[10]
Disadvantages
Of factor analysis :
1. The usefulness depends on the researcher’s
ability to develop a complete and accurate set of
product attributes. If important attributes are
missed the value of procedure is reduced
accordingly.
2. Naming of the factors can be difficult multiple
attributes can be highly correlated with no
apparent reasons.
3. If the observed variables are completely
unrelated the factor analysis is unable to produce
meaningful pattern.
4. It is not possible to know factors actually
represents, only theory can help inform the
researcher’s on this.
Conclusion :
In short, Factor Analysis brings in simplicity after
reducing variables. Factor Analysis, including
Principal Component Analysis, is also often used
along with segmentation studies. In this post, we
understood about the factor analysis method, and
the assumptions made before working on the
method.
1.PCA :
Principal component analysis is a technique to
summarize data, and is highly flexible depending
on your use case. It can be valuable in both
displaying and analyzing a large number of
possibly dependent variables. Techniques of
performing principal component analysis range
from arbitrarily selecting principal components, to
automatically finding them until a variance is
reached.
2.EFA :
EFA is a complex multivariate statistical method
involving many linear and sequential steps.
Exploratory factor analysis (EFA) is generally used
to discover the factor structure of a measure and
to examine its internal reliability. EFA is often
recommended when researchers have no
hypotheses about the nature of the underlying
factor structure of their measure.
3.CFA :
The main advantage of CFA lies in its ability to aid
researchers in bridging the often-observed gap
between theory and observation. For example, an
instrument might be developed by creating
multiple items for each of several specific
theoretical constructs
REFRENCES :
[1] Tabachnick, B.G. and Fidell, L.S., Using multivariate statistics (6th ed.), Pearson, 2013

[2] Verma, J. and Abdel-Salam, A., Testing statistical assumptions in research, John Willey & Sons
Inc., 2019.

[3] Ho, R., Handbook of univariate and multivariate data analysis and interpretation with SPSS,
Chapman & Hall/CRC, Boca Raton, 2006.

[4] Hair, J.F., Anderson, R.E., Tatham, R.L., and Black, W.C., Multivariate data analysis (5th ed.), N J:
Prentice-Hall, Upper Saddle River, 1998.

[5] Pituch, K. A. and Stevens, J., Applied multivariate statistics for the social sciences: Analyses
with SAS and IBM’s SPSS (6th ed.), Taylor & Francis, New York, 2016.

[6] Hair, J. J., Black, W.C., Babin, B. J., Anderson, R. R., Tatham, R. L., Multivariate data analysis,
Upper Saddle River, New Jersey, 2006.

[7] Pallant, J., SPSS survival manual: a step by step guide to data analysis using SPSS, Open
University Press/ Mc Graw-Hill, Maidenhead, 2010.

[8] Retrieved from wikipedia: https://en.wikipedia.org/wiki/Principal_component_analysis

[9] (2023, february 4). Retrieved from indeed: https://www.indeed.com/career-advice/career-

development/confirmatory-factor-analysis.

[10] https://www.du.ac.in/du/uploads/departments/Operational%20Research/24042020_Lect-
8%20Factor%20Analysis.pdf