Factorial Analysis of Variance

Aim : It allows the researcher to “reduce” this mass of numbers to a few representative factors that can
then be used for subsequent analysis.

Factor analysis is based on the assumption that all variables correlate to some degree. Consequently,
those variables that share similar underlying dimensions should be highly correlated, and those variables
that measure dissimilar dimensions should yield low correlations.

The primary role of factor analysis is to identify these clusters of high inter-correlations as independent
factors.

There are three basic steps to factor analysis: [C.E.R]

1. Computation of the correlation matrix for all variables

2. Extraction of initial factors

3. Rotation of the extracted factors to a terminal solution

12.1.1 Computation of the Correlation Matrix

As factor analysis is based on correlations between measured variables. a correlation matrix containing
the inter-correlation coefficients for the variables must be computed.

12.1.2 Extraction of Initial Factors

the researcher must decide on :

(1) the method of extraction, and
(2) the number of factors selected to represent the underlying structure of the data.

12.1.2.1 Method of Extraction

1. principal components analysis

2. common factor analysis.

The choice between these two basic methods of factor extraction lies with the objective of the
researcher.
If the purpose is no more than to “reduce data” in order to obtain the minimum number of factors
needed to represent the original set of data, then principal components analysis is appropriate.

When the main objective is to identify theoretically meaningful underlying dimensions, the common
factor analysis method is the appropriate model.

12.1.2.2 Determining the Number of Factors to Be Extracted

There are two conventional criteria for determining the number of initial unrotated factors to be
extracted:

1. Eigenvalues

2. Scree plot

Criticism of the above 2 methods:

The eigenvalues ≥ 1.00 rule typically overestimates, and sometimes underestimates, the number of
components extracted, and therefore does not always result in factors that are reliable

Other Methods:

1. parallel analysis and

2. Velicer’s minimum average partial (MAP) test.

12.1.2.2.1 Eigenvalues

Only factors with eigenvalues of 1 or greater are considered to be significant; all factors with
eigenvalues less than 1 are ignored.

An eigenvalue is a ratio between the common (shared) variance and the specific (unique)
variance explained by a specific factor extracted.

Common Variance ----> is explained by -----> Extracted Factor

Specific Variance ----> is explained by -----> Single Variable
12.1.2.2.3 Velicer’s Minimum Average Partial (MAP) Test

This test involves a complete principal component analysis (PCA) followed by the examination of
a series of matrices of partial correlations.

The test then calculates the average R-square (systematic variation) of the correlation matrix
(computed from the variables in the data set) after the factors (identified from principal
components analysis) have been partialled out of the matrix.

The number of factors corresponding to the smallest average R-square of the correlation matrix
then represents the number of factors to be retained.

The rationale for this is that the lower the R-square in the correlation matrix (the residual
variance), the stronger the factors extracted.

12.1.2.2.4 Parallel Analysis

This procedure involves extracting eigenvalues from random data sets that parallel the actual
data set with regard to the number of cases and variables

The eigenvalues derived from the actual data are then compared to the eigenvalues derived from
the random data sets. Factors are retained as long as the ith eigenvalue from the actual data is
greater than the ith eigenvalue from the random data.

12.1.3 Rotation of Extracted Factors

Factors produced in the initial extraction phase are often difficult to interpret.

This is because the procedure in this phase ignores the possibility that variables identified to load on or
represent factors may already have high loadings (correlations) with previous factors extracted. This
may result in significant cross-loadings in which many factors are correlated with many variables.
The rotation phase serves to “sharpen” the factors by identifying those variables that load on one factor
and not on another

12.1.4 Rotation Methods

There are two main classes of factor rotation method:

1. Orthogonal: assumes that the extracted factors are independent, and the rotation process
maintains the reference axes of the factors at 90°.

2. Oblique: allows for correlated factors instead of maintaining independence between the
rotated factors. The oblique rotation process does not require that the reference axes be
maintained at 90°.

Oblique rotation is more flexible because: It is more realistic to assume that influences in nature are
correlated.

12.1.5 Orthogonal (Varimax) versus Nonorthogonal (Oblique) Rotation

In choosing between orthogonal and oblique rotation, there is no compelling analytical reason to favor
one method over the other.

 If the goal of the research is no more than to “reduce the data” to more manageable proportions,
regardless of how meaningful the resulting factors may be, and if there is reason to assume that
the factors are uncorrelated, then Orthogonal rotation

 if the goal of the research is to discover theoretically meaningful factors, and if there are
theoretical reasons to assume that the factors will be correlated, then Oblique rotation

Sometimes the researcher may not know whether or not the extracted factors might be correlated. In such
a case, the researcher should try an oblique solution first.

If the correlations between the factors turn out to be very low (e.g., <0.20), the researcher could redo the
analysis with an orthogonal rotation method.

12.1.7 Interpreting Factors

In interpreting factors, the size of the factor loadings (correlation coefficients between the variables and
the factors they represent) will help in the interpretation.
 1. variables with large loadings indicate that they are representative of the factor, while small
loadings suggest that they are not.
2. factor loadings greater than ±0.33 are considered to meet the minimal level of practical
significance.
3. The reason for using the ±0.33 criterion is that, if the value is squared, the squared value
represents the amount of the variable’s total variance accounted for by the factor. Therefore, a
factor loading of 0.33 denotes that approximately 10% of the variable’s total variance is
accounted for by the factor.

12.3 Assumptions

12.3.1 Key Statistical Assumptions

 Normality and linearity—Departures from normality and linearity can diminish the observed
correlation between measured variables, and thus degrade the factor solution.
 Sufficient significant correlations in data matrix—The researcher must ensure that the data
matrix has sufficient correlations to justify

 If visual inspection reveals no substantial number of correlations of 0.33 or greater, then factor
analysis is probably inappropriate.

12.3.2 Key Conceptual Assumptions

Selection of variables—Variables should be selected to reflect the underlying dimensions that

are hypothesized to exist in the set of selected variables.

Homogeneity—The sample must be homogeneous with respect to the underlying factor

structure. If the sample consists of two or more distinct groups (e.g., males and females), separate
factor analysis should be performed.

Bartlett’s Test of Sphericity:

 can be used to test for the adequacy of the correlation matrix, that is, the correlation matrix has
significant correlations.
 Bartlett’s Test of Sphericity tests the hypothesis that the correlation matrix is an identity matrix,
that is, all the diagonal terms are 1 and all off-diagonal terms are 0 .
 If the test value is large and the significance level is small (<0.05), the hypothesis that the
variables are independent can be rejected. (i.e., they are interdependent)
 The test must be Significant (P value <0.05)

Kaiser-Meyer-Olkin (KMO):

 Measure of sampling adequacy.

  Used to examine the appropriateness of factor analysis.
 High values (between 0.5 and 1.0) indicate appropriateness. Values below 0.5 imply not.

Communalities:

 Amount of variance a variable share with all the other factors

 Preferable to be greater than 0.5

Component Matrix:

 Represents the unrotated component analysis factor matrix and presents the correlations that relate the
variables to the three factors using the conventional “eigenvalues ≥ 1.00 rule.” These coefficients,
called factor loadings, indicate how closely the variables are related to each factor.

 Because the factors are unrotated (the factors were extracted on the basis of the proportion of total
variance explained), significant crossloadings have occurred.

 These high cross-loadings make interpretation of the factors difficult and theoretically less meaningful.

SPSS:

Choose from menu [Analyze]  Dimension Reduction  Factor…

Steps:

1. Add all the desired variables from the (left) pan to the (Variables) pan
2. Choose {Descriptive} Select (KMO and Bartlett’s test) Press {Continue}
3. Choose {Extraction}  Choose (Principal Component)
4. Choose {Extract}- Choose {bases on Eigenvalue} Write (1) in Eigenvalue
greater than press {Continue}
5. Choose {Scores} Check (Save as variables)  press {Continue}
6. Choose {options} Check (Sorted by Size)  Check (Suppress small
coefficients) Write (0.3) in Absolute value below  Press {Continue}
7. Press {OK}

1. Correlation Matrix :Examination of the Correlation Matrix to check the most of the
variables are correlated above 0.33
2. Check the KMO & Bartlett’s Test of Sphericity: Must be < 0.5,Must be significant
3. Check the Communalities: shall be greater than 0.5
4. Check the Total Variance Explained table:

5. Check the Component Matrix

factors
F1 F2 F3
V L11 L12 Com
1
V L21 L22 Com
2
V Com
3
V
4
V
Variables

5
V
6
V
7
V
8
V
9
EV EV EV
  Before rotation

 After Rotation
The rotated factor structure is “clean” in that there is no cross-loading of items

Pattern Matrix and Structure Matrix Definition in SPSS

FACTOR output

When the rotation is orthogonal (i.e. the factors are uncorrelated; orthogonal and
uncorrelated are synonymous with centered variables), then the rotated factor matrix
represents both the loadings and the correlations between the variables and factors.

For oblique rotations, where the factors are allowed to correlate (oblimin or promax in
SPSS), then the loadings and correlations are distinct.
  The pattern matrix holds the loadings. Each row of the pattern matrix is
essentially a regression equation where the standardized observed
variable is expressed as a function of the factors. The loadings are the
regression coefficients.
 The structure matrix holds the correlations between the variables and the
factors.

Interpretation of a set of oblique factors involves both the pattern and structure
matrices, as well as the factor correlation matrix. The latter matrix contains the
correlations among all pairs of factors in the solution. It is automatically printed for an
oblique solution when the rotated factor matrix is printed.