Applied Psychometric Strategies Lab

Applied Quantitative and Psychometric

Series

Zijia Li, PhD

Caroline J. Gooden, PhD
Michael D. Toland, PhD
Measurement Invariance with
Categorical Indicators
December 6, 2016
 Today’s Agenda

• What is measurement invariance (MI) with

categorical items?
• Why is testing for MI important?
• What are the levels of MI?
• How is MI tested?
• Provide an applied example

2
 What Is MI?
A technique that helps determine if the
same unobserved variable is being
measured across multiple groups (or time)
within a confirmatory factor analysis (CFA)
framework

3
 Why is Testing for MI
Important?

When MI is evidenced, it assures that

• Group comparisons are meaningful

• We are measuring the same trait across

groups

• Group differences reflect true group

differences
4
 What If There Is No
Evidence for MI?
If MI assumptions aren’t tenable, then

• Scores from multiple groups don’t

represent construct equally well

• Observed group differences cannot be

assumed to be accurate

• Findings may be unsubstantiated or invalid

5
 Why Don’t Some Studies
Test for MI?

• Lack of sample size for each group being

compared to ensure results are
trustworthy

• Lack of knowledge about how to test for

MI

6
 What are Some Examples
Where MI Is Tested?

Compare different groups across the same

instrument
• Gender
• Race/Ethnicity
• Age
• Treatment condition
• Language
• Degree level
• Other

7
 What are the Levels of
MI?
Grimm et al. (2017)
1. Dimensional invariance

2. Configural invariance

3. Weak factorial invariance

4. Strong invariance

5. Strict factorial invariance

NOTE: Each level assumes evidence for MI at the previous level

8
 1. Dimensional Invariance

• Typically tested with exploratory factor

analysis (EFA)

 Same number of latent factor(s)

 Same item-factor structure
 Factor loadings equal across groups
 Item thresholds equal across groups
 Item residuals equal across groups
9
 Conceptual
Representation of
Dimensional Invariance

10
 Levels 2-5 of MI

Tested within a CFA framework:

2. Configural invariance

3. Weak factorial invariance

4. Strong invariance

5. Strict factorial invariance

11
 Levels 2-5 are tested via
a
CFA Framework
Degree of Item Factor
Item Threshold () Item Residuals (u)
Invariance Loading ()

Configural
Free Free Free
Invariance

Weak Factorial
Equal Free Free
Invariance

Strong Invariance Equal Equal Free

Strict Factorial
Equal Equal Equal
Invariance

12
 2. Configural Invariance

 Same number of latent factor(s) across

groups
 Same item-factor structure across
groups
 Factor loadings equal across groups
 Item thresholds equal across groups
 Item residuals equal across groups

13
Conceptual Representation of
Configural Invariance
Group 1 Group 2

ϕ1 ϕ2
η1 η2
1 1
λ41 τ11 λ42
τ21 τ12
λ31 τ31
τ41 λ12 λ22 λ32 τ22
λ11 λ21 τ32 τ42

y11 y21 y31 y41 y12 y22 y32 y42

u11 u21 u31 u41 u12 u22 u32 u42

14
 3. Weak Factorial
Invariance

 Same number of latent factor(s)

 Same item-factor structure
 Factor loadings equal across groups
 Item thresholds equal across groups
 Item residuals equal across groups

15
Conceptual Representation of
Weak Factorial Invariance
Group 1 Group 2

ϕ1 ϕ2
η1 η2
1 1
λ41 τ11 λ42
τ21 τ12
λ31 τ31
τ41 λ32 τ22
λ11 λ21 λ12 λ22 τ32 τ42

y11 y21 y31 y41 y12 y22 y32 y42

u11 u21 u31 u41 u12 u22 u32 u42

16
 4. Strong Invariance

 Same number of latent factor(s)

 Same item-factor structure
 Factor loadings equal across groups
 Item thresholds equal across groups
 Item residuals equal across groups

17
Conceptual Representation of
Strong Invariance
Group 1 Group 2

ϕ1 ϕ2
η1 η2
1 1
λ41 τ11 λ42 τ12
τ21
λ31 τ31 τ22
τ41 λ32
λ11 λ21 λ12 λ22 τ32 τ42

y11 y21 y31 y41 y12 y22 y32 y42

u11 u21 u31 u41 u12 u22 u32 u42

18
 5. Strict Factorial
Invariance

 Same number of latent factor(s)

 Same item-factor structure
 Factor loadings equal across groups
 Item thresholds equal across groups
 Item residuals equal across groups

19
Conceptual Representation of
Strong Invariance
Group 1 Group 2

ϕ1 ϕ2
η1 η2
1 1
λ41 τ11 λ42 τ12
τ21
λ31 τ31 τ22
τ41 λ32
λ11 λ21 λ12 λ22 τ32 τ42

y11 y21 y31 y41 y12 y22 y32 y42

u11 u21 u31 u41 u12 u22 u32 u42

20
 Implications
Degree of Invariance
Recommended Implications

Configural Separate group analyses; no group comparison or full group

Invariance
analyses at observed or latent level are recommended
Weak Factorial Variances and covariances can be compared at the latent level
Invariance
(via SEM)
Means, variances, and covariances can be compared at the
Strong Invariance
latent level (via SEM)
True group differences only source of difference in means,
Strict Factorial
Invariance thus observed and latent comparison can be made for
variances, covariances, and means

21
 Applied Example in Mplus

• 976 girls vs. 1,153 boys

• Child assessment measure of language

– 1 language factor
– 4 items
• 4 Likert categories per item

22
 Model comparison

• Chi-square DIFFTEST (ɑ = .05)

• Modification Indices (Liu, Millsap,
West, Tein, Tanaka, & Grimm, 2016)
• ΔCFI ≤ -.002 (Meade et al., 2008)
• ΔTLI = 0 (Marsh, Lüdtke, Muthén,
Asparouhov, Morin, Trautwein, &
Nagengast, 2010)
• ΔRMSEA ≥ .007 (Meade et al., 2008)

23
Mplus Syntax

24
Mplus Syntax: Configural
Model

25
 Mplus Output: Model Fit
Information for Configural
Model

26
Mplus Syntax: Weak
Factorial Invariance
Model

27
 Mplus Output: Model Fit
Information for Weak
Factorial Invariance Model

28
Mplus Output: Modification
Indices for Weak Factorial
Invariance Model (By
statement)

29
Mplus Syntax: Strong
Invariance Model

30
Mplus Output: Model Fit
Information for Strong
Invariance Model

31
 Mplus Output: Modification
Indices for Strong Invariance
Model ($ statement)

32
 Mplus Syntax: Strict
Factorial Invariance Model (1
of 2)

33
 Mplus Syntax: Strict
Factorial Invariance Model (2
of 2)

34
 Mplus Output: Model Fit
Information for Strict
Factorial Invariance Model (2
of 2)

35
 Mplus Output: Modification
Indices for Strong Invariance
Model (Residual terms)

36
 Conclusions

• Evidence for strict factorial

invariance for language

• Gender difference- only source of

difference found in the mean,
variance, and covariance for
language

37
 Helpful Resources
• Buhs, E. S., McGinley, M. & Toland, M. D. (2010). Overt and relational victimization in Latinos and European
Americans: Measurement equivalence across ethnicity, gender, and grade level in early adolescent groups. The
Journal of Early Adolescence, 30, 171-197. doi: 10.1177/0272431609350923
• Chen, F. F., & West, S. G. (2008). Measuring individualism and collectivism: The importance of considering
differential components, reference groups, and measurement invariance. Journal of Research in Personality, 42,
259-294.
• Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance.
Structural Equation Modeling, 9, 233-255.
• Marsh, H. W., & Grayson, D. (1995). Latent variable models of multitrait–multimethod data. In R. H. Hoyle (Ed.),
Structural equation modeling: Concepts, issues and application (pp. 177–198). Thousand Oaks, CA: Sage.
• Grimm, K. J., Ram, N., & Estabrook R. (2017). Growth modeling: Structural equation and multilevel modeling approaches. New
York, NY: The Guilford Press.
• Millsap, R. E. (2011), Statistical approaches to measurement invariance. NY, NY: Routledge.
• Millsap, R. E. (2010). Testing measurement invariance using item response theory in longitudinal data: An
introduction. Child Development Perspectives, 4: 5–9. doi:10.1111/j.1750-8606.2009.00109.x
• Sessions, J., Finney, S. J., & Kopp, J. P. (2016). Does the measurement or magnitude of academic entitlement
change over time? Measurement and Evaluation in Counseling and Development, 1-15.
• Toland, M. D., & Kupzyk, K. (2007, February). Approaches for evaluating measurement invariance. Presentation
for the Nebraska Center for Research on Children, Youth, Families, and Schools Research Methodology Series,
Lincoln, NE.
• van de Schoot, R., Lugtig, P., & Hox, J. (2012). A checklist for testing measurement invariance. European Journal
of Developmental Psychology, 9, 486-492. doi: 10.1080/17405629.2012.686740
• Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature:
Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3,
4-70. doi: 10.1177/109442810031002
• Widaman, K. F., & Reise, S. P. (1997). Exploring the measurement invariance of psychological instruments:38
Applications in the substance use domain. The Science of Prevention, 281-324.
39