Introduction
It is both a pleasure and honor to introduce the fifth edi- tural causal model (SCM), into the larger SEM fam-
tion of this book. Like the previous editions, structural
equation modeling (SEM) is presented in an acces-
sible way for readers without strong quantitative back-
grounds. Included in this edition are many new exam-
ples of SEM applications in disciplines that include
health, political science, international studies, cognitive
neuroscience, developmental psychology, sport and
exercise, and psychology, among others. Some exam-
ples were selected due to technical problems in the anal-
ysis, but such examples provide a context for discussing
how to deal with challenges that can and do occur in
SEM, especially in samples that are not large. So not all
applications of SEM described in this book are trouble
free, but neither are actual research problems.
WHAT’S NEW
The many changes in this edition are intended to
enhance the pedagogical presentation and cover recent
developments. The biggest changes are summarized
next:
1. The fourth edition of this book was one of the
first introductory works to incorporate Judea Pearl’s
nonparametric approach to SEM, also called the struc-
IntroKline5E.indd 1
ily that dates to the development of path analysis by
Sewall Wright in the 1920s–1930s and to the publica-
tion of LISREL III in 1976 as the first widely available
computer program for covariance structure analysis,
also called covariance-based SEM. In the same tradi-
tion, this fifth edition includes composite SEM, also
referred to as partial least squares path modeling or
variance-based SEM, as the third full member of the
SEM family. Composite SEM has developed from a
set of methods seen in the 1980s–1990s as more suit-
able for exploratory research that emphasized predic-
tion over explanation to a suite of full-fledged modeling
techniques for exploratory or confirmatory analyses,
including theory testing. Both the SCM and composite
SEM offer unique perspectives on causal modeling that
can benefit researchers more familiar with traditional,
covariance-based SEM. This means that researchers
acquainted with all three members of the SEM family
can test a wider range of hypotheses about measure-
ment and causation. I try to make good on this promise
throughout the fifth edition.
2. Traditional SEM and composite SEM are
described within Edward Rigdon’s concept proxy
framework that links data with theoretical concepts
through proxies, which approximate concepts based on
3/22/2023 4:50:11 PM
2 Introduction
correspondence rules—also called auxiliary theory—
about presumed causal directionality between concepts
and data. This point refers to the distinction between
reflective measurement, where proxies for latent vari-
ables are common factors, and formative measurement,
where proxies for emergent variables are composites of
observed variables. The choice between the two mea-
surement models just mentioned should be based on
theory, not by default due to the researcher’s lack of
awareness about SEM techniques for analyzing com-
posites.
3. There are additional new chapters on SEM analy-
ses in small samples and recent developments in media-
tion analysis. Surveyed works about mediation analysis
concern research designs and definitions of mediated
effects, including natural direct and indirect effects and
interventional direct and indirect effects estimated in
clinical trials, among other topics. There is also cov-
erage of new reporting standards for SEM studies by
the American Psychological Association (APA) and
the technique of piecewise SEM, which is based on
concepts from Pearl’s SCM. There are also extended
tutorials on modern techniques for dealing with miss-
ing data, including multiple imputation and full infor-
mation maximum likelihood (FIML), and also about
instrumental variable methods as a way to deal with
the confounding of target causal effects.
4. The topics of specification and identification ver-
sus analysis were described in separate chapters in the
fourth edition. They are now combined into individual
chapters for each technique described in the fifth edi-
tion. I believe this more closely integrated presentation
helps readers to more quickly and easily develop a sense
of mastery for a particular kind of SEM technique.
5. There is greater emphasis on freely available soft-
ware for SEM analyses in this new edition. For exam-
ple, the R package lavaan package was used in most
analyses described in this book. It is a full-featured
computer program for both basic and advanced SEM
analyses. It has the capability to analyze both common
factors and composites as proxies for theoretical con-
cepts. The syntax in lavaan is both straightforward
and used in some other R packages, including cSEM
for composite SEM, to specify structural equation
models, so it has application beyond lavaan.
Other R packages used for detailed examples in the
fifth edition include semTools, piecewiseSEM, MBESS,
MIIVsem, psych, WebPower, systemfit, sem, bmem,
CauseAndCorrelation, dagitty, and ggm. Together with
the lavaan package, a wide variety of analyses for non-
parametric, parametric, and composite models in SEM
is demonstrated, all with no-cost software. Commercial
software for SEM is still described, including Mplus,
which can feature state-of-the-art analyses before they
appear in other computer tools, but free SEM software
is now nearly as capable as commercial products. Also,
I would guess that free software could be used in the
large majority of published SEM studies.
6. Extended presentations on regression fundamen-
tals, significance testing, and measurement and psy-
chometrics beloved by readers of the fourth edition are
freely available in updated form as primers on the book’s
website. This change was necessary to include the new
material in the fifth edition. The topics just mentioned
are still covered in the new edition but in a more con-
cise way. New to the fifth edition in the main text is
a self-test of knowledge about background concepts in
statistics and measurement. There is a scoring key, too,
so readers can check their understanding of fundamen-
tals. Readers with higher scores could directly proceed
to substantive chapters on SEM analyses, and readers
with lower scores can consult any of the primers on the
website for more information and exercises.
BOOK WEBSITE
The address for the book’s website is https://www.
guilford.com/kline-materials. From the site, you can
freely access the computer files—data, syntax, and
output files—for all detailed examples in this book.
The website promotes a learning-by-doing approach.
The availability of both syntax and data files means
that readers can reproduce the analyses in this book
by using the corresponding R packages. Even without
doing so, readers can still open the output file on their
own computers for a particular analysis and view the
results. This is because all computer files are simple
text files that can be opened with any basic text edi-
tor, such as Notepad (Windows), Emacs (Linux/
UNIX), or TextEdit (macOS), among others. Syn-
tax files are annotated with extensive comments.
Even if readers use a different computer tool, such
as LISREL, it is still worthwhile to review the files
on the website generated in the R environment. This
is because it can be helpful to view the same analy-
sis from somewhat different perspectives. Some of the

exercises for this book involve extensions of the analy-
ses for these examples, so there are plenty of opportuni-
ties for practice with real data sets.
PEDAGOGICAL APPROACH
Something that has not changed in the fifth edition
is pedagogical style: I still speak to readers (through
my author’s voice) as one researcher to another, not as
statistician to the quantitatively naïve. For example, the
instructional language of statisticians is matrix algebra,
which conveys a lot of information in a short amount of
space, but readers must already be versed in linear alge-
bra to understand the message. There are other, more
advanced works about SEM that emphasize matrix
presentations (Bollen, 1989; Kaplan, 2009; Mulaik,
2009b), and these works can be consulted when you
are ready. Instead, fundamental concepts about SEM
are presented here in the language of applied research-
ers: words, tabular summaries, and data graphics, not
matrix equations. I will not shelter you from some of
the more technical aspects of SEM, but I aim to cover
fundamental concepts in accessible ways that promote
continued learning.
PRINCIPLES > SOFTWARE
You may be relieved to know that you are not at a dis-
advantage at present if you have no experience using
an SEM computer tool. This is because the coverage
of topics in this book is not based on the symbolism,
syntax, or user interface associated with a particular
software package. In contrast, there are many books
linked to specific SEM computer programs. They can
be invaluable for users of a particular program, but per-
haps less so for others. Instead, key principles of SEM
that users of any computer tool must understand are
emphasized here. In this way, this book is more like a
guide to writing style than a handbook about how to use
a particular word processor. Besides, becoming profi-
cient with a particular software package is just a matter
of practice. But without strong conceptual knowledge,
the output from a computer tool for statistical analy-
ses—including SEM—may be meaningless or, even
worse, misleading.
IntroKline5E.indd 3
Introduction 3
SYMBOLS AND NOTATION
Advanced works on SEM often rely on the symbols
and notation associated with the original matrix-based
syntax for LISREL, which features a profusion of dou-
bly subscripted lowercase Greek letters for individual
model parameters, uppercase Greek letters for matrices
of parameters for the whole model, and two-letter acro-
nyms in syntax for matrices. For example, the symbols
( x)
λ12 , Λ x, and LX
refer in LISREL notation to, respectively, a specific
loading on an exogenous (explanatory) factor, the
parameter matrix of loadings for all such factors, and
LISREL syntax that designates the matrix (Lambda-
X). Although I use here and there some symbols from
LISREL notation, I do not oblige readers to memorize
LISREL notation to get something out of the book.
This is appropriate because LISREL symbolism can be
confusing unless one has learned the whole system by
rote.
ENJOY THE RIDE
Learning a new set of statistical techniques is not every-
one’s idea of fun. (If doing so is fun for you, that’s okay,
I understand and agree.) But I hope the combination
of accessible language that respects your intelligence,
examples of SEM analyses in various disciplines, free
access to background tutorials (i.e., the primers) and
computer files for detailed examples, and the occa-
sional bit of sage advice offered in this book will help to
make the experience a little easier, perhaps even enjoy-
able. It might also help to think of this book as a kind of
travel guide about language and customs, what to know
and pitfalls to avoid, and what lies just over the horizon
in SEM land.
PLAN OF THE BOOK
Part I introduces fundamental concepts, reporting
standards, preparation of the data, and computer tools.
Chapter 1 lays out both the promise of SEM and wide-
spread problems in its application. Concepts in regres-
sion, significance testing, and psychometrics that are
especially relevant for SEM are reviewed in Chapter
2, which also include the self-test in these areas. Basic
3/22/2023 4:50:11 PM
 4 Introduction
steps in SEM and reporting standards are introduced
in Chapter 3 along with an example from a recent
empirical study. How to prepare the data for analysis in
SEM and options for dealing with common problems,
including missing data, are covered in Chapter 4, and
computer tools for SEM, both commercial and free, are
described in Chapter 5.
Part II deals with the fundamentals of hypothesis
testing in SEM for classical path models, which in the
analysis phase feature a single observed measure for
each theoretical variable, also called single-indicator
measurement. It begins in Chapter 6, which introduces
nonparametric SEM as described by Judea Pearl (i.e.,
the SCM). The SCM is graphical in nature; specifically,
causal hypotheses are represented as directed graphs
where theoretical variables are depicted with no com-
mitment to any distributional assumptions or specific
operational definitions for any variable. Graphs in non-
parametric SEM can be analyzed by special computer
tools without data. This capability allows researchers
to test their ideas before collecting the data. For exam-
ple, the analysis of a directed graph may indicate that
a particular causal effect cannot be estimated unless
additional variables are measured. After the data are
collected, it is a parametric model that is typically
analyzed, and such models and their assumptions are
described in Chapter 7. The technique of piecewise
SEM, which connects the two perspectives, nonpara-
metric and parametric, through novel techniques for
analyzing path models, is covered in Chapter 8.
Chapters 9–12 are perhaps the most important ones
in the book. They concern how to test hypotheses and
evaluate models in complete and transparent ways that
respect both reporting standards for SEM and best
practices. These presentations are intended as counter-
examples to widespread dubious practices that plague
many, if not most, published SEM studies. That is, the
state of SEM practice is generally poor, and one of my
goals is to help readers distinguish their work above
this din of mediocrity. Accordingly, Chapter 9 outlines
methods for simultaneous estimation of parameters in
structural equations models and explains how to ana-
lyze means along with covariances. Chapter 10 deals
with the critical issue of how to properly assess model
fit after estimates of its parameters are in hand. A criti-
cal point is that model fit should be routinely adjudged
from at least two perspectives: global or overall fit, and
local fit at the level of residuals, which in SEM con-
cerns differences between sample and predicted asso-
IntroKline5E.indd 4
ciations for each pair of measured variables. Chapters
11–12 extend these ideas to, respectively, the compari-
son of alternative models all fit to the same data and
the simultaneous analysis of a model over data from
multiple groups, also called multiple-group SEM.
Part III deals with the analysis of models where at
least some theoretical concepts are approximated with
multiple observed variables, or multiple-indicator mea-
surement. Such models are often referred to as “latent
variable models,” but for reasons explained in Chapter
13, our models include only proxies for latent variables,
not latent variables themselves. These proxies are of
two general types: common factors based on reflective
measurement models and composites based on forma-
tive measurement models. The analysis of pure reflec-
tive measurement models in the technique of confirma-
tory factor analysis (CFA) is described in Chapter 14,
and Chapter 15 deals with the analysis of structural
regression (SR) models—also called latent variable
path models—where causal effects between observed
variables or common factors are estimated. Chapter 16
is about composite SEM, which analyzes causal models
with multiple-indicator measurement based on forma-
tive, not reflective, measurement and where proxies for
conceptual variables are composites, not common fac-
tors. Application of the technique of confirmatory com-
posite analysis (CCA), the composite analog to CFA, is
demonstrated.
Part IV is about advanced techniques. How to deal
with SEM analyses in small samples is addressed in
Chapter 17, and Chapter 18 concerns the analysis of
categorical data in CFA. Chapter 19 explains how to
analyze nonrecursive models with causal loops that
involve two or more endogenous (outcome) variables
assumed to influence each other, and Chapter 20 sur-
veys recent developments that enhance, improve, and
extend ways to assess hypotheses of causal media-
tion, or indirect causal effects that involve at least one
intervening variable. The state of mediation analysis
in the literature is problematic, but some of the newer
approaches and methods described in this chapter seem
promising. The analysis of latent growth models for
longitudinal data is the subject of Chapter 21, and the
application of multiple-group CFA to test hypotheses
of measurement invariance is dealt with in Chapter
22. The capstone of the book is the summary of best
practices in SEM in Chapter 23. Also mentioned in this
chapter are common mistakes with the aim of helping
you to avoid them.
3/22/2023 4:50:11 PM
 Part I

Concepts, Standards, and Tools

Pt1Kline5E.indd 5 3/22/2023 3:52:35 PM

Pt1Kline5E.indd 6 3/22/2023 3:52:35 PM
 1

Promise and Problems

This book is your guide to the principles, practices, strengths, limitations, and applications of structural equa-
tion modeling (SEM) for researchers and students without extensive quantitative backgrounds. Accordingly,
the presentation is conceptually rather than mathematically oriented, the use of formulas and symbols is kept
to a minimum, and many examples are offered of the application of SEM to research problems in disciplines
that include psychology, education, health sciences, cognitive assessment, and political science, among oth-
ers. When you finish reading this book, I hope that you will have acquired the skills to begin to use SEM in
your own research in an informed, principled way. Here is my four-point plan to get you there:

1. Review fundamental concepts in regression, hypothesis testing, and measurement beginning in the
next chapter. A self-test of knowledge in each area just mentioned is provided with a scoring key.
Additional resources for review of background concepts are freely available on this book’s website.
2. Convey what should be communicated in complete, transparent, and verifiable reporting of the
results in SEM studies. Formal reporting standards for SEM by the American Psychological Associa-
tion (APA) are both explained and modeled by example.
3. Describe all three major members of the SEM family. Each offers unique perspectives that can help
you in every stage of an SEM study, from planning through operationalization to data collection and
then finally to the analysis of a model that faithfully represents your specific hypotheses.
4. Emphasize best analysis practices and provide warnings about questionable practices that are seen
in too many published SEM studies.

To summarize, the main goal is to help you produce research that is distinguished by its accuracy, complete-
ness, and quality by following best practices, that is, SEM done right.

PREPARING TO LEARN SEM Know Your Area

Listed next are suggestions for the best ways to get
ready to learn about SEM. I offer these suggestions
in the spirit of giving you a healthy perspective at the
beginning of our task, one that empowers your sense of
being a researcher.
Strong familiarity with the theoretical and empiri-
cal literature in your research area is the single most
important thing you could bring to SEM. This is
because everything—from the specification of your
initial model to modification of that model in subse-

Pt1Kline5E.indd 7 3/22/2023 3:52:35 PM

 8 Concepts, Standards, and Tools
quent reanalyses to interpretation of the results—must
be guided by your domain knowledge. So, you need,
first and foremost, to be a researcher, not a statistician
or a computer nerd. This is true for most kinds of statis-
tical analysis in that the value of the product (numeri-
cal results) depends on the quality of the ideas (your
hypotheses) on which the analysis is based.
Know Your Measures
Kühnel (2001) reminded us that learning about SEM has
the by-product that researchers must address fundamen-
tal issues of measurement. Specifically, analyzing mea-
sures with strong psychometric characteristics, such as
good score reliability and positive evidence for validity, is
essential in SEM. For example, it is impossible to approx-
imate hypothetical constructs without thinking about
how to operationalize and measure those constructs.
When you have just a single measure of a construct, it is
critical for this single indicator to have good psychomet-
ric properties. Similarly, the analysis of measures with
deficient psychometrics could bias the results.
Eyes on the Prize
The point of SEM is to test a theory by specifying
a model that represents predictions of that theory
among plausible constructs measured with appropri-
ate observed variables (Hayduk et al., 2007). If such a
model does not ultimately fit the data, this outcome is
interesting because there is value in reporting models
that challenge or discredit theories. Beginners some-
times mistakenly believe that the point of SEM is to
find a model that fits the data, but this, by itself, is not
impressive. This is because any model, even one that
is grossly wrong (misspecified), can be made to fit the
data by making it more complicated (adding parame-
ters). In fact, if a structural equation model is specified
to be as complex as possible, it will perfectly represent
the data, not only in a particular sample, but also in any
other sample using the same variables.
There is also real scientific value in learning about
just how and why things went wrong with a model
faithfully based on a particular theory. Schreiber (2017)
put it like this:
One component about SEM analyses that is rarely talked
about is when no model is retained. This is an overlooked
area because researchers fear a lack of being able to publish
it. A well written and argued “no model retained” manu-
Pt1Kline5E.indd 8
script is a joy to read. I have reviewed many in my career
and have always argued for publication. If the models and
data are not working well, that is worthy of announcing it
to the world as students and I have done. (p. 642)
This quote is from a dissertation defense where some
committee members were critical of an SEM analysis
where no model was retained. Asked by the committee
chair for a response, Schreiber (personal communica-
tion, December 1, 2020) replied that retaining no model
is a beautiful thing when it is apparent that predictions
based on theory did not work out in practice. In con-
trast, whether or not a scientifically trivial model fits
the data is irrelevant (Millsap, 2007).
Use the Best Research Computer
in the World . . .
Which is the human brain; specifically—yours. At the
end of the analysis in SEM—or any other type of sta-
tistical analysis—it is you as the researcher who must
evaluate the degree of support for the hypotheses,
explain any unexpected findings, relate the results to
those from prior studies, and consider the implications
of the findings for future research. Without your content
expertise, a statistician or computer nerd could help you
to select data analysis tools or write program syntax,
but could not help with the other things just mentioned.
As aptly noted by Pedhazur and Schmelkin (1991), “no
amount of proficiency will do you any good, if you do
not think” (p. 2).
Get a Computer Tool for SEM
Obviously, you need a computer program to conduct
the analysis. In SEM, there are now many choices of
computer tools, many for no cost. Examples of free
computer software include various SEM packages
such as lavaan, semTools, cSEM, and OpenMx for R,
which is an open-source language and environment for
statistical computing and graphics. Other options are
Wnyx, a graphical environment for creating and test-
ing structural equation models, and JASP, an open-
source computer program with a graphical user inter-
face (GUI) and capabilities for both frequentist and
Bayesian statistical procedures. It also has modules for
SEM, including mediation analysis and latent growth
modeling. Commercial options for SEM include Amos,
Adanco, EQS, LISREL, Mplus, and SmartPLS, among
others.
3/22/2023 3:52:35 PM

In the open science spirit of free access to research
tools and resources, greater emphasis in this book is
placed on free computer programs for SEM than on
commercial options. Specifically, the lavaan package
for R is used in most of the detailed analysis examples.
It has both basic and advanced options for a wide range
of SEM analyses, and its capabilities rival those of
commercial software. Several other R packages that
supplement or extend lavaan analyses, such as sem-
Tools, are also used in examples. Syntax, data, and
output files can be freely downloaded from the website
for this book—see the Introduction—which is also an
exemplar of open access.
Join the Community
An electronic mail network called SEMNET is dedi-
cated to SEM.1 It serves as an open forum for discus-
sion and the whole range of issues associated with
SEM. It also provides a place to ask questions about
analyses or more general issues, including philosophi-
cal ones (e.g., the nature of causality or causal infer-
ence). Subscribers to SEMNET come from various
disciplines, and they range from novices to seasoned
veterans, including authors of many works cited in this
book. Sometimes the discussion gets lively (sparks can
fly), but so it goes in scientific discourse. An archive
of prior discussions on SEMNET can be searched for
particular topics. A special interest group for SEM is
available for members of the American Educational
Research Association (AERA).2 There is even a theme
song for SEM, the hilarious Ballad of the Casual Mod-
eler, by David Rogosa (1988).3 You can blame me if the
song gets stuck in your head.
DEFINITION OF SEM
The term structural equation modeling (SEM) refers
to a set of statistical techniques for estimating the mag-
nitudes and directions of presumed causal effects in
quantitative studies based on cross-sectional, longitu-
dinal, experimental, or other kinds of research designs.
Its application can range from more exploratory and
1 https://listserv.ua.edu/cgi-bin/wa?A0=semnet
2 https://www.aera.net/SIG118/Structural-Equation-Modeling-
SIG-118
3 https://web.stanford.edu/class/ed260/ballad.mp3
Pt1Kline5E.indd 9
Promise and Problems 9
data-driven research, where preliminary causal models
are generated, to more confirmatory research, where
one or more extant models based on a priori hypoth-
eses are tested or compared. While the data in SEM
come from measured variables, they can be treated in
the analysis as approximating hypothetical constructs.
Thus, by analyzing manifest variables as indicators for
target constructs, it is also possible to estimate causal
relations among those constructs.
Pearl (2012, 2023) defined SEM as a causal infer-
ence method that takes three inputs (I) and generates
three outputs (O). The inputs are
I-1. A set of causal hypotheses based on theory or
results of empirical studies that are represented
in the structural equation model. The hypotheses
are typically based on assumptions, only some of
which can actually be tested in the data.
I-2. A set of queries or questions about causal rela-
tions among variables of interest such as, what is
the magnitude of the direct causal effect of X on
Y (represented as X → Y), controlling for other
presumed causes of Y? All queries follow from
model specification.
I-3. Most applications of SEM are in observational
studies, or nonexperimental designs, but data
from experimental or quasi-experimental studies
can be analyzed, too—see Breitsohl (2019) for
examples.
The outputs of SEM are
O-1. Quantitative estimates of model parameters for
hypothesized effects including, for example,
X → Y, given the data.
O-2. A set of logical implications of the model that
may not directly correspond to a specific param-
eter but can still be tested in the data. For exam-
ple, a model may imply that variables W and Y
are unrelated after controlling for certain other
variables in the model.
O-3. The degree to which the testable implications of
the model are supported by the data.
BASIC DATA ANALYZED IN SEM
When unstandardized variables are analyzed in SEM,
the basic datum for continuous variables is the covari-
3/22/2023 3:52:35 PM
 10 Concepts, Standards, and Tools
ance, which is defined for observed variables X and Y
as follows:
covXY = rXY SDX SDY (1.1)
where rXY is the Pearson correlation and SDX and SDY
are their standard deviations.4 A covariance estimates
the strength of the linear relation between X and Y in
their original (raw score) units, albeit with a single
number. Because the covariance is an unstandardized
statistic, its value has no fixed lower or upper bound.
For example, covariances of, say, –1,025.45 or 19.77 are
possible, given the scale of the original scores. The sta-
tistic covXY conveys more information than rXY, which
says something about association but in a standardized
metric only. There are times in SEM when it is appro-
priate to analyze standardized rather than unstandard-
ized variables. If so, then rXY is the basic datum for
variables X and Y. Reasons to analyze unstandardized
versus standardized variables in SEM are described at
various points in the book.
Some researchers, especially those who use ANOVA
(analysis of variance) as their main analytical tool, have
the impression that SEM is concerned only with covari-
ances or correlations. This view is too narrow because
means can be analyzed in SEM, too. But what really
distinguishes SEM is that means of latent variables can
also be estimated. In contrast, ANOVA is concerned
with means of observed variables only. It is also pos-
sible to estimate effects in SEM traditionally associated
with ANOVA, including between-group and within-
group (e.g., repeated measures or longitudinal data)
mean contrasts. For example, in SEM one can estimate
the magnitude of group mean differences on latent vari-
ables, which is not feasible in standard ANOVA. Means
are not analyzed in probably most published SEM stud-
ies, but the option to do so provides additional flex-
ibility. Several examples of the analysis of means are
described later in the book.
Just as in regression analysis, it is possible in SEM to
estimate curvilinear relations for continuous variables
or analyze noncontinuous variables, including nominal
or ordered-categorical (ordinal) variables, among other
variations, as either presumed causes or outcomes.
Interactive effects can also be analyzed in SEM. An
outdated view is that only linear effects of continuous
variables that do not interact can be estimated (Bollen
& Pearl, 2013), but the reality is that SEM is extraordi-
4 Thecovariance of a variable with itself is just its variance, such
as cov XX = s X2 .
Pt1Kline5E.indd 10
narily flexible in terms of effects and types of variables
that can be analyzed as presumed causes or outcomes.
There are other differences between SEM and regres-
sion techniques: Associations for observed variables
only are estimated in standard regression analysis, but
relations for latent variables can also be estimated in
SEM. In regression, the roles of predictor and criterion
are theoretically interchangeable. For instance, there is
no special problem in bivariate regression with specify-
ing X as a predictor of Y in one analysis (Y is regressed
on X) and then in a second analysis with regressing X
on Y. There is no such ambiguity in SEM, where the
specification that X affects Y is a causal link that reflects
theory and also depends on other assumptions in the
analysis. Thus, both the semantics and interpretation of
the results in regression versus SEM are distinct (Bol-
len & Pearl, 2013). Yes, there are situations where stan-
dard regression techniques can be used to estimate pre-
sumed causal effects, but the context for SEM is causal
modeling, not mere prediction.
FAMILY MATTERS
The method of SEM consists of three distinct families of
techniques or approaches to causal inference. All origi-
nated in the pioneering work by the geneticist Sewall
Wright. His method of path coefficients (Wright,
1934)—or path analysis as it is now called—featured
the estimation of causal effects based on hypotheses
represented in a statistical model. Wright’s path mod-
els included both observed and latent variables, and in
graphical form they closely resemble today’s model
diagrams in SEM (e.g., Pearl, 2009, p. 415). Presumed
causal effects were estimated in sample data, and pre-
dictions based on the model were compared with pat-
terns of observed associations in samples. In hindsight,
Wright’s innovations were remarkable, and his work
continues to have influence to this day.
Although all three SEM families date to key innova-
tions in the 1970s–1980s, each was further developed
in relatively distinct sets of research disciplines or
areas. Consequently, other than a handful of scholars
knowledgeable of the histories and intricacies in at least
two different SEM families, many researchers trained
in a particular SEM family were only partially aware
of other possibilities or approaches to causal modeling.
But things are changing, in part due to the increasing
influence of multidisciplinary research that includes
several disciplines in an integrated way under the same
3/22/2023 3:52:35 PM

subject or area of study. The promise is that different
viewpoints or methods over disciplines will comple-
ment each other and provide a more comprehensive
understanding of the problem or novel approaches to
dealing with it (Choudhary, 2015).
The three SEM families are listed next and described
afterward:
1. Researchers in psychology and related disci-
plines are probably most familiar with the SEM fam-
ily called covariance structure analysis, covariance
structure modeling, or covariance-based SEM. All
techniques of this type estimate parameters of causal
models made up of observed variables or proxies for
latent variables by minimizing the difference between
the sample covariance matrix and the predicted covari-
ance for the same measured variables, given the model,
which represents hypotheses about how and why model
variables should be related (covary). Latent variables
are approximated with common factors of the type ana-
lyzed in classical factor analysis techniques that date
to the beginning of the 1900s (e.g., Spearman, 1904).
The technique of confirmatory factor analysis (CFA) is
a member of this family. Although this choice reflects
my own background in psychology, I will use the sim-
pler term “traditional SEM” to refer to the covariance-
based SEM family.5
2. The second member of the SEM family is better
known among researchers in disciplines such as market-
ing, organizational research, business research, or infor-
mation systems, among others. It is called variance-
based SEM, composite SEM, or partial least squares
path modeling (PLS-PM). The term “PLS-PM” also
refers to an estimation algorithm based on regression
techniques that underlie many, but all not, applications
of composite SEM. It analyzes composites, or weighted
combinations of observed variables, to approximate
hypothetical constructs. The term “variance-based”
signals that the goal is not strictly to explain the sample
covariance matrix, although this goal can also be pur-
sued in a composite analysis. Instead, these techniques
analyze total variation among observed variables when
estimating causal effects, given the model. The classi-
cal general linear model (GLM) of multivariate statis-
tics that includes multiple regression, MANOVA (i.e.,
5 Yes,what is considered as tradition by a person with a particu-
lar background may be seen as novelty to another person from a
different background. That’s life and multidisciplinary research.
Pt1Kline5E.indd 11
Promise and Problems 11
multivariate ANOVA), and canonical variate analysis
(canonical correlation), among others, analyzes com-
posites, too, but the focus of composite SEM is aimed
more at causal modeling. The technique of confirma-
tory composite analysis (CCA), a member of this fam-
ily, is the composite-based analogue to CFA. The name
“composite SEM” is used from this point forward.
3. The third family member more familiar to
researchers in epidemiology, computer science, and
medicine is the structural causal model (SCM) or
nonparametric SEM, which originated in Judea
Pearl’s work in the 1970s–1980s on Bayesian prob-
ability networks and later extended to the more gen-
eral problem of causal inference (Pearl, 2009). In the
SCM, causal hypotheses are represented in a directed
acyclic graph (DAG) when unidirectional causation is
assumed or in a directed cyclic graph (DCG) when
certain variables are hypothesized to affect each other,
or reciprocal causation. The method is nonparametric
because the specification of a causal graph requires no
commitment to any particular operational definition,
distributional assumption, or specific functional form
of statistical association, such as linear versus curvilin-
ear, for any pair of variables. Unlike model diagrams in
the other two SEM families, which are basically static
entities that require data to be analyzed, there are spe-
cial methods and computer programs for analyzing a
causal graph with no data. This capability permits the
researcher to analyze alternative causal models in the
planning stage of a study. The results of such analyses
can inform the researcher about how to select covari-
ates that control possible confounding of target causal
effects, among other possible ways to deal with the
problem. This approach has also motivated the devel-
opment of novel approaches to mediation analysis that
are described later in the book. From now onward, this
family is called “nonparametric SEM.”
Traditional SEM
This SEM family is a synthesis of two frameworks:
the path analytic method by Wright, based on regres-
sion techniques to estimate causal effects; and the
factor analytic approach to estimate latent variables
from observed or manifest variables. It dates to (1)
the introduction of path analysis to the social sciences
in the 1950s–1960s by Blalock (1961) and others and
(2) the subsequent integration of regression techniques
and factor-analytic methods into a unified framework
3/22/2023 3:52:35 PM
 12 Concepts, Standards, and Tools
in the 1970s–1980s called the JWK model (Bentler,
1980). The acronym refers to the work of three authors:
K. G. Jöreskog, J. W. Keesling, and D. Wiley. The first
widely available computer program to fit casual mod-
els to sample covariance matrices was LISREL III by
Jöreskog and Sörbom (1976), which was the progenitor
for later versions of LISREL and other computer pro-
grams that include Amos, EQS, lavaan, and Mplus,
among others.
This family of traditional SEM techniques is prob-
ably the most widely used. It offers the potential
benefits summarized next (Bagozzi & Yi, 2012): By
integrating regression techniques with factor analytic
methods, it is possible to estimate causal effects for
any c­ ombination of observed or latent variables speci-
fied as ­ presumed causes or outcomes. The explicit
distinction between observed and latent variables can
take direct account of measurement error, or score
unreliability. This characteristic lends a more realistic
sense to the analysis in that researchers in the behav-
ioral sciences often analyze scores that are subject to
measurement error. As mentioned, traditional SEM
supports a range of applications from exploratory to
more confirmatory, depending on the researcher’s
hypotheses and aims. There is a rich variety of mod-
els that can be analyzed for longitudinal data, more so
than in composite SEM.
There are also limitations of traditional SEM: It
requires large samples, which makes it challenging to
apply the method in research areas where it is difficult to
collect large numbers of cases, such as in studies of rare
disorders. Exactly what is meant by “large samples” is
addressed in a later section of this chapter. There are
certain types of hypotheses that are difficult to test in
this SEM family. An example is when hypothetical con-
structs are defined in ways that contradict the estimation
of latent variables with common factors as one would do
in the technique of CFA. In such cases, the composite
SEM family—which uses composites to approximate
concepts, not common factors—may serve as a better
alternative for reasons explained in Chapters 3 and 16.
Other limitations are due to the misuse of traditional
SEM, which is unfortunately widespread. These prob-
lems are not due to the method per se, but I would guess
that many, if not most, published SEM studies have one
flaw so severe that the results may have little or no inter-
pretative significance. This critical issue is also elabo-
rated later in the chapter. See Bollen, Fisher, Lilly, et al.
(2022), who described the 50-year history of traditional
SEM since 1972, reviewing strengths and vulnerabili-
ties, and outlining future directions.
Pt1Kline5E.indd 12
Traditional SEM itself is part of an extended fam-
ily of methods for analyzing latent variable models
that is briefly outlined next; see the sources cited for
more information. For example, latent variables in
­traditional SEM are assumed to be continuous. There
are other techniques for analyzing models with cat-
egorical latent variables. The levels of a categorical
latent variable are called classes, and they represent
a mixture of subpopulations where membership is not
known but is inferred from the data. Thus, a goal of
the analysis is to estimate the nature and number of
latent classes. The technique of latent class analy-
sis can be viewed as a kind of factor analysis but one
where classes are approximated from observed vari-
ables that could be either categorical or continuous.
Lanza and Rhoades (2013) described applications of
latent class analysis to identify subgroups in treatment
outcome studies.
Muthén (2001) described the analysis of mix-
ture models—also called mixture modeling—with
latent variables that may be continuous or categorical.
When both are present in the same model, the analy-
sis is basically traditional SEM but conducted across
inferred subpopulations. The work just cited is part of
the larger ongoing effort to express all latent variable
models within a common mathematical framework
(Bartholomew, 2002). The Mplus computer program is
especially adept at analyzing a variety of latent variable
models. This is because it can analyze all basic kinds of
traditional SEM models and mixture models, too. Both
kinds of analyses just mentioned can also be combined
in Mplus with multilevel modeling—also called linear
mixed modeling or hierarchical linear modeling—
for analyzing data with repeated measurements or that
are organized in hierarchical levels, such as children
with families, where scores within the same level are
probably not independent (Nezlek, 2008). Computer
programs like Mplus blur the distinction between tradi-
tional SEM and techniques such as latent class analysis,
mixture modeling, and multilevel modeling.
Composite SEM
Composite SEM was developed in the 1970s–1980s by
Herman O. A. Wold (1982). It analyzes composites as
proxies for hypothetical variables, not common fac-
tors. Statistical methods for composites are generally
simpler than methods for common factors, and there
are fewer distributional assumptions and other require-
ments for composite methods. This approach was once
described as a “soft modeling” alternative to “heavy-
3/22/2023 3:52:35 PM

weight” traditional SEM with its complex estimation
algorithms, potential for technical problems to scuttle
the analysis, and the need for large samples. In its early
years, the emphasis in composite SEM was on predic-
tion of target constructs by other variables, including
covariates or other composites for different theoretical
variables. Because estimators were based on standard
regression techniques, these methods generally maxi-
mized R2, or the proportion of explained variance, for
outcome variables. In contrast, traditional SEM tech-
niques do not necessarily maximize R2 for individual
outcomes. Thus, composite SEM at the time was gen-
erally thought of as a prediction method versus tradi-
tional SEM, which was seen as emphasizing explana-
tion through maximizing the similarity of sample and
model-implied data matrices.
In addition to the distinction of prediction versus
explanation, authors of works about composite SEM
published roughly in the 1990s–2010s (e.g., Hair et
al., 2012) highlighted these relative advantages: For
the same number of observed variables in the model,
composite SEM generally required smaller sample
sizes than traditional SEM, and the power of signifi-
cance tests was described as generally higher in com-
posite SEM for the same sample size. Its relative lack
of distributional assumptions was rightly touted as an
advantage, as was its more exploratory nature com-
pared with traditional SEM. Drawbacks included its
lack of a direct way to control for measurement error
or test the overall fit of the model to the data, which are
standard practices in traditional SEM. The inability to
assess overall fit in composite SEM limited its role in
testing theories; specifically, it offered no direct way to
determine how well the model as a whole explained the
data. At the time, composite SEM could be described
as “SEM-lite” for testing hypotheses about latent vari-
ables compared with its older brother, traditional SEM.
Oh how times change. Technical and conceptual
advances in composite SEM since about 2010 have
been a turning point, and this is no hype. (I am not big
on hype; keep it real, please.) Just a few key develop-
ments are summarized next; see Henseler (2021) and
Chapter 16 in this book for more detailed accounts: It is
now possible to apply composite SEM across the whole
range of studies from more exploratory to more con-
firmatory. One reason is that new methods allow the
researcher to test the fit of the whole model to the data,
just as in traditional SEM. The same basic stages of
the analysis, from specification of the model through
its analysis and possible modification based on the data,
Pt1Kline5E.indd 13
Promise and Problems 13
can now be followed in both composite SEM and tradi-
tional SEM. Very recent developments (e.g., Schuberth,
2021) extend the capabilities of standard SEM com-
puter programs like lavaan, Mplus, LISREL, and oth-
ers to analyze composite models, which brings to the
analysis all the advantages formerly associated with
traditional SEM. Composite SEM can also test kinds
of hypotheses about measurement that are difficult to
evaluate in traditional SEM, and in such cases com-
posite SEM is the preferred technique. Relatively new
estimators can directly control for measurement error
in composites. All these developments explain why I
think it is important for you to know about this member
of the SEM family, too.
Nonparametric SEM
Pearl (2009) described his graph-theoretic approach,
the SCM, as unifying two frameworks to causal infer-
ence, traditional SEM and the potential outcomes
model (POM), also called the Neyman–Rubin model
after Jerzy Neyman and Donald Rubin (Rubin, 2005).
Briefly, the POM elaborates on the role of counterfac-
tuals in causal inference. A counterfactual is a hypo-
thetical or conditional statement that expresses not
what has happened but what could or might happen
under different circumstances (e.g., “I would not have
been late, if I had correctly set the alarm”). In treat-
ment outcome studies, for example, there are two basic
counterfactuals: (1) what would the outcome of control
cases be, if they were treated; and (2) what would the
outcome of treated cases be, if they were not treated? If
each case was either treated or not treated, these poten-
tial outcomes would not be observed. This means that
the observed data—outcomes for treatment versus con-
trol—is a subset of all possible combinations.
The POM is concerned with conditions under which
causal effects may be estimated from data that do not
include all potential outcomes. In experimental designs,
random assignment helps with the replicability of stud-
ies in terms of the equivalence of treated and control
groups. Thus, any observed difference in outcome can
be attributed to the treatment, provided there is suffi-
cient replication. But things are more complicated in
quasi-experimental or nonexperimental designs where
the assignment mechanism is both nonrandom and
unknown. In this case, the average observed difference
may be a confounded estimator of treatment effects
versus selection factors. In such designs, the POM
distinguishes between equations for the observed data
3/22/2023 3:52:35 PM
 14 Concepts, Standards, and Tools
versus those for causal parameters. This feature helps
to clarify how estimators based on the data may differ
from estimators based on the causal model (MacKin-
non, 2008). The POM has been applied many times in
randomized clinical trials and in mediation analysis, a
topic covered later in this book.
Some authors describe the POM as a more disci-
plined method for causal inference than traditional
SEM (Rubin, 2009), but such claims are problematic for
two reasons (Bollen & Pearl, 2013). First, it is possible
to express counterfactuals in SEM as predicted values
for outcome variables, once we fix values of its causal
variables to constants that represent the conditions in
counterfactual statements (Kenny, 2021). Second, the
POM and SEM are logically equivalent in that a theo-
rem in one framework can be expressed as a theorem in
the other. That the two systems encode causal hypoth-
eses in different ways—in SEM as functional relations
among observed or latent variables and in the POM as
statistical relations among counterfactual (latent) vari-
ables—is just a superficial difference (Pearl, 2009).
Thus, the SCM as outlined by Pearl is a framework
for causal inference that the extends the capabilities of
both traditional SEM and the POM. This is why Hay-
duk et al. (2003) described the SCM as the future of
SEM and also why I introduce it to readers both in this
edition and in the previous (4th) edition of this book.
Grace et al. (2012) described the SCM as the third gen-
eration of SEM. The first generation of SEM dates to
Sewall Wright, who invented path analysis and corre-
sponding path diagrams as the graphical expression of
causal hypotheses. The second generation of SEM is
the synthesis of path analysis and factor analysis in tra-
ditional SEM.6 Earlier I described the possibility in this
approach of analyzing the graphical model before the
data are collected as a way to help plan the study. These
methods can also locate testable implications implied
by the graph. No special software is needed to analyze
the data. This is because testable implications for con-
tinuous variables can be estimated with standard com-
puter tools for statistical analysis, such as IBM SPSS,
with no need for special SEM software. This particular
approach is called piecewise SEM, which is described
in Chapter 8.
6 Muthén (2001) described second-generation SEM as the capa-
bility in traditional SEM to analyze continuous or categorical
outcomes in models with fixed or random effects, such as latent
growth models.
Pt1Kline5E.indd 14
PEDAGOGY AND SEM FAMILIES
Next, I explain my approach to teaching you about
SEM, given the descriptions of the three SEM families
just considered. First, beginning in Chapter 6, I describe
the logic of causal inference for nonparametric struc-
tural equation models where only theoretical variables
are represented in a causal graph. These variables are
not yet operationalized. Of course, theoretical variables
must eventually be measured. There could be a vari-
ety of ways to operationalize a construct. For example,
there is single-indicator measurement, where a single
observed variable is the proxy for the corresponding
construct. Scores on that indicator could be treated
as ordinal data that measure relative standing without
assuming equal intervals or treated as continuous data
if intervals are assumed to be more or less equal, among
other possibilities. An alternative is multiple-indicator
measurement, where a set of ≥ 2 observed variables
is used to approximate the same theoretical variable.
That is, instead of placing all of one’s measurement
eggs in the basket of a single observed variable, scores
from sets of multiple of variables are combined when
approximating theoretical variables.
But exactly how target constructs should be measured
is a decision that should come after working out your
basic causal hypotheses by specifying and analyzing a
causal graph in nonparametric SEM. Concepts about
ways to control confounding and determine whether
it is possible to estimate specific causal effects in the
graph are reviewed. If analysis of the casual graph indi-
cates that a particular effect cannot be estimated, then
something needs to be done, such as adding covariates
to address confounding. If a causal effect can be esti-
mated, analysis of the graph may indicate that more
than one estimate of that effect could be generated in
the data. Such knowledge helps the researcher to plan
the analysis in ways that prevent unpleasant surprises,
such as discovering after collecting the data that cer-
tain hypothesized effects cannot be estimated without
adding variables to the model (for which it might be
too late).
Next, I describe in Chapter 7 the specification of
parametric causal models that correspond to actual
measured variables. To keep things simpler, the mea-
surement approach is single indicator and the corre-
sponding parametric models are manifest variable
path models analyzed in the classical technique of
path analysis, the most senior member of the tradi-
tional SEM family. I outline in Chapters 8–12 the fun-
3/22/2023 3:52:35 PM

damental principles of estimating model parameters,
evaluating model fit to the data, testing hypotheses
about the model, respecifying the model if its account
of the data is rejected, comparing alternative models fit
to the same data, and analyzing the same model over
multiple groups or conditions. These topics are the core
of traditional SEM. They all generalize to latent vari-
ables models based on multiple-indicator measure-
ment of the kind analyzed in the technique of CFA, but
I want you to understand these fundamentals before we
add the complexities of multiple-indicator measure-
ment to the mix. That is, even if you are most interested
in latent variable modeling, a strong knowledge of path
analysis with observed variables will you help you get
there.
Coverage of latent variable models begins in Chap-
ter 13 with a conceptual treatment of multiple-indicator
measurement. I should caution you that this chapter has
a different structure compared with most other chap-
ters in the book. Specifically, there are no data analysis
examples nor equations. Instead, we deal with concepts
about measurement with multiple indicators that the
researcher should understand before applying specific
analysis techniques. The goal is help you to specify
models where sets of multiple indicators approximate
hypothetical constructs in ways consistent with your
hypotheses.
Described in Chapters 14–15 are, respectively, the
traditional SEM technique of CFA for analyzing mea-
surement models and the analysis of so called latent
variable path models where single or multiple indi-
cators are used to approximate theoretical variables.
These types of models represent the apex in traditional
SEM that can be extended in many ways to test addi-
tional, more advanced hypotheses that are covered
in Chapters 17–22. Chapter 16 is devoted to compos-
ite SEM with an emphasis on explaining the kinds of
hypotheses about measurement that are difficult to test
in traditional SEM but can be evaluated with relative
ease in composite SEM. I do not describe traditional
SEM and composite SEM as competitors; instead, I
emphasize their unique roles and capabilities as com-
plementary approaches.
SAMPLE SIZE REQUIREMENTS
Attempts to adapt SEM techniques to work in smaller
samples are described in Chapter 17, but it is still gener-
ally true that SEM (more so in traditional than in com-
Pt1Kline5E.indd 15
Promise and Problems 15
posite SEM) is a large-sample technique. Implications
of this characteristic are considered throughout the
book, but I can say now that certain types of estimates
in SEM, such as standard errors for effects of latent
variables, may be inaccurate when the sample size is
not large. The risk for technical problems in the analy-
sis is greater, too.
Because sample size is such an important issue, let
us now consider the bottom-line question: What is a
“large enough” sample size in SEM? It is impossible to
give a single answer because the factors that are sum-
marized next can affect sample size requirements:
1. More complex models, or those with more param-
eters, require bigger sample sizes than simpler models
with fewer parameters. This is because models with
more parameters require more estimates, and larger
samples are necessary in order for the computer to esti-
mate the additional parameters with reasonable preci-
sion.
2. In analyses in which all outcome variables are
continuous and normally distributed, all effects are lin-
ear, and there are no interactive effects require smaller
sample sizes. This is in comparison to analyses in which
some outcomes are not continuous or have severely
nonnormal distributions or in which there are curvilin-
ear or interactive effects. Sample size limitations also
curtail the use of estimation methods available in SEM,
some of which need very large samples because of the
assumptions they make—or do not make—about the
data.
3. Larger sample sizes are needed if score reliabil-
ity is relatively low; that is, less precise data require
larger samples in order to offset the potential distorting
effects of measurement error. Latent variable models
can control measurement error better than observed
variable models, so fewer cases may be needed when
there are multiple indicators for constructs of interest.
The amount of missing data also affects sample size
requirements. As expected, higher levels of missing
data require larger sample sizes in order to compensate
for loss of information.
4. There are also special sample size considerations
for particular kinds of structural equation models. In
factor analysis, for example, larger samples may be
needed if there are relatively few indicators per factor,
the factors explain unequal proportions of the variance
across the indicators, some indicators covary appre-
3/22/2023 3:52:36 PM
 16 Concepts, Standards, and Tools
ciably with multiple factors, the number of factors is
increased, or covariances between factors are relatively
low.
Given all of these influences, there is no simple rule
of thumb about sample size that works across all stud-
ies. Also, sample size requirements in SEM can be
considered from at least two different perspectives, (1)
the number of cases required in order for the results to
have adequate statistical precision versus (2) minimum
sample sizes needed in order for significance tests in
SEM to have reasonable power. Recall that power is the
probability of rejecting the null hypothesis in signifi-
cance testing when the alternative hypothesis is true in
the population. Depending on the model and analysis,
sample size requirements needed for power to equal or
exceed, say, .95, can be much greater than those needed
for statistical precision.
Results of a computer simulation (Monte Carlo)
study by Wolf et al. (2013) illustrate the difficulty with
“one-size-fits-all” heuristics about sample size require-
ments in SEM. These authors studied a relatively small
range of structural equation models, including factor
analysis models, manifest-variable versus latent-vari-
able models of mediation, and single-indicator versus
multiple-indicator measurement models. Minimum
sample sizes for both precision and power varied widely
across the different models and extent of missing data.
For example, minimum sample sizes for factor analysis
models ranged from 30 to 460 cases, depending on the
number of factors (1–3), the number of indicators per
factor (3–8), the average correlation between indicators
and factors (.50–.80), the magnitude of factor correla-
tions (.30–.50), and the extent of missing data (2–20%
per indicator).
I describe in Chapter 10 various methods to estimate
target sample sizes in traditional SEM, but here I can
suggest at least a few rough guidelines about sample
size requirements for statistical precision: For latent
variable models where all outcomes are continuous and
normally distributed and where the estimation method
is maximum likelihood—the default method in most
SEM computer tools—Jackson (2003) described the
N:q rule. In this heuristic, Jackson (2003) suggested
that researchers think about minimum sample sizes
in terms of the ratio of the number of cases (N) to the
number of model parameters that require statistical
estimates (q). A recommended sample-size-to-param-
eters ratio would be 20:1. For example, if a total of q
= 10 parameters requires estimates, then a minimum
Pt1Kline5E.indd 16
sample size would be 20q, or N = 200. Less ideal would
be an N:q ratio of 10:1, which for the example just given
for q = 10 would be a minimum sample size of 10q, or
N = 100. As the N:q ratio falls below 10:1 (e.g., N = 50
for q = 10 for a 5:1 ratio), so does the trustworthiness
of the results. The risk for technical problems in the
analysis is also greater.
It is even more difficult to suggest a meaningful
absolute minimum sample size, but it helps to consider
typical sample sizes in SEM studies. A median sample
size may be about 200 cases based on reviews of stud-
ies in different research areas, including operations
management (Shah & Goldstein, 2006) and education
and psychology (MacCallum & Austin, 2000). But
N = 200 may be too small when analyzing a complex
model or outcomes with nonnormal distributions, using
an estimation method other than maximum likelihood,
or finding that there are missing data. With N < 100,
almost any type of SEM may be untenable unless a very
simple model is analyzed, but models so basic may be
uninteresting. Barrett (2007) suggested that reviewers
of journal submissions routinely reject for publication
any SEM analysis where N < 200 unless the popula-
tion studied is restricted in size. This recommendation
is not standard practice, but it highlights the fact that
analyzing small samples in SEM is problematic.
Most published SEM studies are probably based on
samples that are too small. For example, Loehlin and
Beaujean (2017) noted that results of power analyses
in SEM are “frequently sobering” because researchers
often learn that their sample sizes are far too small for
adequate statistical power. Westland (2010) reviewed
a total of 74 SEM studies published in four different
journals on management information systems. He esti-
mated that (1) the average sample size across these
studies, about N = 375, was only 50% of the minimum
size needed to support the conclusions; (2) the median
sample size, about N = 260, was only 38% of the mini-
mum required and also reflected substantial negative
skew in undersampling; and (3) results in about 80% of
all studies were based on insufficient sample sizes. We
will revisit sample size requirements in later chapters,
but many, and probably most, published SEM studies
are based on samples that are too small.
BIG NUMBERS, LOW QUALITY
There is no denying that SEM is increasingly “popular”
among researchers. Thousands of SEM studies have
3/22/2023 3:52:36 PM

been published at an accelerating rate since the 1990s
or so (Hair et al., 2012; Thelwall & Wilson, 2016). The
increasing availability of computer programs, both
commercial and free, has made SEM in its various
forms ever more accessible to applied researchers. It
is not hard to understand the enthusiasm for SEM. As
described by David Kenny in the Series Editor Note in
the second edition of this book, researchers love SEM
because it addresses questions they want answered
and it “thinks” about research problems the way that
researchers do. But there is evidence that many—if
not most—published reports of the application of SEM
have serious flaws as described next.
MacCallum and Austin (2000) reviewed about 500
SEM studies in 16 different psychology research jour-
nals, and they found problems with the reporting in
most cases. For example, in about 50% of the articles,
the reporting of parameter estimates was incomplete
(e.g., unstandardized estimates omitted); in about 25%
the type of data matrix analyzed (e.g., correlation vs.
covariance matrix) was not described; and in about
10% the model specified or the indicators of factors
were not clearly specified. Shah and Goldstein (2006)
reviewed 93 articles in four operations management
research journals. In most articles, they found that it
was hard to determine the model actually tested or the
complete set of observed variables. They also found
that the estimation method used was not mentioned in
about half of the articles, and in 31 out of 143 studies,
the model described in the text did not match the statis-
tical results reported in text or tables.
Zhang et al. (2021) reviewed 144 SEM studies pub-
lished in 12 top organizational and management jour-
nals in 2011–2016. Each article was evaluated against
criteria that included
1. The clarity of the rationale for using SEM over
alternative methods.
2. Whether the statistical models analyzed were
described in sufficient detail.
3. The completeness of the reporting on data integ-
rity, including the rationale for the sample size and
information about distributional assumptions, miss-
ing data, and reliability of the scores analyzed.
4. Whether hypotheses were tested in a clear, specific
order.
5. Whether the statistical results were described in
adequate detail so that readers could evaluate the
trustworthiness of the conclusions.
Pt1Kline5E.indd 17
Promise and Problems 17
Many problems were apparent. For instance, an explicit
justification for using SEM was provided in only
about 40% of the studies; distributional assumptions
were addressed in about 20%; a specific rationale for
the sample size was given in about 30%; and specific
details about the correspondence between model and
data were reported in 20% of reviewed studies. These
results are disheartening, but they are hardly atypical.
Nothing in SEM protects against equivalent models,
which explain the data just as well as the researcher’s
preferred model, but an equivalent model makes dif-
fering causal claims. The problem of equivalent mod-
els is relevant in probably most applications of SEM,
but most authors of SEM studies do not even mention
it (MacCallum & Austin, 2000). Ignoring equivalent
models is a serious kind of confirmation bias whereby
researchers test a single model, give an overly posi-
tive evaluation of the model, and fail to consider other
explanations of the data (Shah & Goldstein, 2006). The
potential for confirmation bias is further strengthened
by the lack of replication, a point considered next.
It is rare when SEM analyses are replicated across
independent samples either by the same researchers
who collected the original data (internal replication)
or by other researchers who did not (external replica-
tion). The need for large samples in SEM complicates
replication, but most of the SEM research literature is
made up of one-shot studies that are never replicated.
It is critical to eventually replicate a structural equation
model if it is ever to represent anything beyond a mere
statistical exercise. Thus, the SEM research literature
is part of the broader replication crisis in psychology
and other disciplines. Kaplan (2009) noted that despite
over 40 years of application of SEM in the behavioral
sciences, it is rare that results from SEM analyses are
used for policy or clinically relevant prediction studies.
The ultimate goal of SEM—or any other type of
method for statistical modeling—should be to attain
what I call statistical beauty, which means that the
final retained model (if any)
1. Has a clear theoretical rationale (i.e., it makes sense).
2. Differentiates between what is known and what is
unknown—that is, what is the model’s range of con-
venience, or limits to its generality?
3. Sets conditions for posing new questions.
That most applications of SEM fall short of these goals
should be taken as an incentive by all of us to do better.
3/22/2023 3:52:36 PM
 18 Concepts, Standards, and Tools
LIMITS OF THIS BOOK
Many advanced applications in SEM are described in
Chapters 17–22, but it is impossible (and undesirable,
too) to cover the whole range of extended or special-
ized analyses in a single volume. Just a few of these
topics are mentioned next with citations for sources
that provide more information. There are special struc-
tural equation models for imaging data, such as func-
tional magnetic resonance imaging (fMRI) (Cooper et
al., 2019), and also for genetic data (Luo et al., 2019).
Bayesian SEM combines the methods of Bayesian esti-
mation with the analysis of structural equations models
(Depaoli, 2021). Bayesian options for SEM are men-
tioned at various points in the book, but their applica-
tion requires strong knowledge of Bayesian statistics.
Two other advanced topics not covered in this book
include multilevel SEM (Castanho Silva et al., 2020)
and the analysis of interactive effects of latent variables
(Cortina et al., 2021), although methods for observed
variables are described in Chapters 12 and 20.
SUMMARY
The SEM families of techniques have their origins
in regression-based analyses of observed variables,
factor-analysis-based evaluations of common factor
models, composite-based analyses of proxies for theo-
retical variables, and methods from computer science
for analyzing causal graphs. Essential features include
the capabilities to analyze causal models before collect-
Pt1Kline5E.indd 18
ing the data, and the potential to distinguish between
observed and latent variables and to test a wide range
of hypotheses about measurement and causation. More
and more researchers are using SEM, but in too many
studies there are serious problems with the way it is
applied or with how analysis results are reported. How
to avoid getting into trouble with SEM is a major theme
in later chapters of this book. The ideas introduced in
this chapter set the stage for review in the next chap-
ter of fundamental principles in statistics that underlie
SEM.
LEARN MORE
Bollen and Pearl (2013) describe myths about traditional
SEM, Hair (2021) outlines the history of composite SEM, and
Wolfle (2003) traces the introduction of path analysis to the
social sciences.
Bollen, K. A., & Pearl, J. (2013). Eight myths about causal-
ity and structural equation models. In S. L. Morgan
(Ed.), Handbook of causal analysis for social research
(pp. 301–328). Springer.
Hair, J. F. (2021). Reflections on SEM: An introspective, idio-
syncratic journey to composite-based structural equation
modeling. SIGMIS Database, 52(SI), 101–113.
Wolfle, L. M. (2003). The introduction of path analysis
to the social sciences, and some emergent themes: An
annotated bibliography. Structural Equation Modeling,
10(1), 1–34.
3/22/2023 3:52:36 PM
 2

Background Concepts and Self‑Test

Newcomers to SEM should have good statistical knowledge in at least three areas: (1) regression analysis;
(2) correct interpretation of results from statistical significance testing including the role of bootstrapping;
and (3) psychometrics, or statistical measures of the properties of scores from psychological tests including
evidence for their reliability or validity. Some estimates in SEM are interpreted exactly as regression coef-
ficients, and these interpretations depend on many of the same assumptions as in regression analysis. The
potential for bias due to omitted predictors that covary with measured predictors or due to measurement
error in predictors or the criterion is similar in both regression and SEM when (1) there is a single observed
measure of each construct, and (2) score reliability is not explicitly represented nor controlled in the analysis.
Results of significance testing both for the whole model and for estimates of its individual parameters are
widely reported in SEM studies, although statistical significance is hardly the sole basis for inference in SEM
for reasons explained throughout the book. It is also true that results in significance testing are widely mis-
understood in perhaps most analyses, including SEM, and you need to know how to avoid making common
mistakes. If predictor or outcome variables are measured with psychological tests, their score reliabilities
should be routinely assessed in the researcher’s sample. Knowledge of psychometrics is also essential when
selecting among alternative measures of the same target concept.
This chapter serves as a check on your understanding of regression fundamentals, significance test-
ing, and psychometrics, and as a gateway to additional resources for self-study. Specifically, after a review
of potential obstacles to a strong command of these topics and a summary of background topics from the
perspective of learning about SEM, a self-test of comprehension with a scoring system is provided. Relatively
low scores (e.g., < 50% correct) in a particular area, such as psychometrics, would signal the need for further
reading. To that end, a total of three supplementary chapters, the Regression Primer, Significance Testing
Primer, and Psychometrics Primer, are freely available on the website for this book. Some advice: Even if you
think that you already know these topics, you should take the self-test. Many readers tell me that they learned
something new after hearing about the issues outlined next.

UNEVEN BACKGROUND States and Canada about training in statistics, measure-

PREPARATION
There is evidence that not all of the background top-
ics just mentioned are adequately covered in graduate
school. For example, Aiken et al. (2008) surveyed just
over 200 psychology doctoral programs in the United
ment, and research methods. They replicated a compa-
rable survey from almost 20 years before on the same
topic. In 2008, the median for training in statistics and
measurement was 1.6 years associated mainly with a
1-year introductory statistics sequence. About 80% of
doctoral programs offered in-depth training in both the

19

Pt1Kline5E.indd 19 3/22/2023 3:52:36 PM

 20 Concepts, Standards, and Tools
“old standards” (e.g., ANOVA) and multiple regres-
sion (MR) with the result that students could generally
perform such analyses themselves (Aiken et al., 2008).
Significance testing in ANOVA, MR, and other tech-
niques would also typically be covered in introductory
graduate statistics courses (Kline, 2020b). But whether
graduate students—or even experienced researchers—
understand significance testing or related concepts,
such as confidence intervals, is questionable, a topic
addressed soon.
Aiken et al. (2008) also described critical gaps in
statistics or measurement training. For instance, only
about 30% of doctoral programs offered in-depth
training in regression diagnostics, or techniques for
assessing whether assumptions are tenable or if there
are scores, such as outliers, with undue influence on the
results. Misunderstanding about assumptions in MR is
common. Examples include being unaware that per-
fect score reliability for predictor variables is assumed,
believing falsely that measurement error necessarily
causes underestimation of individual regression coef-
ficients, and the myth that the requirement for normal
distributions applies to the observed scores instead
of the residuals (Williams et al., 2013). Over 90% of
nearly 900 articles in clinical psychology research jour-
nals reviewed by Ernst and Albers (2017) were unclear
about assumption checks in MR analyses. Another gap
is that logistic regression for dichotomous outcomes is
covered in depth in only about 10% of doctoral pro-
grams (Aiken et al., 2008). This result suggests that
training in other methods for binary data, such as the
probit regression model, or methods for categorical
outcomes with three or more levels, such as ordinal
regression for ordered categories or multinomial logis-
tic regression for unordered categories, is infrequent at
the graduate level.
Another gap is that controversies in significance test-
ing are thoroughly covered in only about 30% of psy-
chology doctoral programs (Aiken et al., 2008). This is
unfortunate because ongoing debate about the proper
role of significance testing, including none, is part of a
larger credibility crisis about psychology research that
includes concerns about replication, reporting, mea-
surement, and the wasting of perhaps most research
funds in some areas (Szucs & Ioannidis, 2017). How to
properly report results from SEM analyses is covered
in the next chapter, but traditional significance test-
ing is seen as lacking in more and more disciplines;
expert and novice researchers alike are unaware of
that decline, which is a serious problem (Kmetz, 2019).
Pt1Kline5E.indd 20
Reasons for this critical assessment include the likeli-
hood that many, if not most, p values reported in the
literature are wrong, the myriad of false beliefs associ-
ated with significance testing, and the concern about p
hacking—see Topic Box 2.1 for elaboration.
Instruction in measurement was available in about
60% of psychology doctoral programs surveyed by
Aiken et al. (2008), but coverage of core topics such
as classical test theory, item response theory (IRT),
and test construction was typically brief with a median
length of just 4.5 weeks for all topics. This is not enough
time to develop any real expertise about measurement.
In fact, I bet that an undergraduate student who has
taken a one-semester (e.g., 3-credit) course in psycho-
metrics knows more about psychological measurement
and test construction than a graduate student without
this background. The meagre amount of doctoral-level
measurement training in 2008 was actually a slight
improvement over results for the same area reported 20
years earlier.
POTENTIAL OBSTACLES TO LEARNING
ABOUT SEM
I do not know whether the results just summarized
about psychology graduate training in statistics, mea-
surement, and research design apply generally to other
disciplines. But in my experience, based on working
with SEM novices from disciplines such as education,
biology, health sciences, operations management, mar-
keting, and commerce, researchers face the following
potential obstacles to learning about SEM:
1. The emphasis in graduate training in regression
techniques is mainly on continuous outcomes, so new
researchers may be relatively unprepared to apply esti-
mators for categorical outcomes in SEM.
2. Overemphasis of statistical significance can blind
researchers to other aspects of the results that are just
as critical, if not even more so, than p values. Antona-
kis (2017) referred to this particular stumbling block
as significosis, or an ordinate focus on statistical sig-
nificance. A related distortion is dichotomania, or the
compulsion to dichotomize continuous p values against
an arbitrary standard, such as p < .05 for results touted
as “significant” versus p ≥ .05 for other results that are
discounted, ignored, or lamented because they are “not
significant.” This binary interpretation is not supported
3/22/2023 3:52:36 PM
 Background Concepts and Self‑Test 21

TOPIC BOX 2.1

Cautionary Tales About Significance Testing:

Inaccuracies, Errors, and Hacking
For two reasons, most p values reported in the research literature could be incorrect:

1. Assumptions of significance testing—­random sampling from population distributions with known

properties (e.g., normality, homoscedasticity) and the absence of all other sources of error other
than sampling error—are generally implausible. Most samples in human studies are ad hoc, or
samples of convenience selected because they happen to be available. Convenience sampling
may have little, if anything, to do with random sampling. Few distributions in real data sets are
normally distributed or homoscedastic, and even slight departures from distributional assumptions
in small, unrepresentative samples can grossly distort p values (Erceg-Hurn & Mirosevich, 2008).
2. There are mistakes in too many journal articles in the reporting of p values. Nuijten et al. (2016)
found that over 50% of reviewed articles published in psychology research journals during the
years 1985–2013 contained at least one incorrect p value, given reported test statistics and
degrees of freedom.

There is ample evidence that most researchers, including those with the highest levels of statistical
training, do not fully understand what p values mean (McShane & Gal, 2016). Psychology professors in
two different surveys endorsed false belief about p values at rates generally no lower than among under-
graduate students (Haller & Krauss, 2002; Oakes, 1986), and about 90% of both groups endorsed at
least one incorrect interpretation. Summarized next are what I call the big five misinterpretations
of p values in significance testing. They are described for the case p < .05 when testing at the .05 level:

1. Most researchers endorse the local Type I error fallacy that the likelihood that the decision just
taken to reject the null hypothesis is a Type I error is less than 5%. This belief is wrong because any
particular decision to reject the null hypothesis is either correct or incorrect, so no probability (other
than 0 or 1.0) is associated with it. Also, it is only with sufficient replication that we could determine
whether or not the decision to reject the null hypothesis in a particular study was correct.
2. Probably most researchers endorse the odds-­against-­chance fallacy, or the false belief that
the probability that a particular result is due to chance (i.e., arose by sampling error alone) is
less than 5%. This belief is wrong because p is calculated by the computer assuming that the null
hypothesis is true, so the probability that sampling error is the only explanation is already taken
to be 1.0. Thus, it is illogical to view p as somehow measuring the likelihood of chance. Besides,
the probability that sample results are affected by error of some kind—­sampling, measurement,
­implementation, or specification error, among others—­is virtually 1.0. From this perspective, basi-
cally all sample results are wrong in that they generate incorrect point estimates of the target
parameter, and significance testing in primary studies does nothing to change this reality (Ioan-
nidis, 2005).
3. Just under half of researchers believe that the probability is less than 5% that the null hypothesis
is true, which is the inverse probability error, also called the fallacy of the transposed
conditional (Ziliak & McCloskey, 2008), and the permanent illusion due to its persistence
over time and disciplines (Gigerenzer & Murray, 1987). This error stems from forgetting that
(continued)

Pt1Kline5E.indd 21 3/22/2023 3:52:36 PM

 22 Concepts, Standards, and Tools

p values are conditional probabilities of data under the null hypothesis, not the other way around.
There are direct methods in Bayesian statistics to estimate conditional probabilities of hypotheses,
but not in classical significance testing.
4. The false belief that 1 – p is the probability of finding another significant result in a future sample
is the replication (replicability) fallacy, which is endorsed by about half of researchers. For
example, if p < .05, then the likelihood of replication is believed to exceed .95 under this myth.
Knowing the probability of replication in hypothetical future samples would be very useful, but the
quantity 1 – p is just the probability of a result or one even less extreme under the null hypothesis.
In general, replication is a matter of experimental design, sampling, and whether some effect
actually exists in the population.
5. The validity (valid research hypothesis) fallacy is the myth that 1 – p is the likelihood
that the alternative hypothesis is true, a false hope endorsed by about half of researchers. As
mentioned, probabilities of hypotheses are not estimated in significance testing, and both p and
its complement, 1 – p, are conditional probabilities of data, not hypotheses.

There are many other cognitive errors about significance testing outcomes. For example, the filter
myth is the false belief that p values sort results into two categories: those “significant” findings that are
not due to chance versus the “not significant” results that are due to chance. This myth is basically an exten-
sion of the odds-­against-­chance fallacy applied to results with higher (not significant) p values. Authors
in just over 50% of nearly 800 published empirical studies reviewed by Amrhein et al. (2019) committed
the zero fallacy—also called the slippery slope of nonsignificance (Cumming & Calin-­Jageman,
2017)—by falsely interpreting the absence of statistical significance as evidence for a zero population
effect size.
The misinterpretations just described plus others—­see the Significance Testing Primer—­can promote
a cognitive style where researchers report low p values with great confidence or even bravado but with
little real comprehension (Lambdin, 2012). The same distortions can also lead to confirmation bias where
statistically significant results are uncritically accepted as proving the researcher’s hypotheses—­see Calin-­
Jageman and Cumming (2019) for examples. False beliefs also pose a challenge to instructors of a first
graduate statistics course: Students’ heads are filled with myth about significance testing that can interfere
with further learning unless those fallacies are identified and remediated (Kline, 2020b).
Confidence intervals are described as ways to summarize results in a more comprehensive way than
is possible in significance testing, and statisticians generally prefer interval estimation over point estima-
tion, or the reporting of results with no margins of error (or error bars in graphical form) (Cumming &
Calin-­Jagerman, 2017). Because significance testing and confidence intervals are based on essentially the
same concepts and assumptions, it is not surprising that similar kinds of errors can affect both. Abelson
(1997) described the law of diffusion of idiocy, which states that every misstep in significant testing
has a counterpart with confidence intervals. There is also evidence that confidence intervals, too, are
widely misinterpreted (Hoekstra et al., 2014). Listed next are some common false beliefs (Morey et al.,
2016):

1. Fundamental confidence fallacy, or the myth that confidence level, such as 95%, indicates
the likelihood (i.e., .95) that the interval contains the value of the target parameter.
2. Precision fallacy, or the misunderstanding that narrower confidence intervals automatically
signal greater precision of knowledge about the parameter.
(continued)

Pt1Kline5E.indd 22 3/22/2023 3:52:36 PM

 Background Concepts and Self‑Test 23

3. Likelihood fallacy, or the untruth that a particular confidence interval includes equally likely
values for the parameter.

The phenomenon of p hacking involves the possibility of presenting any result as “significant”
through decisions, such as covariate selection, transformations of scores, or the treatment of missing data,
that are not always disclosed (Simmons et al., 2011). Most instances of hacking involve lowering p val-
ues so that key results are significant, but p values can also be increased when results that are not sig-
nificant favor the researcher’s goals. Perhaps the most infamous example of hacking to increase p is the
Vioxx calamity, where analyses of data from a clinical trial were manipulated to render nonsignificant an
increased risk of heart attack in the treatment group (Ziliak & McCloskey, 2008). Both p hacking and other
dubious practices that favor the researcher’s hypotheses are probably widespread (John et al., 2012). The
parallel in SEM is model hacking, where significance test results are manipulated to increase the odds
of retaining the model, which represents the researcher’s hypotheses. How to avoid model hacking in SEM
begins with complete and transparent reporting of the results, including explaining the bases for deciding
whether to retain any model.

by what can be trivial differences in continuous p val- SIGNIFICANCE TESTING

ues; that is, a result where p = .04 does not appreciably
differ from another result where p = .06 when testing at
the .05 level.
3. Relatively little formal training in measurement
or psychometrics can put researchers in a difficult
position when selecting measures for their research or
evaluating the quality of data from psychological tests
in their own samples. It can also hinder accurate and
complete reporting about psychometrics.
Whatever researchers lack in their training about
statistics, measurement, or research design when
approaching what is, for them, a new technique such as
SEM can be addressed through self-study or participa-
tion in seminars, summer schools, or other continuing
education experiences. The need to periodically update
one’s data analysis skills is also a normal part of being
a researcher. That is, lifelong learning as self-initiated
education focused on personal or professional devel-
opment is a healthy perspective not just for research-
ers, but for everyone. So let’s get on with knocking the
rust off what you already know or adding to your skill
set. Described next from the perspective of learning
about SEM are special issues in significance testing,
measurement, and regression—see the corresponding
primers for more detailed presentations.
Many effects can be tested for statistical significance in
SEM, ranging from things such as the variance for a sin-
gle variable up to entire models evaluated across mul-
tiple samples. There are four reasons, however, why the
role for significance testing in SEM should be smaller
compared with more standard techniques like ANOVA:
1. The capability to evaluate an entire model at
once brings a higher-level perspective to the analysis.
Although statistical tests of individual effects in the
model may be of interest, at some point the researcher
must make a decision about the whole model: Should
it be rejected?—modified?—if so, how? This deci-
sion should not be based on significance testing alone
because other factors, such as whether statistical results
are meaningful, given the research context and hypoth-
eses, are equally if not more important than statistical
significance. There is also a sense in SEM that the view
of the entire model takes precedence over that of spe-
cific details (individual effects).
2. The technique of SEM generally requires large
samples, but in significance testing, effects with low p
values in very large samples are sometimes of trivial
magnitude. By the same token, virtually all effects that
are not zero could be significant in a sufficiently large
sample. Just the opposite can happen in smaller sam-

Pt1Kline5E.indd 23 3/22/2023 3:52:36 PM

 24 Concepts, Standards, and Tools
ples: Effects of appreciable magnitude that are actually
true in the population may fail to be significant due to
low statistical power.
3. Researchers should be more concerned with esti-
mating effect size and evaluating the substantive signif-
icance of their results than with statistical significance,
which has little to do with scientific or practical impor-
tance. In particular, researchers should not believe that
an effect or association exists just because a result is
significant, especially if an arbitrary threshold, such
as p < .05, is applied to dichotomize continuous p val-
ues. Likewise, researchers should not conclude that an
effect is absent just because it is not significant (Was-
serstein et al., 2019); that is, avoid committing the zero
fallacy—see Topic Box 2.1.
4. Standard errors for the effects of latent vari-
ables are estimated by the computer, and those stan-
dard errors are the denominators of significance tests
for those effects. The value of the standard error could
change if, say, a different estimation method is used or
sometimes even across different software packages for
the same analysis and data. Thus, it can happen that an
effect for a latent variable is “significant” in one analy-
sis but is “not significant” in another analysis with a
different estimator or computer tool. Now, differences
in p values across different computer programs for
the same effect and estimation method are not usually
great, but slight differences in p can make big differ-
ences in significance testing, such as p = .051 versus
.049 for the same effect when testing at the .05 level.
You may be surprised to know there is no require-
ment to dichotomize p values at all. This means that
exact p values are simply reported but are not com-
pared against any bright-line rule or threshold, such as
.05, or any other standard whatsoever. This means that
the stale, musty, and worn-out term “significant” is not
used for results with low p values, so there are no aster-
isks or any other symbol in text or tables that designate
“significant” results just as there is no special demarca-
tion of other effects with higher p values as “not sig-
nificant.” Hurlbert and Lombardi (2009) referred to this
perfectly legitimate reform of traditional significance
testing as neo-Fisherian significance assessments
(NFSA), and it is consistent with the call to researchers
to stop using the term “significant” in point 3 just listed.
You should also know that SEM computer tools gen-
erally print by default standard errors for the estimates
Pt1Kline5E.indd 24
in the unstandardized solution, but standard errors for
standardized estimates might be part of optional out-
put that must be requested by the researcher. Because
the unstandardized and standardized estimates for the
same parameter have their own standard errors, it can
happen that, say, the unstandardized estimate is signifi-
cant but the standardized estimate is not significant at
the same level, or vice versa. This outcome is neither a
computer error nor contradictory because unstandard-
ized and standardized estimates each have their own
distinct sampling distributions. Confusion about this
issue can be minimized by not dichotomizing p values,
such as in NFSA-style reporting of the results.
MEASUREMENT AND PSYCHOMETRICS
Given limited training about measurement in many
graduate programs, it is not surprising that reporting
on psychometrics in the research literature is too often
deficient. This is especially true for reliability coeffi-
cients, which indicate the degree to which test scores
are consistent over variations in testing situations,
times, test administrators or scorers, forms, or selec-
tions of items from the same domain. Lower values of
reliability coefficients indicate less precise scores. If
a reliability coefficient, designated here as rXX, equals
zero, it means that the scores are basically random
numbers, and random numbers measure nothing. The
result rXX = 1.0 indicates flawless consistency, but such
a result in real data would be pretty extraordinary (i.e.,
don’t hold your breath waiting for perfection).
Reporting values of reliability coefficients should be
routine whenever scores from psychological tests are
analyzed, but the reality is unfortunately different. For
example, Vacha-Haase and Thompson (2011) reviewed
nearly 50 meta-analyses of results from about 13,000
primary studies in which test scores were analyzed.
They found that about 55% of authors mentioned noth-
ing about score reliability. In 15% of reviewed studies,
authors merely reported values of reliability coefficients
from other sources, such as test manuals. Inferring
from reliability coefficients derived in other samples,
such as a test’s normative sample, to a different popu-
lation is called reliability induction. Such reasoning
about the generalizability of reliability coefficients
needs explicit justification. But authors of reviewed
studies rarely compared characteristics of their samples
with those from cited studies of score reliability. For
3/22/2023 3:52:36 PM

example, scores from a computer-based task of reaction
time developed in samples of young adults may not be
as precise for elderly adults, who may be unfamiliar
with this method of testing.
A better practice is for researchers to report val-
ues of reliability coefficients in their own samples. It
is critical to do so because reliability is a property
of scores in a particular sample, not an immutable
characteristic of tests. This is because the precision
of scores from the same test varies over samples from
the same population due to sampling error. Variation
in score reliability may be even greater over samples
taken from a population different from the target pop-
ulation for the test. Urbina (2014) used the term rela-
tivity of reliability to emphasize that (1) the quality of
reliability belongs to scores, not tests, and (2) scores
for individual cases are more or less reliable due to
unique characteristics of examinees, such as motiva-
tion or fatigue, and also to examiner qualifications,
such as experience in administering or scoring the
test, and conditions where testing takes place. Some
of these factors are unrelated to the test itself, but they
can still influence the consistency and precision of
scores. Along these lines, researchers should also cite
reliability coefficients in published sources (reliabil-
ity induction) but with comments on the similarities
between samples described in those sources and the
researcher’s sample.
Appelbaum et al. (2018) described revised jour-
nal article reporting standards for quantitative studies
(JARS-Quant) for journals published by the Ameri-
can Psychological Association. Revised guidelines for
reporting on psychometrics (including SEM studies)
call on authors to
1. Report values of reliability coefficients for the
scores analyzed, if possible.
2. Describe the specifics of those reliability coeffi-
cients, such as the length of the retest interval for
test–retest reliabilities, characteristics of scorers
and their training for interrater reliabilities, or the
specific type of internal consistency coefficients for
composite (multi-item) scales.
3. Report the characteristics of external samples if
reporting reliabilities from those samples, such as
original test normative samples.
4. Provide estimates of convergent and discriminant
validity where relevant.
Pt1Kline5E.indd 25
Background Concepts and Self‑Test 25
REGRESSION ANALYSIS
Two topics are addressed next: (1) Effects of measure-
ment error on the results in MR analyses and (2) the
limited usefulness of significance testing when test-
ing hypotheses about incremental validity in the pres-
ence of even moderate amounts of measurement error.
Perfect score reliability (rXX = 1.0) is assumed in MR
for the predictors but not for the criterion, also called
the response, outcome, or dependent variable. This is
because only the criterion has residuals, or differences
between actual and predicted scores on the response
variable. The residuals give random measurement error
a place to “go” for the criterion; that is, error is mani-
fested in the regression residuals. But predictors have
no residuals to “absorb” their measurement error; thus,
it must be assumed that their scores are perfectly con-
sistent.
Measurement error in the criterion only does not
bias unstandardized regression coefficients, if that
error is unrelated to the predictors. But there is down-
ward bias in the standardized coefficients, which also
makes the value of R2 decrease as error in the crite-
rion increases under the same condition (Williams et
al., 2013). Things are more complicated when there is
error in the predictors: Measurement error in a single
predictor that is not shared with any other variable will
attenuate the absolute value of the regression coeffi-
cient for that predictor. But regression coefficients for
the other, error-free predictors can also be affected.
Whether those other coefficients are attenuated, exag-
gerated, or unbiased depends on the values and signs
of (1) correlations between the underlying latent vari-
able imperfectly measured by a predictor and all other
explanatory variables, and (2) the effect of the concep-
tual variable just mentioned on the criterion (Bollen,
1989; Kenny, 1979). Values of these correlations and
their potential distorting effects are rarely known in
practice, so it difficult to anticipate the magnitudes and
directions of this propagation of measurement error
in a single predictor.
The effects of measurement error in multiple predic-
tors are also hard to anticipate because there are two
sources of distortion for each predictor: Attenuation
bias due to error in a particular predictor, plus a term
the value of which is influenced by the degree of unreli-
ability error in other predictors (Kenny, 1979). It is pos-
sible that these two sources of bias could cancel each
other out, but that outcome could be rather unlikely.
3/22/2023 3:52:36 PM
 26 Concepts, Standards, and Tools
Whether bias is positive or negative in a particular coef-
ficient is also generally unknown, but overall distortion
tends to increase as score reliabilities among the pre-
dictors are lower.
Another issue is the presence of measurement error
that is shared over ≥ 2 predictors or with the criterion.
Correlated measurement error can arise when mul-
tiple variables are assessed with a common method,
variables share common informants or stimuli, or the
same variable is measured on ≥ 2 occasions among the
same cases, among other possibilities. The signs and
magnitudes of the error correlations also affect the
degree and direction of bias in regression coefficients
(Bollen, 1989; Williams et al., 2013)—see Trafimow
(2021) for more information and examples. There are
ways to adjust individual regression coefficients for
attenuation, but they assume that measurement error is
not correlated and, as mentioned, measurement error
can attenuate or inflate regression coefficients. Whether
such simple corrections give accurate results in real-
world data is questionable (Williams et. al, 2013).
Incremental validity concerns the relative contribu-
tions of different variables in predicting some criterion.
That contribution is estimated by the regression coeffi-
cients, which statistically control for effects of all other
predictors in the equation. In this way, the coefficients
indicate the relative contribution of each predictor
above and beyond the rest.1 Many researchers interpret
a statistically significant regression coefficient as evi-
dence for the incremental validity of the corresponding
predictor. There are thousands of published regression
studies in which hypotheses about incremental valid-
ity are tested in this way (Hunsley & Meyer, 2003), so
the issue of whether inferences based on p values for
regression coefficients are trustworthy is very relevant.
Based on rational analysis and computer simula-
tion results, Westfall and Yarkoni (2016) demonstrated
that effective Type I error rates for significance tests
of regression coefficients in MR are surprisingly high
even in reasonably large samples (e.g., N = 300) with
at least moderate levels of scores reliability (e.g., rXX
= .80) in analyses with just two predictors. Under the
conditions just stated and assuming that the bivari-
ate correlation of each predictor with the criterion is
1 Other metrics or statistics for measuring incremental validity
include partial correlations, semipartial correlations (also called
part correlations), and sequential increases in R2 when predictors
are added to the equation in a particular order—see Grömping
(2015).
Pt1Kline5E.indd 26
about .50, the error rate in significance testing is just
greater than 65%. In even larger samples, the error
rate can approach 1.0. These counterintuitive results
happen because (1) there is a curvilinear relation such
that Type I error rates generally peak over the range
rXX = .30–.70, and (2) Type I errors also increase with
sample size, holding all else constant. In larger sam-
ples, significance tests have greater power to detect a
true relation between a predictor and the criterion, but
measurement error causes the significance test to con-
flate a common effect of multiple predictors with the
unique contribution for a single predictor (Westfall &
Yarkoni, 2016). Thus, it is possible that inferences in
MR analyses about incremental validity based mainly
on p values are suspect in many, if not most, published
regression studies.
You will learn that a relative strength of SEM over
standard regression techniques is the capability to
explicitly represent in the model the score reliability
for any observed variable or the presence of correlated
measurement errors (if any) for particular pairs of vari-
ables. All other results, including coefficients for puta-
tive causal variables, are estimated given the informa-
tion about psychometrics specified by the researcher.
The potential advantage of SEM for testing hypotheses
about incremental validity is even greater when theo-
retical concepts are specified as measured by multiple
observed variables, or indicators. Thus, hypotheses
about incremental validity are better supported by
methods like SEM that can explicitly model unreli-
ability than in standard regression analysis (Westfall &
Yarkoni, 2016).
SUMMARY
It helps to begin learning about SEM with a good
comprehension of basic concepts in regression, sig-
nificance testing including bootstrapping, and psycho-
metrics. Not all of these topics may be covered with
equal depth in graduate programs, so researchers often
need to supplement their knowledge. There are also
widespread myths about significance testing and psy-
chometrics that can interfere with learning about SEM.
For example, there are many false interpretations of
p values from significance testing that altogether can
make researchers overly confident in their results and
distract them from other aspects of their data, such as
effect size or precision. The black-box view that reli-
ability is a property of tests rather than of scores in a
3/22/2023 3:52:36 PM

particular sample runs counter to the best practice of
reporting psychometrics, including values of score reli-
abilities for the data analyzed. Measurement error in
a single predictor in standard regression analysis can
bias the coefficients for other predictors, too, and its
effects do not always involve attenuation of regression
coefficients, especially if measurement error is shared
over multiple predictors or with the criterion. Presented
next is a self-test of knowledge about concepts in sig-
nificance testing, regression, and psychometrics. After
responding to the questions, score your answers using
the criteria that follow the self-test. Refer to the primers
available on the book’s website for additional informa-
tion about each background topic.
SELF‑TEST
Sorry, no multiple-choice items here. I believe that
essay questions are better able to detect strengths
or weaknesses in knowledge about statistics (Kline,
2020b). Write a concise response to each question. The
number of possible points for each question is indicated
in parentheses, and maximum possible total scores are
listed after the questions for each topic. The scoring cri-
teria for each item are given in the next section.
Significance Testing
1. What is a sampling distribution? (4)
2. For the same continuous variable, interpret SD =
15.00 versus SE = 3.00 (respectively, standard devi-
ation vs. standard error of the mean). (4)
3. What does p = .03 in significance testing mean? (4)
4. What is a in significance testing, and how does it
differ from p? (4)
5. Define power and b in significance testing. (5)
6. List the factors that affect power. (6)
7. State the combination of factors in question 6 that
leads to the highest power. (6)
8. Interpret these results (i.e., each numerical value)
for two independent samples: t(48) = 2.25, p = .029
for a one-tailed hypothesis. (6)
9. Given t = 6.75/3.00 = 2.25 for two independent sam-
ples, interpret the numerator and denominator, and
describe how group size affects each value. (7)
Maximum score = 46
Pt1Kline5E.indd 27
Background Concepts and Self‑Test 27
Regression
Questions 1–4 concern predictors X and W and crite-
rion Y. All variables are continuous. For the same data,
the unstandardized and standardized regression equa-
tions are listed next as, respectively,
Ŷ = 2.15X + 1.30W + 2.34
and
ẑY = .59z X + .34zW
1. Interpret each unstandardized coefficient. (5)
2. Interpret each standardized coefficient. (5)
3. Which variable, X or W, contributes the most to pre-
diction, and why? (3)
4. If X, W, and Y were all measured in a different sam-
ple and new regression analyses conducted, which
estimates—unstandardized or standardized—are
preferred for comparing results for each predictor
over the samples, and why? (4)
5. What is the least squares criterion in standard
ordinary least squares (OLS) regression? What are
advantages and disadvantages? (4)
6. Describe R2 as an estimator of r2, the population
proportion of explained variance. (4)
7. What is a corrected (adjusted, shrunken, shrinkage-
corrected) R2, and what is a potential complication
in its interpretation? (6)
8. What is the problem of overfitting (overparameter-
ization) in regression analysis? (4)
9. Describe the effects of omitting a predictor that
covaries with the criterion above and beyond all the
included (measured) predictors (also called omit-
ted-variable bias or left-out variables error). (5)
Maximum score = 40
Psychometrics
All questions assume classical test theory.
1. What is the difference between reliability and valid-
ity? (4)
2. Give a general interpretation for rXX = .80. (4)
3. Given rXX = 1.0 for test–retest reliability, explain
what is wrong with this statement: Each case
obtained exactly the same score at both occasions.
(3)
3/22/2023 3:52:37 PM
 28 Concepts, Standards, and Tools
4. What is the relation between the (Cronbach’s) alpha
coefficient and the split-half reliability coefficient
for scores from the same test? (6)
5. What does the alpha coefficient measure, that is,
what determines its value? Comment on this state-
ment: Alpha = .90; thus, the items are unidimen-
sional (they measure a single factor). (5)
6. What is required to evaluate alternate-forms reli-
ability? What is a purpose for alternate forms?
What does rXX = .75 mean for immediate versus
delayed administration of the forms? (6)
7. What is the standard error of measurement (SEm)?
How does rXX affect its value? What does SEm = 7.50
mean? What is a practical application of SEm? (6)
8. Why is it a misconception that the value of the Pear-
son correlation rXY always has a range from –1.0 to
1.0? What is a risk of this myth? (4)
9. What is convergent validity versus discriminant
validity, and what is the concern about common
method variance when evaluating either type of
validity? (4)
Maximum score = 42
SCORING CRITERIA
Award 1 point for each underlined definition or phrase
mentioned in your answer. Calculate the total score
for each area and then calculate the proportions of the
maximum possible total score.
Significance Testing
1. A sampling distribution is the probability distribu-
tion for a statistic over many random samples all the
same size (N) and drawn from the same population.
2. The term SD = 15.00 is the square root of the aver-
age squared distance between the scores and the
mean,2 and it estimates the population standard
deviation, s. The quantity SE = 3.00 is the esti-
mated standard deviation in the sampling distribu-
tion of random means each based on N = 25 cases
(i.e., SE = SD/N1/2). Thus, the estimated square root
of the average squared distance between sample
means and the population mean is 3.00.
2A common but incorrect response is that SD is the average abso-
lute distance between the scores and the mean—see Huck (2016).
Pt1Kline5E.indd 28
3. If the null hypothesis is true and all distributional
assumptions are correct, the results in 3% of ran-
dom samples would be as extreme as the sample
result or even more so. Whether this result is “sig-
nificant” is unknown because no criterion level for
statistical significance was stated. In NFSA, there
is no criterion level (i.e., p values are not dichoto-
mized), so the terms “significant” or “not signifi-
cant” do not apply.
4. The quantity a is the criterion level of statistical
significance specified before the data are collected.
It is the conditional probability of rejecting the null
hypothesis over random samples when the null
hypothesis is true, or a Type I error. The value of
p is the conditional probability of a sample result
or one even more extreme assuming that the null
hypothesis is true, sampling is random, and all dis-
tributional assumptions hold.
5. Power is the probability of correctly rejecting the
null hypothesis over random samples when the
alternative hypothesis is true. Its complement, or
1 – power, is b, or the probability of failing to reject
(retaining) the null hypothesis when the alternative
hypothesis is true.3
6. Power is determined by sample size, the magnitude
of the true effect in the population, the level of a,
the reliability of the scores, whether the effect is
between-subjects or within-subjects, and whether
the test statistic is parametric or nonparametric.
7. Bigger samples, a larger population effect size,
lower levels of a (e.g., .05 instead of .01), higher
score reliability, a within-subject effect, and a para-
metric test statistic are all associated with higher
power.
8. The degrees of freedom are 48, which means that
the total sample size is N = 50, but the size of each
group is unknown.4 The mean difference is 2.25
standard errors in magnitude. Assuming random
sampling from two different populations with nor-
mal distributions and equal variances, the probabil-
ity of observing a mean difference as large as 2.25
standard errors or even greater is .029.
9. The group mean difference is 6.75, and its value is
not affected by group size. The standard error of the
3 It is just as correct to say that 1 – b = power.
4 Equal group sizes for the independent-samples t test are not
required.
3/22/2023 3:52:37 PM

mean difference is 3.00, and it estimates the stan-
dard deviation in a sampling distribution of ran-
dom mean differences, where samples in each pair
were selected from two different populations with
normal distributions and equal variances. Its value
is affected by group size: Larger groups lead to
smaller standard errors, keeping all else constant.
Regression
1. All answers for this question refer to the origi-
nal (raw score) metric of each variable: A 1-point
increase in X predicts an increase in Y of 2.15
points, while controlling for W; a 1-point increase
in W predicts an increase in Y of 1.30 points, con-
trolling for X; and the predicted score on Y is 2.34
when X = W = 0.
2. Given an increase in X of 1 standard deviation, the
expected increase in Y is .59 standard deviations,
controlling for W; and Y is predicted to increase by
.34 standard deviations, given an increase in W of
one standard deviation while we are controlling for
X. In the standardized solution, the intercept equals
zero because the mean for all variables is zero.
3. Unstandardized coefficients for X and W—respec-
tively, 2.15 and 1.30—are not directly comparable
unless the two variables have the same raw score
metric. Because both variables do have the same
metric in the standardized solution (i.e., mean = 0,
standard deviation = 1.0), their standardized coef-
ficients can be directly compared: The relative
contribution of predictor X is largest, or about 1.75
that times of W (i.e., 59/.34) in a standard deviation
scale.
4. It is the unstandardized coefficients that are gener-
ally preferred. If two independent samples differ in
their variances for the same measures, the basis for
the standardized estimates is not comparable over
samples.5 Standardized coefficients are better for
comparing the relative contributions of the predic-
5 Here is an example: Suppose that the same multiple-choice test
is administered to each of two different classes. Scores in each
class are reported as the proportion correct, but relative to the
highest raw score in each group, not the total number of items.
Although the proportions in each class are standardized and have
the same range (0–1.0), they are not directly comparable across
the classes if the highest scores in each group are unequal. Only
the raw scores (total number of items correct) would have the
same meaning over classes.
Pt1Kline5E.indd 29
Background Concepts and Self‑Test 29
tors within the same sample—see Grace and Bollen
(2005).
5. The least squares criterion defines a unique solu-
tion for the regression coefficients, including the
intercept, such that the sum of squared residuals is
minimized; that is, S (Y – Ŷ)2 is as small as possible.
This solution is the best possible solution in a par-
ticular sample, one that maximizes the value of R2.
A drawback is capitalization on chance: The “best”
solution in one sample will not be so in a different
sample unless the covariance matrices and means
are identical in both samples.
6. The statistic R2 is a positively biased estimator of r2
(i.e., R2 generally overestimates r2). Bias increases
as the sample size decreases for a fixed number of
predictors or bias increases as predictors are added
for a fixed number of cases. In very large samples,
the expected value of R2 is essentially equal to that
of r2.
7. An adjusted R2 estimates r2 by reducing the value
of R2 as a function of sample size (N) and the num-
ber of predictors (k). In general, there is greater
reduction in smaller samples than in larger samples
just as there is greater reduction as the ratio k/N
increases (e.g., predictors are added while sample
size is constant). Also, there is generally greater
reduction for smaller than for larger values of R2.
It can happen that adjusted R2 values are < 0 (nega-
tive); if so, then it is usually treated as though the
corrected R2 is zero (Cohen et al., 2003).
8. Overfitting refers to a regression equation with
too many predictors relative to the sample size. It
inflates R2 values to the point where they start to
describe random variation more so than any real
relations among variables. Overfitting also reduces
generalizability of the results to another sample,
that is, the results may not replicate because they so
heavily reflect idiosyncratic variation in the origi-
nal sample that is not repeated in other samples. For
example, if N = 100 and there are k = 99 predictors,
then the value of R2 must equal 1.0 because there is
no error variance with so many predictors, and this
is true even if the data are random numbers.
9. Omitting a relevant predictor increases standard
errors for measured predictors, reduces the value
of R2 , and biases the intercept unless the mean on
the omitted predictor is zero. If the omitted variable
is uncorrelated with all measured predictors, the
3/22/2023 3:52:37 PM
 30 Concepts, Standards, and Tools
regression coefficients for the measured predictors
are unbiased; otherwise, their coefficients will be
biased (Mauro, 1990). This bias can be positive or
negative, and it generally increases as correlations
between omitted and included predictors increase.
Psychometrics
1. Reliability concerns the precision or consistency of
scores whereas validity concerns the accuracy of
interpretations for those scores in a particular con-
text of use. Reliability is a prerequisite for valid-
ity—very inconsistent scores measure nothing and
thus have no meaningful interpretation—but reli-
ability does not guarantee validity.6 That is, just
because scores are consistent and repeatable does
not mean that they actually measure the target con-
struct.
2. A total of 1 – .80 = .20, or 20% of the observed
variation in scores is due to the type of random
error measured by the method used to derive the
coefficient (e.g., test–retest, alternate-form, inter-
rater, internal consistency). The remaining 80% is
systematic, or not due to the kind of measurement
error estimated by the coefficient. Some of that
remaining variance could be due to sources of error
not estimated by the coefficient. If so, then addi-
tional reliability studies would be needed (i.e., those
that depend on a different method).
3. Assuming there are individual differences among
cases, the pattern where each case obtains exactly
the same score at each occasion would generate the
outcome rXX = 1.0. But so would any other pattern
where the magnitudes of differences between the
cases are perfectly preserved over the occasions
even though perhaps no case obtained the same
score both times. For example, if the values in
parentheses listed next represent scores from each
case at the first and second occasions, or
(12, 16), (14, 18), (15, 19), (18, 22),
and (20, 24)
then rXX = 1.0. Thus, test–retest coefficients measure
whether individual differences are perfectly main-
tained over the two occasions, not just whether the
scores are the same.
6 Saying that reliability is a necessary but insufficient condition
for validity means the same thing.
Pt1Kline5E.indd 30
4. Both methods are based on a single administration
of the test. Both are measures of internal consis-
tency but at different levels: Alpha measures consis-
tency at the item level, but the split-half coefficient
measures consistency at the level of test halves, or
consistency in total scores over two halves of the
test corrected for overall test length. There is a
single value of the alpha coefficient, but there are
as many possible split-half coefficients as there are
ways to split test items into two halves. If all items
have the same variance, the alpha coefficient equals
the average of all possible split-half coefficients.
5. Alpha measures the combined effect of test length
and the interrelatedness of responses over test
items, which can also be expressed as the ratio of
variability in examinees’ responses across all test
items over the variance of test total scores. Increas-
ing either factor just mentioned will increase the
value of alpha. Coefficient alpha assumes that the
items are unidimensional (homogenous), so a high
value of alpha does not confirm that hypothesis. It
is possible to obtain a relatively high value of alpha
even when the items are multidimensional (Schmitt,
1996; Streiner, 2003; Urbina, 2014).
6. The alternate-forms method requires at least two
different complete versions of the test that should be
comparable in terms of length, difficulty, and con-
tent. The forms are meant to be used interchange-
ably and for the same purpose. In situations where
people are required to take the same test on multiple
occasions, administration of alternate forms may
reduce practice effects. Immediate administration,
1 – .75 = .25, or 25% of observed variation is due
to content sampling error, or idiosyncratic effects
of item wording, participant familiarity with item
content, or inadvertent selection of items from dif-
ferent domains that lead to inconsistent scores over
the forms. Delayed administration, a total of 25%
of the variance, is due to the combined effects of
content sampling error and time sampling error, but
with no additional information, the two sources of
error are confounded.
7. The statistic SEm estimates the standard deviation
of observed scores (X) around a hypothetical true
(error-free) score T, where observed scores have
a random error term, or X = T + E, such that the
average of the error component of the scores, E, is
zero and normally distributed. One context is a dis-
3/22/2023 3:52:37 PM

tribution of many repeated measures from a hypo-
thetical case around their true score, and another
is a distribution of observed scores for all cases in
the population with the same true score. The result
SEm = 7.50 says that the square root of the average
squared distance between the observed scores and
the true score is 7.50. As rXX increases, the value
of SE m decreases, and if rXX = 1.0, then SEm = 0.
The term SEm is used in constructing confidence
intervals for true scores, given observed scores for
individual cases.
8. The value of rXY can be perfect (i.e., –1.0 or 1.0) only
if the distributions of X and Y have exactly the same
shape, their association is strictly linear, and their
score reliabilities are perfect (rXX = rYY = 1.0); other-
wise, the maximum absolute value for rXY is < 1.0.
A risk is that a researcher is unable to accurately
judge the strength of association if they are unaware
of the range of possible values for rXY. For example,
what is thought to be a relatively low value of rXY,
such as .30, may actually be close to its maximum
value, given the score reliabilities for variables X
and Y (Huck, 2016).
Pt1Kline5E.indd 31
Background Concepts and Self‑Test 31
9. Convergent validity involves the hypothesis that
scores from ≥ 2 tests claimed to measure the same
construct should appreciably covary, and discrimi-
nant validity concerns the prediction that scores
from ≥ 2 tests claimed to measure different con-
structs should not appreciably covary. Method vari-
ance is due to use of a particular measurement or
source of information, and common method vari-
ance can inflate validity coefficients for two tests
based on the same method. Thus, it is better to
evaluate convergent or discriminant validity when
no two tests are based on the same measurement
method.
Well, how did you do? Don’t fret if any of your three
scores expressed as a percentage are < 50%. A score
in this range is not uncommon, especially if your sta-
tistics knowledge is rusty—see Kline (2020b). Scores
around 70% suggest at least a basic understanding, so
your review of the corresponding primer could be more
focused on areas of relative weakness. Aced it (> 90%)?
Nice, but don’t get cocky: You should still browse the
primers to check for any missing tidbits in your knowl-
edge.
3/22/2023 3:52:37 PM
 3

Steps and Reporting

Described in this chapter are the basic steps in SEM and journal article reporting standards for SEM analyses
recently published by the American Psychological Association. The first step in SEM, model specification, is
the most important step of all. This statement is true because results from all later steps assume that the model
analyzed is reasonably correct; otherwise, there may be little meaningful interpretation of the results. The
state of reporting in SEM studies is poor—no, that’s too mild—it is actually in a state of crisis. This is because
too little information about model specification, correspondence between model and data, or possible limita-
tions to conclusions is given in many—and, I would guess, unfortunately, most—published SEM studies. With-
out complete reporting, readers are unable to judge whether the findings are trustworthy, including whether
the model actually fits the data, if a model was retained. Helping you to distinguish your own reporting so
that common mistakes and omissions are avoided is thus the major goal of this chapter.

BASIC STEPS 3. Select the measures (operationalize the concepts)

Six basic steps comprise most SEM analyses, and in a
perfect world two additional optional steps would be
carried out in every analysis. Review of the basic steps
will help you to understand the relation of specification
to later steps and to recognize the utmost importance
of specification. The basic steps are actually iterative
because problems at a particular step may require a
return to an earlier step. Note that a possible outcome
is the decision to retain no model, which is a legitimate
conclusion to an SEM analysis. Remember that the goal
in SEM is not to retain a model at any cost—it is instead
to a test a theory to the best of the researcher’s ability.
Basic steps are listed next and discussed afterward.
Later chapters elaborate on specific issues at each step
beyond specification for particular SEM techniques:
1. Specify the model.
2. Evaluate whether the model is identified (if not, go
back to step 1).
and collect, prepare, and screen the data.
4. Analyze the model:
a. Evaluate the fit of the model; if it is poor, respec-
ify the model, provided that doing so is justified
by theory (skip to step 5); otherwise, retain no
model (skip to step 6).
b. Assuming that a model is retained, interpret the
parameter estimates.
c. Consider equivalent or near-equivalent models
(skip to step 6).
5. Respecify the model, which is assumed to be identi-
fied (return to step 4).
6. Report the results.
Specification
In SEM, specification involves the representation of
the researcher’s hypotheses as a series of equations or
as a model diagram (or both) that define the expected

32

Pt1Kline5E.indd 32 3/22/2023 3:52:37 PM


relations among observed or unobserved variables.
Depending on the theory, these relations could be
specified as causal or noncausal. The latter (noncausal)
refers to statistical relations that arise due to spurious
associations, such as when two variables are affected
by a common cause, or confounder, but are not caus-
ally related to each other. The model as a whole should
thus represent all the ways the variables are expected to
relate to one another.
Outcome (dependent) variables in SEM are referred
to as endogenous variables, each of which has at
least one presumed cause among other variables in
the model. Endogenous variables usually have error
terms, which represent variation that is not explained
by the causes of those variables. Given the hypotheses,
an endogenous variable could be specified as a cause
of a different endogenous variable. Endogenous vari-
ables as just described are intervening (intermediate)
variables that act as a causal link between other vari-
ables. That is, intermediates are specified as affected
by causally-prior variables, and in turn they affect other
variables further “downstream” in the causal pathway.
For reasons explained in later chapters, an interven-
ing variable is not synonymous with a mediator—also
called a mediating variable—but a mediator is always
an intervening variable. Other causes of some endog-
enous variables in the model are independent variables,
called exogenous variables in SEM, which themselves
are strictly causal. This is because whatever causes
exogenous variables are not represented in the model;
that is, their causes are unknown as far as the model is
concerned.
Whether a variable is endogenous or exogenous is
determined by the theory being tested, not by analysis.
This means that (1) the model is specified before the
data are collected, and the model represents the total
set of hypotheses to be evaluated in the analysis and
(2) the technique of SEM is not generally a method for
causal discovery so that if given a true causal model,
then SEM could be applied to estimate the directions
and magnitudes of causal effects represented in the
model. However, this is not how SEM is typically used:
Instead, a causal model is hypothesized, and the model
is fitted to sample data assuming that it is correct.
Therefore, specification is the most important step.
There are three general contexts for model specifica-
tion in SEM (Jöreskog, 1993). In a strictly confirma-
tory application, the researcher has a single model that
is either retained or rejected based on its correspon-
Pt1Kline5E.indd 33
Steps and Reporting 33
dence to the data, and that’s it. But this scope of model
testing is so narrow that it occurs only on relatively few
occasions. A second, somewhat less restrictive context
involves the testing of alternative models, and it refers
to situations in which ≥ 2 a priori models are available.
Alternative models usually include the same observed
variables but represent different patterns of hypoth-
esized relations among them. This context requires
sufficient bases to specify more than one model, such
as at least two competing theories that make different
predictions for the same variables. Another example is
in relatively new research areas when there is uncer-
tainty about expected patterns of relations. In this more
exploratory case, the researchers might test a range of
models from simpler to more complex instead of com-
paring models based on different theories (MacCallum,
1995). In either case, the particular model with the best
acceptable fit to the data may be retained, but the rest
will be rejected.
Probably the most frequent specification context in
SEM is model generation, where an initial model is
fitted to the data. If model fit is found to be unsatis-
factory, it is respecified usually by adding effects, or
parameters, to the original model, which makes the
model more complex and also generally improves its
fit. But if the initial model has acceptable fit, that model
might be simplified by dropping parameters, or making
the model simpler, which generally worsens fit. In both
scenarios just described, the respecified model is tested
again with the same data. The goal of model generation
is to “discover” a model with three attributes: It makes
theoretical sense, it is reasonably parsimonious, and it
has acceptably close correspondence to the data.
Because the initial model in model generation is
not always retained, I suggest that researchers make,
in the specification step, a list of possible modifica-
tions that would be justified according to theory. This
means to prioritize the hypotheses, representing just
the very most important ones in the model, and leave
the rest for a “backup list,” if needed. This is because
it is often necessary in SEM to respecify models (step
5), and respecification should respect the same prin-
ciples as specification. Preregistration of the analysis
plan would make a strong statement that changes to
the initial model were not made after the examin-
ing the data (Nosek et al., 2018); that is, the basis for
changing model was a priori, not post hoc in order to
get a model—at worst, any arbitrary model—to fit the
data.
3/22/2023 3:52:37 PM
 34 Concepts, Standards, and Tools
Identification
Although graphical models are useful heuristics for
organizing knowledge and representing hypotheses,
they must eventually be translated into a statistical
model that can be analyzed using a computer program.
A statistical model in SEM is a set of simultaneous
equations, where the computer must eventually derive
a single set of estimates that resolve unknown values, or
model parameters, over all equations, given the data.
Statistical models must generally respect certain
rules or restrictions. One requirement is that of iden-
tification, or whether every model parameter can be
expressed as a function of the variances, covariances,
or means in a hypothetical data matrix. Meeting this
requirement means it is possible to find a unique value
for that parameter, given a particular estimation method
and its statistical criteria. But if a parameter is not iden-
tified, any particular value might not be unique. This
means that the researcher could not distinguish true
versus false values for that parameter even with access
to population data (Bollen, 2002).
Mathematical proof for identification generally
involves solving the equations for a parameter in terms
of symbols for elements in a hypothetical data matrix,
and these symbols generally do not refer to specific
numerical values in any given sample. For example,
Dunn (2005) described a symbolic proof of the claim
that least squares estimators of regression coefficients
and intercepts minimize the sum of squared residuals
(i.e., the least squares criterion) in standard regression
analysis. Note that identification is an inherent property
of a parameter in a particular model. This means that
a parameter that is not identified remains so regardless
of both the data matrix and the sample size (N = 100,
1,000, etc.). Attempts to analyze a model with at least
one parameter that is not identified may be fruitless;
therefore, a model that is not identified should be
respecified (return to Step 1). Presented in Topic Box
3.1 is an intuitive account of model identification. You
can use an online simultaneous equations calculator to
work through the examples.1
Structural equation models are often sufficiently
complex that is impractical for applied researchers
without strong backgrounds in linear algebra to inspect
model equations or parameters for identification.
1 https://www.symbolab.com/solver/simultaneous-equations-
calculator
Pt1Kline5E.indd 34
Instead, there are graphical methods and identifica-
tion heuristics, or rules of thumb, that can determine
whether some, but not all, kinds of models are identi-
fied. There are also computer tools that analyze graphi-
cal representations of some, but not all, kinds of path
models for identification (Textor et al., 2021), which is
both convenient and less subject to error compared with
manual application of heuristics. Dealing with identi-
fication is one of the biggest challenges for newcom-
ers to SEM (Kenny & Milan, 2012); accordingly, it is a
recurrent theme throughout this book.
Suppose that a researcher specifies a structural equa-
tion model that is true to a particular theory, but the
resulting model is not identified. In this case, there is
little choice in SEM but to respecify the model so that
it is identified, but respecifying the original model can
be akin to making an intentional specification error
from the perspective of the theory. There are two basic
options: (1) Apply graphical identification methods to
determine which parameters in the original model are
identified, and then estimate only those parameters;
that is, analyze what is possible and skip the rest (i.e.,
the underidentified parameters). This first option means
that the whole model is not analyzed. (2) Respecify the
model by adding variables, such as covariates or instru-
mental variables, so that all parameters are identified
while still respecting the original theory. This second
option is thus a balancing act between identification
and fidelity to theory. But the main point is that identi-
fication should be evaluated while planning the study
and before the data are collected. Otherwise, it may
be difficult—if not impossible—to add variables to the
model to fix an identification problem, if the data are
already collected.
Measure Selection and Data Collection
The various activities for this stage—select good mea-
sures, collect the data, and screen them—are discussed
in the next chapter on data preparation and in the Psy-
chometrics Primer available on the website for this
book.
Analysis
This step involves using an SEM computer program
to conduct the analysis. Computer tools for SEM, both
commercial and those freely available, are described in
Chapter 5. Here is what takes place during this step:
3/22/2023 3:52:37 PM
 Steps and Reporting 35

TOPIC BOX 3.1

Conceptual Explanation of Identification

Consider the following formula

a+b=6 (3.1)

where a and b are unknowns that require estimates. Because there are more unknowns than formulas in
Equation 3.1, it is impossible to find a unique set of estimates. In fact, there are an infinite number of solu-
tions for (a, b), including

(4, 2), (8, –2), and (2.5, 3.5)

and so on. Thus, Equation 3.1 is not identified; specifically, it is underidentified or underdetermined,
which here signals the excess of unknowns over the number of formulas or, respectively, 2 > 1. A similar
thing happens when a computer tries to derive unique estimates for an underidentified set of equations: It
is not possible to do so, and the attempt fails.
The next example shows that having an equal number of unknowns and formulas does not guarantee
identification. Consider the following set of formulas:

a+b=6 (3.2)
3a + 3b = 18

Although Equation 3.2 has 2 unknowns and 2 formulas, it does not have a unique solution. Actually, an
infinite number of solutions satisfy Equation 3.2, such as (4, 2), and so on. This happens due to an inherent
characteristic: The second formula in Equation 3.2 is linearly dependent on the first formula (i.e., multiply
the first formula by the constant 3), so it cannot narrow the range of solutions that satisfy the first formula.
Thus, Equation 3.2 is underidentified because the effective number of formulas is 1, not 2.
Now consider the following set of 2 formulas with 2 unknowns where the second formula is not lin-
early dependent on the first:

a+b=6 (3.3)
2a + b = 10

Equation 3.3 has a unique solution; it is (4, 2), and none other. Thus, Equation 3.3 is just-­identified,
just-­determined, or saturated with equal numbers of unknowns and distinct (i.e., not wholly depen-
dent) formulas, or 2 for each. That unique solution (4, 2) also perfectly reproduces the constants in Equa-
tion 3.3 (6, 10).
Let’s see what happens when there are more formulas than unknowns. Consider the following set of
formulas with 3 equations and 2 unknowns:

a+b=6 (3.4)
2a + b = 10
3a + b = 12
(continued)

Pt1Kline5E.indd 35 3/22/2023 3:52:37 PM

 36 Concepts, Standards, and Tools

There is no single solution that satisfies all three formulas in Equation 3.4. For example, the solution (4, 2)
works only for the first two formulas in Equation 3.4, and the solution (2, 6) works only for the last two
formulas. But there is a way to find a unique solution: Impose a statistical criterion that leads to an overi-
dentified or overdetermined set of equations with more formulas than unknowns. An example for
Equation 3.4 is the least square criterion from regression analysis but with no constant (intercept) in the
prediction equation. Expressed in words:

Find values of a and b in Equation 3.4 that yield total scores such that the sum of squared
differences between the constants 6, 10, and 12 and these total scores is as small as possible.

Applying the criterion just stated to the estimation of a and b yields a solution that not only gives the
smallest possible difference (.67) but that also is unique, or (3, 3.33). This solution does not perfectly
reproduce the constants in Equation 3.4 (6, 10, 12): the total scores generated by the solution just stated
are 6.33, 9.33, and 12.33, but no other solution comes closer, given the least squares criterion applied
in this example.
Definitions of underidentified, just-­identified, and overidentified models in SEM are elaborated on in
later chapters, but some implications can be stated now: An unidentified model has at least one param-
eter that cannot be algebraically expressed as a unique function of the covariances, variances, or means
among the observed variables. Such models may have other parameters that are identified, and these
identified parameters could potentially be estimated with sample data. But a model with ≥ 1 underidenti-
fied parameter(s) cannot be analyzed as a whole unless it is respecified to reduce the excess number of
unknowns (parameters) or certain constraints are imposed by the researcher in the analysis; otherwise,
underidentified models are too complex to be analyzed with available data.
The unique solution for a just-­identified model can perfectly reproduce the data, but that feat is not
very impressive because model and data are equally complex. Any respecification of a just-­identified
model (within requirements about identification) will also perfectly explain the same data even though
such respecifications oftenmake opposing causal claims (i.e., the models are equivalent). Also, a justified-­
identified model will perfectly fit any arbitrary sample data matrix for the same variables. For all these
reasons, (1) just-­identified models are not discomfirmable, and (2) such models usually have little scientific
merit (MacCallum, 1995), but Pearl (2009) discussed some exceptions.
Overidentified models typically do not perfectly explain the data because such models are simpler
than the data. The possibility for imperfect fit makes such models discomfirmable, and a question in the
analysis is whether the degree of model–data discrepancy warrants rejecting the model (i.e., go to the
respecification step). The principle of disconfirmability also implies a preference for models that are
not highly parameterized, or have so many unknowns relative to the data that they can hardly “disagree”
with those data (MacCallum, 1995). Box (1976, p. 792) put it like this. “Just as the ability to devise simple
but evocative models is the signature of the great scientist so overelaboration and overparameterization
is often the mark of mediocrity.” Other aspects of parameter identification in SEM are elaborated in later
chapters.

Pt1Kline5E.indd 36 3/22/2023 3:52:37 PM


1. Evaluate model fit, which means determine how well
the model explains the data. Perhaps more often than
not, an initial model does not fit the data very well.
Not if, but when this happens to you, skip the rest of
this step and consider the question, “Can a respecifi-
cation of the original model be justified, given rel-
evant theory and results of prior empirical studies?”
2. Assuming the answer is “yes” and given satisfac-
tory fit of the respecified model, then interpret the
parameter estimates.
3. Next, consider equivalent or near-equivalent mod-
els. Recall that equivalent models explain the same
data just as well as the researcher’s preferred model
but with a contradictory pattern of causal effects
among the same variables. For a given model,
there may be many—and in some cases infinitely
many—equivalent versions. Thus, the researcher
should explain why their favorite model should not
be rejected in favor of equivalent ones. There may
also be near-equivalent models that fit the same
data nearly as well as the researcher’s preferred
model, but not exactly so. Near-equivalent models
are often just as critical a validity threat as equiva-
lent models, if not even more so.
When testing alternative models, it is not the model
with the best relative fit to the data that would be auto-
matically retained. This is because the best-fitting
model among alternatives may not itself have accept-
able fit to the data when viewed from a more absolute
standard. Also, the researcher in this context should
not be solely concerned with model fit. This is because
the parameter estimates for a candidate model should
also make theoretical sense. As noted by MacCallum
(1995), a model that fits well but generates nonsensi-
cal parameter estimates is of little scientific value. This
principle is also true in strictly confirmatory and model
generation applications of SEM.
Respecification
A researcher usually arrives at this step because the fit
of their initial model is poor. In the context of model
generation, now is the time to refer to that backup list of
theoretically acceptable changes I suggested when you
specified the initial model. If there is no such list—or
if the researcher has exhausted their backup list and yet
the respecified model still has poor fit—it may be bet-
Pt1Kline5E.indd 37
Steps and Reporting 37
ter (and more honest, too) to retain no model (Hayduk,
2014). This is because there are risks to respecifying
models based more on improving fit to the data in a
particular sample than on substantive considerations
(MacCallum, 1995):
1. Data-driven respecification can so greatly capi-
talize on sampling error that any retained model
is unlikely to replicate, especially when complex
models are analyzed in small samples.
2. Estimates of some parameters in purely data-driven
respecified models may have little or no substantive
meaning.
3. Because the model is evaluated based on the same
data used to modify it, any retained model should
be validated with data from a new sample; that is,
evidence for model validation should come from a
different sample. Thus, a respecified model should
not be treated as confirmed without fitting it to new
data.
MacCallum (1995) noted that it would be appropriate
for journal editors to reject manuscripts for SEM stud-
ies based on model generation if the concerns just listed
are not addressed. This advice strikes me as reasonable
in that the direct acknowledgment of study limitations
is part of reporting, which is considered next.
Reporting
The last basic step is an accurate and complete descrip-
tion of the analysis in a written report. The fact that so
many published articles that concern SEM are flawed
in this regard was discussed earlier. These blatant
shortcomings are surprising considering that there have
been, for some time, published guidelines for reporting
SEM results. For example, recent APA standards for
SEM analyses (Appelbaum et al., 2018) described later
in this chapter were based on earlier SEM standards
for the journal Archives of Scientific Psychology by
Hoyle and Isherwood (2013). McDonald and Ho (2002),
Jackson et al. (2009), and Boomsma et al. (2012) dis-
cussed general principles for reporting results in SEM,
and there are works on how to apply SEM in various
disciplines, including travel and tourism, social and
administrative pharmacy, and marketing, among oth-
ers (respectively, Nunkoo et al., 2013; Schreiber, 2008;
Richter et al., 2016).
3/22/2023 3:52:37 PM
 38 Concepts, Standards, and Tools
OPTIONAL STEPS
Two optional steps could be added to the six basic ones
just described (i.e., steps 1–6):
7. Replicate the results.
8. Apply the results.
The requirement for large samples in SEM complicates
replication. This is because it may be difficult enough
to collect a single sample large enough for SEM analy-
ses much less collect twice as many cases or so to also
have a replication sample. Poor reporting of results in
SEM studies may also contribute to the problem: If
the sample, measures, model, and analysis are not all
clearly described, replication efforts could be stymied.
That fact that many, and perhaps most, claims in SEM
studies are made with little evidence for their general-
izability outside the original sample(s) should give us
all pause, though.
A bright spot about replication in SEM concerns the
evaluation of measurement invariance, or whether a set
of indicators measures the same theoretical concepts
over different populations, times, methods of adminis-
tration, and so on. Measurement invariance can be esti-
mated in CFA by testing whether a measurement model
has similar fit over data collected from samples drawn
from different populations, such as women and men,
among other variations described later in this book
(Chapter 22). There are now many CFA studies of mea-
surement invariance (Dong & Dumas, 2020), and most
involve replication over different samples collected by
study authors.
There are fewer examples of replication over dif-
ferent samples collected by different researchers, or
external replication. One is Tunca (2019), who con-
ducted a pre-registered replication of an earlier SEM
study by Hollebeek et al. (2014). The original analysis
involved a statistical model of customer brand engage-
ment conceptualized as three dimensions—cognitive
processing, brand-related affection, and behavioral
interaction—presumed to affect self–brand connection
and brand usage intent (Hollebeek et al., 2014, p. 157).
Tunca (2019) fitted the same model to data from a new
sample. Although some of the original findings were
consistent with the original and replication samples,
evidence for the discriminant validity of customer
brand engagement dimensions was more problematic.
Pt1Kline5E.indd 38
Gender differences in consumer brand engagement
were noted by Tunca (2019) as a possible explanation
for differences in results compared with those reported
by Hollebeek et al. (2014).
Issues concerning the external replication of SEM
studies are summarized next; see Porte (2012) for more
information about general issues in replication: Recall
that unstandardized estimates of the same param-
eter should generally be compared over independent
samples, not standardized results. How to simultane-
ously fit the same structural equation model to the data
from multiple samples is described in Chapter 12. This
process involves specifying equality constraints that
should be imposed on unstandardized estimates for the
same parameter over different samples as a way to test
for group differences. This method (among others) is
used in CFA studies of measurement invariance, and
it could also be applied by a researcher who collects
new data to test the same model from a prior study by a
different researcher. There are formal measures of the
similarity of factor solutions over different samples,
such as the Tucker coefficient of congruence, a stan-
dardized measure of proportionality in factor loadings
over different groups, that can be applied in CFA—see
Moss et al. (2015) for an example.
There are hundreds, probably thousands, of studies
where SEM has been applied to test theories (Thelwall
& Wilson, 2016). One has to look much harder to find
practical applications of SEM, but such studies exist.
For example, the latest revision of the Stanford–Binet
Intelligence Scales, Fifth Edition (Roid, 2003) was
influenced by results of factor analytic studies, includ-
ing CFA findings, about its internal structure (DiSte-
fano & Dombrowski, 2006). Rebueno et al. (2017) used
the technique of path analysis to evaluate the attributes
of a pre-graduate clinical training program in a ran-
domly selected sample of nursing students. Verdam et
al. (2017) described effect sizes that can be computed
in SEM analyses on different types of response shifts
in reported outcomes over time among patients in
treatment. Effect sizes are derived from decomposing
observed change into elements that include three dif-
ferent types of response shift—including (1) change in
patients’ internal standards (recalibration), (2) change
in patients’ values about the importance of particu-
lar outcomes (reprioritization), and (3) change in the
meaning of the outcome to patients (reconceptualiza-
tion)—versus estimated true change in the underlying
construct.
3/22/2023 3:52:38 PM

REPORTING STANDARDS
Reporting standards are envisioned as sets of best prac-
tices established by authority figures, such as experi-
enced researchers, journal editors (who also tend to
be experienced researchers), and methodologists, pre-
sented with a rationale that also allows for discretion in
their application (Cooper, 2011). They are not intended
as requirements, or mandatory practices that must be
adopted by all, but they are also not merely recommen-
dations, or endorsements of good reporting practices
with no obligation to actually carry them out. Instead,
the goal of reporting standards is to improve the quality
of journal articles through clear, accurate, and trans-
parent reporting, while simultaneously not interfering
too much with researcher creativity and the normal
workings of science (Appelbaum et al., 2018). The hope
is that better reporting will help readers to more clearly
understand the bases for claims made in articles and
facilitate study replication.
A drawback to reporting standards is that there is not
always consensus about exactly what to report in cer-
tain research areas. This is true in SEM in which there
is longstanding debate about the adequacy of various
statistics about model fit, especially about the useful-
ness of significance tests of global model fit versus
continuous measures of fit (Vernon & Eysenck, 2007).
When experts in the field disagree, no reporting stan-
dards can resolve the issue. The approach taken in this
book is to (1) address directly the areas of disagreement
beginning especially in Chapter 10 about model testing
and (2) offer what I believe are principled arguments for
best practices, including reporting. I also believe that
the existence of reporting standards for SEM helps to
focus the conversation among both experienced practi-
tioners and students.
The original APA journal article reporting standards
(JARS) did not include SEM analyses (APA Publica-
tions and Communications Board Working Group on
Journal Article Reporting Standards, 2008). Hoyle and
Isherwood (2013) developed SEM reporting standards
for the journal Archives of Scientific Psychology in the
form of questions to be addressed by authors or review-
ers. Their guidelines were reworded as statements,
slightly edited for consistency, and then incorporated
into the revised APA standards, JARS-Quant for quan-
titative studies (Appelbaum et al., 2018). Hundreds
of reporting standards for health-related research are
described on the Enhancing the Quality and Transpar-
Pt1Kline5E.indd 39
Steps and Reporting 39
ency of Health Research (EQUATOR) network, which
offers a searchable database.2
Summarized in Table 3.1 are the APA JARS-Quant
standards for SEM analyses. Authors should explain
the theoretical bases for model specification, includ-
ing directionalities of presumed effects. The analysis
context—for example, whether an initial model will be
respecified (model generation)—should also be stated.
Describe the model in complete detail, including its
parameters, associations between observed and latent
variables, and whether means were analyzed along
with covariances. Verify that the model is identified,
and explicitly tabulate the model degrees of freedom
(df M). How to do so for different kinds of models is
described in later chapters, but this basic reporting is
problematic in too many studies: Cortina et al. (2017)
reviewed nearly 800 models described in 75 published
SEM studies. They found that the information needed
to calculate df M was reported about 75% of the time, but
reported df M was matched by the article information
only 62% of the time. Shah and Goldstein (2006) found
a similar problem: In 31 out of a total of 143 reviewed
articles detailing SEM studies, the model described in
the text failed to match the statistical results presented
in tables or text. Such discrepancies raise questions
about the model described versus the model actually
analyzed.
State whether samples are empirical or generated,
such as in a computer simulation study, and describe
any loss of samples due to technical problems, such as
inadmissible solutions (i.e., illogical estimates) or fail-
ure of iterative estimation to converge—see Table 3.1.
Give the rationale for the sample size, including the
parameters for a power analysis. Power analysis and
other methods to specify target sample size in SEM
are described in Chapter 10. Describe all measures in
detail, including psychometrics, for scores from psy-
chological tests analyzed in the researcher’s sample.
Describe the data screening in complete detail,
including (1) the extent, patterns, and assumptions about
missing data; (2) the status of distributional assump-
tions; (3) whether any other problems, such as outliers,
were detected; and (4) corrective actions taken, if any
(Table 3.1). A strong demonstration of transparency
occurs when both the syntax for the analysis and the
data file are accessible to other researchers. Access to
original data is part of the open-science movement in
2 https://www.equator-network.org/
3/22/2023 3:52:38 PM
 TABLE 3.1. Summary of JARS-Quant Reporting Standards for Structural Equation
Modeling Analyses
Section Topic and description
Abstract If a model is retained, report values of at least two global fit statistics with brief comment on local fit, and
state whether the initial model was respecified.
Introduction Justify directionality assumptions (e.g., X → Y instead of the reverse).
State whether respecification is planned, if the initial model is rejected.
Specification Describe context (i.e., model generation, model comparison, strictly confirmatory).
Give a full account for model specification, including latent variables and indicators (i.e., measurement
model), status of parameters (i.e., free, fixed, or constrained), or how the model or analysis deals with
score non-independence.
Describe the mean structure, if means are analyzed.
Justify error correlations in the model, if present.
Verify that tested models are identified.
Explicitly tabulate numbers of observations, free parameters, and dfM.
Methods State whether data were simulated or collected from actual units of study (i.e., cases).
Describe measures (e.g., taken from a single vs. multiple questionnaires), specify whether they are items or
scales (i.e., total scores), and report psychometrics.
Outline how sample size was determined (e.g., power analysis, accuracy in parameter estimation, resource
limitations), give details for power analysis (e.g., H0, H1, a, effect size).
Indicate whether results or samples were discarded due to nonconvergence or inadmissible solutions.
Data screening Describe data loss patterns (i.e., MCAR, MAR, MNAR) and corrective actions (e.g., single or multiple
and summary imputation, FIML).
State distributional assumptions (e.g., multivariate normality), report evidence that assumptions were met or
describe actions taken to address violations.
Report sufficient summary statistics that allow secondary analysis (e.g., covariance matrix), or make the raw
data file available.
Estimation State the computer tool (including version) used in the analysis and the estimation method used, or make the
syntax available.
Describe default criteria that were changed to obtain a converged solution.
Report whether the solution is admissible (e.g., negative error variances) and describe corrective actions.
Model fit Interpret values of global fit statistics according to evidence-based criteria.
Describe local fit, or the residuals (e.g., standardized, normalized, correlation).
Explain the decision to retain or reject any model.
State criteria for evaluating parameter estimates, including whether results are compared over groups.
If comparing alternative models, state criteria for selecting a preferred model.
Respecification Indicate whether respecifications were a priori or post hoc (i.e., arrived at before or after examining the data).
Give a theoretical rationale for any parameters fixed or freed in respecification.
Estimates Report unstandardized and standardized estimates with standard errors for all free parameters, if possible.
Report standardized and unstandardized indirect effects with standard errors, outline analysis strategy.
Report estimates of interactions with standard errors, describe follow-up analyses.
Discussion Summarize changes to the initial model and rationale, if a model is retained.
Justify the preference for retained models over equivalent models or near equivalent models that explain the
same data just as well or nearly so.

Note. JARS-Quant, journal article reporting standards for quantitative studies; MCAR, missing completely at random; MAR, missing at random;
MNAR, missing not at random; FIML, full information maximum likelihood; dfM, model degrees of freedom. Underlining emphases the most
essential parts for each standard. From “Journal Article Reporting Standards for Quantitative Research in Psychology: The APA Publications and
Communications Board Task Force Report,” by Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., & Rao, S. M. (2018).
Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board Task Force report.
American Psychologist, 73(1), 3–25. https://doi.org/10.1037/amp0000191. Copyright © 2018 American Psychological Association. All rights re-
served.

40

Pt1Kline5E.indd 40 3/22/2023 3:52:38 PM


which all components of research, including data, meth-
ods, peer review, and publications, are freely available.
Increasing numbers of journals encourage the sharing
of the data, and some journals award badges to authors
for open-science practices (Chambers, 2018).
A fact of life in SEM is that computer analyses can,
and sometimes do, go wrong. How to deal with vari-
ous technical problems in the analysis are addressed
throughout the book, but the nature of such problems
and corrective steps taken to resolve them should be
reported—see Table 3.1. Reassure readers that the solu-
tion is admissible, that is, the estimates do not include
anomalous results that would invalidate the solution.
Model fit should be described at two different levels,
global and local. Global fit concerns the overall or aver-
age correspondence between the model and the sample
data matrix. Just as averages do not indicate variability,
it can and does happen in SEM that models with appar-
ently satisfactory global fit can have problematic local
fit, which is measured by residuals computed for every
pair of observed variables in the model. Residuals are
the differences between observed (sample) and pre-
dicted (by the model) covariances or correlations, and
as absolute residuals increase, local fit becomes worse
for those pairs of variables. Evidence for problematic
local fit goes against retaining the model.
The analogy in regression analysis is the differ-
ence between R2, or overall explanatory power (global
fit), and regression residuals, or differences between
observed and predicted scores at the case level, not
at the variable level as in SEM. Aberrant patterns of
regression residuals, such as severely nonnormal distri-
butions or heteroscedasticity, indicate a problem even if
the value of R2 is reasonably high—see Fox (2020). Just
as reports about regression analyses without descrip-
tion of the residuals are incomplete, so too are reports
in SEM in which only global fit is described. This is a
common shortcoming in many, if not most, published
SEM studies. For example, only 17 out of 144, or 12%,
of SEM studies published in organizational or manage-
ment research journals reviewed by Zhang et al (2021)
contained information about local fit. The remedy is
straightforward: Present a matrix of residuals in the
article, or at least describe the pattern of residuals in
the article and make the residuals available in supple-
mentary materials. Without the details about local fit,
readers cannot understand the whole picture about
model–data correspondence.
Explain the basis for a decision to either reject or
retain a model. That is, state the statistical criteria,
Pt1Kline5E.indd 41
Steps and Reporting 41
degree of correspondence of empirical estimates with
theoretical predictions, or other reasons for the decision
(Table 3.1). If a model is retained, a common mistake
is to report only the standardized solution. Although
standardized estimates are easier to interpret, they are
generally less useful for comparing results for the same
model and variables but fitted to data in a different sam-
ple, especially if samples, such as men versus women,
differ in their variances (Grace & Bollen, 2005). Thus,
also report the unstandardized solution with stan-
dard errors. If analyzing indirect effects or interaction
effects, then describe the analysis strategy, including
follow-up analyses. Another common mistake is the
failure to acknowledge the existence of equivalent ver-
sions of any retained model with an explanation about
why the researcher’s version would be preferred.
As a reviewer of SEM studies submitted to research
journals, I see many examples of deficient, poor report-
ing, so much so that over the years I developed a set of
boilerplates, or standardized text, that I can adapt to a
particular manuscript. Presented in Topic Box 3.2 are
examples of some of my review templates. I hope that you
can learn a few things about better, more complete report-
ing from these cautionary tales on inadequate reporting.
REPORTING EXAMPLE
Several colleagues and I wrote an SEM study with the
(at the time) upcoming APA reporting standards for
SEM in mind. The study, data, research aim, and model
are introduced next. Do not worry at this you point if
you do not understand everything about the model or
its specification because we focus on issues related to
reporting next. Analysis results for this example are
described in Chapter 15, so you’ll get to see this model
again along with the data described here.
In a sample of 193 patients with first-episode psy-
chosis (FEP) (schizophrenia, schizoaffective disorder)
or recurrent multiple-episode psychosis (MEP), Sauvé
et al. (2019) administered the CogState Schizophrenia
Battery (CSB) (Pietrzak et al., 2009), a computerized
test of cognitive ability intended to measure domains
affected by psychosis (e.g., working memory, attention-
vigilance, social reasoning), and the Scale to Assess
Unawareness of Mental Disorder (SUMD) (Amador
et al., 1993), an interview-based measure of the rela-
tive absence of understanding about one’s own mental
condition, also called lack of illness insight or anosog-
nosia. Higher scores on the SUMD indicate greater
3/22/2023 3:52:38 PM
 42 Concepts, Standards, and Tools

TOPIC BOX 3.2

Review Boilerplates for Common Problems in SEM Manuscripts

I have reviewed hundreds of manuscripts of SEM studies for about 50 different journals or publishers. After
seeing repetitions of similar problems in reporting, I wrote a set of more-or-less standardized comments
that could be adapted to the specifics of a particular work. These boilerplates cover incomplete reporting
about model fit, the interpretation of values of global fit statistics in ways that are not supported by recent
evidence, or the lack of a rationale for retaining a model (see Table 3.1), among other problems. Some of
these comments refer to global fit statistics that are defined in Chapter 10 on model testing and indexing,
but issues in their misuse are raised next:

1. Failure to describe local fit. It is inappropriate to base model evaluation solely on values of global
fit statistics without also considering local fit, or the residuals, which provide additional details
about model fit. It can and does happen that models can generate favorable values of global fit
statistics even though there is evidence for appreciable model–data discrepancy at the level of the
residuals. Recent reporting standards for SEM call on authors to describe both global and local
fit—see Appelbaum et al. (2018), Greiff and Heene (2017), and Vernon and Eysenck (2007) for
more information.
2. Failure to report unstandardized estimates. It is a mistake to omit the reporting of unstandardized
parameter estimates, which is generally the basis for comparing results from the same model and
variables over different samples—­see Grace and Bollen (2005). Please also report the unstan-
dardized estimates with their standard errors.
3. Incorrect model degrees of freedom. The reported value of df M is impossible, given the variables
and effects represented in the model diagram. I suspect something is wrong here, ranging from
a simple typographical error to something more serious, including the analysis of a model that
differs from the model described in the manuscript. Please explicitly tally the numbers of observa-
tions, freely estimated model parameters, and df M. Next, resolve any discrepancies in the presen-
tation.
4. Ignored failed significance tests for the whole model. The model failed the chi-­square test, which
signals covariance evidence against the model, but this outcome is discounted by the incorrect
statement that the chi-­square test is “biased” by sample size. There are two problems here: The
sample size in this study is not large for SEM analyses, and the chi-­square test is affected by sam-
ple size only when the model is incorrect. This problem is compounded by the failure to describe
the residuals, which could indicate poor local fit for certain pairs of variables—­see Hayduk (2014)
and Ropovik (2015).
5. No rationale for the decision to retain a model. Why the respecified model is retained was not
explained. Instead, the author(s) report(s) values of global fit statistics with no interpretation. That
is, how do the author(s) get from these results in the text of the Results section, or

c2 (272) = 587.52, p < .01, CFI = .97, TLI = .946, RMSEA = .057

to the unjustified conclusion that the model has “acceptable” or “excellent” fit? There is no con-
nection, argument, or logic that links the two. It is possible that the model does not actually fit the
data at level of the residuals but, as mentioned, the reader is told nothing about this area.

Pt1Kline5E.indd 42 3/22/2023 3:52:38 PM

 Steps and Reporting 43

anosognosia. The main goal of Sauvé et al. (2019) was
to estimate the association between level of general
cognitive ability and symptom unawareness where both
concepts were modeled as latent variables. The context
is model generation.
Presented in Figure 3.1 is the final model retained by
Sauvé et al. (2019). A total of six tasks from the CSB
are represented as indicators of a general cognitive
ability factor. Each measured variable is represented in
the figure with a rectangle, a standard graphical con-
vention in SEM. The lines with single arrowheads that
point from the cognitive ability factor to the indicators
signal the assumption of reflective measurement, or the
hypothesis that general ability as a theoretical variable
affects task performance. The factor as a theoretical—
and thus unmeasured—variable is represented in the
figure with an oval, another oft-seen graphical symbol.
Each indicator has an error term designated in the fig-
ure by a line with a single arrowhead oriented at a 45°
angle that points to that indicator, and the error term
represents the unique variation in each indicator, such
as that due to measurement error, not explained by their
common ability factor.
The errors for two pairs of indicators are connected
by the symbol for a covariation, or curved lines with
arrowheads at each end. These symbols represent cor-
related errors, or the hypothesis that the correspond-
ing indicators share something unique to that pair. For
instance, the International Shopping List Test is a list
of items read aloud to examinees. Two scores from this
task were analyzed, immediate versus delayed recall of
list items or, respectively ISL and ISLR in Figure 3.1.
Because they come from the same task, indicators ISL
and ISLR may covary even after controlling for their
common ability factor. This same could be true for the
two scores from the Groton Maze Chase Test, GML
and GMR (see Figure 3.1).
The SUMD is represented in Figure 3.1 as the single
indicator of a symptom unawareness factor. This speci-
fication acknowledges that (1) an observed variable is
not identical to a theoretical concept and (2) scores on
the SUMD are assumed to be affected by measurement
error (i.e., the score reliability coefficient is < 1.0). The
constant in the figure that appears next to the symbol for
the error term of the SUMD, .360, is a value related to
empirical reliability coefficients reported in the litera-
ture for this measure; how the particular value of “.360”
was computed for the SUMD is explained in Chapter
15. The line with the single arrowhead that connects
the factors in the figure represents the presumed causal

ISL .360

ISLR SUMD
1
1
GML
Cognitive Symptom
Unaware
GMR

OCL

CPAL

FIGURE 3.1. Final model of cognitive capacity and symptom unawareness. ISL, International Shopping List; ISLR, Interna-
tional Shopping List Immediate Recall; GML, Groton Maze Learning task; GMR: Groton Maze Learning task Delayed Recall;
OCL, One-Card Learning task; CPAL, Continuous Paired Associate Learning task; SUMD, Scale to Assess Unawareness of
Mental Disorder. From “Cognitive Capacity Similarly Predicts Insight into Symptoms in First- and Multiple-Episode Psychosis,”
by G. Sauvé et al., 2019, Schizophrenia Research, 206, p. 239. Copyright © 2019 Elsevier B.V. Adapted with permission.

Pt1Kline5E.indd 43 3/22/2023 3:52:38 PM

 44 Concepts, Standards, and Tools
effect of cognitive ability on symptom unawareness. Its
value is estimated controlling for measurement error
not only in the SUMD variable but also in all six indi-
cators of the cognitive ability factor. The outcome vari-
able in this analysis, the symptom unawareness factor,
itself has an error term called a disturbance that repre-
sents variation not explained by cognitive ability—see
Figure 3.1.
The features of reporting in Sauvé et al. (2019) that
respected SEM reporting standards are listed next.
Some of these details are reported in the main text;
others are available in the supplementary materials for
the article:
1. The methodology for preparing the data for analy-
sis under the assumption of normality is described,
including normalizing transformations applied to
scores from individual cognitive tasks. Distribu-
tional characteristics are verified.
2. The rules by which the initial single-factor mea-
surement model for cognitive capacity was respeci-
fied are stated. Briefly, each indicator was required
to share at least 30% of its variance with the com-
mon factor, and it is predicted that error terms for
two scores from the same task, such as immediate
versus delayed recall of the same stimuli (e.g., ISL,
ISLR in Figure 3.1), might covary.
3. The syntax, data, and complete output files for all
SEM analyses in lavaan are available to readers.
Thus, readers can reproduce the original analyses
or fit models to the same data not considered by
Sauvé et al. (2019).
6. The fact that all solutions are admissible is reported.
Model fit is described at both the global and local
levels. Full matrices of residuals are available so
readers have full access to complete information
about fit.
5. Unstandardized estimates with standard errors are
reported for all model parameters, and standard-
ized estimates are reported, too.
6. An equivalent model is directly acknowledged.
Other aspects of reporting by Sauvé et al. (2019) are
not ideal. For example, we did not preregister the analy-
sis plan. The model degrees of freedom are not explic-
itly tabulated (they are df M = 12 for Figure 3.1), and
the sample size was not determined based on a priori
considerations, such as power analysis. The sample size
Pt1Kline5E.indd 44
is relatively small for SEM, N = 193, but the population
base rate of psychotic disorders is relatively low, about
1–2% or so, so even this sample size is reasonably large
among comparable studies of insight in psychotic dis-
orders (e.g., Phahladira et al., 2019). Briefly, the results
indicated that cognitive capacity explains about 6%
of the variation in symptom unawareness and that, as
expected, lower levels of cognitive ability predict less
illness awareness. Follow-up regression analyses indi-
cated that the relation between cognitive capacity and
symptom unawareness did not vary appreciably over
patients with first-time versus multiple-episode psycho-
sis.
SUMMARY
Specification is the most important step in SEM because
the quality of the ideas that underlie the model affects
all subsequent steps, including the analysis phase. Mod-
els should be evaluated for identification, or for whether
model parameters can be expressed as functions of the
elements in a data matrix, in the study planning stage.
This is because adding variables to the model is one
way to rectify underidentified causal effects, but doing
so may be difficult if the data are already collected.
The context of model generation is the most common
in SEM studies, and respecification should be guided
by the same principles as the specification of the origi-
nal model. This means that changes to an initial model
require substantive justification; otherwise, a risk is that
respecification will generate a model that fits the data in
a particular sample but does not replicate. The state of
reporting in too many SEM studies is deficient such that
readers are not given enough information about model
specification, respecification, or fit to the data to fully
comprehend the findings. For example, both global fit
and local fit, or the residuals, should be described in
written reports. The existence of reporting standards
for SEM analyses should help authors to write better,
more complete summaries of SEM analyses. The next
chapter covers preparation of the data.
LEARN MORE
Recommendations for reporting SEM results as part of JARS-
Quant reporting standards by the APA are described in
Appelbaum et al. (2018), the classic work by MacCallum
(1995) about model specification offers helpful advice, and
3/22/2023 3:52:38 PM

Schreiber (2008) describes the information that should be
reported in SEM analyses in clear, accessible terms.
Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E.,
Nezu, A. M., & Rao, S. M. (2018). Journal article report-
ing standards for quantitative research in psychology:
The APA Publications and Communications Board Task
Force report. American Psychologist, 73(1), 3–25.
Pt1Kline5E.indd 45
Steps and Reporting 45
MacCallum, R. C. (1995). Model specification: Procedures,
strategies, and related issues. In R. H. Hoyle (Ed.), Struc-
tural equation modeling: Concepts, issues, and applica-
tions (pp. 16–36). Sage.
Schreiber, J. B. (2008). Core reporting practices in struc-
tural equation modeling. Research in Social and Admin-
istrative Pharmacy, 4(2), 83–97.
3/22/2023 3:52:38 PM
 4

Data Preparation

Besides good ideas (hypotheses), effective SEM practice requires careful data preparation and screening.
Just as in other types of multivariate analyses, data preparation is critical in SEM for three reasons: First,
it is easy to make a mistake entering data into computer files. Second, some estimation methods in SEM
make specific distributional assumptions about the data. These assumptions must be taken seriously because
violation of them could result in bias. Third, data-related problems can make SEM computer tools fail to
yield a logical solution. A researcher who has not carefully screened the data could mistakenly believe
that the model is at fault, and confusion ensues. How to deal with missing data is a problem in many, if not
most, empirical studies. Classical (i.e., obsolete) methods for handling incomplete data sets are contrasted
with modern techniques such as multiple imputation, for which a tutorial is offered for readers who are less
familiar with the method. Other data screening issues addressed in this chapter include outliers, extreme
collinearity, and distributional characteristics. We begin with review of basic options for inputting data to
SEM computer tools.

FORMS OF THE INPUT DATA
Most primary researchers—those who conduct origi-
nal studies—analyze raw data files, but sometimes the
raw data themselves are not required. For example,
when analyzing continuous outcomes with methods
that assume normal distributions, a matrix of summary
statistics can be input to an SEM computer tool instead
of the raw data. In fact, you can replicate most of the
analyses described in this book using the matrix sum-
maries that accompany them. This is a great way to
learn because you can make mistakes using someone’s
data before analyzing your own. Some journal articles
about the results of SEM analyses contain enough
information, such as correlations, standard deviations,
and means, to create a matrix summary, which can then
be submitted to a computer program for analysis. Thus,
readers of these works can, with no access to the raw
data, replicate the original analyses or estimate alterna-
tive models not considered in the original work (i.e.,
conduct a secondary analysis).
Most SEM computer tools accept either a raw data
file or a matrix summary. If raw data are analyzed, the
computer will create its own matrix, which is then ana-
lyzed. Consider the issues listed next when choosing
between raw data and a matrix summary as program
input:
1. Certain kinds of analyses require raw data files.
One situation is when distributions for continuous out-
comes are severely nonnormal and the data are ana-
lyzed with a method that assumes normality, but stan-
dard errors and model test statistics are computed that
adjust for nonnormality. A second situation involves
missing data. Both classical and modern techniques for
missing data require the analysis of raw data files. The

46

Pt1Kline5E.indd 46 3/22/2023 3:52:38 PM

 Data Preparation 47

third case is when outcome variables are categorical, TABLE 4.1. Raw Data and Matrix Summaries
either unordered (nominal) or ordered (ordinal). Such in Lower Diagonal Form
outcomes can be analyzed in SEM, but raw data files X W Y
are generally needed.
Raw scores
2. Matrix input offers a potential economy over raw
3 65 24
data files. Suppose that 1,000 cases are measured on 10 8 50 20
continuous variables. The data file may be 1,000 lines 10 40 22
(or more) in length, but a matrix summary for the same 15 70 32
data might be only 10 lines long. 19 75 27
3. Sometimes a researcher might “make up” a data
Covariances
matrix using theory or results from a meta-analysis,
so there were never raw data, only a matrix summary. 38.5000
42.5000 212.5000
Submitting a made-up data matrix to an SEM com-
17.5000 51.2500 22.0000
puter tool is a way to diagnose certain kinds of techni-
cal problems that can occur when analyzing a complex Correlations, standard deviations
model. This idea is elaborated later in the book.
1.0000
.4699 1.0000
If means are not analyzed, there are two basic sum- .6013   .7496 1.0000
maries of raw data for continuous variables—covari- 6.2048 14.5774 4.6904
ance matrices and correlation matrices with standard
deviations. For instance, listed in the top part of Table Note. MX = 11.0000, MW = 60.0000, MY = 25.0000.
4.1 are scores on three continuous variables for five
cases. Presented in the lower part of the table are the
two summary matrices just mentioned in lower diago- researcher submits just a correlation matrix for analy-
nal form, where only unique values are reported in the sis. There are special methods for analyzing correlation
lower left-hand side of the matrix. Both the covariance matrices without standard deviations, but they are not
matrix and the correlation matrix with standard devia- available in all software. Thus, it is generally safer to
tions in the table encapsulate all covariance informa- analyze a covariance matrix (or a correlation matrix
tion in the raw data. Computer tools for SEM generally with standard deviations). The pitfalls of analyzing cor-
accept for input lower diagonal matrices as alternatives relation versus covariance matrices are the reason why
to full ones, with redundant entries above and below you must state in written reports the specific kind of
the diagonal, and can “assemble” a covariance matrix data matrix analyzed and the estimation method used.
given just the correlations and standard deviations. Matrix summaries of raw data must consist of
Four-decimal accuracy is recommended for matrix covariances and means whenever means are analyzed.
input to minimize rounding error. Exercise 1 asks you For example, either the correlation matrix with stan-
to reproduce the covariance matrix from the correla- dard deviations or the covariance matrix in Table 4.1
tions and standard deviations in Table 4.1. would be submitted with an extra row for the means of
It may be problematic to submit just a correlation all three variables, if means are analyzed. Even if your
matrix with no standard deviations for analysis, specify analyses do not concern means, you should neverthe-
that all standard deviations equal 1.0 (which standard- less report the means of all continuous variables. You
izes the variables), or convert raw scores to normal may not be interested in analyzing means, but someone
deviates (z scores) and then submit the data file of stan- else might be. Always report sufficient descriptive sta-
dardized scores. This is because most SEM estimation tistics (including means) so others can reproduce your
methods assume the analysis of unstandardized vari- results. If the analysis requires raw data, make those
ables. If the variables are standardized, then the results data available to readers, such as including the data
can be incorrect, including wrong values for standard file in the supplemental materials for a journal article.
errors or model test statistics. Some SEM computer Some journals host data files for articles in databases
tools issue warning messages or terminate the run if the that are accessible to all readers.

Pt1Kline5E.indd 47 3/22/2023 3:52:38 PM

 48 Concepts, Standards, and Tools
POSITIVE DEFINITENESS
The data matrix that you submit—or the one calculated
by the computer from your raw data—should be posi-
tive definite (PD), which is required for most estima-
tion methods. A data matrix that lacks this character-
istic is nonpositive definite (NPD); therefore, attempts
to analyze such a matrix would probably fail. A PD
data matrix has the properties summarized next:
1. The matrix is nonsingular, which means that it has
an inverse. A matrix with no inverse is singular.
2. All eigenvalues for the matrix are positive (> 0),
which says that the matrix determinant is also posi-
tive.
3. There are no out-of-bounds correlations or covari-
ances.
In most kinds of multivariate analyses (SEM
included), the computer needs to derive the inverse of
the data matrix as part of its linear algebra operations. If
the matrix is singular, these operations fail. If v equals
the number of observed variables, the computer should
also be able to generate v linear combinations of the
variables that (1) are pairwise uncorrelated (orthogo-
nal) and (2) reproduce all the covariance information in
the original data matrix. The v weighted combinations
are called eigenvectors, and the amount of variance
explained in the original data matrix by each eigenvec-
tor is its corresponding eigenvalue.1 It is impossible to
derive more eigenvectors than the number of observed
variables because no information remains once all v
eigenvectors are extracted from the data matrix.
If all v eigenvalues are > 0, the matrix determinant
will be positive, too. The determinant is the serial
product (the first times the second times the third, and
so on) of the eigenvalues. If all eigenvalues are posi-
tive, the determinant is a kind of matrix variance, or
the volume of the multivariate space “mapped” by the
observed variables.2 If any eigenvalue equals zero, then
(1) the data matrix has no inverse (it is singular), (2) the
1 Inprincipal components analysis, a type of exploratory factor
analysis where the factors are linear combinations of observed
variables (principal components), eigenvectors correspond to
principal components and eigenvalues are the variances of those
components.
2 Thereis a good illustration at https://en.wikipedia.org/wiki/
Determinant
Pt1Kline5E.indd 48
determinant equals zero, and (3) there is some pattern
of perfect collinearity that involves two variables (e.g.,
rXY = 1.0) or ≥ 3 variables in a more complex pattern
(e.g., RY•XW = 1.0). Perfect collinearity means that the
denominators of some calculations will be zero, which
results in “illegal” (undefined) fractions in computer
analysis (estimation fails). Near-perfect collinearity,
such as rXY = .97, manifested as near-zero eigenvalues
or determinants, can cause the same problem.
Negative eigenvalues (< 0) may indicate a data
matrix element—a correlation or covariance—that
is out of bounds. It is mathematically impossible for
such an element to occur if all elements were calculated
from the same cases with no missing data. For exam-
ple, the value of rXY, the Pearson correlation between
two continuous variables, is limited by the correlations
between these two variables and a third variable W. It
must fall within the range defined next:
2
(rXW × rWY ) ± (1 − rXW 2
)(1 − rWY ) (4.1)
For example, given a value of rXW = .60 and of rWY = .40,
the value of rXY must fall within the range
.24 ± .73, or from –.49 to .97
Any other value for rXY would be out of bounds. Equa-
tion 4.1 specifies a kind of triangle inequality for val-
ues of correlations among three variables. In a geomet-
ric triangle, the length of a given side must be less than
the sum of the lengths of the other two sides but greater
than the difference between the lengths of the two sides.
In a PD data matrix, the maximum absolute value
of covXY, the covariance between two continuous vari-
ables X and Y, must respect the limits defined next:
22 22
XY ≤ s XX × sYY
max cov XY (4.2)
2 2
where s X and sY are, respectively, the sample variances.
In other words, the maximum absolute value for the
covariance between two variables is less than or equal
to the square root of the product of their variances. For
example, given
2 2
covXY = 13.00, s X = 12.00, and sY = 10.00
The covariance between X and Y is out of bounds
because
13.00 > 12.00 × 10.00 =
10.95
3/22/2023 3:52:39 PM

which violates Equation 4.2. The value of rXY is also
out of bounds because it equals 1.19, given these vari-
ances and covariance. Exercise 2 asks you to verify
this fact.
Because the matrix determinant is the serial prod-
uct of the eigenvalues, the determinant will be nega-
tive if some odd number of eigenvalues (1 or 3 or 5,
etc.) is negative. A matrix with a negative determinant
may have an inverse, but the whole matrix is neverthe-
less NPD, perhaps due to out-of-bounds correlations or
covariances. See Topic Box 4.1 for more information
about causes of NPD data matrices and possible solu-
tions.
Before analyzing either raw data or a matrix sum-
mary, the original data file should be screened for the
problems considered next. Some of these difficulties are
causes of NPD data matrices, and most are also con-
cerns in data screening for other kinds of multivariate
statistical analyses—see Tabachnick and Fidell (2019).
If scores from psychological tests are to be analyzed,
the reliabilities of those scores should be estimated
in the researcher’s samples—see the Psychometrics
Primer on this book’s website.
MISSING DATA
The topic of how to deal with missing observations is
complex, and entire books are devoted to it (Enders,
2010; Graham, 2012; Rubin & Little, 2020). There are
also articles or chapters about dealing with incomplete
data in SEM (Enders, 2013, 2023; Jia & Wu, 2019),
which is fortunate because it is not possible here to
give a comprehensive account. The goal instead is to
describe obsolete versus modern options and explain
their relevance in SEM.
Ideally, researchers would always work with com-
plete data sets; otherwise, prevention is the best strat-
egy. For example, questionnaire items that are clear and
unambiguous may prevent missing responses, and com-
pleted forms should be reviewed for missing responses
before participants submit a computer-administered
survey. Little et al. (2012) offered suggestions for reduc-
ing missing data in clinical trials, including routine
follow-up after treatment discontinuation, allowing for
flexible treatment that accommodates side effects or
differences in efficacy, offers of monetary value or other
incentives for completeness of records, and targeting an
underserved population, which provides an incentive to
remain in the study.
Pt1Kline5E.indd 49
Data Preparation 49
In the real world, missing values occur in many, if
not most, data sets, despite best efforts at prevention.
A few missing values, such as < 5% in the total data
set, may be of little concern. This is because selection
among alternative methods to deal with missing data
tends to make little difference when rates of missing
data are low (Schafer, 1999). Higher rates of data loss
present greater challenges, especially if the data loss
mechanism is not random (or at least predictable).
There is no critical rate of missing data that signals a
severe problem, but as that rate exceeds about 10% or
so, there is an increasing likelihood of bias. In this case,
the choice of method can affect the results. Best prac-
tices involve the steps listed next (Johnson & Young,
2011; Little et al., 2012):
1. Report the extent of missing data with a tabular
summary or diagram, such as a participant flow.
Describe procedures used to prevent or mitigate
data loss.
2. Diagnose missing data patterns, which are related
to assumptions about data loss mechanisms. Note
that different variables in the same data set can be
affected by different data loss mechanisms.
3. Use a modern, principled statistical method for ana-
lyzing incomplete data, one that takes full advan-
tage of the structure in the data and does not rely on
implausible assumptions.
4. Acknowledge the reality that certain assumptions
about data loss are untestable unless new data are
collected.
5. Conduct a sensitivity analysis, where the extant
incomplete data are reanalyzed using a different
method. If the results differ appreciably from the
original findings, they are not considered robust
compared to alternative assumptions about missing
data.
Data Loss Mechanisms
and Missingness Graphs
Rubin (1976) described three basic data loss mecha-
nisms. The least troublesome in terms of bias—and
perhaps also the most unlikely in real data sets—is
missing completely at random (MCAR). This means
that missingness (i.e., missing yes or no) on a variable
is unrelated to (1) all other variables in the data set and
(2) the variable itself with no missing observations.
The second point just stated means that the propensity
3/22/2023 3:52:39 PM
 50 Concepts, Standards, and Tools

TOPIC BOX 4.1

Causes of Nonpositive Definite Data Matrices and Solutions

The causes of NPD data matrices described by Wothke (1993) are listed next. Some causes can be
detected through data screening:

1. Extreme bivariate or multivariate collinearity among observed variables.

2. The presence of outliers that force the values of correlations to be extremely high.
3. Sample correlations or covariances that barely fall within the limits defined by Equations 4.1 and
4.2 but nevertheless cause the analysis to fail with a warning or error message about an NPD
matrix.
4. Pairwise deletion of missing data.
5. Making a typing mistake when transcribing a data matrix from one source, such as a table in
a journal article, to another, such as a syntax file for computer analysis, can result in an NPD
data matrix. For example, if the value of a covariance in the original matrix is 15.00, then typing
150.00 in the transcribed matrix could generate an NPD matrix.
6. Plain old sampling error can generate NPD data matrices, especially in small or unrepresentative
samples.
7. Sometimes matrices of estimated Pearson correlations, such as polyserial or polychoric correla-
tions derived for observed variables that are not continuous, can be NPD.

Here are some tips for diagnosing whether a data matrix is PD before submitting it for analysis to an
SEM computer tool: Copy the full matrix (with redundant entries above and below the diagonal) into a text
(ASCII) editor, such as Microsoft Windows Notepad. Next, point your Internet browser to a free, online
matrix calculator (many are available) and then copy the data matrix into the proper window on the cal-
culating page. Finally, select options on the calculating page to derive the eigenvalues, eigenvectors, and
determinant. Look for results that indicate an NPD matrix, such as near-zero, zero, or negative eigenvalues.
A useful matrix and vector calculator is available at https://www.symbolab.com/solver/matrix-calculator.
An alternative is to analyze the matrix in R using its native matrix algebra functions (i.e., no special pack-
age is needed). An example follows.
Suppose that the covariances among variables X, W, and Y, respectively, are

1.00 .30 .65

.30 2.00 1.15
.65 1.15 .90

The R syntax that turns scientific notation off, defines and displays the covariance matrix just listed, and
generates the eigenvalues and eigenvectors, the determinant, and the inverse (if any) is given here. The last
command converts the covariance matrix to a correlation matrix:
(continued)

Pt1Kline5E.indd 50 3/22/2023 3:52:39 PM

 Data Preparation 51

options(scipen = 999)
a <- matrix(c(1,.3,.65,.3, 2, 1.15,.65, 1.15,.9),
nrow = 3, byrow = TRUE)
a
eigen(a)
det(a)
solve(a)
cov2cor(a)

The eigenvalues generated using the R code just listed for the covariance matrix are

(2.918, .982, 0)

The third eigenvalue is zero, so the covariance matrix has no inverse and the determinant equals 0. Let us
inspect the weights for the third eigenvector, which for X, W, and Y, respectively, are

(–.408, –.408, .816)

Some online matrix calculators report the eigenvector weights as

(–1, –1, 2) or (–.5, –.5, 1)

but the values just listed are proportional to the weights computed in R. None of these weights equals zero,
so all three variables are involved in perfect collinearity. The pattern for these data is

RY•XW = RW•XY = RX•YW = 1.0

To verify this pattern, you should calculate the multiple correlations just listed from the bivariate correlations
for X, W, and Y, respectively, computed in R and presented next in lower diagonal form—see the Regres-
sion Primer on this book’s website for the equations:

1.0
.2121 1.0
.6852 .8572 1.0

The LISREL program offers an option for ridge adjustment, which multiples the diagonal elements in a
covariance matrix by a constant > 1.0 until negative eigenvalues disappear (the matrix becomes PD). These
adjustments increase the variances until they are large enough to exceed any out-of-­bounds covariance
element in the off-­diagonal part of the matrix. This technique “fixes up” a data matrix so that the necessary
algebraic operations can be performed (Wothke, 1993), but parameter estimates, standard errors, and fit
statistics are biased after ridge adjustment. A better solution is to try to solve the problem of nonpositive
definiteness through data screening. There are other contexts where you may encounter NPD matrices in
SEM, but these generally concern (1) matrices of parameter estimates for your model or (2) matrices of
correlations or covariances predicted from your model. A problem is indicated if any of these matrices is
NPD. We will deal with these contexts in later chapters.

Pt1Kline5E.indd 51 3/22/2023 3:52:39 PM

 52 Concepts, Standards, and Tools
for an observation to be missing does not depend on a
case’s true status on that variable. Thus, there is no sys-
tematic process anywhere that would make some data
more likely to be missing than other data. If so, then
the observed incomplete data are just a random sample
of scores that the researcher would have analyzed if the
data were complete (Enders, 2010). Results based on
the complete cases only should not be biased, although
power of significance tests may be reduced due to a
smaller effective sample size.
Data are missing at random (MAR) when missing-
ness is (1) correlated with other variables in the data set,
but (2) does not depend on the level of the variable itself
with no missing observations. This means that data are
actually missing conditionally at random, or after con-
trolling for values of other measured variables. Sup-
pose that men are less likely to report on their health
status in a particular area than women, but there is no
real difference in health between men with no missing
data versus men who opted not to disclose. After con-
trolling for gender, the data loss pattern is random. The
dependence of missingness solely on other observed
variables explains why Mohan and Pearl (2021) used
the term v-MAR, where “v” stands for “observed
variables.” Information lost due to an MAR process is
potentially recoverable through imputation in which
missing scores are replaced by values predicted from
other variables in the data set. Options for imputation
are described momentarily, but statistical methods suit-
able for the MAR data loss pattern can also be applied
when the pattern is MCAR. The reverse is not true,
though: Methods that assume MCAR are not generally
appropriate when the missing data mechanism is MAR.
The third pattern is missing not at random (MNAR),
also known as nonignorable missing data, where the
probability of missingness is related to the true level
of the variable itself even after controlling for other
variables in the data set. That is, there is a structure
to the missing data, and it cannot be treated as though
it were random. Unlike the MAR pattern, which is a
recoverable process, the MNAR pattern is not because
it is latent (not directly measured) (Little, 2013). An
example of an MNAR missing data pattern is when
men who fail to answer questions about their health
status are more likely to be ill than men with no miss-
ing responses on the same variable. Another is when
respondents with either very low or very high incomes
are less likely to answer questions about their income.
Thus, the challenge presented by the MNAR pattern is
Pt1Kline5E.indd 52
that the missing values depend on information that is
not directly available in the analysis. Thus, (1) results
based on the complete cases only can be severely biased
when the data loss pattern is MNAR, and (2) the choice
of methods to deal with the missing data can make a
big difference in the results. Some of this bias may be
reduced if other measured variables happen to covary
with unmeasured causes of data loss, but whether this is
true in a particular sample is typically unknown.
Mohan and Pearl (2021) described missingness
graphs or m-graphs that visually represent the data
loss mechanisms just outlined. An m-graph is a spe-
cial type of directed acyclic graph (DAG) in which uni-
directional causal effects are represented with arrows
that point from presumed causes to their outcomes.
Presented in Figure 4.1(a) is causal DAG for variables
with no missing data, where Xo and Wo are specified
as uncorrelated causes of Yo. Figures 4.1(b)–4.1(d) are
m-graphs in which Ym is an incomplete variable with
missing scores for some, but not all, cases. Variable R
in the m-graphs is the missingness mechanism that
determines whether a particular score is missing or not
missing on the observed variable Y*, which is a proxy
for Ym. The relation among Yo, R, and Y* can be sum-
marized as follows:
Y if R = 0
Y* =  0 (4.3)
if R = 1
m
where m means “missing.” That is, when R = 0, the true
value for Y is observed, but when R = 1 that value is
hidden (not observed).
In Figure 4.1(b), the data loss mechanism is MCAR
because missingness occurs with no relation to any vari-
able whether it is completely measured (Xo, Wo) or par-
tially measured (Ym). The independencies just stated are
represented in Figure 4.1(b) by the absence of arrows
that point to R from all other variables. The MAR pat-
tern is depicted in Figure 4.1(c), where missingness is
specified as being caused by the fully observed variable
Wo but not by the incomplete variable Ym. For example,
if variable Wo in the figure is gender, then the whole
m-graph says that although the likelihood of missing
data varies by gender, there is no systematic difference
between responders and nonresponders on the partially
measured variable after controlling for gender. The
m-graph in Figure 4.1(d) represents an MNAR pattern,
where missingness is caused by the partially measured
variable itself (i.e., Ym → R), and controlling for any
3/22/2023 3:52:39 PM
 Data Preparation 53

(a) Complete (b) MCAR (c) MAR (d) MNAR

Xo Wo Xo Wo Xo Wo Xo Wo

Yo Ym R Ym R Ym R

Y* Y* Y*

FIGURE 4.1. No missing data (a). Missingness graphs (m-graphs) under conditions of missing completely at random (MCAR)
(b), missing at random (MAR) (c), and missing not at random (MNAR) (d). Variables Xo, Wo, and Yo are observed in all cases;
Ym has missing data for some cases; Y* is the proxy variable that is actually measured; and R is the missingness mechanism.
If R = 0, then Y* = Yo, but if R = 1, then Y* = missing. From “Graphical Models for Processing Missing Data,” by K. Mohan
and J. Pearl (2021), Journal of the American Statistical Association, 116(534), p. 1025. Copyright © 2021 by Taylor & Francis.
Adapted with permission.

fully measured variable, Xo or Wo, can neither break nor
disrupt this association.
Diagnosing Missing Data
It is not easy in practice to determine whether the data
loss mechanism is random or systematic, especially
when each variable is measured once. One reason is
that all three data loss mechanisms—MCAR, MAR,
and MNAR—can be involved in causing case attri-
tion or nonresponse, and their relative influence can
change over variables in the same data set. Another
reason is that while there is a way to determine
whether the assumption of MCAR is reasonable,
there is no specific test that provides direct evidence
of either MAR or MNAR if the hypothesis of MCAR
is rejected. The only way to distinguish MAR and
MNAR data loss mechanisms is to measure the miss-
ing data. For example, if survey nonrespondents are
later followed up by phone to get the missing infor-
mation, then respondents and nonrespondents in the
first round can now be compared. If these two groups
differ appreciably on the recovered data, there is evi-
dence for MNAR; otherwise, the hypothesis of MAR
may be viable. Whenever the recovery of missing data
is impractical, analysis of the original incomplete data
is the only option.
Little (1988) described a multivariate significance
test for the hypothesis of MCAR. It compares cases
with observed versus missing observations on all other
variables in the data set while controlling for their
intercorrelations. In the bivariate case, where missing
data are confined to a single variable and the other vari-
able is continuous, the Little MCAR Test reduces to
an independent-samples t test. Assuming normal distri-
butions and homoscedasticity, the test statistic for the
Little MCAR Test is distributed over large samples as
a central chi-square statistic. A significant result (e.g.,
p < .05 when testing at the .05 level) means that the null
hypothesis of MCAR is rejected; that is, the data loss
pattern is either MAR or MNAR. A problem with the
test is that its power can be low in small samples (i.e.,
the null hypothesis of MCAR is retained too often), and
it can flag trivial departures from MCAR as significant
in large samples.
Estimating effects sizes in comparisons of com-
plete versus incomplete cases over other variables can
help the researcher to interpret results from the Little
MCAR Test. One tactic involves creating a binary
(dummy) variable, where a score of “1” indicates a
missing observation and a score of “0” means not miss-
ing (complete). Next, the means for the two groups just
defined are compared to other variables that are con-
tinuous. Magnitudes of univariate group mean differ-
ences can be estimated using either standardized mean
differences, d, or point-biserial correlations, rpb (Kline,
2013a, chap. 5). In large samples, a significant Little
MCAR Test result when magnitudes of group differ-
ences on other variables are considered trivial would
bolster the hypothesis of MCAR. The opposite pattern

Pt1Kline5E.indd 53 3/22/2023 3:52:39 PM

 54 Concepts, Standards, and Tools
in a small sample—a nonsignificant Little MCAR Test
when magnitudes of group differences on other vari-
ables are meaningful—would provide evidence against
the hypothesis of MCAR.
Observing differences between complete versus
incomplete cases on other variables is helpful for a
method that imputes scores for the incomplete cases
when predicting from those other variables, assuming
the data loss pattern is MAR (Figure 4.1(c)). A strat-
egy that anticipates this pattern is to measure auxiliary
variables that may not be of theoretical interest except
they are expected to be potential causes or correlates
of missingness on other variables. For example, family
socioeconomic status (SES) could be a potential aux-
iliary variable in a longitudinal study, if it is expected
that lower SES families are more likely to drop out of
the study. Including family SES in the analysis may
help to reduce bias due to differential attrition related
to this variable. An auxiliary variable could also be one
that simply covaries appreciably with variables that
have missing observations or with their causes whether
or not they are related to the missingness mechanism.
Suppose that information on family SES is not avail-
able, but addresses for places of residence are mea-
sured. If place of residence, or neighborhood, is related
to SES, then including residence in the analysis may
recover some of the information about SES as a miss-
ing data mechanism, thus reducing bias related to SES.
Auxiliary variables require care in their selection.
For example, including too many such variables in
small samples can decrease precision and create down-
ward bias in estimates of regression coefficients, espe-
cially if absolute correlations between auxiliary and
other variables are < .10 (Hardt et al., 2012). Enders
(2010) recommended that auxiliary variables should
have absolute correlations of about .40 with incomplete
variables, although that particular value is not a golden
rule. Ideally, there should be no missing observations
on auxiliary variables; otherwise, their potential role in
recovering information due to data loss on other vari-
ables is diminished (Dong & Peng, 2013). Thoemmes
and Rose (2014) described situations where auxiliary
variables might actually increase bias. One case is
when the auxiliary variable is an outcome of a partially
measured variable and where controlling for the aux-
iliary variable induces a spurious association between
the missing mechanism and the partially measured
variable, although Rubin (2009) speculated that the
pattern just described would be rare in real data sets.
Pt1Kline5E.indd 54
CLASSICAL (OBSOLETE) METHODS
FOR INCOMPLETE DATA
Classical techniques for handling missing data are
simple to understand and easy to implement in statisti-
cal software, but they are increasingly seen as obsolete
because such methods
1. Assume that the data loss mechanism is MCAR,
and results with these methods can be seriously
biased when this rather improbable assumption
does not hold.
2. Take little or no advantage of structure in the data
when investigating missing data patterns.
3. Basically ignore or minimize the problem by allow-
ing an incomplete data file to be produced.
Classical methods are briefly described next so that
readers can better understand their limitations, but I
can’t recommend their use. Now, to be fair, if the rate
of missing data is very low (e.g., 1%), then it doesn’t
really matter whether classical or more modern tech-
niques are used because the results will be similar. But
as the rate of data loss increases, the trustworthiness of
classical methods decreases, especially if the data loss
pattern is not MCAR.
There are two broad categories of classical tech-
niques: available case methods, which analyze avail-
able data through removal of incomplete cases from
the analysis, and single-imputation methods, which
replace each missing score with a single calculated
(imputed) score. Available case methods include list-
wise deletion in which cases with missing scores on
any variable are excluded from all analyses. The effec-
tive sample size with listwise deletion includes only
cases with complete records, and this number can be
much smaller than the original sample size if miss-
ing observations are scattered across many records.
An advantage is that all analyses are conducted with
the same cases. This is not so with pairwise deletion
in which cases are excluded only if they have missing
values on variables involved in a particular analysis.
Suppose that N = 300 for an incomplete data set. If
250 cases have no missing scores on variables X and Y,
then the effective sample size for covXY is this number.
If fewer or more cases have valid scores on X and W,
however, the effective sample size for covXW will not
be 250. This property of pairwise deletion can give
3/22/2023 3:52:39 PM

rise to NPD data matrices, which is demonstrated in
Exercise 3.
Mean substitution, where the overall sample (grand)
mean replaces any missing score on that variable, is
the simplest single-imputation method. A variation is
group-mean substitution, where the missing score is
replaced by a group mean on that variable, such as the
mean for men when the record for a male participant is
incomplete. Neither method takes account of the infor-
mation about the individual case except group member-
ship for group-mean substitution. Another problem is
that mean substitution distorts the score distribution by
reducing variability. This happens because the imputed
score equals the mean, so they contribute nothing to
sum of squares (numerator) of the variance while
increasing the degrees of freedom (denominator); that
is, variance decreases after imputation. Consequently,
covariances are weakened and error variances can be
underestimated (Liu, 2016).
Regression substitution is a bit more sophisticated
because each missing score is replaced by a predicted
score from regression analyses based on variables with
no missing data. The method uses more information
than mean substitution, but it assumes that incom-
plete variables can be predicted reasonably well from
other variables in the data set. Error variance is still
underestimated in this method because (1) the imputed
score for cases having the same values on the predic-
tors is a constant, and (2) sampling error affects pre-
dicted scores, too, and this uncertainty is not estimated
in single-imputation methods. A variation is to add a
randomly sampled error term from the normal distri-
bution or other user-specified distribution in stochastic
regression imputation.
Other single-imputation methods include
1. Last observation carried forward (LOCF), where
in clinical trials the last observation is the most
recent score for participants who drop out of the
study.
2. Pattern matching, in which the computer replaces
a missing observation from a case with the most
similar profile on other variables.
3. Random hot-deck imputation (RHDI), which
separates complete from incomplete cases, sorts
both sets so that cases with similar profiles on back-
ground variables are grouped together; randomly
interleaves the incomplete and complete cases; and
Pt1Kline5E.indd 55
Data Preparation 55
replaces missing scores with those on the same
variable from the nearest complete record.
All the methods just listed have limitations: Pattern
matching and RHDI require large samples. The LOCF
method assumes that patients in treatment generally
improve and that the last measurement before dropout
is a conservative estimate of eventual outcome. But if
patients drop out because they are becoming more ill
despite treatment or even die, the LOCF approach can
grossly overestimate treatment efficacy (Liu, 2016).
MODERN METHODS
FOR INCOMPLETE DATA
Modern techniques for handling missing data include
multiple imputation (MI), which generates multiple
predicted scores for each missing observation, and a
special version of full information maximum likeli-
hood (FIML) estimation for incomplete data files that
neither imputes missing observations nor deletes cases.
Both methods assume that the data loss mechanism
is MAR, which is less strict than the assumption of
MCAR by classical techniques. Depending on the SEM
computer tool or procedure, one or both of the modern
methods just mentioned will be available. How the spe-
cial FIML option deals with missing data is described
in Chapter 9.
The technique of MI uses variables of theoretical
interest and optional auxiliary variables to generate for
each missing observation a set of k ≥ 2 imputed scores
from predictive distributions that model the data loss
mechanism. The result is a total of k data sets, each
with a unique imputed value for any missing observa-
tion. Next, all k imputed data sets are analyzed with
standard statistical techniques, such as fitting the same
structural equation model to each generated data set.
Finally, the resulting k different estimates of each model
parameter are pooled, and the corresponding standard
errors reflect sampling error due both to case selection
and imputation, and these standard errors are typically
larger than those from single imputation. The three
basic steps of MI just summarized are described in more
detail in Appendix 4.1. Readers already very familiar
with MI can skip this appendix; otherwise, the presen-
tation there is somewhat technical, but it is worth the
effort to learn more about this modern and statistically
principled approach to dealing with incomplete data
3/22/2023 3:52:39 PM
 56 Concepts, Standards, and Tools
files under the assumption of MAR data loss patterns.
The technique has widespread application in SEM and
in many other kinds of statistical analyses, too.
OTHER DATA SCREENING ISSUES
Considered next are ways to deal with extreme collin-
earity, outliers, violation of distributional assumptions,
and very heteroscedastic data matrices.
Extreme Collinearity
Extreme collinearity—also called extreme multi-
collinearity—refers to high levels of interdependence
among predictors of the same outcome. It can lead to
inflation of standard errors, instability in the results
such that small changes in sample covariance patterns
can generate very different solutions, or analysis fail-
ure due to linear dependence. Extreme bivariate col-
linearity can occur because what appear to be separate
variables really measure the same thing. Suppose that
X measures accuracy and W measures speed for the
same task. If rXW = .97, for example, then X and W are
redundant notwithstanding their different labels (i.e.,
accuracy is speed, and vice versa). Either one could be
included in the same regression equation, but not both.
Although there is no gold standard, it seems to me that
absolute bivariate correlations > .90 signal a potential
problem, but others have suggested even lower thresh-
olds, such as .80 (Abu-Bader, 2010).
Researchers can inadvertently cause extreme col-
linearity when total scores (composites) and their
constituents are analyzed together. Suppose that a life
quality questionnaire has five individual scales and a
total score that is summed across all scales. Although
the bivariate correlation between the total score and
each of the individual scales may not be very high, the
multiple correlation between the total score and the five
scale scores must equal 1.0 when there is no missing
data, which is multivariate collinearity in the extreme.
A straightforward method to detect extreme col-
linearity among three or more continuous variables
is based on the variance inflation factor (VIF). It is
often available in regression diagnostic procedures of
computer programs for general statistical analyses. It
is computed as
1 that “999” means the observation is missing.
VIF = (4.4)
1 − R 2j
Pt1Kline5E.indd 56
where R 2j is the proportion of variance in the jth pre-
dictor explained by all other predictors. Tolerance is
1 – R 2j , or the proportion of variance that is unique, or
not explained by other predictors, so VIF is the recipro-
cal of tolerance, and vice versa. Thompson et al. (2017)
described how to calculate VIF in secondary analyses
given just a correlation matrix with no raw data.
Values for the VIF range from 1.0, which indicates
all predictor variance is unique, to increasingly positive
values with no theoretical upper bound, which indicates
higher and higher levels of collinearity. Again, there
is no gold standard but some authors have suggested
that VIF > 10.0 signals possible extreme collinearity
(Chatterjee & Price, 1991). This threshold corresponds
to R 2j > .90 and tolerance < .10. Others have expressed
skepticism about whether any single threshold for the
VIF (10.0, 20.0, or even higher) that ignores other
factors, such as sample size, is meaningful (O’Brien,
2007). Thompson et al. (2017) reminded us that any
cutting point applied to the VIF should not be treated
as a hard dichotomy in the sense that, for instance, one
falsely believes that VIF = 9.9 versus VIF = 10.1 makes
a practical difference.
Outliers
Outliers are scores that are very different from the rest.
A univariate outlier is a score on a single variable that
falls outside the expected population values (Mowbray
et al., 2019). There is no single definition of a univariate
outlier for continuous variables. For example, Tabach-
nick and Fidell (2019) suggested that scores that are
more than 3.29 standard deviations above or below
the mean are possible outliers. Expressed in terms of
normal deviates, | z | > 3.29 describes this guideline.
Elsewhere I suggested the more conservative heuristic
of | z | > 3.0 (Kline, 2020a), but there is no magic cutoff
point. A limitation is that outlier detection based on nor-
mal deviates is not robust against extreme scores—see
Topic Box 4.2 for a description of an alternative method.
More important than any particular numerical
threshold for detecting univariate outliers is that the
researcher investigates extreme scores, which can arise
due to
1. Mistakes in data entry or coding, such as typing
“95” instead of “15” for a score or failing to specify
2. Intentional distortion or careless reporting, such as
3/22/2023 3:52:39 PM
 Data Preparation 57

when research participants respond randomly to
questionnaire items as a covert way to be uncoop-
erative or lie in response to questions about socially
undesirable behaviors such as cheating or drug use.
3. Administration of measures in ways that violate
standardization, such as giving examinees hints
that are not part of task instructions.
4. Selection of samples that are unrepresentative of
the target population or under faulty distributional
assumptions.
5. Natural variation within a population, or an extreme
score belongs to a case selected from a different
population (Osborne, 2013).
The last point just mentioned assumes that an
extreme score is correct (e.g., it is not invalid). If it
can be determined that a case with univariate outli-
ers is not from the same population, such as a gradu-
ate student who completes a survey while auditing an
undergraduate class, then it is best to remove that case.
It is more difficult when extreme scores come from the
target population; that is, although infrequent, such
scores arise naturally, so removing them could affect
the generalizability of the results. The basic options are
(1) do nothing; (2) remove the outlier from the analy-
sis; or (3) minimize its influence through substitution,
such as converting extreme scores to a value that equals
the closest scores not considered extreme (e.g., within
3.0 standard deviations from the mean); or (4) apply a
monotonic transform that pulls extreme scores closer to
the center of the distribution. Another option is a sensi-
tivity analysis in which results based on different deci-
sions about extreme scores are explicitly compared.

TOPIC BOX 4.2

Robust Univariate Outlier Detection

Suppose that scores for five cases are 19, 25, 28, 32, and 10,000. The last score (10,000) is obviously
an outlier, but it so distorts the mean and standard deviation that even the more conservative | z | > 3.00
rule fails, also called masking:
10,000 − 2,020.80
M = 2,020.80 SD = 4,460.51
= and z = 1.79
4, 460.51
A more robust decision rule for detecting univariate outliers is

| X − Mdn |
> 2.24 (4.5)
1.4826 (MAD)
where Mdn designates the median—­which is more robust against outliers than the mean—and MAD is the
median absolute deviation of all scores from the median. The product of MAD and the scaling factor
1.4826 is an unbiased estimator of s in a normal distribution. The whole ratio is the distance between a
score and the median expressed in robust standard deviation units. The constant 2.24 is the square root of
the approximate 97.5th percentile in a central chi-­square distribution with a single degree of freedom. A
potential outlier thus has a value on Equation 4.5 that exceeds 2.24.
For the five scores in this example, Mdn = 28.00, and the absolute values of median deviations are,
respectively, 9.00, 3.00, 0, 4.00, and 9,972.00. The median of the deviations just listed is MAD = 4.00,
and so for X = 10,000 we calculate

9,972.00
= 1,681.51
1.4826 (4.00)
which clearly exceeds 2.24 and thus detects the score of 10,000 as an outlier. See Rousseeuw and Hubert
(2018) for additional methods of robust outlier detection.

Pt1Kline5E.indd 57 3/22/2023 3:52:39 PM

 58 Concepts, Standards, and Tools
A multivariate outlier has extreme scores on ≥ 2
variables or an atypical pattern of scores. For example,
a case may have scores between 2–3 standard devia-
tions above the mean on all variables. Although no
individual score might be considered extreme, the case
could be a multivariate outlier if this pattern is unusual.
Here are some options for detecting multivariate outli-
ers with no univariate outliers:
1. Some SEM computer tools, such as IBM SPSS
Amos and EQS, list cases that contribute the most
to multivariate nonnormality, and such cases may
be multivariate outliers.
2. Calculate for each case its squared Mahalonbis dis-
tance, D 2, which indicates the distance in variance
units between the profile of scores and the vector of
sample means, or centroids, adjusting for correla-
tions among the variables. In large samples, D2 is
distributed as a central chi-square with degrees of
freedom equal to the number of variables. A rela-
tively high value of D 2 and low p value may lead
to the rejection of the null hypothesis that the case
comes from the same population as the rest. A con-
servative level of statistical significance is usually
recommended for this test, such as .001. Leys et al.
(2018) described a robust version of the multivariate
test just mentioned.
Visual methods to detect univariate outliers include
the inspection of histograms or box plots (Mowbray
et al., 2019). In both types of displays, extreme scores
are represented as further away from the main body
of scores, or the rest of the distribution. Tukey (1977)
developed box plots, also called box-and-whisker
plots, as a way to graphically display the spread of the
data throughout their whole range. The outer parts of
the “box” are defined by the hinges, which correspond
approximately to the first quartile (Q1), or the 25th
percentile, and the third quartile (Q3), or the 75th per-
centile. The second quartile (Q2), or the median (50th
percentile), is represented by a line in the box that is
parallel to two hinges. The whiskers are lines that con-
nect the hinges with the lowest and highest scores that
do not exceed 1.5 times the positive difference between
the hinges (i.e., approximately 1.5 times the interquar-
tile range, or Q3 – Q1). Any scores that fall outside of
the limits just stated—the lower and upper fences—are
represented as outliers. Exercise 4 asks you to generate
a boxplot for a small data set with an extreme score.
Pt1Kline5E.indd 58
Majewska (2015) described graphical displays for mul-
tivariate outliers based on robust D2 statistics. Because
interpretation of graphical displays can be rather sub-
jective, they are not substitutes for numerical methods.
Distributions
The default method in most SEM computer tools is a
form of ML estimation for either complete raw data
files or summary matrices (e.g., Table 4.1) that assumes
multivariate normality, also called multinormality,
for continuous outcome (endogenous) variables. This
means that
1. All the univariate frequency distributions are nor-
mal.
2. All joint distributions of any pair of variables are
bivariate normal; that is, each variable is normally
distributed for each value of every other variable.
3. All bivariate scatterplots are linear with homo­
scedastic residuals.
Other variations on the definition of multivariate nor-
mality in SEM are described in Chapter 9 on global
estimation methods. Because it is often impractical to
examine all joint frequency distributions, it can be dif-
ficult to assess all aspects of multivariate normality.
Fortunately, many instances of multivariate nonnor-
mality can be detected through evaluation of univariate
frequency distributions. This is because univariate nor-
mality is necessary but insufficient to guarantee multi-
variate normality (Pituch & Stevens, 2016).
Early SEM computer programs had few estimators
other than default ML, but the situation today is very
different: Most SEM computer programs now offer
multiple estimators that accommodate different kinds
of outcome variables, such as continuous, ordinal,
binary, count, or censored variables, with various types
of distributions, such as normal or nonnormal distri-
butions for continuous outcomes and Poisson distribu-
tions for count variables, and so on.3 A robust option for
ML estimation for continuous outcomes with nonnor-
mal distributions is described in Chapter 9, and other
3A count variable is the number of times a discrete event happens
over a period of time, such as the number of hospitalizations over
the past 5 years. In a Poisson distribution, the mean and variance
are equal. A censored variable is one for which values occur out-
side the range of measurement, such as a scale that registers the
value of weight between 1 and 300 pounds only.
3/22/2023 3:52:39 PM

options for noncontinuous outcomes are covered later
in the book. Thus, normality assumptions are less criti-
cal in modern SEM. Instead, the challenge is for the
researcher to select an appropriate estimator, given
the distributional characteristics of their data.
But there are still occasions in SEM when data
should be screened for multivariate normality. One is
when using the default ML method. Note that raw data
are required to evaluate multivariate normality. If just a
covariance matrix is submitted for analysis with default
ML, it must be assumed that the original distributions
for continuous outcomes are multivariate normal.
Other occasions arise when using a method of MI that
assumes multivariate normality or when applying the
original Little MCAR Test. Quantitative measures of
nonnormality are described next, with the assumption
that readers already know how to inspect scatterplots,
distributions of regression residuals, or other kinds of
graphical displays or numerical summaries used with
bivariate linearity and homoscedasticity; otherwise,
see Cohen et al. (2003, chap. 4).
Significance tests intended to detect multivariate
nonnormality, such as Mardia’s (1970) test, or detect
univariate nonnormality, such as the Kolmogorov-
Smirnov (K-S) test, among others (Oppong et al., 2016),
have limited usefulness. One reason is that slight depar-
tures from normality could be significant in large sam-
ples, and power in small samples may be low, so larger
departures could be missed. An alternative for assessing
univariate normality are quantitative measures of skew-
ness and excess kurtosis, the two ways a distribution can
be nonnormal, and they can occur either separately or
together in the same variable. A normal distribution has
zero skewness and excess kurtosis.
Skewness is the degree of asymmetry in a probabil-
ity distribution. It is defined as the third standardized
moment, which in the population is
m
g1 = 33 (4.6)
s The term mesokurtic refers to distributions with zero
where s is the population standard deviation and m3 is
the third central moment, a particular instance of the
rth moment defined as
mr =
∑ ( X − m)r (4.7)
N that could be printed by software for general statisti-
where m is the population mean and r is an integer ≥ 1.
In Equation 4.7, r = 3 for the third moment. In symmet-
rical distributions, the sum of deviations raised to the
third power for scores above the mean will balance the
Pt1Kline5E.indd 59
Data Preparation 59
sum of deviations raised to the same power for scores
below the mean, so m3 = g1 = 0 in a normal curve. In
unimodal distributions where most of the scores are
below the mean such that the distribution has a longer
right tail than the left tail, then m3 > 0 and g1 > 0, which
indicates positive skew. But if most scores are above
the mean such that the left tail is longer than the right
tail, then m3 < 0 and g1 < 0, which indicates negative
skew.
In unimodal distributions, kurtosis concerns the
combined weight of the both tails relative to the center
of the distribution. Thus, kurtosis measures the rela-
tive presence of outliers in both tails of the distributions
compared with a normal distribution. It is the fourth
standardized moment, which in the population distri-
bution is
m
g 2 = 44 (4.8)
s
where m4 is the fourth moment about the mean,
defined by Equation 4.7 for r = 4. The expected value
for g2 in a normal distribution is 3.0. Excess kurtosis
is defined as
g2 – 3 (4.9)
so that its value equals 0 in a normal distribution. Thus,
positive excess kurtosis, or
g2 – 3 > 0
means that the tails of the distribution are heavier rela-
tive to a normal curve, and such distributions are called
leptokurtic. Negative excess kurtosis, or
g2 – 3 < 0
means just the opposite (lighter tails), and distribu-
tions with this characteristic are called platykurtic.
excess kurtosis, such as the normal curve. All refer-
ences to “kurtosis” from this point concern excess kur-
tosis. Skewed distributions are generally leptokurtic.
Joanes and Gill (1998) described the three sample
estimators for skew and kurtosis listed in Table 4.2
cal analyses. Statistics g1 for skew and g2 are based on
sample moments and standard deviations computed
as S with N in the denominator. Other statistics in the
table feature adjustments for small sample size in the
3/22/2023 3:52:40 PM
 60 Concepts, Standards, and Tools

computation of sample standard deviations as s with
N – 1 in the denominator (e.g., b1, b2) or in calculations
for central moments (e.g., G1, G 2). In normal popula-
tion distributions, all three skewness statistics in Table
4.2 are unbiased, but only G 2 for kurtosis is unbiased
in such distributions. Also shown in the table are the
relative values of error variances for each statistic in
normal distributions. Estimators G1 for skew and G 2
for kurtosis tend to have the greatest expected variation
over random samples, but differences among their val-
ues and error variances narrow for samples sizes > 100.
In very small samples, though, differences among these
estimators can be striking, including results for kurto-
sis that are both positive and negative in the same vari-
able—see Cain et al. (2017) for more information.
Just as there are no golden rules for detecting outliers,
there are also no universal absolute values for skewness
or kurtosis statistics that indicate severe nonnormal-
ity. Finney and DiStefano (2013) noted that absolute
univariate skewness and kurtosis values greater than,
respectively, 2.0 and 7.0, have been described as indi-
cating severe nonnormality in some computer simula-
tion studies, but exceptions are easy to find. For exam-
ple, Lei and Lomax (2005) treated absolute skewness
and kurtosis values > 2.30 as indicating severe nonnor-
mality. The point is that there is no magic demarca-
tion between trivial and appreciable nonnormality that
will fit all models and data sets, but the assumption of
normality becomes increasingly less plausible as there
is more and more skewness or kurtosis. Exercise 5
addresses a common misinterpretation in this area. Sig-
nificance testing where skewness or kurtosis statistics
are divided by their standard errors is another method,
but it is problematic for reasons already stated (e.g., low
power in small samples).
Normalizing transformations (normalization), or
monotonic arithmetic operations that compress some
parts of a distribution more than others while preserv-
ing rank order so that the transformed scores are more
normally distributed, are an option, but you should
think about the variables of interest. Some variables,
including reaction times, reports of alcohol or drug use,

TABLE 4.2. Estimators of Skewness and Kurtosis and Relative

Error Variances for Normal Samples
Statistic Equation Unbiased Error variance rank
Skewness
g1 m3 Yes 2
S3

G1 N ( N − 1) Yes 1
g1
N −2

b1 m3 Yes 3
s3

Kurtosis
g2 m4 No 3
−3
S4

G2 N −1 Yes 1
(( N + 1)g2 + 6)
( N − 2)( N − 3)

b2 m4 No 2
−3
s4
Note. Error variances are ranked from highest to lowest; m3 = S (X – M)3/N; m4 = S (X – M)4/N;
and S and s are the sample standard deviations computed with, respectively, N or N – 1 in the
denominator.

Pt1Kline5E.indd 60 3/22/2023 3:52:40 PM

 Data Preparation 61

and health care costs, are expected to have nonnormal
distributions (Bono et al., 2017). Normalizing an inher-
ently nonnormal variable could mean that the target
variable is not actually studied. Another consideration
is whether the metric of the original variable is mean-
ingful, such as postoperative survival time in years.
Transformation means that the original meaningful
metric is lost, which could be a sacrifice. Described in
Topic Box 4.3 are types of normalizing transformations
that might work—there is no guarantee—with practical
suggestions for using them. Exercise 6 asks you to find
a normalizing transformation for a small data set.

TOPIC BOX 4.3

Normalizing Transformations
Three kinds of normalizing transformations are described next with suggestions for their use:

1. Positive skewness. Before applying these transformations, add a constant to the scores so that
the lowest modified score is 1.0. A basic operation is the square root transformation, or X1/2,
which works by compressing differences between scores in the upper end of the distribution
more than the differences between lower scores. Logarithmic transformations are another option.
A logarithm is a power (exponent) to which a base number must be raised to get the original
number, such as 102 = 100, so the logarithm of 100 in base 10 is 2. Distributions with extremely
high scores may require a transformation with a higher base, such as log10 X, but a lower base
may suffice for less extreme cases, such as the natural base e (approximately 2.7183) for the
transformation loge X = ln X. The inverse function 1/X is an option for even more severe skewness.
Because inverting scores reverses their order, (1) reflect (reverse) the original scores (multiply them
by –1.0) and (2) add a constant to the reflected scores so that the maximum score is at least 1.0
before taking the inverse.
2. Negative skewness. All the transformations just mentioned also work for negative skewness when
they are applied as follows: First, reflect the scores, and then add a constant so that the lowest
score equals 1.0. Next, apply the transformation, and reflect the scores again to restore their
original order.
3. Other types of nonnormality. Odd-root functions (e.g., X1/3) and sine functions tend to bring in
outliers from both tails of the distribution toward the mean. Odd-­powered polynomial functions,
such as X3, may help for negative kurtosis. If the scores are proportions, the arcsine square root
transformation, or arcsin X1/2, may help to normalize the distribution.

There are other types of normalizing functions, and this is one of their problems: It can be difficult to
find a transformation that works with a particular distribution. The Box–Cox transformations (Box &
Cox, 1964) may require less trial and error. The most common form is defined next for positive scores only:
X l −1
(l )  , if l ≠ 0;
X = l (4.10)
 log X , if l =0.

where the exponent l is a constant that normalizes the scores. Computer software for Box–Cox transforma-
tions attempts to find the optimal value of l for a particular distribution. There are other variations of the
Box–Cox transformation, some of which can be applied in regression analyses to deal with heteroscedas-
ticity (Osborne, 2013).

Pt1Kline5E.indd 61 3/22/2023 3:52:40 PM

 62 Concepts, Standards, and Tools
Relative Variances
In an ill-scaled covariance matrix, the ratio of the
largest to the smallest variance is greater than, say,
100.0. Most estimation methods in SEM are iterative,
which means that initial estimates are derived by the
computer and then modified through subsequent cycles
of calculation. The goal is to derive better estimates
at each stage, estimates that progressively improve
the fit between model and data. When improvements
from step to step become sufficiently small (i.e., they
fall below the convergence criterion), iteration stops
because the solution is stable. But if the estimates do
not settle down to stable values, the process may fail.
One cause is variances that are very different: When
the computer adjusts the estimates from one step to the
next in an iterative method, the sizes of these changes
may be huge for variables with small variances but
trivial for variances with large variances. The whole
process may head toward worse rather than better fit.
To prevent the problem just described, variables with
extremely low or high variances can be rescaled by
multiplying their scores by a constant, which changes
the variance by a factor that equals the squared con-
stant. For example,
s X2 = 12.0 and sY2 = .12
so their variances differ by a factor of 100.0. Using the
constant .10, we can rescale X as follows:
2
s.10 X = .10 × 12.0 = .12
2 LEARN MORE
so now variables .10X and Y have the same variance, or
.12. Next, we rescale Y so that it has the same variance
as X, or 12.0, by applying the constant 10.0, or
2
s10.0Y = 102 × .12 = 12.0
Multiplying a variable by a constant is a linear transfor-
mation that changes its average and variance but not its
correlation with other variables. This is because linear
transformations do not alter relative distances between
scores. An example with real data follows.
Roth et al. (1989) administered measures of exercise,
hardiness (resiliency, tough mindedness; referred to as
“hardy” from this point), fitness, stress, and level of ill-
ness in a sample of 373 university students. Table 4.3
provides a summary matrix of these data. The largest
and smallest variances in this matrix (see the table) dif-
Pt1Kline5E.indd 62
fer by a factor of > 27,000, so the covariance matrix is
ill-scaled. I have seen older SEM computer tools fail to
analyze this matrix due to this characteristic. To pre-
vent this problem, I multiplied the original variables by
the constants listed in the table (including 1.0; i.e., no
change) in order to make their variances more uniform.
Among the rescaled variances, the largest variance is
only about 13 times greater than the smallest variance.
The rescaled matrix is not ill-scaled.
SUMMARY
The 80/20 rule of data analysis is a variation on the
Pareto principle, named after the Italian economist Vil-
fredo Pareto, that 80% of a nation’s wealth is owned by
20% of the people. In the context of data analysis, it
means that researchers should invest more time (i.e., at
least four times as much) screening and preparing their
data than actually conducting the substantive analy-
ses. Data screening in SEM includes the evaluation of
missing data patterns and extent; detection of outliers,
extreme collinearity, and whether data matrices are ill-
scaled; assessment of whether distributional assump-
tions for particular estimators are consistent with the
data; and complete reporting in written summaries of
the results on how particular problems were dealt with
in the analysis. Making the data file available to other
researchers is a strong statement of transparency.
Dong and Peng (2013) offer clear and accessible descrip-
tions of techniques for missing data, Manly and Wells (2015)
describe best practices for reporting about the use of MI,
and van Ginkel et al. (2020) review common misunderstand-
ings about MI.
Dong, Y., & Peng, C.-Y. J. (2013). Principled missing data
methods for researchers. SpringerPlus, 2(1), Article 222.
Manly, C. A., & Wells, R. S. (2015). Reporting the use of
multiple imputation for missing data in higher educa-
tion research. Research in Higher Education, 56(4),
397–409.
van Ginkel, J. R., Linting, M., Rippe, R. C. A., & van der
Voort, A. (2020). Rebutting existing misconceptions
about multiple imputation as a method for handling
missing data. Journal of Personality Assessment, 102(3).
297–308.
3/22/2023 3:52:40 PM
 Data Preparation 63

TABLE 4.3 Example of an Ill-Scaled Data Matrix

Variable 1 2 3 4 5
1. Exercise —
2. Hardy –.03 —
3. Fitness .39 .07 —
4. Stress –.05 –.23 –.13 —
5. Illness –.08 –.16 –.29 .34 —

Original M 40.90 0.00 67.10 4.80 716.70

Original s2 4,422.25 14.44 338.56 44.89 390,375.04

Constant 1.00 10.00 1.00 5.00 .10

Rescaled M 40.90 0.00 67.10 24.00 71.67

Rescaled s2 4,422.25 1,440.00 338.56 1,122.25 3,903.75

Rescaled SD 66.50 38.00 18.40 33.50 62.48

Note. These data (correlations, means, and variances) are from Roth et al. (1989); N = 373. Note that
low scores on the hardy measure used by these authors indicate greater hardiness. To avoid confusion
due to negative correlations, the signs of the correlations that involve the hardy measure were reversed
before they were recorded in this table.

EXERCISES

1. Reproduce the covariance matrix in the middle of Locate in your figure values for the extreme of the
Table 4.1 from the correlations and standard devia- lower whisker, lower hinge (H1), median, upper
tions at the bottom of the table. hinge (H2), extreme for the upper whisker, and any
outlier beyond 1.5 × (H2 – H1) from its respective
2. Given covXY = 13.00, s X = 12.00, and sY = 10.00,
2 2 hinge:
show that the corresponding correlation is out of
bounds. Score Frequency Score Frequency
10 5 15 5
11 15 16 4
3. Listed next as (X, W, Y) with missing observations 12 14 17 1
indicated by “—” are scores for 7 cases: 13 13 27 1
(42, 13, 8), (34, 12, 10), (22, —, 12), (—, 8, 14) 14 5
(24, 7, 16), (16, 10, —), (30, 10, —)
5. A researcher finds for a continuous variable that
Compute the covariance matrix for these data using
skewness is 1.97 and kurtosis is 6.90, and concludes
pairwise deletion. Show that the resulting covari-
that the distribution is normal. Comment.
ance matrix is NPD and also that the corresponding
correlation matrix has an out-of-bounds value.
6. Find a normalizing transformation for the data in
Exercise 4. Do not remove the outlier.
4. Generate a box plot for the N = 63 scores listed next.

Pt1Kline5E.indd 63 3/22/2023 3:52:40 PM

 64 Concepts, Standards, and Tools
Appendix 4.A
Steps of Multiple Imputation
The three basic steps in MI—the imputation step, the
analysis (estimation) step, and the pooling (combina-
tion) step—and corresponding decisions required at
each point are summarized next. It is not possible in
this overview to give detailed descriptions about sta-
tistical options, especially in the imputation step, but
see Allison (2012), Dong and Peng (2013), van Buuren
(2018), or van Ginkel et al. (2020) for more informa-
tion.
STEP 1. IMPUTATION
The imputation step involves the analysis of the vari-
ables that make up the imputation model and also
the method by which random variation is modeled
and incorporated in generating imputed scores. That
method should match the level of measurement for
incomplete variables (e.g., continuous vs. categorical),
whether data loss is univariate or multivariate (occurs
on a single vs. multiple variables), and whether the pat-
tern is monotone or general. Monotone missing data
means that variables can be ordered and data loss at
a particular point means that all subsequent observa-
tions are missing. Dropout in a longitudinal study is an
example of monotone data loss; any other pattern is a
general (nonmonotone) missing data.
An option for univariate data loss is the regres-
sion method, which is based on linear regression and
assumes multivariate normal distributions for continu-
ous variables. It works by regressing an incomplete
variable on other variables with complete data. Next,
imputed regression coefficients are selected from the
sampling distributions for the coefficients in the first
analysis, and imputed scores are generated from the
values of the imputed coefficients and a random error
term. These steps are repeated for k times, the num-
ber of imputed data sets. A variation is the predic-
tive mean matching method, which does not require
normality and imputes values “borrowed” from other
records called donor cases. This method generates pre-
dicted scores for all cases, including those with com-
plete data, and randomly selects as an imputed score
Pt1Kline5E.indd 64
from among cases with observed scores similar to the
predicted score for an incomplete case. The number of
complete cases in the donor set can be specified by the
researcher with about 5–10 cases as a typical range. In
small samples, though, 10 cases in the donor set may
include too many dissimilar records, especially for few
predictors (Allison, 2015).
A method for multivariate data loss is fully condi-
tional specification, also called multivariate imputa-
tion by chained equations (MICE) and sequential
regression multivariate imputation. It generates a
series of conditional models, one for each incomplete
variable in the data set, and the method requires speci-
fication of the imputation model for each incomplete
variable. Because the method is based on the separate
distributions of each incomplete variable, it can be
applied to continuous variables with nonnormal dis-
tributions or to variables that are not continuous, such
as ordinal data. Initial imputed values are randomly
selected from the observed data, and these values are
subsequently improved for each incomplete variable
through iterative selection from conditional distribu-
tions estimated by the observed and imputed scores.
The whole process then cycles through all incomplete
variables. The method is flexible but can converge to
incompatible conditional models depending on the
order of the univariate imputation steps (van Buuren,
2018).
Another multivariate method for arbitrary data loss
patterns is based on the Markov Chain Monte Carlo
(MCMC) approach that randomly samples from theo-
retical probability distributions, in this case from pre-
dictive distributions for the missing data, and these
draws become the imputed scores. A Markov Chain is
a probability model in which the likelihood of an event
depends only on the state of the previous event. That
is, the probability of a future event can be estimated
just as well from the current state of the event as when
knowing the full history of events. A “chain” thus con-
cerns multiple simulated random draws from the same
distribution.
In the application of the MCMC method in MI, it
is assumed that the underlying complete data follow a
multivariate normal distribution, and the computer sim-
ulates draws from such distributions over an iterative
sequence of paired steps called the I-step (imputation,
but not meaning the first step in MI) and the P-step
(posterior). At the I-step, imputed scores are drawn for
each incomplete case from the predictive distribution
in the current iteration. Next, in the P-step, the param-
3/22/2023 3:52:40 PM

eters of the predictive distribution are updated by draws
from the posterior distribution. The “chain” consists of
I-step/P-step pairs over iterations. Some implementa-
tions allow for a burn-in period, or default number
of iterations (e.g., 200) before the first set of imputed
values is drawn. One rationale for this option is to dis-
sipate the effects of the distribution from the prior itera-
tion that may differ appreciably from those of the target
distribution, but burn-in is not an inherent feature of
the MCMC method, and there are other methods to
find good starting points for drawing imputed scores
(Geyer, 2011).
The MCMC method is generally robust against mul-
tivariate nonnormality in large samples (Demirtas et
al., 2008), but its use can be problematic with categori-
cal variables. For example, the practice of imputing
values based on normal distributions and then rounding
to the nearest integer or to the nearest plausible value
can yield very biased results for categorical variables
(Allison, 2012). The chained equations method (MICE
algorithm) described earlier is an alternative. Another
possibility is to use a method for the imputation step
based on logistic, loglinear, or other statistical model
for categorical variables—see Audigier et al. (2017) for
more information and examples.
Yet another multivariate method for arbitrary miss-
ing data patterns is the expectation–maximization
(EM) algorithm, which is a general purpose itera-
tive procedure to find ML estimates for parameters
that involve latent variables. In the context of MI, the
method alternates between the E-step (expectation),
in which missing scores are imputed from predicted
distributions for the missing data, and the M-step
(maximization), where parameters for the distribution
of both the missing and observed data, such as means
and covariances, are estimated using ML before the
cycle repeats over the E- and M-steps until there are k
imputed scores for each missing observation. Because
the EM algorithm is a general method, it can be used
outside of MI to directly estimate parameters for latent
variables without imputing scores for individual cases
(Dempster et al., 1977). Estimation of standard errors
may be more accurate in the FIML method for incom-
plete data sets that is described in Chapter 9 (Dong &
Peng, 2013).
Besides the algorithm, the researcher must also
specify the number of scores to be imputed for each
missing observation, or k. Suggestions in older works
were roughly k = 3–10 based on the relative efficiency
of imputing these numbers of times against a theoreti-
Pt1Kline5E.indd 65
Data Preparation 65
cal infinite number of imputations (Patrician, 2002).
Greater numbers of imputations, such as k = 100–200,
are recommended in more recent works (Graham et al.,
2007; Little, 2013) based on statistical power and the
fraction of missing information (FMI), which is not
the rate of missing data. The FMI is instead the propor-
tion of variation in parameter estimates due to nonre-
sponse. That is, it quantifies the amount of parameter
information lost to nonresponse (Lang & Little, 2018),
so in this way the FMI is analogous to an R2 statistic for
missing data (Enders, 2010). For example, if FMI = .03
for a particular parameter, then the loss of efficiency
due to missing data is 3%; that is, the estimate based
on incomplete data is 97% as efficient compared to
what it would have been with no missing data (Sava-
lei & Rhemtulla, 2012). Wagner (2010) suggested that
researchers should consistently report the value of the
FMI when analyzing incomplete data sets.
The FMI is a function of k and the ratio of the
between-imputation variance over the total variance,
or the sum of between-imputation variance and the
within-imputation variance. The within-imputation
variance is the average error variance associated with
parameter estimates within all imputed data sets. Con-
ceptually, it estimates what the error variance would be
if there were no missing scores. The between-imputa-
tion variance is the variation in estimates over the k
imputations. As more information is lost due to missing
data, the between variance will increase, but when the
data set includes good predictors of the variables with
missing data, both the between variance and the FMI
are expected to decrease (Wagner, 2010). In general, the
greater the covariances among the observed variables,
the lower the value of the FMI (Little, 2013). Dong and
Peng (2013, p. 5) described equations for computing the
FMI, and some computer procedures for MI print the
value of this statistic in the output.
STEP 2. ANALYSIS
After k complete data sets are generated in the impu-
tation step, next comes the analysis step, which con-
cerns the analysis model. Parameters are estimated for
effects of substantive interest in each of the k imputed
data sets. Ideally, the variables in the imputation model
would be the same as those in the analysis model except
for auxiliary variables (if any). But as the imputation
and analysis models are based on increasingly smaller
sets of overlapping variables, then results from the
3/22/2023 3:52:40 PM
 66 Concepts, Standards, and Tools
imputation step may not be very meaningful in the
analysis step.
The imputation model should also be general enough
in terms of assumptions about distributions or func-
tional relations to reflect the data in the analysis model.
Suppose that normality is assumed for a variable in the
imputation model, but the actual distribution is severely
nonnormal. Results in the analysis model based on
means and covariances, such as regression coefficients,
may not be grossly inaccurate, but estimates of p val-
ues or bounds for confidence interval could be severely
distorted (Schafer, 1999). Enders (2010) offers sugges-
tions for representing interactive effects of continuous
or categorical variables in both models.
STEP 3. COMBINATION
Results from the analysis step from each of the k
imputed data sets are synthesized in the combination
(pooling) step. The final parameter estimate is the
average over the k results for that parameter, and its
standard error is estimated based on both the within-
and between-imputation variances. You should also be
aware of the issues listed next:
1. There is no unique set of results in MI: This is
because imputed scores are generated through
simulated random sampling. Thus, results for the
Pt1Kline5E.indd 66
same model and incomplete data file can and will
change each time the procedure is repeated unless a
random seed that determines the starting point for
generating random numbers is specified.
2. Indeterminacy also applies to the data: The research-
er’s model is analyzed in k imputed data sets, so
there is no definitive summary data matrix. Thus,
the results could not be reproduced in a secondary
analysis conducted with a single data matrix (all k
matrices would be required). But the researcher can
still make available the original raw data file.
3. There is little doubt that MI generates better esti-
mates than classical techniques in reasonably large
samples when the data loss pattern is MAR instead
of MCAR (Schafer & Graham, 2002): It might also
reduce bias compared with classical techniques
when data loss patterns are both MAR and MNAR,
if variables can be added that measure the miss-
ing data mechanism, but there is no guarantee (van
Ginkel et al., 2020).
There are special statistical techniques for analyzing
MNAR data (Rubin & Little, 2020), but they are not yet
widely used and are more difficult to apply than meth-
ods that assume MAR. Methods for MNAR data loss
mechanisms are becoming available in SEM computer
tools such as Mplus. Galimard et al. (2016) described
adaptations of MI for MNAR mechanisms.
3/22/2023 3:52:40 PM
 5

Computer Tools

Two categories of computer tools for traditional SEM are described in this chapter: freely available software
and commercial software. Free software includes packages for conducting SEM analyses in the R comput-
ing environment (e.g., lavaan, semTools) and stand-alone computer tools with a graphical user interface
(GUI) that require no larger software environment (e.g., JASP, Wnyx). Free computer tools have become
increasingly capable to the point where they can replace commercial tools for basically all but the very most
advanced kinds of analyses. Modern SEM computer programs—both free and commercial—are generally
easier to use than their predecessors. But greater user-friendliness of contemporary SEM computer tools
should not lull the researcher into thinking that SEM is easy or requires minimal conceptual understanding.
Features of computer programs can change quickly with new versions, so check the sources listed next for
the most up-to-date descriptions. Computer tools for nonparametric SEM are described in Chapter 6, and
Chapter 16 covers software options for composite SEM. In a work describing the ideas of the Canadian
communication theorist Herbert Marshall McLuhan, Culkin (1967, p. 70) wrote, “We shape our tools and
thereafter they shape us.” I hope that computer use sharpens, rather than dulls, your ability to think critically
about SEM.

EASE OF USE, NOT SUSPENSION by drawing it in onscreen using geometric symbols

OF JUDGMENT
The first widely available SEM computer tool was
LISREL III (Jöreskog & Sörbom, 1976). At the time,
LISREL and related applications were not easy to use
because they required the generation of rather arcane
code, and were available only on mainframe comput-
ers with stark command-line user interfaces. The abun-
dance of relatively inexpensive, yet capable, personal
computers greatly changed things. Statistical software
with a GUI is generally easier to use than their char-
acter-based counterparts. Indeed, user-friendliness in
modern SEM computer tools is a near-revolution com-
pared with older programs.
Most SEM computer tools still permit the user to
write code in that application’s native syntax. Some
programs offer the alternative of specifying the model
such as boxes, circles, and lines with arrowheads on
one or both ends. Next, the program automatically
translates the model graphic into lines of code, which
are then executed. Thus, (1) the user need not know
very much (if anything) about how to write syntax in
order to run an SEM analysis, and (2) the role for tech-
nical programming skills is reduced. For researchers
who understand the basic concepts of SEM, this devel-
opment can only be a plus—anything the reduces the
drudgery and gets one to the results quicker is a benefit.
But there are potential drawbacks to push-button
modeling. For example, no- or low-effort program-
ming could encourage the use of SEM in uninformed
or careless ways. This is why it is more important than
ever to be familiar with the conceptual and statistical
bases of SEM. Computer programs, however easy to
use, should only be the tools of your knowledge and

67

Pt1Kline5E.indd 67 3/22/2023 3:52:40 PM

 68 Concepts, Standards, and Tools
not its master. Steiger (2001) noted that emphasis on
ease of use of statistical computer tools can give begin-
ners the impression that SEM is easy, but the reality is
that things can and do go wrong in SEM. Beginners
often quickly realize that analyses fail because of tech-
nical problems, including a terminated program run
with cryptic error messages or uninterpretable output.
These things happen because (1) actual research prob-
lems can be quite technical, and the availability of user-
friendly software does not change this fact. Also, (2)
computer tools are not perfect and, thus, are incapable
of detecting or preventing all failure conditions. That
is the reason why this book places so much emphasis
on conceptual knowledge instead of teaching you how
to use a particular computer tool: In order to deal with
problems in the analysis when—not if—they occur,
you must understand what went wrong and why.
HUMAN–COMPUTER INTERACTION
There are three basic ways to interact with SEM com-
puter tools:
1. Batch processing, where the user writes syntax
that specifies the model, data, analysis, and output.
Next, the syntax is executed through some form of
a “run” command.
2. Drawing editor, where the user draws the model on
screen. When the diagram is finished, the analysis
is run in the GUI.
3. Templates or menus, where the model and analy-
sis are specified as the user clicks with the mouse
on interface elements such as text fields, pull-down
menus, or radio buttons.
Batch mode is for users who know the syntax for a
particular SEM computer tool. In contrast, knowledge
of syntax is generally unnecessary when using a draw-
ing editor or templates. But even here some knowledge
of syntax can help. In LISREL, for example, both its
drawing editor and template-based mode of interac-
tion automatically write syntax in a window that must
be run by the user. That syntax can be edited, and
sometimes a problem in model specification is appar-
ent in the syntax, which the user can correct, if they
understand the syntax. In contrast, the drawing edi-
tor in IBM SPSS Amos analyzes the model and data
without generating editable (or even viewable) syntax
Pt1Kline5E.indd 68
(Arbuckle, 2021), although Amos does offer a separate
syntax editor. Although drawing editors are popular
with beginners, there are potential drawbacks—see
Topic Box 5.1.
TIPS FOR SEM PROGRAMMING
Listed next are suggestions for using SEM computer
tools; see also Little (2013, pp. 25–27):
1. Learn from the examples of others. Annotated syn-
tax, data, and output files for all detailed analyses
can be downloaded from the book’s website. Read-
ers can open these files on their own computers
without installing any special software. There are
additional online resources with syntax examples
for lavaan,1 Mplus,2 LISREL,3 and other SEM
computer tools.
2. Annotate your syntax files. Comments are usu-
ally designated by special symbols, such as *, #,
or !, that are ignored by the computer. Use com-
ments to describe the specified model, data, and
requested output. Explain the creation of any new
variable. Such information is useful for colleagues
or students who did not conduct the analysis, but
who need to understand it. Annotation also helps
researchers to know just what they did in a particu-
lar analysis days, weeks, months, or even years ago.
Without sufficient comments, one quickly forgets.
3. Keep it simple. Sometimes beginners try to analyze
models that are too complicated, and such analyses
are more likely to fail. With more syntax or screen
space in a drawing editor for a complex model,
there are also more opportunities for making mis-
takes. It can also be hard to tell whether a very com-
plex model is identified. If the researcher does not
know that their model is not really identified, the
failure of the analysis may be wrongly attributed to
a syntax error or problem with the data.
4. Build it up. Start instead with a simpler model that
you know is identified. Try to get the analysis of
the initial model to successfully run. Then build up
1 https://www.lavaan.ugent.be/tutorial/index.html
2 https://www.statmodel.com/ugexcerpts.shtml
3 https://ssicentral.com/index.php/products/lisrel/lisrel-
examples/
3/22/2023 3:52:40 PM
 Computer Tools 69

TOPIC BOX 5.1

Graphical Isn’t Always Better

The potential drawbacks of graphical editors in SEM computer tools are outlined next. They explain why
some researchers switch from a drawing editor when first learning about SEM to working in batch mode
as they gain experience:

1. It can be tedious to draw onscreen a complex model with many variables, such as numerous
repeated measures variables in a longitudinal design or dozens of items on a questionnaire in a
factor analysis. This is because the screen quickly fills up with graphical elements. The resulting
visual clutter can make it difficult to keep track of what you are doing.
2. Specifying analyses where models are simultaneously fitted to data from two or more samples can
be difficult. This is because it may be necessary to look through different screens or windows in
order to find information about data or model specification for each sample.
3. It is easier to annotate the analysis by putting comments in a syntax file compared with working
in a graphical editor, which may not support user-­supplied comments. It is so easy to lose track of
what you have done in an analysis without detailed comments. Thus, using a drawing editor that
does not allow annotations can engender carelessness in record keeping (Little, 2013).
4. Debugging a syntax file after a failed analysis is generally simpler than doing so in a graphical
editor, which may require clicking on multiple graphical elements to inspect specifications in sepa-
rate dialogs boxes or windows. In contrast, everything that specifies the analysis (i.e., commands)
can be viewed in a single place when editing a syntax file.
5. The format of files generated in graphical editors is typically proprietary, which means that they
generally cannot be opened or edited with a different computer tool. In contrast, syntax files are
usually text (ASCII) files that can be opened or edited in any basic text editor, including word
processors. Thus, sharing an analysis with other researchers is generally easier with syntax files.
Including syntax files in supplemental materials for journal articles supports both transparency in
reporting and accessibility for readers.
6. Certain kinds of advanced analyses or options may be available in syntax only. One reason is
that although model diagrams for more basic SEM analyses are more or less standard, this is
less true for more advanced applications such as multilevel analyses, for which there is no single
graphical standard (Curran & Bauer, 2007).
7. There is evidence that graphical user interfaces can help novices who are not trained in informa-
tion technology or in computer science to learn and perform basic tasks faster and in fewer steps.
But graphical interfaces can also impede or slow down expert users compared with text-based
interfaces, and just because an interface is graphical does not guarantee ease of use or reduced
cognitive load (Chen & Zhang, 2007).
8. It seems that it would be easy to generate a publication-­quality diagram in an SEM drawing edi-
tor, but this is not exactly true. Drawing editors may use a fixed symbol set that does not include
special symbols that you want to appear in the diagram. There may be limited options for adjust-
ing the appearance of this diagram (e.g., changing font or line widths). Graphs generated by
drawing editors may be rendered in relatively low resolution that is fine for the computer monitor
but not for display in a printed (paper or virtual) document. There are R packages that can gener-
(continued)

Pt1Kline5E.indd 69 3/22/2023 3:52:41 PM

 70 Concepts, Standards, and Tools

ate high-­quality model diagrams, including semPlot (Epskamp, 2022), but they can be frustrating
to use: The diagram is specified in syntax, but the diagram generated by executing that syntax
cannot be directly edited, so if something is wrong, then it is back to the syntax (repeat).

Another option is a professional diagramming and graphics computer tool, such as Microsoft Visio,
that features templates and tools for drawing predefined shapes, such as circles, squares, or lines with
optional arrowheads on one or both ends. A drawback is that professional graphics software can be
relatively expensive, although there are free alternatives like LibreOffice Draw with generally more modest
capabilities. But using comprehensive drawing software to create diagrams of structural equation models
can seem like overkill. This is because only a small fraction of the program functionalities is used to create
model diagrams, which are composed of just a few kinds of graphical elements.
Here is a trade secret I’ll share with you: All model diagrams in this book were created using nothing
other than Microsoft Word Shapes (rectangles, ovals, text boxes, etc.) that are grouped together. Maybe
I am biased, but I think these diagrams are not too shoddy. Sometimes you can do a lot with a simple but
flexible tool. Yes, it takes a lot of time to make a publication-­quality model diagram in Word—or in any
graphical computer tool—but once you make a few examples, you can reuse graphical elements, such as
those for common factors and indicators, in future diagrams. Mai et al. (2022) described semdiag, a free,
open source web application for drawing model diagrams in SEM. An online version of semdiag is avail-
able at https://semdiag.psychstat.org/ There is also a free, open-­source diagramming application that can
be used online through a web browser or as a downloaded application for Windows, Apple (macOS),
Linux, or Google Chrome OS platform computers available at https://www.diagrams.net/

the model by adding parameters that reflect your
hypotheses until the target model is eventually
specified. If the analysis fails after adding a par-
ticular parameter, the reason may be identification.
5. Comment out instead of delete. In analyses where
models are progressively simplified (trimmed)
instead of built, the researcher can comment out
part of the syntax, or deactivate it, by designating
those commands as comments in the next analysis.
This method preserves the original code as a record
of the changes.
6. Lend the computer a hand. Sometimes iterative
estimation fails because the computer needs better
starting values, or initial estimates of model param-
eters, to find a converged solution. Most SEM com-
puter tools generate their own starting values, but
computer-derived values do not always lead to con-
verged solutions. Fortunately, SEM computer tools
generally allow users to specify starting values that
override computer defaults. Suggestions for speci-
fying starting values for different types of models
are offered later in the book.
COMMERCIAL VERSUS FREE
COMPUTER TOOLS
I am often asked, what is the best software package
for SEM? My answer has two parts: (1) There is no
single “best” package because there are now many
capable options available. (2) What is available to you
at the lowest cost? For example, many universities have
site licenses that permit researchers and students to
use commercial SEM computer tools free of charge.
Advantages of commercial SEM programs include reg-
ular updates, user support including a single point of
contact for problems, and complete program manuals
and documentation, sometimes with numerous analysis
examples and data sets.
Commercial applications are less advantageous for
users without institutional or grant funds for software.
As with other specialized software for multivariate sta-
tistical analyses, costs to purchase or license SEM com-
puter tools can be rather expensive and can range from
several hundred to over $1,000 a year or more, although
discounts may be offered for academic users. There are
also researchers who simply prefer using free or open-

Pt1Kline5E.indd 70 3/22/2023 3:52:41 PM

 Computer Tools 71

source software. Open source means that software
development is decentralized by making the source
code available so that individual users can modify the
software and redistribute or publish their version back
to the community. Most, but not all, open-source soft-
ware is free, and just because a computer tool is free
does not mean that it is also open source.
Among researchers in the behavioral sciences, the
best-known computing environment for data manage-
ment, statistical analysis, and graphics that is both free
and open source is R (R Core Team, 2022). A basic
installation of R has many of the same capabilities for
statistical analysis as commercial products such as
SPSS or SAS/STAT, and thousands of free packages
extend its range even further. This includes several
packages for SEM, some of which can analyze a wide
range of structural equation models with capabilities
similar to those of commercial software. This is espe-
cially true for lavaan, described momentarily. Its
extensive capabilities are why I used lavaan in most
detailed analysis examples described in this book.
There are potential obstacles to using the noncom-
mercial software programs just mentioned. One is doc-
umentation. Although there are articles, chapters, or
even entire books written for applied researchers who
use lavaan for SEM, standard R package documenta-
tion is written in a technical way that assumes knowl-
edge of object-oriented programming. Specifically,
the typical “manual” for an R package consists of an
alphabetical listing of functions with terse references to
objects, methods, and data types. Such documents can
be pretty cryptic for researchers without strong pro-
gramming skills. It is also true that numerical output
in R is not always “pretty,” that is, the output is format-
ted in ways that are not easy to read at first glance. An
example is scientific notation in which very small or
very large numbers are represented in a simpler form
(e.g., 1.0E-4 for the number .0001). For researchers
unwilling to deal with these challenges, a commercial
tool may be a better option, if cost is no issue.
R PACKAGES FOR SEM
Listed in Table 5.1 are major R packages for SEM anal-
yses with citations and descriptions. All packages in

TABLE 5.1. Major R Packages for SEM

Name Citation Description
General model fitting
sem Fox et al. (2022) Basic but broad capabilities for analyzing structural
equation models

lavaan Rosseel et al. LAtent VAriable ANalysis, estimates a wide range of

(2023) models with capabilities that rival commercial tools

OpenMx Boker et al. Matrix processor and numerical optimizer that can be
(2022) used with multicore computers or networked computers

piecewiseSEM Lefcheck (2020) Single-equation estimation of path models with global fits
test based on predicted conditional independencies

Utilities and tools

semTools Jorgensen et al. Extends lavaan capabilities, includes utilities for power
(2022) analysis and other special types of analyses

systemfit Henningsen & Estimates simultaneous linear equations with ordinary

Hamann (2022) least squares and instrumental variable methods

bmem Zhang & Wang Estimators and bootstrapped confidence intervals for
(2022) indirect effects with incomplete data

Pt1Kline5E.indd 71 3/22/2023 3:52:41 PM

 72 Concepts, Standards, and Tools
the table except OpenMx were used in analyses for this
book. The sem package was probably the very first for
analyzing a wide range of structural equation models,
but there are no plans for major development beyond
its current form.4 It is still quite usable, though, and
it offers a range of estimators, including instrumental
variable methods. The sem package can also be inte-
grated with external R packages for multiple imputa-
tion (MI), bootstrapping, and estimating polychoric
correlations, which further extend its capabilities (Fox
et al., 2022).
The lavaan package for SEM (see Table 5.1) has
extensive analysis capabilities and has been updated
several times since it was first released in 2011. The
lavaan package has estimators for continuous, binary,
and ordinal data, and there are capabilities for item
response theory (IRT), latent class, mixture model-
ing, and multilevel analyses. Modern options for han-
dling missing data include full information maximum
likelihood (FIML) and MI, but use of an external R
package, such as semTools, is required for the latter.
Resources for using lavaan include books (Beaujean,
2014; Gana & Broc, 2019), journal articles (Andersen,
2022; Svetina et al., 2020), a website,5 and a discussion
group.6
The OpenMx package is a powerful matrix proces-
sor and numerical optimization tool with capabilities
for analyzing structural equation models, multilevel
models, mixture models, and models of genetic relat-
edness—see Table 5.1. Models can be interactively
built one step at a time, which allows code debugging
in smaller steps. The package supports multicore com-
puters, where multiple central processing units in the
same computer operate in parallel on an analysis, and
distributed processing over separate computers work-
ing in parallel clusters. These capabilities support the
analysis of extremely large data sets. Missing data are
handled through the FIML method. There is an online
user guide,7 and the OpenMx community wiki offers
tutorials, example models, and forums.8 The syntax is
relatively complex but highly flexible once mastered—
see Neale et al. (2016) for examples.
4 J. Fox (personal communication, November 20, 2020).
5 https://www.lavaan.ugent.be/
6 https://groups.google.com/g/lavaan
7 https://vipbg.vcu.edu/vipbg/OpenMx2/docs//OpenMx/latest/
8 https://openmx.ssri.psu.edu/wiki/main-page
Pt1Kline5E.indd 72
The eponymously named package piecewiseSEM
(Table 5.1) supports the method of piecewise SEM,
described in Chapter 8. Briefly, the method features
(1) single-equation (local) estimation of observed vari-
able path models and (2) a global test of model–data
correspondence (i.e., goodness-of-fit) based on Pearl’s
(2009) approach to nonparametric structural equation
modeling covered in the next chapter. Local estimation
means that the equation for each outcome variable is
separately analyzed instead of the computer attempt-
ing to simultaneously estimate all model parameters,
or global estimation. Piecewise SEM is best known in
biology and ecology, but it offers potential benefits for
researchers in the behavioral sciences, too.
Listed in the lower part of Table 5.1 are “toolbox”
R packages for special kinds of analyses in SEM and
other statistical techniques. The semTools package
extends the capabilities of lavaan to exploratory fac-
tor analysis (EFA) or estimation of interactive effects of
latent variables. The same package can also estimate
power, conduct Monte Carlo simulations, correct sig-
nificance test results for small sample sizes, estimate
the reliability of factor measurement, and combine
results from MI. It has a special function that uses the
Kaiser and Dickman (1962) method to generate a raw
data file of a specified size from an input covariance
matrix, where all descriptive statistics of generated
raw scores exactly match those of the specified cova-
riance matrix (i.e., the scores are generated with no
added sampling error). This capability is handy when
the researcher has a summary covariance matrix but
no raw scores, such as from a journal article, and the
researcher wants to conduct a secondary analysis with a
computer tool that requires raw scores. A GitHub wiki
for semTools is available.9
The R package systemfit (Table 5.1) estimates sys-
tems of linear equations for observed variables. It has
capabilities for OLS and instrumental variable methods.
Versions of the latter method can be used for an analysis
called seemingly unrelated regressions (SUR), which
is actually a misnomer because the regressions for sep-
arate outcome variables are correlated due to overlap-
ping (correlated) error terms. In contrast, error terms
for multiple outcomes in standard OLS regression are
assumed to be independent. There are also functions in
systemfit that evaluate the overall fit of regression mod-
els to the data and conduct diagnostic tests for instru-
mental variable methods. The bmem package supports
9 https://github.com/simsem/semTools/wiki
3/22/2023 3:52:41 PM
 Computer Tools 73

the generation of confidence intervals for estimates of
indirect causal effects in mediation analyses when there
are missing data (see the table). It also has capabilities
for estimating statistical power for tests of indirect
effects in a variety of mediational analyses, including
ones where the intervening variable is modeled as a
latent variable.
FREE SEM SOFTWARE
WITH GRAPHICAL USER INTERFACES
The :nyx (pronounced “onyx”) program for SEM runs
under the Java Runtime Environment (version 1.6 or
later) on Windows, Apple (macOS), or Unix/Linux plat-
form computers (von Oertzen et al., 2015). It is a graph-
ical computing environment for creating and analyzing
structural equation models that can be freely down-
loaded.10 There is no native programming language
in :nyx. Instead, the user draws the model onscreen.
After associating a data file with the diagram, estima-
tion of model parameters automatically begins. Miss-
ing data are handled by the FIML estimator. There is a
post-analysis option to synthesize raw data files where
cases are selected with sampling error from a hypo-
thetical population where model parameters equal the
sample estimates. The program can also automatically
generate syntax for Mplus, sem, lavaan, or OpenMx
that specifies the model represented in the diagram.
10 http://onyx.brandmaier.de/
The JASP program is an open-source, integrated
application for both traditional (frequentist) and Bayes-
ian statistical analyses (JASP Team, 2022).11 There
are versions for Windows, MacIntosh, and Linux plat-
form computers.12 It features a GUI that is intuitive
even for relatively inexperienced users of statistical
computer tools. Statistical capabilities include uni-
variate and multivariate analysis of variance (ANOVA,
MANOVA), regression analysis (linear, linear mixed,
logistic) and EFA, among others. There is a special
module for SEM, for which users enter lavaan syn-
tax to specify the model in a special text window, but
output options are controlled in the JASP user inter-
face. The SEM module also offers separate options for
analyzing mediation models and latent growth models.
COMMERCIAL SEM COMPUTER TOOLS
Listed in the top part of Table 5.2 are stand-alone com-
mercial programs for SEM that do not require a larger
computing environment. All four of these computer
tools—EQS, IBM SPSS Amos (hereafter just “Amos”),
LISREL, and Mplus—allow the user to work in batch
mode (syntax) or specify the model through a draw-
ing editor or templates. The drawing editors in EQS,
11 The
acronym stands for Jeffreys’s Amazing Statistics Program,
named after the British mathematician Harold Jeffreys for his
work on Bayesian probability theory.
12 https://jasp-stats.org/

TABLE 5.2. Major Commercial Software for SEM

Software Environment needed Batch (syntax) Drawing editor Template or menu
Stand-alone programs
EQS — 9 9 9
IBM SPSS Amos — 9 9 9
LISREL — 9 9 9
Mplus — 9 9 9

Procedures or commands in larger environments

Builder, sem, gsem Stata 9 9 9
CALIS SAS/STAT 9
SEPATH STATISTICA 9 9
CFA SYSTAT 9
RAMONA SYSTAT 9
 74 Concepts, Standards, and Tools
LISREL, and Mplus automatically write program syn-
tax in a separate window that can be edited, run, or
saved. The EQS and LISREL programs can be used in
all stages of the analysis from data entry and screening
to exploratory analyses to SEM. The Amos and Mplus
programs have somewhat more limited capabilities for
manipulating raw data files, so users of these applica-
tions may elect to prepare their data using other com-
puter tools for general statistical analyses.
There are versions of EQS (Equations) for Windows,
Apple (Mac/Unix), and Linux platform computers
(Bentler & Wu, 2020).13 Its syntax is straightforward
and based on the Bentler–Weeks representational
system, in which model parameters are regression
coefficients for effects on dependent variables and the
variances and covariances of independent variables
when means are not analyzed. All types of models
are thus set up in a consistent way. Special features of
EQS include the availability of estimators for ellipti-
cal distributions with varying degrees of kurtosis, but
not skew. Options for missing data include FIML for
either normal or nonnormal data or a method based on
the expectation–maximization (EM) algorithm. Other
features include bootstrapping, the ability to correctly
analyze a correlation matrix with no standard devia-
tions, EFA capabilities with parallel analysis and bifac-
tor rotation, and special syntax for multilevel analyses.
Books by Blunch (2016) and Dunn et al. (2020) are for
EQS users.
The Amos (Analysis of Moment Structures) pro-
gram is for Windows platform computers and does not
require the IBM SPSS environment to run (Arbuckle,
2021).14 It has two main parts, Amos Graphics, in which
users control the analysis by drawing the model on the
screen, and Amos Basic, its syntax editor that works in
batch mode. Amos Basic is also a language interpreter
and debugger for Microsoft Visual Studio VB.NET
or C# (“C-sharp”). Users with programming experi-
ence can write VB.NET or C# scripts that modify the
functionality of Amos Graphics. Other utilities include
a file manager, a random seed manager for bootstrap-
ping, and viewers for data and output files. The FIML
method is used for incomplete data files. Amos has
additional capabilities for Bayesian estimation, includ-
ing the generation of graphical posterior distributions.
13 https://mvsoft.com/
14 https://www.ibm.com/products/structural-equation-
modeling-sem
Pt1Kline5E.indd 74
Amos can also analyze mixture models with latent cat-
egorical variables either with training data, where some
cases are already classified but not the rest, or without
training data. Books by Blunch (2013), Byrne (2016),
and Collier (2020) support Amos users.
The LISREL (Linear Structural Relations) program
is the forerunner to all SEM computer programs.15
Available for Windows platform computers, LISREL
is actually a suite of applications that includes PRE-
LIS (“pre LISREL”), which generates and prepares raw
data files for analysis. It also has capabilities for diag-
nosing missing data patterns, MI, bootstrapping, and
Monte Carlo simulation (Jöreskog & Sörbom, 2021).
The FIML method for missing data is also available in
LISREL. There are additional bundled applications for
analyzing multilevel models and for fitting generalized
linear models to data from complex survey designs.
There are two LISREL programming languages, its
original syntax based on matrix algebra and SIMPLIS
(“simple LISREL”), which is not based on matrix alge-
bra nor does it require familiarity with the classic syn-
tax except to specify certain output options. The clas-
sic syntax is not easy to use until one has memorized
the whole system, but it is efficient: One can specify
a complex model with relatively few lines of code.
Some advanced capabilities, such as nonlinear param-
eter constraints, are unavailable when using the SIM-
PLIS language. Recent books about LISREL include
Jöreskog et al. (2016) and Viera (2011).
The Mplus program for Windows, Apple (macOS),
and Linux platform computers is divided into a base
program and three optional add-on modules (Muthén
& Muthén, 1998–2017).16 The Base Program for SEM
can analyze models with outcomes that are any combi-
nation of dichotomous, nominal, ordinal, censored, or
count variables. It can also analyze discrete- and con-
tinuous-time survival models. There are capabilities for
conducting exploratory factor analysis, bootstrapping,
Monte Carlo simulation, Bayesian estimation, and IRT
analyses. Special syntax supports the specification of
sampling weights—also called survey weights—that
correct for systematic differences in probability sam-
pling between target and sample proportions of cases
with specific demographic or other characteristics.
There is also special syntax for specifying latent growth
models and the analysis of indirect casual effects. Both
15 https://ssicentral.com/
16 https://www.statmodel.com/
3/22/2023 3:52:41 PM

MI and FIML methods are available for handling miss-
ing data under the assumption of MAR. Special meth-
ods in Mplus for data missing not at random (MNAR)
are described in Chapter 9.
The Multilevel Add-On estimates multilevel ver-
sions of the kinds of models analyzed in the Base Pro-
gram, and the Mixture Model Add-On estimates mix-
ture model versions, where the data are assumed to be
sampled from a mix of subpopulations that correspond
to levels of a latent categorical variable. The Combina-
tion Add-On contains all the features of the multilevel
and mixture model analyses just mentioned. Of all SEM
computer tools, Mplus can analyze perhaps the widest
range of statistical models, and some of the very newest
analysis capabilities can show up first in Mplus. Recent
books about Mplus include Finch and Bolin (2017),
Geiser (2021), Heck and Thomas (2015), and Wickrama
et al. (2022). See Geiser (2023) for examples of SEM
analyses conducted with both Mplus and lavaan.
Listed in the bottom part of Table 5.2 are SEM pro-
cedures or functions within larger software environ-
ments. Builder is the drawing editor for SEM in Stata,
and the commands sem and gsem are for specifying
models in syntax (StataCorp, 1985–2021).17 The sem
command analyzes models with continuous outcomes,
and the gsem (generalized SEM) command analyzes
outcomes that are continuous, dichotomous, categorical
(ordered or unordered), count, or censored variables.
The gsem command also has capabilities for multilevel
modeling in an SEM framework and for analyzing
models based on IRT. Stata automatically generates for
Builder diagrams the corresponding syntax, which can
be edited and saved as a text file. Special symbols in
Builder designate the underlying distribution (Gauss-
ian, Poisson, etc.) for observed variables and the cor-
responding link function (logit, probit, etc.). Missing
data are handled by FIML. The book by Acock (2013)
covers SEM in Stata.
The procedure CALIS (Covariance Analysis of Lin-
ear Structural Equations) in SAS/STAT for Windows
and Unix platform computers (SAS Institute, 2022)
is for SEM.18 It works in batch mode, and users can
specify their models using one of seven different pro-
gramming languages, including LISMOD, a matrix-
based syntax that corresponds to LISREL’s original
language, and LINEQS, an equation-based syntax
17 https://www.stata.com/
18 https://www.sas.com/
Pt1Kline5E.indd 75
Computer Tools 75
related to EQS’s original language, among others. The
missing data method in CALIS is FIML, but MI is also
available through the larger SAS/STAT environment.
The diagram for the analyzed model can be drawn
onscreen, but the researcher must specify the diagram
in syntax. O’Rourke and Hatcher (2013) describe exam-
ples of SEM analyses in SAS/STAT.
J. Steiger’s SEPATH (Structural Equation model-
ing and Path Analysis) is the SEM module in Statis-
tica, an integrated environment for data visualization,
simulation, and statistical analysis.19 There is a desk-
top version for Windows platform computers (TIBCO
Statistica, 2022) and an enterprise version with compa-
nywide server support. Models are specified using the
PATH1 programming language based on a represen-
tational system for SEM by McArdle and McDonald
(1984) introduced in Chapter 7. There are also tem-
plate-based options that are preprogrammed sequences
of graphical dialog boxes for specifying common types
of structural equation models—see Table 5.2. Special
features include the capabilities to correctly analyze a
correlation matrix with no standard deviations, gener-
ate simulated random samples for Monte Carlo studies,
and precisely control parameter estimation. A separate
power analysis module (also by J. Steiger) estimates the
power of various significance tests in SEM.
There are two SEM procedures in SYSTAT for
Windows platform computers (Systat Software Inc.,
2018).20 The user interacts with RAMONA (Reticular
Action Model or Near Approximation) by submitting
batch files in the general SYSTAT environment. Syn-
tax for RAMONA is relatively straightforward and
involves only two parameter matrices, one for covari-
ances between independent variables and the other for
direct effects on dependent variables. A second pro-
cedure is CFA, where the user specifies measurement
models and analysis options through graphical dialogs
or templates (Table 5.2). A special feature of both pro-
cedures just described includes the ability to correctly
fit a model to a correlation matrix only. There is a
“Restart” command that automatically takes parameter
estimates from a prior analysis as starting values in a
new analysis. This capability is convenient when evalu-
ating whether a complex model is identified. There
are relatively few other advanced features for SEM in
19 https://www.statistica.com/
20 https://systatsoftware.com/
3/22/2023 3:52:41 PM
 76 Concepts, Standards, and Tools
SYSTAT including the capability to simultaneously fit
a model to data from multiple groups.
SEM RESOURCES FOR OTHER
COMPUTING ENVIRONMENTS
Resources for conducting SEM analyses in more spe-
cialized computing environments are listed next:
1. The freely available semopy (Structural Equation
Models Optimization in Python) package for the
object-oriented programming language Python
relies on lavaan-like syntax, features relatively
fast processing times, and has capabilities for impu-
tation of missing values, analysis of ordinal out-
comes, and estimation of random coefficients mod-
els (Igolkina & Meshcheryakov, 2020).21 A second
edition of semopy is recently available (Meshch-
eryakov et al., 2021).
2. Mathematica is a software system for technical and
symbolic computation with capabilities for inter-
facing with programs written in other languages
(Wolfram Research, 2022).22 Oldenburg (2020)
describes Mathematica code for estimating mod-
els in both the traditional way through covariance
matrix-based computations and through methods
based on least squares optimization that can also be
applied to nonlinear models.
21 https://semopy.com/
22 https://www.wolfram.com/
Pt1Kline5E.indd 76
3. The Matlab program is a computing environment
and programming language for data analysis, visu-
alization, and simulation (Mathworks, 2022).23
Williams (2021) describes the Toolbox for SEM, a
set of functions for estimating structural equation
models with continuous outcomes.
SUMMARY
Modern SEM computer tools are generally no more
difficult to use than other computer programs for mul-
tivariate statistical analyses. The capability to specify
a model by drawing it onscreen helps beginners to be
productive right away, but with experience they may
find that specifying models in syntax is actually more
straightforward. Problems can be expected in the anal-
ysis of complex models, and no amount of user-friend-
liness in the interface of a computer tool can negate this
fact. When things in the analysis go wrong, you need,
first, to have a good understanding of the problem and,
second, basic computer skills to correct the problem.
You should not let ease of computer tool use lead you
to carry out unnecessary analyses or select analytical
methods or options you do not understand. The con-
cepts and tools covered in Part I of this book set the
stage for considering the specification and analysis of
basic kinds of structural equation models in Part II.
23 https://www.mathworks.com/
3/22/2023 3:52:41 PM
 Part II

Specification, Estimation,
and Testing

Pt2Kline5E.indd 77 3/22/2023 3:43:59 PM

Pt2Kline5E.indd 78 3/22/2023 3:43:59 PM
 6

Nonparametric Causal Models

The principal concepts in Pearl’s (2009) nonparametric approach to SEM (i.e., the structural causal model)
are introduced in this chapter. A causal model as described next corresponds to a directed acyclic graph
(DAG), which depicts hypotheses of unidirectional causation or temporal ordering among variables in graph-
ical form. A DAG does not allow for causal loops, where ≥ 2 variables are specified to have direct or indirect
effects on each other. In contrast, a directed cyclic graph (DCG) includes at least one causal loop, but we
will deal with such models later in the book. A directed graph as a nonparametric causal model implies two
things: (1) The researcher makes no commitment to distributional assumptions for any variable. (2) A direct
causal effect represents all forms of the functional relation between a putative cause and effect. If variables
X and Y are both continuous, for example, the specification X → Y in a nonparametric model represents the
linear and all curvilinear trends of the causal effect of X. But in parametric causal models—introduced in the
next chapter—the specification X → Y represents just the linear trend for continuous variables.
Another difference is that there are methods and computer programs for analyzing directed acyclic
graphs before any data are collected. Such analyses can alert the researcher to the presence of confounding,
including whether it would be necessary to measure additional variables in order to estimate any specific
causal effect with less bias. It is easier to address the problem of omitted variables when the study is being
planned than after the data are collected. Even if no new variables are required, analysis of the graph can
indicate options for estimating a target causal effect, including which covariates or instruments to select, in
the presence of confounding. A tutorial on instruments (i.e., instrumental variables) is offered in this chapter.
Analysis of a DAG can also help researchers to find testable implications of their causal hypotheses. No
special software is needed to analyze the data. This is because testable implications for linear models with
continuous variables can be evaluated using partial correlations, which can be estimated with standard com-
puter tools for statistical analysis. Thus, researchers who understand nonparametric causal models are better
prepared for all stages of SEM, from model specification through data collection to analysis and reporting
of the results. So if the ideas considered next seem at first unfamiliar, trust me, it is worth persevering. The
payoff will be apparent in later chapters when we apply the ideas outlined next.

GRAPH VOCABULARY pair of variables is adjacent if they are connected by

AND SYMBOLISM
Variables in directed graphs are also called nodes or
vertices. Some variables are connected by arcs, also
known as edges or links, that designate presumed
functional or statistical dependencies in the graph. A
an edge; otherwise, that pair is nonadjacent. An arrow
or directed edge represents a presumed direct causal
effect between two variables, such as
X→Y (6.1)

79

Pt2Kline5E.indd 79 3/22/2023 3:43:59 PM

 80 Specification, Estimation, and Testing
where X is a presumed cause of Y. In contrast, a bidi-
rectional edge, often symbolized with an arc rendered
as a dashed line instead of a solid line with arrowheads
at each end, such as
X Y (6.2)
designates a spurious (noncausal) association between
X and Y due to ≥ 1 unmeasured (latent) common causes.
An alternative symbolism in a causal DAG is
X UC Y (6.3)
where UC explicitly represents all unmeasured com-
mon causes of X and Y, and the arrows are rendered as
dashed lines because the corresponding causal effects
on both X and Y are not directly observed.
By convention, error terms for outcome variables are
not included in a causal DAG. This is in part because
it is assumed that all variables, causal or outcome,
will have idiosyncratic error due to unobserved fac-
tors that can vary over time or units (cases, settings,
regions, etc.). Also, error terms for outcome variables
are implied by arrows in the graph that point to them
from other variables, such as in Equation 6.1, where Y
is the outcome. The hypothesis of overlapping or cor-
related error is represented by Equation 6.2, where
a bidirectional edge connects two variables X and Y
(Equation 6.3 is an alternative specification). Corre-
lated errors are always explicitly represented in a DAG;
otherwise, error terms for outcomes are assumed to be
independent.
The direct causes of a variable in a DAG are it par-
ents, and all direct or indirect causes of a variable are
its ancestors. All variables directly caused by a given
variable are its children, and its descendants include
all variables directly or indirectly caused by that same
variable. All parents in a DAG are ancestors just as all
children are descendants. A variable with no parents
is exogenous, and a variable with at least one parent is
endogenous, just as in parametric structural equation
models.
A path is a sequence of adjacent edges that con-
nect ≥ 2 variables regardless of the directions of those
edges (i.e., unidirectional or bidirectional). It passes
through any variable along the path just once. In a
directed path, all edges are unidirectional arrows that
point away from a cause toward an outcome at the end
of the path through possibly ≥ 1 intervening variables
(i.e., indirect effects). Thus, directed paths transmit
Pt2Kline5E.indd 80
causal effects from variables at the beginning of the
path to “downstream” variables at the end the path. A
directed path is also called a front-door path because
it starts with an arrow pointing away from the cause,
and all subsequent arrows in the path are oriented
in the same direction. An undirected path is a path
where the arrows do not all point in the same direc-
tion. Undirected paths might convey statistical associa-
tion, but not causation between variables at either end
of the path. The goal of specifying a causal DAG is the
same as for any parametric model in SEM: The graph
represents all hypothesized connections, causal or non-
causal, between any pair of variables in the model.
CONTRACTED CHAINS
AND CONFOUNDING
Presented in Figure 6.1(a) is the smallest causal struc-
ture, the contracted chain X → Y (i.e., Equation 6.1).
The two variables in a contracted chain are uncondi-
tionally dependent because there are no intervening
variables that could disrupt or block the causal coor-
dination between them. Represented in Figure 6.1(a) is
the total causal effect—hereafter called just the total
effect—of X on Y. Besides the hypothesis about direc-
tionality, the figure also assumes there are no unmea-
sured confounders, or latent common causes of both X
and Y. A variation on this assumption is that all omitted
causes of Y are uncorrelated with X. For this reason,
variable X can be described as an exogenous regressor
in the prediction of Y.
If the assumptions for Figure 6.1(a) just stated are
correct, the coefficient from the regression of Y on X
would estimate the total effect without bias. Because
the model is nonparametric, though, no particular
regression technique can be specified. For instance, if
both X and Y were continuous and their relation strictly
linear, the method would be bivariate ordinary least
squares (OLS) regression. But if Y were dichotomous,
the regression method could be logistic regression or
probit regression and, if time-to-event data were also
collected for varying risk periods, a proportional haz-
ards model, such as a Cox model, could be specified,
among other possibilities for binary outcomes (Lee et
al., 2009). The point is that the choice of regression
technique depends on assumptions about the functional
form of the causal effect and on the level of measure-
ment for both X and Y, but nonparametric models are
not concerned with such details.
3/22/2023 3:43:59 PM