Applied Neuropsychology: Child

ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/hapc20

Rethinking “gold standards” and “best practices” in

the assessment of autism

Noah K. Kaufman

To cite this article: Noah K. Kaufman (2020): Rethinking “gold standards” and “best practices” in
the assessment of autism, Applied Neuropsychology: Child, DOI: 10.1080/21622965.2020.1809414

To link to this article: https://doi.org/10.1080/21622965.2020.1809414

Published online: 27 Aug 2020.

Submit your article to this journal

Article views: 19

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=hapc20
APPLIED NEUROPSYCHOLOGY: CHILD
https://doi.org/10.1080/21622965.2020.1809414

Rethinking “gold standards” and “best practices” in the assessment of autism

Noah K. Kaufman
Center for Neuropsychological Studies, Texas Tech University Health Sciences Center, Las Cruces, New Mexico, USA

ABSTRACT KEYWORDS
Failure to correctly diagnosis autism is problematic in both the false-positive and false-negative ADI-R; ADOS; assessment;
directions. Diagnosing autism when it is not truly present can direct limited resources away from autism; autism spectrum
those who actually need the services, while also creating stress and confusion for individuals and disorder; “best practices”;
“gold standard”;
families. In contrast, failure to correctly identify autism when it is indeed present can prevent indi- neuropsychological;
viduals and families from receiving needed support, including early intervention services. Those psychological
familiar with current trends in autism assessment are likely aware of “gold standards” involving
specific autism tests and “best practices” involving multi-disciplinary autism teams. Curiously, these
“gold standard” and “best practice” proclamations have not been adequately scrutinized. The pre-
sent article aims to address this gap in the literature by (a) discussing the value of autism tests/
tools; (b) drawing attention to biasing influences in autism assessment; (c) identifying methodo-
logical flaws in “gold standard” autism assessment research; and (d) proposing that more assess-
ment, not less, might be better in autism assessment. It is concluded that it is time to rethink
“gold standards” and “best practices” in the assessment of autism.

Introduction neuropsychology. For these and other reasons explored

below, it is time to rethink “gold standards” and “best
Current “best practices” in autism assessment equates to
practices” in the assessment of autism.
multi-disciplinary autism teams using “gold standard” tests
to make a diagnosis of autism. While this sounds reassuring,
the scientific support underlying these proclamations is Clinical judgment and autism tests/tools
characterized by conflicts of interest that are easily missed
Autism is a diagnosis that is defined in the Diagnostic and
by non-experts who, understandably, do not meticulously
Statistical Manual of Mental Disorders-Fifth Edition (DSM-
pore through the unexciting nooks of the supporting scien-
5; American Psychiatric Association, 2013) based on the
tific manuscripts. For example, many of the same research-
presence of certain behaviors. Although genetic testing for
ers studying the autism tools also benefit from the
those with autism is recommended, because tuberous scler-
widespread acceptance, and sale, of the tools. Likewise, the
osis and fragile X syndrome both overlap with autism at a
overly optimistic claims of diagnostic accuracy are riddled
rate that exceeds chance (Bailey et al., 1996; Patzer and
with methodological flaws. For example, diagnostic accuracy
Volkmar, 1999), there are no genetic tests for autism. In
statistics are calculated using samples with extremely high
fact, there are no “autism tests,” meaning that a diagnosis of
autism base rates and then generalized to clinical settings,
autism ultimately comes down to the diagnostic criteria
where the base rate for autism is much lower. And, in the
found in DSM-5 and clinical judgment. Even the assessment
case of multi-disciplinary autism teams being “best practice,”
tools described by many as “gold standards”—viz., the
data collection and treatment planning is, arguably,
Autism Diagnostic Inventory-Revised (ADI-R; Lord et al.,
enhanced through inclusion of professionals from varied 1994) and the Autism Diagnostic Observation Scale-Second
backgrounds. However, no research has been done to sup- Edition (ADOS-2; Lord et al., 2012)—are to be used in con-
port a blanket “best practices” assertion. Moreover, if the junction with clinical judgment:
social psychological literature on group decision-making is
of any value, there are powerful reasons to question a group The ADI-R and ADOS provide unique and critical overlapping
information that informs clinical judgments [emphasis added] in
decision-making process when diagnosing autism. While making an ASD diagnosis. When results from these instruments
there are defensible reasons to believe that tests and tools are consistent and correspond with clinical impression [emphasis
can improve diagnostic decision-making in the assessment added], diagnostic decision-making is straightforward.
of autism, tests are no substitute for an active awareness of Additional testing and alternative hypotheses must be
biasing influences and other general principles of assess- considered when there is inconsistency in results between the
ment, which are at the core of fields like psychology and

CONTACT Noah K. Kaufman noah71@pm.me Center for Neuropsychological Studies, Texas Tech University Health Sciences Center, 1188 West Hadley, Las
Cruces, NM 88005, USA; Department of Psychiatry, Paul L. Foster School of Medicine, 4615 Alameda, El Paso, TX 79905, USA.
ß 2020 Taylor & Francis Group, LLC
2 N. K. KAUFMAN
instruments and when the instrument results deviate from
clinical impression [emphasis added] (Risi et al., 2006, p. 1102).
It is reasonable to ask why autism assessment tools are
needed in the diagnostic process, given that there is no
mention of testing in the DSM-5 diagnostic criteria for aut-
ism spectrum disorder. This is not to imply that testing is
not beneficial in the assessment of autism; indeed, as will be
explained below, testing frequently improves diagnostic deci-
sion-making. Rather, it is noteworthy that a diagnosis of
autism spectrum disorder in DSM-5, in comparison to some
other diagnoses, does not require a certain score on any aut-
ism assessment test or tool. For a comparative perspective,
the diagnosis of intellectual disability, another neurodevelop-
mental disorder described in DSM-5, does involve actual
testing, which is explained in DSM-5 as follows: “The diag-
nosis of intellectual disability is based on both clinical
assessment and standardized testing of intellectual and adap-
tive functions … Individual cognitive profiles based on
neuropsychological testing are more useful for understand-
ing intellectual abilities than a single IQ score.” (p. 37).
Whereas a DSM-5 diagnosis of intellectual disability does
hinge, to some extent, on IQ, adaptive functioning, and
sometimes even neuropsychological testing results, a DSM-5
diagnosis of autism spectrum disorder does not overtly rely
on any testing whatsoever. And yet, the trend in the field is
to use “gold standard” assessment tools (viz., some version
of the ADOS and the ADI-R).
To illustrate, it is not unheard of for insurance company
representatives to authorize more units if the ADOS is used.
Consider also that it is very common for a practicing neuro-
psychologist or psychologist to be asked by a referral source,
or the parent of a referred child, whether or not the ADOS
will be utilized. This theme was recently addressed by
Gwynette et al. (2019) as follows:
A concerning trend has emerged in the diagnosis and treatment
of autism spectrum disorder (ASD) that has a negative impact
on care. Quite often, a clinician’s diagnosis of ASD using DSM-
5 criteria is no longer sufficient for individuals with ASD to
access services. Insurance companies, school districts, and
developmental disability agencies commonly require an Autism
Diagnostic Observation Schedule (ADOS) to be eligible for
services. (p. 1222)
Even though the DSM-5 criteria for autism spectrum dis-
order do not call for, let alone require, the use of any assess-
ment tools, clinicians serving those with
neurodevelopmental disorders know that there is now tre-
mendous pressure to make the diagnosis of autism using the
ADOS, if not also the ADI-R.
Taking into consideration the scientific literature on the
limits of subjective clinical judgment, as compared with clin-
ical judgment based on structured data collection (e.g. a
structured clinical interview) and empirical weighting of
data (Faust & Ahern, 2012; Meehl, 1954), it would seem to
make sense to incorporate certain tests and tools into the
diagnosis of autism, as has been done with other diagnoses
in DSM-5 (e.g. intellectual disability, specific learning disor-
ders, language disorder, neurocognitive disorders). While a
psychometric test is not an actuarial model, per se, it is a
highly structured way of collecting clinical data, making it
much more similar to actuarial decision-making methods
than subjective clinical judgment based on unstructured
interview. Therefore, it stands to reason that a diagnosis of
autism should require data from assessment tools with
acceptable psychometric properties. Hence, use of the ADOS
and ADI-R, assuming acceptable psychometric properties,
also stands to reason.
Beyond the limits of unstructured clinical judgment, there
are other implied reasons to include assessment tools as a
part of the diagnostic process when autism is under consid-
eration. One obvious example relates to ascertaining whether
or not the symptoms are exclusively due to intellectual dis-
ability, or if autism and intellectual disability are both pre-
sent. To accomplish this, an IQ test and an assessment of
adaptive functioning is required. As conveyed by the follow-
ing from DSM-5, the diagnostic process terminates with a
diagnosis of intellectual disability, not autism spectrum dis-
order, if social and/or communication skills are approxi-
mately as low as the low IQ:
A diagnosis of autism spectrum disorder in an individual with
intellectual disability is appropriate when social communication
and interaction are significantly impaired relative to the
developmental level of the individual’s nonverbal skills (e.g. fine
motor skills, nonverbal problem solving). In contrast,
intellectual disability is the appropriate diagnosis when there is
no apparent discrepancy between the level of social-
communicative skills and other intellectual skills. (p. 58)
Similarly, DSM-5 now defines autism spectrum disorder
in a way that requires consideration of “intellectual
impairment” and “language impairment” (p. 51), both of
which imply a need for psychometric testing. Even though
DSM-5 does not specifically mention assessment tools in
making a diagnosis of autism spectrum disorder, it implies
the need for psychometric assessment of IQ, adaptive func-
tioning, and speech-language functioning.
Other reasons to include assessment tools in the diagnos-
tic process of autism are supported by the fundamentals of
psychological and neuropsychological assessment. Much of
the impetus for these two fields came from difficulty clini-
cians experienced when trying to correctly identify those
with actual neurodevelopmental deficits. Alfred Binet and
Theodore Simon of France, early developers of the IQ test,
wrote about how doctors were unable to correctly identify
children with learning difficulties (Binet & Simon, 1908). As
a result of the unsystematic, random clinical decisions doc-
tors were making about which children were educatable and
which were not, Binet and Simon developed the
Binet–Simon Scale, considered to be the first IQ test
(Kaufman, 2009). More recently, Dahlstrom (1993) empha-
sized the limitations of human judgment when it is unaided
by methods to collect and interpret clinical data in a more
structured manner:
People are poor judges of other people. In our dealings with
one another we are subject to a variety of systematic and
random errors … Positive and negative halos … serve to cloud
our judgments and evaluations … this fallibility in the judgments
made by humans about fellow humans is one of the primary
reasons that psychological tests have been developed and

applied in ever-increasing numbers over the past century.
(p. 393).
In short, psychometric assessment tests and tools can
improve clinical decision-making, above and beyond what
can be accomplished only using unstructured clinical inter-
view, semi-structured or structured interviews, structured
behavioral observations, and so forth. Importantly, however,
the degree of improvement in clinical decision-making
depends on the specific test or tool and how it is utilized by
the clinician. Therefore, psychometric assessment in the con-
text of autism would certainly seem to have its place, even
though the DSM-5 does not explicitly call for it. So, again,
use of the ADOS and ADI-R in the assessment of autism
makes sense, insofar as these tools truly improve the quality
of data upon which diagnoses are based.
Biasing influences in autism assessment
All assessment is influenced by bias, which is important irre-
spective of what particular assessment test or tool is used to
improve diagnostic decision-making. Accordingly, it is
essential for mental health professionals conducting autism
assessments to minimize bias, no matter what test or tool
they do, or do not, use. One way to conceptualize bias dur-
ing assessment is to separate biasing influences that operate
during data collection and those that operate during data
interpretation (Kaufman & Bush, 2020a, 2020b).
Bias during data collection
Of the many possible biasing influences at work during data
collection, the following are particularly salient in the assess-
ment of autism: medication; age; emotions; dissimulation
(Rogers & Bender, 2013); response expectancy (Suhr & Wei,
2013); and jurisogenic effects (Weissman, 1990). More
explanation of these biasing influences is provided next,
beginning with the biasing influence of medications.
Some psychotropic medications can cause individuals to
seem as if they meet diagnostic criteria for autism. While
many clinicians are skilled at factoring in medication effects,
thereby minimizing a confounding influence, this point is
still worth emphasizing, especially since some individuals
have been heavily medicated for years, making it harder for
even close family members to appreciate the impact of the
medication. To illustrate, antipsychotic medication, espe-
cially when combined with a stimulant, can make an other-
wise animated, erratic youngster appear socially detached
and robotic. This needs to be considered during assessment.
Age is important because, for example, it is known that
“the process of brain maturation is long, lasting at least into
early adulthood.” (Kolb & Fantie, 2012, p. 41). Simply put,
it is sometimes very difficult to know if a young child has
autism, as opposed to another neurodevelopmental disorder,
making it important to proceed cautiously when trying to
understand and characterize the workings of an immature
nervous system.
Next, an individual in a major depressive episode might
present with deficits in social communication and social
APPLIED NEUROPSYCHOLOGY: CHILD 3
interaction. But these deficits might be exclusively due to
emotional functioning at the time of the assessment—not
autism. Alternatively, depression may interact with neurode-
velopmental deficits, blurring the clinical picture.
Rogers and Bender (2013) defined dissimulation as “a
general term to describe an inaccurate portrayal of symp-
toms and associated features. It is typically used when more
precise terms (e.g. malingering and defensiveness) are
inapplicable.” (p. 518). Dissimulation can erode the quality
of clinical data when, for example, a parent overreports
symptoms in their child for reasons that may not be defini-
tively known. Alternatively, some parents may not openly
share the full magnitude of their child’s challenges and diffi-
culties with the clinician, thereby inaccurately representing
their child’s symptoms. Similarly, if it were established
through validity testing and other methods that an individ-
ual, or a parent, were intentionally reporting fake symptoms
of autism, the data could lead the clinician down a diagnos-
tic rabbit hole, possibly resulting in the delivery of costly
services that are needed elsewhere.
In contrast, response expectancies are “expectations for
automatic emotional, physical, or behavioral responses as
reactions to situational cues” (Suhr & Wei, 2013, p. 183). If,
for example, a parent came to believe that their child has
autism, depending on the child’s age and level of develop-
ment, it is conceivable that the child would present with
symptoms of autism due to parental influence. Similarly, it
is conceivable that the parent would describe their child in
accord with beliefs that are influenced by inaccurate infor-
mation the parent has come to rely on. Suhr and Wei
(2013) described this phenomenon as follows:
As in other response expectancies, disease-specific beliefs may
be learned through not only personal experience but also the
suggestions of others (news media, public health
announcements, physician suggestion) and can even be learned
through observation (e.g. epidemic hysteria; Hahn, 1999). In
other words, when one is given diagnosis X and then reads
about diagnosis X, hears about it on television, attends support
groups for diagnosis X, and meets others with diagnosis X, etc.,
this can create response expectancy templates that include how
X might affect cognitive abilities and performance on tasks
within a neuropsychological assessment. (p. 189)
The jurisogenic effect is perhaps a close cousin to
response expectancy, but it only arises in the context of liti-
gation. In essence, involvement in protracted litigation,
along with the potential for some sort of monetary compen-
sation if a case is won, can exacerbate a litigant’s personality
in such a way that they seem to meet diagnostic criteria for
autism. To illustrate, consider a scenario where a family sues
a school for not diagnosing their child with autism, which
occurs with noteworthy frequency (Hill et al., 2011;
Zirkel, 2011).
Bias during data interpretation
Of the many possible biasing influences at work during data
interpretation, the following stand out with regard to autism
assessment: ignoring the base rate; dilution effects; illusory
correlation; pressure from the family, an individual being
assessed, an institution (e.g. a school), or retaining legal
4 N. K. KAUFMAN
counsel; diagnosis momentum; hindsight bias; confirmation
bias; regression toward the mean; and clinician overconfi-
dence (Croskerry, 2002). More in-depth explanation of these
biasing influences follows, beginning with ignoring the
base rate.
Ignoring the base rate (Croskerry, 2002) is relevant to
autism diagnosis because autism is extremely rare. An esti-
mated 1 in 59 children have autism at age eight years (Baio
et al., 2018; Centers for Disease Control, 2014), which corre-
sponds with a base rate of 1.69%. This is very low. The base
rate according to the DSM-5 is even lower, at 1%.
Therefore, it is always important to appreciate that the odds
of any particular individual having autism are also low. In
fairness, it is necessary to recognize that the base rate of
those who truly have autism is higher in certain settings
(e.g. all children referred for outpatient pediatric neuro-
psychological assessment). Nonetheless, one must remain
mindful that autism is still not a common disorder, which
means correctly identifying it is more challenging, especially
when other neurodevelopmental disorders might explain
the symptoms.
Dilution effects (Tetlock & Boettger, 1989) can bias aut-
ism assessment when clinical information with high rele-
vance is underweighted during the assessment process, in
favor of clinically irrelevant information. For example, some
parents may not be able to remember important historical
details about their child’s development that are very import-
ant (e.g. severe lack of oxygen during delivery), while simul-
taneously becoming very invested in information with low
clinical relevance (e.g. vaccines as a cause of symptoms, or
the perception that the individual has savant-like abilities,
when such may not be the case). If this occurs, the highly
relevant information will get diluted. As it turns out, accur-
acy in decision-making, the underlying purpose for assess-
ment (Sattler, 2001), is optimized when clinicians properly
weight a relatively small number of highly relevant variables:
A ceiling on predictive accuracy is commonly approached or
reached once the three to five most valid, and least redundant,
variables have been identified. Beyond this point, additional
information tends to increase confidence, not accuracy. More
problematically, at times continuing to add information may
decrease accuracy because weaker predictors may dilute the
stronger predictors. (Faust et al., 2009, p. 14)
This is why “more is better” does not exactly apply to
assessment. In making this point, Frederick (2012) stated,
“Mental health treatment records are potentially misleading
because diagnoses are often arbitrarily assigned, without
rigorous application of diagnostic guidelines.” (p. 536);
Frederick also pointed out that “Reports of friends, family
members, and neighbors also have variable consistency and
reliability. Such reports should not be casually integrated
into reports.” (p. 537). The key word in the foregoing is
“casually.” Report from those who know the individual can
be extremely valuable and should be sought; however, auto-
matically weighting it strongly, in the absence of support
from validity testing of raters, may dilute other more valid
data points. Stated plainly, if a parent provides invalid data
and the clinician does not assess for the validity of the
parent report, the clinician can end up down a diagnostic
rabbit hole, as noted above.
The symptoms of autism, and many other mental disor-
ders, occur in the general population more often than might
be realized. For example, most teens and adults have, at
some point in their life, displayed “inflexible adherence to
routines,” “rigid thinking patterns,” and/or “extreme distress
at small changes” (p. 50). Because of this, it would be a clin-
ical mistake to conclude, based only on the presence of these
symptoms, that an individual satisfies DSM-5 diagnostic cri-
teria for autism spectrum disorder. Making this erroneous
associative connection between symptoms and the diagnosis
of autism is called an illusory correlation (Chapman &
Chapman, 1969). Clinicians conducting assessments can
reduce the likelihood of bias from an illusory correlation by
considering how often symptoms are present, and absent, in
relation to those who truly have the disorder, versus those
who do not have the disorder. If large numbers of people
have the symptoms, but not the disorder, an illusory correl-
ation may be present.
Pressure from retaining legal counsel is known to bias
the opinions of legal experts (Murrie & Boccaccini, 2015).
Similarly, it is not uncommon for individuals or family
members to present for an assessment with preexisting ideas
about the nature of the individual’s difficulties. This might
be a result of response expectancies, as explained above.
Similarly, a clinician working within a school or agency—or
as a part of a multi-disciplinary team—might be influenced
by pressure to make certain clinical conclusions. Under any
of these circumstances, the quality of the assessment may
suffer as a result of this outside pressure.
Diagnosis momentum (Croskerry, 2002) and hindsight
bias (Fischhoff, 1975) are similar concepts. Diagnosis
momentum is when a clinician automatically re-applies a
diagnosis to an individual, without engaging in their own
diagnostic decision-making process. Hindsight bias is when
a clinician engages in their own diagnostic decision-making
process, but is heavily influenced by a diagnosis that was
already made. With hindsight bias, there is an “I knew it all
along” quality to the decision-making, which can be particu-
larly influential if the original diagnosis was made by a
prominent clinician or group of clinicians. In studies seeking
to establish diagnostic accuracy statistics for autism tests,
best-estimate diagnoses of autism are frequently made by
clinicians who have access to the results of autism tests.
With this unfortunate research design, hindsight bias can
quickly eat away at the methodological core of the study,
artificially boosting the diagnostic accuracy statistics for the
autism test.
Confirmation bias (Wason, 1960) has relevance to autism
assessment. It is known that decisionmakers frequently do
not manage competing information particularly well. To
reduce the cognitive dissonance (Festinger, 1957) created by
competing information, decision-makers often seek out
information that accords with their beliefs or hypotheses.
This is termed confirmation bias. Diagnostic decisions about
whether or not an individual has autism are sometimes very
straightforward, making the decision relatively easy.

For example, consider the following individual characteris-
tics: male; age 7 years; positive for Fragile X Syndrome;
macrocephalic (i.e. very large head); full-scale IQ ¼ 53;
extremely poor verbal and non-verbal pragmatics; extremely
limited speech; echolalia; intolerant to schedule changes; fix-
ated on trains; and hypersensitive to loud noises. In actual
practice, however, a high percentage of cases are much more
complex, a topic nicely addressed by Wolff et al. (2018). For
example, consider the following individual characteristics:
male; age 44; negative for Fragile X Syndrome and Tuberous
Sclerosis; macrocephalic; always lived with mother; was
placed in special education due to “borderline retardation;”
never held a job; cannot manage money without help; often
gives money to those who ask for it; has a driver’s license;
full-scale IQ ¼ 69; very limited adaptive functioning based
on brother report; never married; no children; has had three
girlfriends in his adult life; and primary interest is singing.
In contrast to the first case described, where autism is very
probable, the second case is much more challenging because
intellectual disability might better explain the symptoms.
When diagnostic decision-making is complex, diagnostic
decisions are more vulnerable to confirmation bias, which
can creep in to help resolve the cognitive dissonance created
by competing information. Notably, this biasing effect may
be particularly powerful when multi-disciplinary teams col-
laborate to make the diagnostic decision, given what is
known about group decision-making (Myers, 2008):
In groups, we become more aroused, more stressed, more tense,
more error-prone on complex tasks. Submerged in a group that
gives us anonymity, we have a tendency to loaf or have our
worst impulses unleashed by deindividuation … Discussion in
groups often polarizes our views … it may also suppress dissent,
creating a homogenized groupthink that produces disastrous
decisions. (p. 297)
Of note, zero-sum thinking about individual versus team-
based assessment of autism is not the point of emphasizing
confirmation bias in groups, given that professionals from
different educational backgrounds can make independent,
and therefore valuable, clinical contributions (Goldstein &
Ozonoff, 2018).
Regression toward the mean, an important term from Sir
Francis Galton, was described by Campbell and Kenny
(1999) as follows: “scores that are extreme in standard devi-
ation units on one measure are not likely to be as extreme
when measured on another measure” (p. 170). In their dis-
cussion of the term, Faust and Ahern (2012) described it as
follows: “There is a well-known tendency for extreme occur-
rences to be followed by less extreme occurrences merely
due to the operation of chance.” (p. 203). Insofar as chance
contributes to the occurrence of an event, extreme events
are more likely to be followed up by less extreme events.
Given the low base rate for autism, having autism is an
uncommon, and therefore, extreme event. Similarly, obtain-
ing a test score on a psychometric test that supports a diag-
nosis of autism is also an uncommon and extreme event.
And since psychometric tests lack perfect reliability, chance
will always play a role in obtained psychometric test scores.
It follows, therefore, that diagnostic decision-makers are well
APPLIED NEUROPSYCHOLOGY: CHILD 5
advised to keep regression toward the mean in mind when
interpreting information that supports a diagnosis of autism.
It might be that what is observed during one assessment is
not observed during a subsequent assessment, especially to
the extent that what is observed is extreme and measured
using methods with imperfect reliability.
Clinician overconfidence (Croskerry, 2002) can be par-
ticularly problematic in autism assessment. Clinicians
involved in diagnosing autism are likely familiar with other
clinicians who document bold, highly confident diagnostic
impressions in their reports about the presence, or absence,
of autism. Or, clinicians involved in diagnosing autism may
themselves boldly propound diagnostic statements about the
presence, or absence, of autism. The research shows that
overconfidence about diagnostic decision-making can be
very problematic: “confidence accounts for 2% of variance
in judgment accuracy.” (Miller et al., 2015, p. 9). Stated
plainly, confidence about one’s diagnostic decision does not
equate to accuracy about one’s diagnostic decision, which
has particular relevance to autism, given that many cases are
very complex. Reframed differently, a high degree of cogni-
tive dissonance (Festinger, 1957) can accompany autism
assessments, because cases can be complex (Wolff et al.,
2018). Some clinicians will resolve this cognitive dissonance
in the direction of “being right.” But the research shows that
such confidence about “being right” does not correlate with
accuracy—a counterintuitive research finding that those try-
ing to correctly diagnose autism are encouraged to consider.
In summary, bias during data collection and data inter-
pretation can erode the quality of mental health assessments.
This includes assessments directed at the correct identifica-
tion of autism. No matter what method a clinician chooses
to utilize, assessments will suffer if biasing influences are not
actively taken into consideration. Hence, even if a clinician
uses “gold standard” assessment tools, conclusions can be
incorrect if biasing influences are not dispassionately built
into the clinical decision-making process. The same holds
true for “best practices” involving multi-disciplinary autism
teams, in light of the compelling social psychological
research on group decision-making (e.g. higher stress among
team members, more error on complex tasks, social loafing,
deindividuation, polarization of views, suppression of dis-
sent, and homogenized groupthink). In other words, groups
of individuals attempting to make an important decision—
i.e. whether or not an individual has autism—are not
beyond the influence of bias. In fact, because of the social
psychology of group decision-making, multi-disciplinary aut-
ism teams may introduce more sources of bias into the diag-
nostic process.
Problems with research supporting current “gold
standard” autism assessment tools
In their recent discussion of assessment tools for autism,
Pennington et al. (2019) stated, “The two main diagnostic
tools for ASD are the Autism Diagnostic Observation
Schedule-Second Edition (ADOS-2; Lord et al., 2012) and
the Autism Diagnostic Interview-Revised (ADI-R; Lord
6 N. K. KAUFMAN
et al., 1994). Both are standardized measures that have
become the “gold standard” in the ASD field. Both the ADI-
R and ADOS-2 have rigorous training procedures to reach
reliability standards.” (p. 292). Notably, however, this “gold
standard” proclamation is not supported by a specific refer-
ence with scientific evidence. In their defense, Pennington,
McGrath, and Peterson are not alone in making this bold
proclamation. For years now, many others involved in aut-
ism assessment have described these assessment tools as
“gold standards,” raising an important question: Why?
It is known, for example, that the term “gold standard” is
to be avoided in the social sciences. In making this point,
Lilienfeld et al. (2015) wrote, “In the domains of psycho-
logical and psychiatric assessment, there are precious few, if
any, genuine ‘gold standards.’ Essentially all measures, even
those with high levels of validity for their intended purposes,
are necessarily fallible indicators of their respective con-
structs … ” (p. 4).
A review of the autism literature over the past couple of
decades points to a study by Risi et al. (2006), in which evi-
dence of diagnostic accuracy is provided for both the ADI-R
and the first version of the ADOS (Lord et al., 1999). The
authors of this study, some of whom are also authors of the
ADI-R and/or ADOS, conclude that use of the ADI-R and
ADOS in the assessment of autism improves diagnostic deci-
sion-making: “The Autism Diagnostic Interview-Revised and
Autism Diagnostic Observation Schedule make independent,
additive contributions to the judgment of clinicians that
result in a more consistent and rigorous application of diag-
nostic criteria.” (p. 1094). Largely as a result of this conclu-
sion, this study has subsequently been heavily relied upon to
justify use of the “gold standard” terminology when describ-
ing the ADI-R and ADOS/ADOS-2. However, a closer
inspection of the data in the Risi et al. (2006) study leads to
more questions about why these autism assessment tools are
automatically embraced by so many as “gold standards.”
To illustrate, Risi et al. (2006) reported that combining
the ADI-R and ADOS in their U.S. sample (age 3þ years)
resulted in sensitivity of 82.0% (95% CI ¼ 78–85) and speci-
ficity of 86.0% (95% CI ¼ 83–89) in the prediction of aut-
ism. This means that the combined, overall accuracy rate,
based on Youden’s J Index (Biggerstaff, 2000; Youden, 1950)
is 0.68 (Youden’s J Index ¼ [0.82 þ 0.86]-1 ¼ 0.68). This
value is the likelihood of making a correct diagnostic deci-
sion about autism, or a 68% overall likelihood of being cor-
rect. While this is better than flipping a coin to see if
someone has autism, it is not high enough to instill much
confidence about the diagnostic decision, let alone to justify
a “gold standard” declaration. Moreover, these diagnostic
accuracy scores are based on a combined mathematical algo-
rithm of best fit for using the ADI-R and ADOS in combin-
ation—something practitioners seldom, if ever, actually do.
More frequently, because “time and cost concerns are often
a barrier” (Pennington et al., 2019, p. 293), clinicians use
one or the other of these assessment tools, but not both.
This is very important because when sensitivity (i.e. the
percentage of those correctly identified by the test as having
the condition) and specificity (i.e. the percentage of those
correctly identified by the test as not having the condition)
for the ADI-R and ADOS are considered separately, for
each of the two assessment tools, the Youden’s J Index for
each tool drops further. To illustrate using the Risi et al.
(2006) data, Youden’s J Index equals 0.32 for the ADI-R
among those in the U.S. sample with intellectual disability.
Similarly, Youden’s J Index equals 0.182 for the ADOS in
this same sample.
To further appreciate why the ADI-R and ADOS may
not be “gold standards,” consider the Canadian data (age
3þ years) from the Risi et al. (2006) study: sensitivity ¼
77.2% (95% CI ¼ 70–83); specificity ¼ 75.0% (60–86); and
Youden’s J Index ¼ 0.522. This means the likelihood of
making a correct diagnostic decision about the presence of
autism among Canadians age 3þ years, using the ADI-R
and ADOS in combination, is at about a chance level. Now
consider the likelihood of making a correct diagnostic deci-
sion about the presence of autism among those in the U.S.
with a diagnosis of intellectual disability. Given sensitivity of
91.1% (95% CI ¼ 83–98) and specificity of 50.0% (95% CI
¼ 31–75), Youden’s J Index is .411, which falls below a
chance level.
Youden’s J Index is a valuable overall index of diagnostic
accuracy, in part because it is independent of the sample
base rate (i.e. the number of those who truly have the condi-
tion [autism]). But it has been criticized as hard to interpret
(Glas et al., 2003). An alternative to Youden’s J Index is the
Hit Rate, another overall index of diagnostic accuracy com-
puted as follows: (True Positives þ True Negatives)/(True
Positives þ True Negatives þ False Positives þ False
Negatives). Simply put, the Hit Rate is the percentage of
those correctly identified by the test, both in terms of true
positives and true negatives. However, the Hit Rate is
affected by the sample base rate; the higher the sample base
rate, the higher the Hit Rate, and vice versa. Therefore, if
research is conducted on a sample where those with the
condition of interest (i.e. autism) is high, the Hit Rate will
be higher. But if the research is performed on a sample
where those with the condition of interest (i.e. autism) is
low, the Hit Rate will be lower. Hence, the Hit Rate is
higher when the condition of interest is abundant and easy
to identify, but lower when the condition of interest is rare
and hard to spot.
Using data in Risi et al. (2006), the base rate for autism
in each of the four samples is uncommonly high: 540/
960 ¼ 0.563 for participants 3þ years old in the U.S.; 162/
270 ¼ 0.600 for participants <3 years old in the U.S.; 45/
66 ¼ 0.620 for participants with profound intellectual disabil-
ity in the U.S.; and 184/232 ¼ 0.793 for participants 3þ years
old in Canada. As expected, using these high sample base
rates for autism, the Hit Rates are also higher (Table 1).
This is notable because, as pointed about above, the popula-
tion base rate for autism is currently estimated to be
between 1 and 1.69%, which is much lower than the base
rates in the Risi et al. (2006) study.
Therefore, correctly identifying autism with the ADI-R-
ADOS combination is more challenging than Risi et al.
(2006) would suggest, especially in real-world clinical

Table 1. Indices of diagnostic accuracy for autism using the ADI-R þ ADOS.
Sensitivity (%) Specificity (%) Youden’s J Index
U.S.: <36 mo. 80.9 87.0 0.679
U.S.: ID 91.1 50.0 0.411
U.S.: 3þ yrs. 82.0 86.0 0.680
Canada: 3þ yrs. 77.2 75.0 0.522
ADI-R: Autism Diagnostic Inventory-revised; ADOS: Autism
Observation Scale.
settings. For this reason, the Hit Rate as an overall index of
diagnostic accuracy has limitations that do not apply to
Youden’s J Index, which is independent of the sample base
rate. When Youden’s J Index is computed on the Risi et al.
(2006) data, the combination of the ADI-R and the ADOS
often adds little to the diagnostic decision-making process,
thereby challenging the widely adopted view that these
assessment tools are “gold standards.” Moreover, further
consideration of all the data in the Risi et al. (2006) paper
reveals an overall pattern demonstrating that the ADI-R and
ADOS, especially used separately, do not improve overall
diagnostic decision-making as much as one might think.
There are limitations with indices of overall diagnostic
accuracy that attempt to combine sensitivity and specificity
(Grimes & Shulz, 2002), which extends to both Hit Rate and
Youden’s J Index, so it is common to focus on interpreting
sensitivity and specificity, as well as positive predictive value
and negative predictive value. Briefly, sensitivity is the ratio
of true positives divided by the true positives plus the false
negatives (TP/(TP þ FN)); in other words, sensitivity is the
percentage of those correctly identified by the test as having
the condition. In comparison, specificity is the true negatives
divided by the true negatives plus the false positives (TN/
(TN þ FP)); in other words, specificity is the percentage of
those correctly identified by the test as not having
the condition.
Notably, sensitivity and specificity describe how a certain
assessment tool performed, in the past, with a particular
sample and under specific circumstances. As explained by
Grimes and Shulz (2002), sensitivity and specificity “look
backward” (p. 882) at results gathered on a particular group
of individuals and under particular circumstances, making
these indices of less value to clinicians, who want to make
predictions about what will happen in the future with those
they assess, who may differ dramatically from the sample
originally studied. Because they are forward-looking, the
positive and negative predictive values are frequently relied
upon, instead of sensitivity and specificity, which describe
historical information. Briefly, the positive predictive value
is the ratio of true positives divided by true positives plus
false positives (TP/(TP þ FP)); this is the chance of truly
having the condition (i.e. autism) if the autism test is posi-
tive. In contrast, the negative predictive value is the ratio of
true negatives divided by true negatives plus false negatives
(TN/(TN þ FN)); this is the chance of not having the condi-
tion (i.e. autism) if the autism test is negative.
In Risi et al. (2006), what appear to be very high positive
and negative predictive values are reported. But this is mis-
leading because the reported base rates of autism in this par-
ticular study are uncommonly high: .563 for participants 3þ
APPLIED NEUROPSYCHOLOGY: CHILD 7
years old in the U.S.; .600 for participants <3 years old in
Hit rate the U.S.; .620 for participants with profound intellectual dis-
0.833 ability in the U.S.; and .793 for participants 3þ years old in
0.776
0.838
Canada. Therefore, while the positive and negative predictive
0.767 values reported in Risi et al. (2006) are reasonably high (e.g.
Diagnostic PPV ¼ 88.3% and NPV ¼ 78.8% for participants 3þ years
old in the U.S. using both ADI-R and ADOS to predict aut-
ism), it is not realistic to generalize these predictive values
to settings where the base rate for autism is signifi-
cantly lower.
The crucial message here is that positive and negative
predictive values are a function of the base rate of the con-
dition of interest. Therefore, assessment tools with impres-
sive predictive values in settings where the base rate (of
what is being predicted) is high will cease to have impressive
predictive values in a setting where the base (of what is
being predicted) is low. To fully appreciate why positive and
negative predictive values are a function of the base rate, the
following formulas are helpful:
PPV ¼ ðsensitivity  baserateÞ=½ðsensitivity  baserateÞ
þ ðð1–specificityÞ  ð1–base rateÞÞ
NPV ¼ ðspecificityxð1–base rateÞÞ=½ðspecificity
 ð1–baserateÞÞ þ ðð1–sensitivityÞ  baserateÞ
An additional, perhaps more serious, concern with the
Risi et al. (2006) study pertains to how sensitivity and speci-
ficity were computed. Namely, scores on the ADI-R and
ADOS were compared to a best-estimate diagnosis, which
differed between their U.S. and Canadian samples. The best-
estimate diagnoses in the U.S. samples were made using the
ADI-R and ADOS, meaning that the consensus best-estimate
diagnosis was confounded with the ADI-R and ADOS
scores, as opposed to being independent of these scores.
Stated another way, scores on the ADI-R and ADOS were
not compared to some other independent real “gold stand-
ard,” but were, instead, compared to scores on the ADI-R
and ADOS. This means that the diagnostic accuracy of the
ADI-R and ADOS, in the U.S. samples, were derived by
checking to see how much scores on the ADI-R and ADOS
compared to scores on the ADI-R and ADOS. As might be
expected, diagnostic accuracy studies with this sort of
research design have been heavily criticized
(Wilczynski, 2008).
In the Canadian samples, the best-estimate diagnoses
were made by clinicians who had not themselves adminis-
tered the ADI-R and ADOS, but who had access to the
ADI-R and ADOS scores, raising concerning questions
about hindsight bias (Fischhoff, 1975). In other words, clini-
cians used clinical judgment to form an opinion about
which participants had autism, but these clinicians knew
whether or not the ADI-R and ADOS pointed toward, or
away from, an autism diagnosis, setting the stage for a hind-
sight bias scenario. This too is a methodological flaw with
significant implications, which Risi et al. (2006) acknow-
ledge: “In an ideal design, different examiners would admin-
ister the ADI-R and ADOS in random order, and examiners
8 N. K. KAUFMAN
blind to ADI-R and ADOS scores would make the BE [best-
estimate] diagnoses … Circumstances were closer to the ideal
in study 2 [i.e. the Canadian sample].” (pp. 1101–1102). If
one looks past this flaw, the Canadian research design has
better external validity than the U.S. research design; how-
ever, even with the Canadian study, the external validity is
specious. And, as expected since the research design was
slightly more realistic, this is why the diagnostic accuracy of
the ADI-R used in addition to the ADOS drops substantially
with the Canadian participants, down to chance-level accur-
acy (Youden’s J Index ¼ .522 for the Canadian sample; see
Table 1).
Recently, Randall et al. (2018) set out to “find out which
of the commonly used tools is most accurate for diagnosing
ASD in preschool children.” (p. 3). Despite the availability
of many more than six “commonly used tools” (p. 3) to
help diagnose autism, these researchers began with six diag-
nostic tools. They then proceeded to study three of these six
tools—viz., the ADOS, the ADI-R, and the Childhood
Autism Rating Scale (CARS; Schopler et al., 2010)—on the
basis that the other three tools did not meet inclusion crite-
ria for the study. Notably, however, because developers of
the three included diagnostic tools were also the authors of
the research studies, developers of the index tools being
studied, as well as trainers for use of the instruments, there
were conflicts of interests in nearly 40% of the 21 studies
included, causing Randall et al. (2018) to state that “the
presence of conflicts of interest in some publications may
result in ADOS, CARS, and ADI-R appearing more accurate
than they really are.” (p. 3).
Another major concern with the Randall et al. (2018)
study is that the base rate for autism was 74%, using a best-
estimate approach, similar to that described in the Risi et al.
(2006) study. As a result, diagnostic accuracy statistics
reported by Randall et al. (2018) cannot be generalized to
real-world clinical settings where the base rate for autism is
lower, which is a crucial consideration, given that the popu-
lation base rate for autism is quite low (i.e. between 1% and
1.69%). This important message was explained by Grimes
and Shulz (2002) as follows:
Clinicians must know the approximate prevalence of the
condition of interest in the population being tested; if not,
reasonable interpretation is impossible. Consider a new PCR test
for chlamydia, with a sensitivity of 0.98 and specificity of 0.97 (a
superb test) … a doctor uses the test in a municipal sexually
transmitted disease clinic, where the prevalence of Chlamydia
trachomatis is 30%. In this high-prevalence setting … 93% of
those with a positive test actually have the infection. Impressed
with the new test, the doctor now takes it to her private practice
in the suburbs, which has a clientele that is mostly older than
age 35 years … Here, the prevalence of chlamydial infection is
only 3%. Now the same excellent test has a predictive positive
value of only 0.50 … Here, flipping a coin has the same
predictive positive value (and is considerably cheaper and
simpler than searching for bits of DNA). This message is
important, yet not widely understood: when used in low-
prevalence settings, even excellent tests have poor predictive
positive value. (p. 883)
Also of note, Randall et al. (2018) acknowledged (a) that
the results of best-estimate diagnosis of autism “does not
always replicate the multi-disciplinary assessment recom-
mended for clinical diagnosis” (p. 3) and (b) that “Emerging
evidence suggests that there is low agreement between indi-
vidual clinician and transdisciplinary team diagnoses
(Stewart et al. 2014), with both underdiagnosis and over-
diagnosis of ASD.” (p. 7). In other words, the best-estimate
method used by the researchers to establish the diagnostic
accuracy rates of the ADOS and ADI-R does not mesh with
the results of multi-disciplinary autism teams; at the same
time, Stewart et al. (2014) report that the results of multi-
disciplinary teams do not agree with the conclusions made
by individual clinicians. Can accuracy rates not established
using a multi-disciplinary team approach be generalized to
multi-disciplinary team approach settings? Similarly, how do
we know that the individual-clinician approach is not more
similar to the best-estimate approach used in the research,
given that the best-estimate approach does not involve a
multi-disciplinary team decision?
At present, researchers use a best-estimate approach to
establish which individuals have autism; however, as Randall
et al. (2018) have acknowledged, what the researchers estab-
lish is not always replicated by the multi-disciplinary teams.
Likewise, Randall et al. (2018) note low diagnostic agree-
ment between individual clinicians and multi-disciplinary
teams. It would seem, therefore, that the multi-disciplinary
teams are making diagnostic conclusions that differ from
both the researchers and the individual clinicians. And yet,
Randall et al. (2018) describe the multi-disciplinary team
approach involving a pediatrician, a speech pathologist, and
a psychologist as “best practice for this preschool age group”
(p. 23). Is this “best practice” proclamation supported by
the research?
As stated above, zero-sum thinking about individual ver-
sus team-based assessment of autism is not the point of
emphasizing potential problems with group decision-making
about autism. There is no reason to doubt the valuable con-
tributions of professionals from varying educational back-
grounds—with regard to data collection during autism
assessment. The doubt arises from low diagnostic agreement
between researchers and multi-disciplinary teams and low
diagnostic agreement between individual clinicians and
multi-disciplinary teams. In other words, the possible prob-
lem might be with the final diagnostic decision-making pro-
cess utilized by a team of professionals.
Rather than asserting that multi-disciplinary assessment is
“best practice,” it makes sense to test the hypothesis that
multi-disciplinary teams make less accurate diagnostic con-
clusions than the researchers and/or the individual clini-
cians. This hypothesis seems especially worth exploring,
given the social psychological research on worse decision-
making by groups. Put simply, it is reasonable to want repli-
cated scientific support for bold assertions about “best
practices” that, unfortunately, have the effect of (a) exclud-
ing individual clinicians from the autism diagnostic deci-
sion-making process and (b) forcing individuals and families
to wait until a multi-disciplinary autism team is available,
which can greatly delay needed mental health services.

If one does not blindly accept that the ADOS and ADI-R
are “gold standard” assessment tools in the assessment of
autism, and instead seeks scientific support for these instru-
ments, not only will reasonable questions emerge about
overall diagnostic accuracy, but other limitations will
become evident. For example, interscorer reliability for
ADOS-2 items are frequently poor, challenging the need for
lengthy and expensive ADOS-2 trainings. In other words, if
the costly ADOS-2 trainings are not resulting in adequate
interscorer reliability, it is reasonable to question
Pennington, McGrath, and Peterson’s assertion that “the
ADI-R and ADOS-2 have rigorous training procedures to
reach reliability standards” (2019, p. 292).
More, not less, might be better in autism assessment
Overall, when conducting autism assessments, it is import-
ant to consider the purpose of assessment in a general sense
(Sattler, 2001), because in so doing, clinicians are less ham-
strung by the known limitations of the categorical, diagnos-
tic nosology used in DSM-5 (Frances, 2009):
It has long been realised that the mental disorders described in
our diagnostic system are fuzzy sets that lack clear boundaries
between themselves and with normality. The traditional DSM
(and ICD) categorical approach is necessarily forced to carve
nature at awkward joints and loses much information in the
process. (p. 391)
Insofar as Frances (2009) and others (Galatzer-Levy &
Bryant, 2013; Hyman, 2007, 2010; Mirowsky & Ross, 1989)
are correct about the limitations of DSM-5, consideration of
diagnostic criteria for disorders can be viewed as but one
part of an overall assessment process, which includes the
collection of other valuable data. For example, symptom and
performance validity (Larrabee, 2012) data can make or
break an assessment. Similarly, test scores that rule out other
diagnoses (e.g. intellectual disability) or more precisely char-
acterize an autism diagnosis (e.g. “with intellectual
impairment” or “with language impairment”) can tremen-
dously improve and enrich an autism assessment. Likewise,
data on the quality of the pregnancy and delivery (Kaufman
et al., 2018) can establish much-needed context for test
scores and other data. This is precisely why other assessment
tools can be so valuable to the assessment process.
Therefore, in contrast to the “gold standard” perspective on
autism assessment (Huerta & Lord, 2012; Pennington et al.,
2019; Perry et al., 2002; Randall et al., 2018; Risi et al.,
2006), it is necessary to emphasize that there are more than
two autism tests available to the clinician wishing to enrich
and improve the autism assessment process.
Moreover, it is important to disabuse consumers of aut-
ism assessment services of the scientifically unsupported
belief that “best practices” dictate that autism is only prop-
erly diagnosed by a multi-disciplinary autism team. This is
not to say that professionals from different backgrounds
(e.g. pediatricians, speech-language pathologists, psycholo-
gists, medical geneticist) do not have much to offer with
regard to data collection and treatment planning. Nor is it
to say that this work is easy, straightforward, or
APPLIED NEUROPSYCHOLOGY: CHILD 9
uncomplicated. Indeed, it can become extremely complex, as
emphasized above and written about by Wolff et al. (2018).
Rather, it simply means that general principles of assessment
remain vital (e.g. dispassionate consideration of bias), which
summons into action fields like psychology and neuropsych-
ology that have long been devoted to reliably measuring
abnormal human behavior using a wide range of methods.
And it remains to be proven that decision-making by an
autism team is more accurate than decision-making by an
individual clinician, especially one equipped with fundamen-
tal knowledge about the general principles of assessment.
Only two autism assessment tools, arbitrarily described as
“gold standards,” should not be used to block the clinical
judgments of individual clinicians using other assessment
tools with equal, if not better, scientific support. For
example, Zhou et al., (2020) published a study on the diag-
nostic utility of the Behavioral Assessment System for
Children-Third Edition (BASC-3; Reynolds & Kamphaus,
2015) in discriminating between children with autism versus
attention-deficit/hyperactivity disorder (ADHD). Zhou et al.
(2020) also reported diagnostic accuracy rates for autism
that are slightly better than a recent large-scale study
(N ¼ 1,080) of the ADOS-2 involving infants, children, and
young adults (Kamp-Becker et al., 2017). To the extent that
these BASC-3 findings are generalizable to clinical settings,
it suggests that lengthy and expensive assessment tools like
the ADI-R and ADOS-2 may not be justified in the assess-
ment of autism. Or, alternatively, it might be possible to
obtain very similar results using a more cost-effective meas-
ure like the BASC-3, which has the added benefit of testing
symptom validity (Larrabee, 2012) and is more accessible
than the ADOS-2 and ADI-R, both of which require lengthy
and expensive training that many clinicians would
rather avoid.
Applying this same reasoning, multi-disciplinary autism
teams, arbitrarily described as “best practice,” should not
unquestioningly be viewed as any more accurate than indi-
vidual clinicians, especially in light of the social psychology
of group decision-making. This is especially relevant, insofar
as individual clinicians are, through their professional back-
grounds and training (e.g. neuropsychological board-certifi-
cation), equipped with a deep awareness of general
assessment principles, including the various ways bias can
skew diagnostic accuracy. Similarly, it is known that neuro-
psychological methods lead to high accuracy rates in the
detection of brain dysfunction (Fargo et al., 2008; Garb &
Schramke, 1996), which is common among many
with autism.
Concluding thoughts
It is time to rethink “gold standards” and “best practices” in
the assessment of autism, for the reasons articulated above
and because other methods may yield as good or better
results. There are many other autism assessment tools that
are more cost-effective, more accessible, and frequently less
time-consuming. Among these other measures are the fol-
lowing: Childhood Autism Rating Scale-Second Edition
10 N. K. KAUFMAN
(CARS-2; Schopler et al., 2010); Diagnostic Interview for
Social and Communication Disorders-Tenth Revision
(DISCO-10; Wing, 2006); Gilliam Autism Rating Scale-Third
Edition (GARS-3; Gilliam, 2013); Developmental,
Dimensional, and Diagnostic Interview (3di; Skuse et al.,
2004); Social Responsiveness Scale-Second Edition (SRS-2;
Constantino & Gruber, 2012); the Social Communication
Questionnaire (SCQ; Rutter et al., 2003); the Modified
Checklist for Autism in Toddlers (M-CHAT; (Robins et al.,
2014); the Social Language Development Test (SLDT;
Bowers et al., 2008, 2010); various subtests from the
NEPSY-II (Korkman et al., 2007); Social Cognition subtests
from the Advanced Clinical Solutions (Pearson, 2009); the
Benton Facial Recognition Test (Benton et al., 1994); and
the Autism Spectrum Rating Scales (ASRS; Goldstein &
Naglieri, 2009).
Additionally, when other more traditional psychological
and neuropsychological methods are utilized to characterize
the functioning of those suspected of autism—that is, tests
of personality, executive functioning, memory, attention,
processing speed, sensory-motor functioning, and so on—
the clinical picture frequently comes into focus in a way that
informs not only the DSM-5 diagnostic criteria of autism
spectrum disorder, but other disorders in DSM-5. This is
important because such an approach can reveal, or help rule
out, deficits that support an autism disorder, as well as mul-
tiple other disorders. Moreover, using neuropsychological
methods in the context of a possible autism spectrum dis-
order is in keeping with the research demonstrating that
neuropsychological methods lead to high accuracy rates in
the detection of brain dysfunction (Fargo et al., 2008; Garb
& Schramke, 1996). Hence, even in scenarios where categor-
ical diagnosis produces unsatisfying answers, the clinician
using psychological and neuropsychological methods will
have rich clinical data on how the examinee is functioning,
relative to others in their respective comparison group. By
using a “gold standard” and “best practices” rationale to
limit what assessment tests and tools can be used in the
diagnosis of autism, it is likely that methods with scientific
support will not be recognized as valid, which increases the
likelihood that the assessment results will not be accepted.
This is harmful because it prevents individuals and families
from accessing needed services.
Whenever autism is suspected, individuals and family
members deserve high-quality, thoughtful, and scientifically-
supported assessments. There are self-evident problems with
rushed, incomplete assessments that generate assailable aut-
ism diagnoses that will be questioned at every turn, by
teachers, by future clinicians, and by others invested in
understanding the individual. These false-positive conclu-
sions are particularly problematic because, as touched on
above, they “may cause family stress, lead to unnecessary
investigations and treatments, and place greater strain on
already limited service resources.” (Randall et al., 2018; p.
3). Arguably, this is more likely to occur with an individual
clinician, as opposed to a multi-disciplinary team. That said,
there are also problems with inflexible thinking about how
autism must be diagnosed, particularly when this thinking
emphasizes only two “gold standard” tests, as opposed to
the thought processes of those interpreting the tests. For this
and the other reasons articulated above, it is time to rethink
“gold standards” and “best practices” in the assessment
of autism.
Disclosure statement
No potential conflict of interest was reported by the author(s).
References
American Psychiatric Association. (2013). Diagnostic and statistical
manual of mental disorders (DSM-5). American Psychiatric
Publishing.
Bailey, A., Phillips, W., & Rutter, M. (1996). Autism: Towards an inte-
gration of clinical, genetic, neuropsychological, and neurobiological
perspectives. Journal of Child Psychology and Psychiatry, and Allied
Disciplines, 37(1), 89–126. https://doi.org/10.1111/j.1469-7610.1996.
tb01381.x
Baio, J., Wiggins, L., Christensen, D. L., Maenner, M. J., Daniels, J.,
Warren, Z., & White, T. (2018). Prevalence of autism spectrum dis-
order among children aged 8 years—autism and developmental dis-
abilities monitoring network, 11 sites, United States, 2014. Mortality
and Morbidity Weekly Report Surveillance Summaries, 67(6), 1.
https://doi.org/10.15585/mmwr.ss6706a1
Benton, A. L., Sivan, A. B., deS. Hamsher, K., Varney, N. R., & Spreen,
O. (1994). Contrubutions to neuropsychological assessment (2nd ed.).
Oxford University Press.
Biggerstaff, B. J. (2000). Comparing diagnostic tests: A simple graphic
using likelihood ratios. Statistics in Medicine, 19(5), 649–663.
https://doi.org/10.1002/(SICI)1097-0258(20000315)19:5<649::AID-
SIM371>3.0.CO;2-H
Binet, A., & Simon, T. (1908). Les enfants anormaux. Colin.
Bowers, L., Huisingh, R., & LoGiudice, C. (2008). Social language devel-
opment test: Elementary. Pro-Ed.
Bowers, L., Huisingh, R., & LoGiudice, C. (2010). Social language devel-
opment test: Adolescent. Pro-Ed.
Campbell, D. T., & Kenny, D. A. (1999). A primer on regression arti-
facts. Guilford Publications.
Centers for Disease Control and Prevention. (2014). Prevalence of aut-
ism spectrum disorder among children aged 8 years–Autism and
developmental disabilities monitoring network, 11 sites, United
States, 2010. Morbidity and Mortality Weekly Report Surveillance
Summaries, 63(2), 1–21.
Chapman, L. J., & Chapman, J. P. (1969). Illusory correlation as an
obstacle to the use of valid psychodiagnostic signs. Journal of
Abnormal Psychology, 74(3), 271–280. https://doi.org/10.1037/
h0027592
Constantino, J. N., & Gruber, C. P. (2012). Social Responsiveness Scale-
Second Edition (SRS-2) Manual. Western Psychological Services.
Croskerry, P. (2002). Achieving quality in clinical decision making:
Cognitive strategies and detection of bias. Academic Emergency
Medicine: Medicine, 9(11), 1184–1204. https://doi.org/10.1197/aemj.
9.11.1184
Dahlstrom, W. G. (1993). Tests: Small samples, large consequences.
American Psychologist, 48(4), 393–399. https://doi.org/10.1037/0003-
066X.48.4.393
Fargo, J. D., Schefft, B. K., Szaflarski, J. P., Howe, S. R., Yeh, H.-S., &
Privitera, M. D. (2008). Accuracy of clinical neuropsychological ver-
sus statistical prediction in the classification of seizure types. The
Clinical Neuropsychologist, 22(2), 181–194. https://doi.org/10.1080/
13854040701220093
Faust, D., & Ahern, D. C. (2012). Clinical judgment and prediction. In
D. Faust (Ed.), Coping with psychiatric and psychological testimony
(6th ed, pp. 147–208). Oxford University Press.

Faust, D., Bridges, A. J., & Ahern, D. C. (2009). Methods for the iden-
tification of sexually abused children: Issues and needed features for
abuse indicators. In K. Kuehnle & M. Connell (Eds.), The evaluation
of child sexual abuse allegations: A comprehensive guide to assessment
and testimony (p. 3–19). John Wiley & Sons Inc.
Festinger, L. (1957). A theory of cognitive dissonance. Stanford
University Press.
Fischhoff, B. (1975). Hindsight is not equal to foresight: The effect of
outcome knowledge on judgment under uncertainty. Journal of
Experimental Psychology: Human Perception and Performance, 1(3),
288–299. https://doi.org/10.1037/0096-1523.1.3.288
Frances, A. (2009). Whither DSM–V? The British Journal of Psychiatry,
195(5), 391–392. https://doi.org/10.1192/bjp.bp.109.073932
Frederick, R. I. (2012). Malingering/Cooperation/Effort. In D. Faust
(Ed.), Coping with psychiatric and psychological testimony (6th ed.,
pp. 229–247). Oxford University Press.
Galatzer-Levy, I. R., & Bryant, R. A. (2013). 636,120 ways to have post-
traumatic stress disorder. Perspectives on Psychological Science: A
Journal of the Association for Psychological Science, 8(6), 651–662.
https://doi.org/10.1177/1745691613504115
Garb, H. N., & Schramke, C. J. (1996). Judgment research and neuro-
psychological assessment: A narrative review and meta-analyses.
Psychological Bulletin, 120(1), 140–153. https://doi.org/10.1037/0033-
2909.120.1.140
Gilliam, J. E. (2013). Gilliam Autism Rating Scale (3rd ed.). Pro-Ed.
Glas, A. S., Lijmer, J. G., Prins, M. H., Bonsel, G. J., & Bossuyt, P. M.
(2003). The diagnostic odds ratio: a single indicator of test perform-
ance. Journal of Clinical Epidemiology, 56(11), 1129–1135. https://
doi.org/10.1016/S0895-4356(03)00177-X
Goldstein, S., & Naglieri, J. A. (2009). Autism diagnostic spectrum rat-
ing scales manual. MHS Assessment.
Goldstein, S., & Ozonoff, S. (2018). Assessment of autism spectrum dis-
orders (2nd ed.). The Guilford Press.
Grimes, D. A., & Schulz, K. F. (2002). Uses and abuses of screening
tests. Lancet, 359(9309), 881–884. https://doi.org/10.1016/S0140-
6736(02)07948-5
Gwynette, M., McGuire, K., Fadus, M. C., Feder, J. D., Koth, K. A., &
King, B. H. (2019). Overemphasis of the Autism Diagnostic
Observation Schedule (ADOS) evaluation subverts a clinician’s abil-
ity to provide access to autism services. Journal of the American
Academy of Child & Adolescent Psychiatry, 58(12), 1222–1223.
https://doi.org/10.1016/j.jaac.2019.07.933
Hahn, R. A. (1999). Expectations of sickness: Concept and evidence of
the nocebo phenomenon. In I. Kirsh (Ed.), How expectancies shape
experience (pp. 333–356). American Psychological Association.
Heilbronner, R. L., Sweet, J. J., Morgan, J. E., Larrabee, G. J., & Millis,
S. R., 1., C. P, Conference Participants (2009). American Academy
of Clinical Neuropsychology Consensus Conference Statement on
the neuropsychological assessment of effort, response bias, and
malingering. The Clinical Neuropsychologist, 23(7), 1093–1129.
https://doi.org/10.1080/13854040903155063
Hill, D. A., Martin, E. D., Jr,., & Nelson-Head, C. (2011). Examination
of case law (2007–2008) regarding autism spectrum disorder and
violations of the Individuals With Disabilities Education Act.
Preventing School Failure: Alternative Education for Children and
Youth, 55(4), 214–225. https://doi.org/10.1080/1045988X.2010.
542784
Huerta, M., & Lord, C. (2012). Diagnostic evaluation of autism spec-
trum disorders. Pediatric Clinics of North America, 59(1), 103–111,
xi. https://doi.org/10.1016/j.pcl.2011.10.018
Hyman, S. E. (2007). Can neuroscience be integrated into the DSM-V?
Nature Reviews. Neuroscience, 8(9), 725–732. https://doi.org/10.1038/
nrn2218
Hyman, S. E. (2010). The diagnosis of mental disorders: the problem
of reification. Annual Review of Clinical Psychology, 6(1), 155–179.
https://doi.org/10.1146/annurev.clinpsy.3.022806.091532
Kamp-Becker, I., Langmann, A., Stehr, T., Custodis, K., Poustka, L., &
Becker, K. (2017). Diagnostic accuracy of the ADOS-2 taking
account of gender effects. Zeitschrift Fur Kinder– Und
APPLIED NEUROPSYCHOLOGY: CHILD 11
Jugendpsychiatrie Und Psychotherapie, 45(3), 193–207. https://doi.
org/10.1024/1422-4917/a000492
Kaufman, A. S. (2009). IQ testing 101. Springer Publishing Company.
Kaufman, N. K., & Bush, S. S. (2020a). Validity assessment in military
psychology. In U. Kumar (Ed.), The Routledge International
Handbook of Military Psychology and Mental Health (pp. 211–223).
NY. Routledge Taylor & Francis Group.
Kaufman, N. K., & Bush, S. S. (2020b). Concussion in pediatric neuro-
psychology. Journal of Pediatric Neuropsychology, 6(1), 14–26.
https://doi.org/10.1007/s40817-020-00078-3.
Kaufman, N. K., Davis, A. S., Sandoval, H., Tonarelli, S., Mullins., &
Aguilar, Y. (2018). Perinatal complications: Pregnancy, delivery
problems and 2D:4D ratios. Journal of Pediatric Neuropsychology, 4,
63.
Kolb, B., & Fantie, B. D. (2009). Development of the child’s brain and
behavior. In C. R. Reynolds & E. Fletcher-Janzen (Eds.), Handbook
of clinical child neuropsychology (pp. 19–46). Springer.
Korkman, M., Kirk, U., & Kemp, S. (2007). NEPSY-II: Administration
Manual. The Psychological Corporation.
Larrabee, G. J. (2012). Performance validity and symptom validity in
neuropsychological assessment. Journal of the International
Neuropsychological Society: JINS, 18(4), 625–630. https://doi.org/10.
1017/S1355617712000240
~
Lilienfeld, S. O., SauvignA#, K. C., Lynn, S. J., Cautin, R. L., Latzman,
R. D., & Waldman, I. D. (2015). Fifty psychological and psychiatric
terms to avoid: a list of inaccurate, misleading, misused, ambiguous,
and logically confused words and phrases. Frontiers in Psychology, 6,
1–15. https://doi.org/10.3389/fpsyg.2015.01100
Lord, C., Rutter, M., & Le Couteur, A. (1994). Autism diagnostic inter-
view-revised: A revised version of a diagnostic interview for care-
givers of individuals with possible pervasive developmental
disorders. Journal of Autism and Developmental Disorders, 24(5),
659–685. https://doi.org/10.1007/BF02172145
Lord, C., Rutter, M., DiLavore, P., & Risi, S. (1999). Manual: Autism
diagnostic observation schedule. Western Psychological Services.
Lord, C., Rutter, M., DiLavore, P. C., Risi, S., Gotham, K., & Bishop, S.
(2012). Autism diagnostic observation schedule (2nd ed., ADOS-2).
Western Psychological Services.
Meehl, P. E. (1954). Clinical versus statistical prediction: A theoretical
analysis and a review of the evidence. University of Minnesota Press.
Miller, D. J., Spengler, E. S., & Spengler, P. M. (2015). A meta-analysis
of confidence and judgment accuracy in clinical decision making.
Journal of Counseling Psychology, 62(4), 553–567. https://doi.org/10.
1037/cou0000105
Mirowsky, J., & Ross, C. E. (1989). Psychiatric diagnosis as reified
measurement. Journal of Health and Social Behavior, 30(1), 11–25.
https://doi.org/10.2307/2136907
Murrie, D. C., & Boccaccini, M. T. (2015). Adversarial Allegiance
among Expert Witnesses. Annual Review of Law and Social Science,
11(1), 37–55.
Myers, D. G. (2008). Group influence social psychology. (9th ed., pp.
261–297). McGraw Hill.
Patzer, D., & Volkmar, F. (1999). The neurobiology of autism and the
pervasive developmental disorders. In D. S. Charney, E. J. Nestler, &
B. S. Bunney (Eds.), Neurobiology of mental illness (pp. 761–778).
Oxford University Press.
Pearson. (2009). Advanced clinical solutions for WAIS-IV and WMS-IV:
Administration and scoring manual. Pearson.
Pennington, B. F., McGrath, L., & Peterson, R. (2019). Autism spectrum
disorder diagnosing learning disorders (3rd ed., pp. 258–311). The
Guilford Press.
Perry, A., Condillac, R. A., & Freeman, N. L. (2002). Best practices and
practical strategies for assessment and diagnosis of autism. Journal
on Developmental Disabilities, 9(2), 61–75.
Randall, M., Egberts, K. J., Samtani, A., Scholten, R. J., Hooft, L.,
Livingstone, N., & Williams, K. (2018). Diagnostic tests for autism
spectrum disorder (ASD) in preschool children. Cochrane Database
of Systematic Reviews, (7), 1–103.
Reynolds, C. R., & Kamphaus, R. W. (2015). Behavior assessment sys-
tem for children (3rd ed.). Pearson.
12 N. K. KAUFMAN
Risi, S., Lord, C., Gotham, K., Corsello, C., Chrysler, C., Szatmari, P.,
& Pickles, A. (2006). Combining information from multiple sources
in the diagnosis of autism spectrum disorders. Journal of the
American Academy of Child & Adolescent Psychiatry, 45(9),
1094–1103.
Robins, D. L., Casagrande, K., Barton, M., Chen, C.-M A., Dumont-
Mathieu, T., & Fein, D. (2014). Validation of the modified checkist
for autism in toddlers. Pediatrics, 133(1), 37–45. https://doi.org/10.
1542/peds.2013-1813
Rogers, R., & Bender, S. (2013). Evaluation of malingering and related
response styles. In A. M. Goldstein (Ed.), Handbook of psychology,
11 (Forensic Psychology) (pp. 517–540). John Wiley & Sons, Inc.
Rutter, M., Bailey, A., Berument, S. K., Lord, C., & Pickles, A. (2003).
Social communication questionnaire (SCQ). Western Psychological
Services.
Sattler, J. M. (2001). Assessment of children: Cognitive applications (4th
ed.). Jerome M. Sattler, Publisher, Inc.
Schopler, E., Van Bourgondien, M. E., Wellman, G. J., & Love, S. R.
(2010). The Childhood Autism Rating Scale, second edition (CARS-2).
Western Psychological Services.
Skuse, D., Warrington, R., Bishop, D., Chowdhury, U., Lau, J., Mandy,
W., & Place, M. (2004). The Developmental, Dimensional and
Diagnostic Interview (3di): a novel computerized assessment for aut-
ism spectrum disorders. Journal of the American Academy of Child
and Adolescent Psychiatry, 43(5), 548–558. https://doi.org/10.1097/
00004583-200405000-00008
Stewart, J. R., Vigil, D. C., Ryst, E., & Yang, W. (2014). Refining best
practices for the diagnosis of autism: a comparison between individ-
ual healthcare practitioner diagnosis and transdisciplinary assess-
ment. Nevada Journal of Public Health, 11(1), 1–13.
Suhr, J., & Wei, C. (2013). Response expectancies and their potential
influence in neuropsychological evaluation. In P. Arnett (Ed.),
Secondary influences on neuropsychological test performance (pp.
182–200). Oxford University Press.
Tetlock, P. E., & Boettger, R. (1989). Accountability: A social magnifier
of the dilution effect. Journal of Personality and Social Psychology,
57(3), 388–398. https://doi.org/10.1037/0022-3514.57.3.388
Wason, P. C. (1960). On the failure to eliminate hypotheses in a con-
ceptual task. Quarterly Journal of Experimental Psychology, 12(3),
129–140. https://doi.org/10.1080/17470216008416717
Weissman, H. N. (1990). Distortions and deceptions in self presenta-
tion: Effects of protracted litigation in personal injury cases.
Behavioral Sciences and The Law, 8(1), 67–74. https://doi.org/10.
1002/bsl.2370080108
Wilczynski, N. L. (2008). Quality of reporting of diagnostic accuracy
studies: no change since STARD statement publication—before-and-
after study. Radiology, 248(3), 817–823. https://doi.org/10.1148/
radiol.2483072067
Wing, L. (2006). Diagnostic Interview for Social and Communication
Disorders (11th ed.). Centre for Social and Communication
Disorders.
Wolff, M., Bridges, B., & Denczek, T. (2018). The complexity of autism
spectrum disorders. Routledge.
Youden, W. J. (1950). Index for rating diagnostic tests. Cancer, 3(1),
32–35. https://doi.org/10.1002/1097-0142(1950)3:1<32::AID-
CNCR2820030106>3.0.CO;2-3
Zhou, X., Reynolds, C., Zhu, J., & Kamphaus, R. W. (2020).
Differentiating autism from ADHD in children and adolescents
using BASC-3. Journal of Pediatric Neuropsychology, 6(2), 61–65.
https://doi.org/10.1007/s40817-020-00082-7
Zirkel, P. A. (2011). Autism litigation under the IDEA: A new meaning
of “disproportionality”. Journal of Special Education Leadership,
24(2), 92–103.