The Incremental Value of Neuropsychological Assessment: A Critical Review

The Clinical Neuropsychologist
ISSN: 1385-4046 (Print) 1744-4144 (Online) Journal homepage: https://www.tandfonline.com/loi/ntcn20
The incremental value of neuropsychological

assessment: a critical review
Jacobus Donders
To cite this article: Jacobus Donders (2019): The incremental value of

neuropsychological assessment: a critical review, The Clinical Neuropsychologist, DOI:
10.1080/13854046.2019.1575471
To link to this article: https://doi.org/10.1080/13854046.2019.1575471
Published online: 23 Apr 2019.
Submit your article to this journal
View Crossmark data
Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=ntcn20
THE CLINICAL NEUROPSYCHOLOGIST
https://doi.org/10.1080/13854046.2019.1575471
REVIEW ARTICLE
The incremental value of neuropsychological assessment:

a critical review
Jacobus Donders
Department of Psychology, Mary Free Bed Rehabilitation Hospital, Grand Rapids, MI, USA
ABSTRACT ARTICLE HISTORY

Objective: The purpose of this critical review was to evaluate the Received 21 August 2018
current state of research regarding the incremental value of Published online 26 April
neuropsychological assessment in clinical practice, above and 2019
beyond what can be accounted for on the basis of demographic,
KEYWORDS
medical, and other diagnostic variables. The focus was on neuro-
Neuropsychological
logical and other medical conditions across the lifespan where assessment; incremental
there is known risk for presence or future development of cogni- value; outcomes;
tive impairment. prediction; validity
Method: Eligible investigations were group studies that had been
published after 01/01/2000 in English in peer-reviewed journals
and that had used standardized neuropsychological measures and
reported on objective outcome criterion variables. They were
identified through PubMed and PsychInfo electronic databases on
the basis of predefined specific selection criteria. Reference lists of
identified articles were also reviewed to identify potential add-
itional sources. The Grades of Recommendation, Assessment,
Development and Evaluation Working Group’s (GRADE) criteria
were used to evaluate quality of studies.
Results: Fifty-six studies met the final selection criteria, including
2 randomized-controlled trials, 9 prospective cohort studies, 12
retrospective cohort studies, 21 inception cohort studies, 2 case
control studies, and 10 case series studies. The preponderance of
the evidence was strongly supportive with regard to the incre-
mental value of neuropsychological assessment in the care of per-
sons with mild cognitive impairment/dementia and traumatic
brain injury. Evidence was moderately supportive with regard to
stroke, epilepsy, multiple sclerosis, and attention-deficit/hyper-
activity disorder. Participation in neuropsychological evaluations
was also associated with cost savings.
Conclusions: Neuropsychological assessment can improve both
diagnostic classification and prediction of long-term daily-life
outcomes in patients across the lifespan. Future high-quality
prospective cohort studies and randomized-controlled trials are
necessary to demonstrate more definitively the incremental value
of neuropsychological assessment in the management of patients
with various neurological and other medical conditions.
CONTACT Jacobus Donders Jacobus.Donders@maryfreebed.com Department of Psychology, Mary Free Bed

Rehabilitation Hospital, Grand Rapids, MI, USA
ß 2019 Informa UK Limited, trading as Taylor & Francis Group
2 J. DONDERS
Introduction
There is increasing pressure on health care providers in general, and on neuropsychol-
ogists in particular, to provide empirical evidence of the value of their services. Several
authors have suggested the importance of studies of clinical (Chelune, 2010; Prigatano
& Morrone-Strupinsky, 2010) as well as financial cost (Prigatano & Pliskin, 2003) out-
comes in this regard. Others have discussed ways in which neuropsychological assess-
ment can assist with the diagnosis and management of various disorders (Braun et al.,
2011) and how neuropsychologists can function as part of integrated care teams
(Kubu, Ready, Festa, Roper, & Pliskin, 2016).
In a recent review, Watt and Crowe (2018) evaluated the literature on the degree to
which neuropsychological assessment assists with diagnosis and prognosis in adults,
as well as the degree to which consumers feel satisfied with provided services. They
concluded that the preponderance of the evidence did support the value of neuro-
psychological assessment but they also indicated that the quality of the empirical
studies was fairly modest. In addition, although many of the cited studies indicated
that neuropsychological measures have some predictive value with regard to out-
comes, they often did not clearly demonstrate that such measures actually improved
diagnosis, prediction, or satisfaction – above and beyond basic demographic, medical,
and other diagnostic variables. It is one thing to demonstrate that specific neuro-
psychological tests are sensitive to severity of traumatic brain injury (Donders &
Strong, 2015), match with seizure lateralization in patients with epilepsy (Keary,
Frazier, Busch, Kubu, & Iampietro, 2007), or outperform interview (Wyman-Chick,
Martin, Barrett, Manning, & Sperling, 2017) as well as general screeners (Cohen et al.,
2012) to identify cognitive dysfunction in persons with Parkinson disease. It is quite
another thing to demonstrate that the neuropsychological assessment actually adds
unique value to prediction or management of future patient outcomes. This is import-
ant because improved prediction can potentially lead to earlier recognition of persons
at risk for poor outcomes and/or earlier intervention. The purpose of this critical
review was to specifically evaluate the state of the current research of that potential
incremental value of neuropsychological assessment in neurological and other medical
samples who are at risk for cognitive impairment. In addition, the scope of this review
also included pediatric studies. For purposes of this review, neuropsychological assess-
ment was defined as a formal evaluation of patients that included not only interview,
records review, and general screening measures or rating scales but also standardized
face-to-face tests of specific cognitive domains for which appropriate norms were
available. Incremental value was defined as objective evidence that, after controlling
for the effect of non-neuropsychological variables, neuropsychological test data made
a statistically significant difference in either the prediction or modification of objective
patient outcomes in functional, economic, disease progression, and/or other medical
or psychological domains.
Method
Table 1 presents the inclusion criteria for studies that were considered for this review.
A search of the PubMed and PsychInfo databases was conducted for all journal articles
THE CLINICAL NEUROPSYCHOLOGIST 3
Table 1. Inclusion criteria.

Human participants only
Group studies only; no case studies or expert opinions
Published after 01/01/2000, in English, in a peer-reviewed journal
Used standardized neuropsychological measures for which norms were available.
Reported data on objective criterion variables (e.g. measurable outcomes, disease progression) as opposed to
subjective levels of satisfaction or concurrent correlations with other measures only
Included data that allowed evaluation of the incremental contribution of neuropsychological measures to such
criterion variables
Open access journal papers were eligible as long as there was documentation of formal peer review.
that had been published between January 2000 and December of 2018, with the last
search date being 10 December 2018. The 2000 cut-off was chosen in order to focus
on relatively recent literature with studies that had comparable access to neuroimag-
ing and biomarkers. The initial search used a combination of the following terms:
“neuropsychological” AND/OR “added value” AND/OR “incremental value”. The initial
search terms were kept fairly specific in an attempt to focus on papers that explicitly
addressed the potentially incremental contribution of neuropsychological data to pre-
diction or management of outcomes while controlling for various demographic, illness,
and other diagnostic variables. However, this resulted in lower than expected numbers
of studies with regard to various conditions that are commonly seen by clinical neuro-
psychologists. For example, only a few studies were found in the areas of stroke and
multiple sclerosis, even though several professional organizations, including the
National Stroke Foundation (2010) and the National Multiple Sclerosis Society (2006)
have explicitly endorsed guidelines for the use of neuropsychological assessment in
the cognitive care of persons with such conditions. This raised concern that the initial
literature search was too restrictive. Therefore, an additional search was run in
PubMed and PsychInfo; this time with the terms “neuropsychological OR cognitive”
AND “outcomes” AND “prediction”. There was no restriction on nature of the condition
being investigated. Papers that were listed as advance electronic publications online
were considered to be eligible for inclusion in this review. Reference lists of articles
that were identified by both searches were also reviewed to identify potential add-
itional articles. In addition, article authors were contacted directly on some occasions
in order to obtain further clarification and suggestions.
After the final search of potential studies was completed, abstracts of each of the
articles were reviewed according to the criteria listed in Table 1. This led to exclusion
of a considerable number of papers. For example, there were several dozens of papers
that dealt exclusively with the outcome of cognitive behavioral therapy (e.g. Stiles-
Shields, Corden, Kwasny, Schueller, & Mohr, 2015).
Journal articles that passed the abstract review stage were then selected for full-
text review. This led to the elimination of more papers, most often because they failed
to address the potential incremental value of such assessment in the evaluation of
future patient outcomes when compared to standard demographic, medical, and other
diagnostic variables. For example, one study demonstrated that within-person, across-
test variability added to knowledge of level of neuropsychological performance, but it
did not evaluate whether it improved the prediction of incident dementia over things
such as serum biomarkers or neuroimaging (Holtzer, Verghese, Wang, Hall, & Lipton,
2008). Similar problems with, on the one hand, demonstrating some predictive value
4 J. DONDERS
Figure 1. Flow diagram of literature search and study selection.
of neuropsychological assessment when considered in isolation in the context

of prediction of outcome while not addressing incremental value after controlling
for demographic, medical, and/or imaging variables also excluded various studies
in other domains ranging from multiple sclerosis (e.g. Pardini et al., 2014) to stroke
(e.g. Park, Sohn, Jee, & Yang, 2017). Other studies were eliminated because they
did not use formal neuropsychological tests. For example, Denti, Agosti, and
Franceschini (2008) reported that cognitive status at admission predicted patients’
discharge status, but they only included a mental status exam and ordinal rankings
of cognitive functioning. Finally, a fair number of studies (e.g. Hanks, Jackson, &
Crisanti, 2016; Leitner, Miller, & Libben, 2018; Struchen et al., 2008) were eliminated
because they reported only on concurrent relationships between neuro-
psychological variables and measures of disability or need for supervision, and not
on prediction of future outcomes. Figure 1 details the initial literature search and
study selection process.
The quality of evidence of studies that were eventually included in the review was
first evaluated according to the methods of the Grades of Recommendation,
Assessment, Development and Evaluation Working Group (GRADE; Ryan & Hill, 2016).
The GRADE starts with ratings ranging from “high” for randomized-controlled trials to
“low” for case studies. Initial higher ratings can then be adjusted downward, based on
five variables: risk of bias due to methodological limitations, inconsistency of results
due to clinical or methodological heterogeneity, indirectness of evidence due to select-
iveness in populations or comparisons, imprecision of effect estimates due to limits in
sample size or unusually large confidence intervals, and publication bias due to small
participant- and/or event-to-variable ratios. Alternatively, initial lower GRADE ratings
can also be adjusted upward. Although this is relatively less common, it can be done
with well-designed studies, based on three variables: effect sizes that are large in mag-
nitude, presence of clear dose-response relationships, and clear accounting for plausible
confounding factors.
The level of evidence of studies included in the review was then finally graded
in five categories, based on the nature of the study with criteria modified by Burns,
Rohrich, and Chung (2011), with high and low quality in several of these categories
determined on the basis of GRADE criteria. Level I included high-quality randomized-
controlled trials. Level II pertained to lesser-quality randomized-controlled trials and
high-quality prospective cohort studies. Level III was comprised of lesser-quality
prospective cohort studies, retrospective cohort studies, and high-quality inception
cohort and case control studies. Level IV involved lesser quality inception cohort and
case-control studies, as well as case series studies. Level V, which is typically assigned
to expert opinions and case studies, was not applicable for purpose of this review
because such publications were not considered eligible. Table 2 presents a summary
of all the 56 studies that were included in this review.
It should be noted that many of the papers that were retrieved for this review did
not report sufficient data to allow determination of effect size and other variables that
would have been necessary for meta-analysis. This was one way in which this review
deviated from Preferred Reporting Items for Systematic Reviews and Meta-Analyses
(PRISMA) guidelines (Moher, Liberati, Tetzlaff, & Altman, 2009) for a systematic review.
Other differences were that the review was not registered and that there was no
calculation of principal summary measures or of consistency across studies.
Results
Mild cognitive impairment and dementia
The field of progression from normal cognition to mild cognitive impairment (MCI)
and subsequent conversion to dementia has seen the most extensive evidence
of the incremental value of neuropsychological assessment variables with regard
to conditions included in this review. Due to the nature of the condition, all studies
in this area pertained to adults.
Some studies investigated how much such variables improved the prediction
of outcomes in this population, after accounting for various demographic,
genetic, neuroimaging, and/or serum biomarkers. Other studies have taken a different
approach, investigating instead to what degree which other biomarkers actually
improved prediction of outcomes, as compared to neuropsychological variables only.
Gomar, Bobes-Bascaran, Conejero-Goldberg, Davies, and Goldberg (2011) conducted
a study that was representative of the first approach. They prospectively studied
a cohort of patients with MCI who did (n ¼ 116) or did not (n ¼ 204) convert to
Alzheimer disease (AD) over a two-year period. They evaluated through a series
of stepwise logistic regression analyses of the predictive utility of various demo-
graphic, genetic, cerebrospinal, brain volumetric, and cognitive marker variables.
Conversion from MCI to AD was best predicted by a combination of two neuropsycho-
logical variables (i.e. Logical Memory – Delayed and Auditory Verbal Learning Test –
Delayed) and one neuroimaging variable (i.e. left-middle temporal lobe thickness),
with the neuropsychological variables accounting for more of the variance than the
neuroimaging one. This study, thus, demonstrated that neuropsychological variables
at baseline were more robust predictors of patient outcomes than various genetic
6 J. DONDERS
Table 2. Papers reporting on the incremental value of neuropsychological assessment.

Authors Type of study Participants Level of evidence Main findings of interest
Lincoln et al. (2002) Randomized-con- 211 patients with MS I No differences in outcomes
trolled trial at eight months between
NP assessment, NP asses-
sment þ intervention, and
control groups.
Alves de Moraes Prospective cohort 188 persons with inci- II NP measures did not contrib-
et al. (2003) dent stroke ute to prediction of stroke
incidence after controlling
for cardiovascular risk and
other confounds.
Breaux et al. (2016) Prospective cohort 168 preschoolers II NP variables at age four
years improved prediction
of ADHD at age six years,
above and beyond family
income and baseline
symptoms at age
three years.
Cervilla et al. (2004) Prospective cohort 387 older adults II No added benefit of APOE
genotype to the value of
NP variables in the predic-
tion of development
of dementia.
Ganguli et al. (2014) Prospective cohort 1129 community- II Memory scores were predict-
dwelling elders ive of outcome over four
years whereas APOE status
and stroke risk were not.
Gomar et al. (2011) Prospective cohort 320 persons with MCI II NP measures of delayed
recall were more robust
predictors of conversion
to dementia than MRI and
CSF biomarkers.
McKinney et al. (2002) Randomized-con- 228 patients II No difference in functional
trolled trial with stroke outcomes at 3–6 months
between patients who did
versus did not have a NP
evaluation at 3–4 weeks.
Wiberg et al. (2010) Prospective cohort 105 persons with inci- II NP function predicted inci-
dent dent infarction, independ-
brain infarction ent of education, social
group and traditional
stroke risk factors.
Wiberg et al. (2012) Prospective cohort 155 patients with first- II NP variables were predictors
time stroke of post-strokemortality,
independent of education,
social group and
cardiovascular risk.
Arnett et al. (2013) Inception cohort 132 adolescents III NP variables added to injury
with TBI severity in the prediction
of educational compe-
tence at 12- month
follow-up.
Bertolin et al. (2018) Retrospective cohort 498 persons III Brief NP screening instru-
with stroke ment was an independent
predictor of cognitive and
communication outcomes
at six months.
Bondi et al. (2014) Retrospective cohort 1150 elderly III Actuarial NP criteria reduced
participants false positive error rate in
diagnosis of MCI, com-
pared to conven-
tional criteria.
(continued)
Table 2. Continued.
Brewster et al. (2012) Retrospective cohort 504 older adults III NP measures contributed to
prediction of development
of Alzheimer versus vascu-
lar dementia, independent
of vascular risk.
Callahan et al. (2015) Retrospective cohort 494 elderly III Small incremental benefit of
participants APOE e4 allele markers
over NP variables in pre-
diction of development of
Alzheimer disease.
Devitt et al. (2006) Retrospective cohort 306 adults with TBI III NP measures contributed
independently to commu-
nity outcomes at a
median of 14 years
post-injury.
Devore et al. (2017) Inception cohort 126 older adult with III NP variables were the stron-
post-opera- gest predictors of long-
tive delirium term cognitive decline
after controlling for living
status and informant rat-
ings at baseline.
Edmonds et al. (2015) Retrospective cohort 1109 elderly III Actuarial NP criteria provided
participants improved differentiation
of MCI subtypes as well as
prediction of disease
progression.
Elkins et al. (2005) Prospective cohort 516 persons with inci- III Neuropsychological measures
dent cardiovascu- predicted incident cardio-
lar events vascular events, independ-
ent of various vascular
risk factors.
Elshorst et al. (2009) Retrospective cohort 59 persons III Pre-operative NP measure of
with epilepsy verbal learning and MRI
ratings of temporal scler-
osis were both independ-
ent predictors of post-
surgical cognitive decline.
Emerson et al. (2012) Inception cohort 100 elderly drivers III NP measures predicted mov-
ing violations and motor
vehicle collisions after
controlling for education
and baseline
weekly mileage.
Hanks et al. (2008) Inception cohort 176 adults with TBI III NP tests during inpatient
rehabilitation were more
predictive of 1-year out-
comes than baseline func-
tional and injury variables.
Ito et al. (2015) Inception cohort 88 patients with MCI III Both PET and NP variables
contributed to the predic-
tion of conversion from
MCI to dementia.
Jedynak et al. (2012) Retrospective cohort 687 elderly III Delayed recall NP variable
participants was the earliest biomarker
to become abnormal.
Lange et al. (2018) Retrospective cohort 402 persons with MCI III Incremental benefit of neuro-
imaging and CSF bio-
markers in predicting
disease progression
depended on patients’
(continued)
8 J. DONDERS
Table 2. Continued.
preceding cogni-
tive status.
Leininger et al. (2014) Inception cohort 49 patients with III NP measure of visual
mild TBI scanning and attention
contributed to prediction
of functional outcome at
discharge, after account-
ing for baseline status
and prior history.
Miller & Inception cohort 58 children with TBI III NP variables were better
Donders (2003) predictors of special
education placement at
48 months than demo-
graphic or injury variables.
Morrow et al. (2010) Inception cohort 97 patients with MS III After controlling for
demographics and MS
characteristics, NP
variables predicted change
in employment status
at follow-up.
Nys et al. (2005) Inception cohort 111 patients III NP variables significantly
with stroke improved prediction of
outcomes in IADL over
what could be accounted
for by vascular and other
medical variables
at baseline.
Pastorek et al. (2004) Inception cohort 105 adults with TBI III NP testability at 1 month
improved prediction of
functional outcomes at
6 months post-injury.
Portaccio et al. (2009) Prospective cohort 61 patients with MS III NP variables contributed
statistically significantly
to prediction of disease
evolution over a
5-year period.
Richard et al. (2013) Retrospective cohort 181 participants III After measure of immediate
with MCI verbal recall, MRI and CSF
variables did not improve
prediction of disease
progression.
Scott et al. (2016) Inception cohort 50 patients with III Level of depressed mood
mild TBI prior to treatment pre-
dicted functional out-
comes at 3 months post-
discharge, after account-
ing for baseline functional
status and premorbid
history.
Sieg et al. (2018) Inception cohort 280 inpatients; mixed III Refusal to undergo neuro-
clinical sample psychological evaluation
was associated with
increased inpatient service
utilization and cost.
Stroup et al. (2003) Retrospective cohort 132 patients III Pre-operative NP measures
with epilepsy of verbal retention added
to side of surgery and
MRI findings in predicting
post-operative decline
in verbal memory.
Thomas et al. (2018) Retrospective cohort III
(continued)
Table 2. Continued.
525 elderly NP process scores contrib-
participants uted unique value to pre-
diction of
cognitive decline.
van der Holst Inception cohort 494 older patients III NP variables did not contrib-
et al. (2016) with small ves- ute statistically signifi-
sel disease cantly to mortality over a
period of eight years.
Van Kampen Case control 102 concussed and 77 III NP testing improved sensitiv-
et al. (2006) non-con- ity to sequelae of concus-
cussed athletes sion by 19%.
Van Kirk et al. (2013) Inception cohort 439 veterans, mixed III Reduction in health care util-
clinical sample ization and cost following
NP evaluation.
Wagle et al. (2011) Inception cohort 163 patients III RBANS was a significant and
with stroke independent predictor of
functional status at
13 months.
Williams et al. (2013) Inception cohort 288 adults with TBI III NP tests during inpatient
rehabilitation improved
prediction of two-year
outcomes, above and
beyond, functional and
injury variables, whereas
CT variables did not.
Alosco et al. (2012) Case series 122 persons with IV NP variables improved the
heart failure prediction of independ-
ence with driving and
medications.
Aretouli et al. (2013) Inception cohort 85 persons with MCI IV NP measures of executive
functioning only demon-
strated a trend toward
statistically significant
multivariate prediction of
development of dementia.
Benedict et al. (2005) Case series 122 patients with MS IV NP variables discriminated
and 44 persons with MS who
healthy controls were v. were not disabled
from work, after control-
ling for disease duration.
De Lepeleire Case series 152 elderly partici- IV Clock drawing but not com-
et al. (2005) pants, 50 of whom puterized NP tests
had dementia improved differen-
tial diagnosis.
Devanand et al. (2007) Case control 139 persons with MCI IV Small incremental benefit of
and 63 controls volumetric MRI measures
over NP variables in pre-
diction of
Alzheimer disease.
Ferreira et al. (2012) Case series 50 elderly drivers IV Trend for more complex NP
variables to improve dis-
crimination of safe v.
unsafe drivers.
Gansler et al. (2017) Case series 170 patients IV NP measures improved dif-
with dementia ferential diagnosis by
13–30% over base rates.
Geroldi et al. (2008) Case series 474 patients at centers IV Completion of both NP
for cogni- assessment and neuroi-
tive impairment maging improved differ-
ential diagnostic accuracy
of type of dementia.
(continued)
10 J. DONDERS
Table 2. Continued.
Jansen et al. (2017) Case series 221 memory IV Inclusion of NP variables
clinic patients improved accuracy both
syndromal and etiological
diagnosis and some
aspects of prognosis.
Kavaliunas et al (2018) Case series 2080 patients with MS IV Cognitive impairment
affected income, inde-
pendent of clinical and
sociodemographic
variables.
Kringle et al. (2018) Case series 175 patients IV RBANS Visuospatial index
with stroke improved concurrent pre-
diction of patient engage-
ment over baseline stroke
severity and disability.
Lee et al. (2006) Inception cohort 69 older adults with IV Trend level of ability of NP
questionable measures to predict con-
dementia version to dementia
MacNeill et al. (2000) Inception cohort 194 elderly IV NP variable was the single
participants best predictor of medical
rehabilitation dis-
charge status.
Pritchard et al. (2014) Inception cohort 119 youth with ADHD IV Those who received a neuro-
psychological evaluation
were more likely to subse-
quently begin behavioral
or medical treatment.
Ruet et al. (2013) Inception cohort 48 patients with MS IV Baseline NP results predicted
vocational status at 7
years, after controlling for
baseline vocational and
physical variables.
Schmand et al. (2014) Case series 62 memory IV Composite NP measure was
clinic patients more responsive to dis-
ease progression
than MRI.
Studies listed by level of evidence (I ¼ highest, IV ¼ lowest).
ADHD: attention-deficit/hyperactivity disorder; APOE: apolipoprotein E; CSF: cerebrospinal fluid; IADL: instrumental
activities of daily living; MCI: mild cognitive impairment; MRI: magnetic resonance imaging; MS: multiple sclerosis;
NP: neuropsychological; PET: positron emission tomography; RBANS: Repeatable Battery for the Assessment of
Neuropsychological Status; TBI: traumatic brain injury.
(e.g. apolipoprotein E [APOE] e4 status) and biological (e.g. cerebrospinal fluid [CSF]
tau) markers. In a related retrospective study that used a larger data set from the
same source (i.e. the Alzheimer’s Disease Neuroimaging Initiative, ADNI), Jedynak et al.
(2012) reported that over time, the Auditory Verbal Learning Test – delayed variable
was also the earliest marker to become abnormal, followed by bilateral hippocampal
volume and protein concentrations of amyloid b and tau, when participants were fol-
lowed up to three years. In addition, Thomas et al. (2018), used the ADNI dataset to
demonstrate that neuropsychological process scores (i.e. intrusion errors on the
Auditory Verbal Learning Test) contributed unique value in predicting progression
from cognitively normal to MCI as well as dementia over a five-year period, above and
beyond what could be accounted for on the basis of genetic and CSF biomarkers.
Measures of verbal learning as well as verbal fluency have also been identified in
other studies as having predictive value with regard to the differential prediction of
various dementias. Brewster, McDowell, Moineddin, and Tierney (2012) found that
such measures allowed for a double dissociation that could not be accounted for by
Hachinski Ischemic Score alone. Specifically, relative to those who developed
Alzheimer disease, older adults with poorer phonemic fluency and better immediate
verbal recall at baseline were more likely to develop vascular dementia over a five-
year span. That study did not include neuroimaging or serum biomarkers. In another
study that did include neuroimaging, Ito and colleagues (2015) reported that although
F18 fluorodeoxyglucose (F-FDG) positron emission tomography (PET) was the strongest
single predictor for distinguishing convertors from MCI to Alzheimer disease, the best
multivariate model included both PET findings and results from neuropsychological
measures of delayed verbal memory.
Other studies have focused instead on neuropsychological measures of executive
functioning. Aretouli, Tsilidis, and Brandt (2013) reported that such measures contrib-
uted marginally to the prediction of conversion from MCI to dementia over a four-year
period, after accounting for demographic background and clinical ratings of daily func-
tioning. However, that particular study was hampered by considerable attrition as well
as a suboptimal event to variable ratio in the multivariate models.
An example of the other approach to potential incremental value was the study
by Schmand et al. (2014). They used neurologists’ ratings of cognitively normal
(n ¼ 28) versus clinically impaired (n ¼ 34) as the standard against which to compare
the predictive validity of neuroimaging and neuropsychological measures in a mem-
ory clinic. They found that a composite neuropsychological score, based on measures
of verbal fluency, learning and memory, and executive functioning, was more sensi-
tive to disease progression than measures of hippocampal atrophy or cortical thick-
ness. Although small in sample size, a relative strength of this particular study was
that the authors screened for performance validity, which was a relatively rare
method in studies included across domains in this critical review. In a separate study
using a different sample, several of the same authors found that, after consideration
of performance on a measure of immediate verbal recall, MRI and CSF variables did
not substantially affect diagnostic accuracy of progression to AD in persons with MCI
who were followed for a mean of three years (Richard, Schmand, Eikelenboom, &
Van Gool, 2013).
In contrast, Lange et al. (2018), who used a large sample of persons with MCI
from the ADNI project, found that neuroimaging and CSF biomarkers were able to
provide modest incremental benefit over initial assessment of cognitive status (as
assessed with the ADAS-13; Mohs et al., 1997) during follow-up period of up to five
years. They clarified that the potential value of such markers depended strongly on
the patient’s status of each preceding step when considered in a stepwise fashion.
For example, F-FDG PET variables improved risk stratification only in persons with
relatively preserved cognitive performance at baseline. At the same time, this study
confirmed that psychometric variables had the highest univariate effect size for differ-
entiating persons who did versus did not progress from MCI to early dementia. This
was consistent with the findings of Gomar et al. (2011), who focused on a different
neuropsychological instrument. In addition, the findings from Lange et al. were con-
sistent with those from an earlier study by Devanand et al. (2007) who found that
12 J. DONDERS
baseline MRI measurements of hippocampal and entorhinal volumes had only a small
added value in the prediction of conversion from MCI to AD after accounting for age
and neuropsychological variables such as Selective Reminding Test delayed recall and
WAIS–R Digit Symbol.
The relative value of genetic markers, as compared to neuropsychological and other
variables, in AD development has been inconsistent across studies. For example,
Devanand et al. (2007) reported that APOE e4 allele carrier status did not alter the pre-
dictive accuracy of their MRI measures. In contrast, Callahan, Ramirez, Berezuk,
Duchesne, and Black (2015) found that having two copies of the APOE e4 allele
improved the prediction of AD development in persons with MCI, above and beyond
neuropsychological variables of delayed recall, even though the latter variables had
relatively greater odd ratios in the final prediction model. Some of these discrepancies
may have been due to differences in methodology. For example, Devanand et al.
defined MCI as performance at least 1.5 SD below the normative mean on a single
test within any domain whereas Callahan et al. used a cut-off of <1 SD on at least
two memory measures.
Other studies have also found that APOE status did not add to the predictive value
of neuropsychological measures in the prediction of long-term cognitive development
in elderly persons. For example, Cervilla, Prins, Joels, Lovestone, and Mann (2004) found
that cognitive tests (Trail Making in particular) increased the area under the curve for
prediction of incipient Alzheimer disease, above and beyond age and family history.
Importantly, this predictive value was not enhanced by addition of APOE genotype.
Similar findings were reported by Ganguli and colleagues (2014), who found that mem-
ory scores, based on the measures of logical memory and visual reproduction, were
strong predictors of development of cognitive impairment as well as dementia in a
large, population-based cohort of older adults. At the same time, the addition of varia-
bles such as APOE e4 status and stroke risk did not further improve this prediction. Lee
and colleagues (2006) had reported similar findings with regard to word list learning,
where the prediction of outcome was not improved by the addition of APOE e4 status.
However, in that study, the findings for the neuropsychological variables were only at a
trend level of statistical significance, which was likely related to small sample size.
A study by Jansen et al. (2017) exemplified yet another way of demonstrating the
incremental value of neuropsychological assessment in the evaluation of possible
development of MCI or dementia. They had non-psychologist health professionals pro-
vide a diagnosis (i.e. subjective cognitive complaints only, MCI or dementia) in patients
attending a memory clinic, both before and after neuropsychological assessment
results were made available. The provision of neuropsychological data resulted in stat-
istically significant changes in diagnosis of the cognitive syndrome (22%) and of the
underlying etiology as well as the disease prognosis (15%). This was, in turn, associ-
ated with statistically significant increases in correctly classified cases (based on con-
sensus panel) by 18% for the syndrome and 5% for the underlying etiology. The
findings confirmed the incremental value of neuropsychological assessment with
regard to diagnosing etiology in patients with either initial subjective cognitive com-
plaints only or initial MCI, and for predicting disease course in patients with initial
MCI, over a two-year period.
Geroldi et al. (2008) had used a similar design but focused on diagnosis of type of
dementia (i.e. Alzheimer, vascular, frontotemporal, or Lewy body). They noted statistic-
ally significant improvement in accuracy of diagnosis when neuropsychological assess-
ment as well as neuroimaging results became available. Unfortunately, they did not
evaluate the potential additive or interactive effects of each of these two test techni-
ques. Therefore, the independent incremental value of neuropsychological variables
could not be determined from this study.
There is also evidence that diagnostic criteria based on more comprehensive
neuropsychological variables can improve characterization of MCI as compared to
conventional methods. Bondi et al. (2014) used Jak et al.’s (2009) criteria to classify
persons as cognitively normal or having MCI, based on six neuropsychological
measures in three domains and one functional measure, and compared this classifi-
cation to results from conventional criteria based on subjective memory com-
plaints, clinical interviews, cognitive screening and rating scales, and impairment
on a single objective memory measure. The actuarial neuropsychological approach
identified significantly fewer patients as having MCI, while at the same time identi-
fying a larger percentage of patients with MCI who progressed to dementia over a
period of up to seven years. This study suggested that neuropsychological assess-
ment can significantly reduce the rate of false positive diagnostic errors in the
diagnosis of MCI.
Edmonds et al. (2015) expanded on the work by Bondi et al. (2014), using 825 per-
sons who had been identified in the ADNI project as having MCI based on conven-
tional criteria. They used cluster analysis of a set of more extensive neuropsychological
variables to explore the possibility of subtypes in this group. Importantly, in addition
to dysnomic (n ¼ 153), dysexecutive (n ¼ 102), and amnestic (n ¼ 288) clusters, they
also found a cognitively normal (n ¼ 282) subtype. Compared to the other subgroups,
this cognitively normal cluster had fewer APOE e4 carriers and fewer patients who pro-
gressed to dementia at follow-up. In addition, the profile of CSF biomarkers of this
cluster did not differ from that of a group who had initially been classified by conven-
tional ADNI criteria as cognitively normal (n ¼ 284). This study demonstrated that
neuropsychological variables can not only identify heterogeneity in MCI profiles that is
not captured by conventional criteria but also significantly improve specificity with
only minimal loss of sensitivity. Very similar results were reported by Eppig et al.
(2017) who used much of the same sample but applied different multivariate subtyp-
ing procedure (i.e. latent profile analysis) and also included measures of visual con-
structional abilities (pentagon and clock drawing), which had not been part of Bondi
et al. (2014) and Edmonds et al.’s (2015) studies.
Whereas most studies have focused on progression from MCI to AD, Gansler, Huey,
Pan, Wasserman, and Grafman (2017) focused on the differentiation of persons with
behavioral variant frontotemporal dementia (n ¼ 77), primary progressive aphasia
(n ¼ 25), and cortico-basal syndrome (n ¼ 68). Neuropsychological measures included
performance-based measures as well as standardized caregiver reports. The combined
use of these two sets of variables improved differentiation of behavioral variant fronto-
temporal dementia from the two other conditions by 13–30% over base rates. It was
noteworthy that the performance-based and caregiver report variables did not
14 J. DONDERS
strongly correlate with each other, reinforcing the importance of the conjoint use of
both sets of data.
It should be noted that not all studies have supported the incremental value of
specific neuropsychological tests. De Lepeleire, Heyrman, Baro, and Buntinx (2005)
found that although clock drawing improved the diagnosis of dementia in a cross-sec-
tional study, additional neuropsychological tests did not. However, this study used
uncommon computerized test variables and did not include more widely validated
measures of learning and delayed recall.
In summary, there is strong evidence for the incremental value of neuropsycho-
logical assessment in the diagnosis and prediction of outcomes in elderly individuals
who are at risk for MCI and/or progression to dementia. This may potentially add to
earlier and more effective intervention with such individuals in the future. Most of the
studies providing this support were cohort studies, with no randomized-con-
trolled trials.
Traumatic brain injury

There are several studies that have supported the incremental value of neuropsycho-
logical assessment with regard to prediction of outcomes in persons with traumatic
brain injury (TBI). Miller and Donders (2003) reported on a group of children with TBI
who had been seen in a rehabilitation setting, had no premorbid complicating factors
such as learning disability or depression, and were seen for neuropsychological evalu-
ation at a median of three months post-injury. They evaluated predictors of whether
or not the children were in special education at 24 months post-injury. They found
that performance on the children’s version of the California Verbal Learning Test at
three months was a statistically significantly better predictor of special education
placement at 24 months than any combination of demographic (age, gender, parental
education) and/or injury (duration of coma, neuroimaging findings) variables.
Pastorek, Hannay, and Contant (2004) demonstrated that even determining whether
adults were able to complete neuropsychological assessment as an inpatient following
TBI was informative with regard to prediction of outcomes. “Testability” at 1-month
post-injury improved the prediction of functional outcomes at six months, above and
beyond the variance accounted for by demographic and neurological injury variables.
When adult patients are testable following TBI, performance on the Trail Making
Test (TMT) has been shown to be predictive of long-term outcomes. Devitt and col-
leagues (2006) reported that worse performance on this test was an independent pre-
dictor of community integration, including occupational productivity and leisure
participation, at a median of 14 years post-injury; even after accounting for variables
such as premorbid history, baseline physical deficits and current lack of access to
transportation.
Hanks et al. (2008) followed adults with complicated mild-to-severe TBI who had
been admitted to inpatient rehabilitation and examined predictors of functional out-
comes (e.g. level of disability or need for supervision, as assessed by standardized rat-
ing scales) at one year post-injury. A model that was based on a brief battery of nine
neuropsychological tests that was administered during inpatient stay predicted those
outcomes statistically significantly better than a model based on baseline functional

and injury severity variables. This particular study did not directly investigate whether
adding the neuropsychological variables to the model comprised of baseline func-
tional and injury severity characteristics made a difference. As such, it provided only
indirect evidence of the incremental value of neuropsychological assessment.
Williams, Rapport, Hanks, Millis, and Greene (2013) provided more direct evidence
of such incremental value in a different sample of adults with complicated mild-to-
severe TBI who had been admitted for rehabilitation. Whereas CT scan variables were
not predictive of functional disability at two years post-injury, neuropsychological
measures obtained during inpatient rehabilitation contributed statistically significant
to such prediction, even after controlling for baseline demographic and injury severity
characteristics. The TMT and a visuospatial composite based on Benton Judgment of
Line Orientation and/or Visual Form Discrimination were the neuropsychological tests
that accounted for most of the unique variance.
It is important to appreciate that focusing only on cognitive variables as part of
neuropsychological evaluations of persons with TBI may potentially be misleading. For
example, when Leininger, Strong, and Donders (2014) examined multiple predictors of
outcomes of treatment in a post-concussion program, it appeared that performance
on part B of the TMT at the beginning of treatment predicted outcomes as measured
with the Mayo-Portland Adaptability Inventory–Fourth Edition (MPAI–4), above and
beyond what could be accounted for by baseline MPAI–4 ratings and psychosocial
and medical history variables. However, in a subsequent study with a new sample,
Scott, Strong, Gorter, and Donders (2016) found that when emotional distress at base-
line as measured by the Beck Depression Inventory–Second Edition was included in
the model, TMT was no longer a statistically significant predictor of MPAI–4 outcomes
whereas baseline emotional distress was.
There have also been studies of the benefit of neuropsychological assessment in TBI out-
side of a rehabilitation context. Van Kampen, Lovell, Pardini, Collins, and Fu (2006) evaluated
high school and college athletes who had sustained a concussion and compared their sub-
jective symptom reports as well as their neurocognitive test performance at two days post-
injury to pre-injury baseline scores as well as to those of an age- and education-matched
non-injury athlete control group. A relative strength of this study was the use of reliable
change index scores to define clinically significant change between pre- and post-injury
symptoms and performance. They found that the addition of neurocognitive test scores
resulted in a 19% increase in sensitivity to reliable change in athlete health status.
Finally, Arnett et al. (2013) demonstrated that parental ratings on the Behavior
Rating Inventory of Executive Function as well as children’s performance on the child-
ren’s version of the California Verbal Learning Test within the first six months after
moderate-to-severe TBI improved the prediction of educational competence at 12
months post-injury, above and beyond what could be accounted for by injury severity
and socio-economic status. The participants in this investigation were originally
enrolled in a randomized-clinical intervention trial, but this particular part of the study
was an inception cohort design.
In summary, several studies have convincingly indicated that neuropsychological
test variables add uniquely to the prediction of outcomes after TBI, in both children
16 J. DONDERS
and adults. Better prediction of educational and functional outcomes may allow for
earlier flagging of individuals at risk for poor outcomes, and potential acceleration of
compensatory intervention. The studies in this domain were all inception cohort or
case control studies and there were no randomized-controlled trials.
Stroke
In the area of stroke, findings with regard to the value of neuropsychological assess-
ment were initially mixed. On the one hand, McKinney et al. (2002) found no evidence
that such assessment improved the functional outcomes of patients with stroke,
although there was a trend for reduction of strain in those who cared for those
patients. On the other hand, Nys et al. (2005) reported that neuropsychological meas-
ures of visual perception and construction at three weeks post-stroke statistically
significantly improved the prediction of instrumental activities of daily living at 6 to
10-month follow-up. The study with a relatively higher quality from a methodological
point of view (i.e. McKinney et al., 2002) was the one that had non-supportive results.
The only caveat is that the study also had a relatively higher attrition rate, with fairly
uneven numbers of participants for various outcome variables.
Part of the discrepancy in findings may have been related to differences in goals
and statistical analysis. McKinney et al. compared the outcomes of two groups, only
one of whom had received a neuropsychological evaluation. Nys et al., on the other
hand, had neuropsychological data on all of their patients. McKinney et al. addressed
whether having a neuropsychological evaluation eventually changed patient out-
comes, whereas Nys et al. focused on whether neuropsychological data could predict
(but not necessarily modify) patient outcomes. Another possible explanation for the
differences may have been that McKinney et al. reported on global findings with
regard to activities of daily living, whereas Nys et al. differentiated between basic and
instrumental activities of daily living. They found an incremental contribution of neuro-
psychological assessment only for the latter. At the same time, it must be appreciated
that McKinney et al. also reported on other outcomes that were not addressed by Nys
et al. (such as levels of distress and satisfaction with care) and found no differences
between their two groups on those variables, either. Regardless of the explanation for
the differences in conclusions, it is important to appreciate that these two studies
tried to answer distinctly different questions. McKinney et al. wanted to know whether
those who had a neuropsychological evaluation had better or different outcomes after
stroke than those who did not. Nys et al., on the other hand, evaluated whether
neuropsychological findings improved the prediction of specific outcomes in a cohort
of patients who all had such an evaluation along with other (e.g. vascular risk factors)
information.
Some of the findings in the area of stroke appeared to differ with the age range of
the population studied and with the nature of the outcome variable. Alves de Moraes,
Szklo, Tilling, Sato, and Knopman (2003) reported that in a study of relatively young
persons, aged 48–67 years, who were followed for up to eight years, three different
neuropsychological tests (Digit Symbol, Word Fluency, Delayed Word Recall) did not
contribute to the prediction of ischemic stroke, after controlling for cardiovascular risk
factors, demographics, and tobacco and alcohol use variables. In a subsequent re-ana-
lysis of the same cohort data, however, Elkins, Knopman, Yaffe, and Johnston (2005)
reported that those same tests actually were independent predictors of incident car-
diovascular events, which included not only strokes but also myocardial infarctions
and coronary heart disease deaths. An important difference between the two studies
was that Alves de Moraes and colleagues excluded persons taking psychoactive medi-
cations whereas Elkins et al. did not. The latter study also controlled for fewer risk fac-
tors. Furthermore, combined myocardial events and deaths were about twice as
common as strokes in that cohort. It should be noted that there were no studies in
the area of pediatric stroke that met the inclusion criteria for this review, so the
degree to which age differences extend to that population cannot be ascertained at
this time.
Several studies by Wiberg and colleagues did offer more consistent support for a
role of neuropsychological assessment in the prediction of outcome in elderly patients
with stroke. In the first study, they followed 70-year-old men for a median of 11 years
(Wiberg et al., 2010). Performance on part B of the TMT was a robust predictor of
brain infarction during the follow-up period, even after accounting for education, soci-
oeconomic status, and a range of cardiovascular risk variables. The second investiga-
tion involved tracking for a median of 2.5 years of those persons from the original
investigation who experienced a stroke or TIA (Wiberg, Kilander, Sundstro €m, Byberg, &
Lind, 2012). TMT parts A and B were both predictors of post-stroke mortality, again
independent of education, socioeconomic status, and baseline cardiovascular risk.
It should be noted that not all studies have supported a role of neuropsychological
measures in the prediction of mortality after cerebral vascular disease. van der Holst
and colleagues (2016) reported that cognitive variables did not contribute to the pre-
diction of mortality over eight years in persons with small vessel disease, after control-
ling for age, baseline motor ability, and MRI measures of gray matter volume and
white matter diffusivity. To some extent, though, their findings may have been
affected by the combination of various psychometric variables (some of which were
not commonly used neuropsychological tests) into a composite index. That may have
obscured the potential contribution of variables that have shown promise in other
studies of post-stroke outcome, such as the Auditory Verbal Learning Test. It is also
quite possible that small vessel disease had other correlates than stroke, and that the
role of neuropsychological assessment may vary with the respective conditions. In
another recent study, Bertolin, Van Patten, Greif, and Fucetola (2018) found evidence
that a brief screening measure (the Short Blessed Test) was an independent predictor
of cognitive and communication outcomes at six months post-stroke whereas some
domain-specific neuropsychological tests (e.g. Trail Making) were not.
Several other recent studies have been supportive of the value of neuropsychological
assessment after stroke when it comes to predicting functional outcomes. Wagle et al.
(2011) first evaluated two models to predict functional status 13 months post-stroke.
One was a biological model that included sociodemographic variables, pre-stroke gen-
etic and vascular risk factors, lesion characteristics and stroke-related neurological
impairment. The other model was a functional one that included measures of pre- and
early post-stroke cognitive functioning, early measures of activities of daily living, and
18 J. DONDERS
early measures of depressive symptoms. The authors then combined statistically signifi-
cant variables from those two models into a final, combined model. They found that
early post-stroke cognitive functioning, as assessed by the Repeatable Battery for the
Assessment of Neuropsychological Status, was a statistically significant independent pre-
dictor of functional outcome at 13 months, after accounting for premorbid level of
functional independence, age- and stroke-related neurological impairment.
Kringle, Terhorst, Butters, and Skidmore (2018) examined predictors of patient
engagement, as assessed with the Pittsburgh Rehabilitation Participation Scale, during
inpatient rehabilitation after stroke. They reported that measures of executive func-
tioning and visuospatial skills, in combination with male gender and impairment in
mood, were predictive of levels of engagement after controlling for behavioral inter-
vention group status, baseline stroke severity, and baseline level of disability. A relative
limitation of their methods was that they used a fairly liberal level of alpha (p < .10)
in a stepwise backward regression, which may have capitalized on chance fluctuations
in the data. Although the participants in this study had originally been part of a
randomized-controlled trial with regard to the effect of behavioral intervention after
stroke, this particular set of analyses constituted a case series.
In summary, one early randomized-controlled trial did not find evidence for an
incremental value of neuropsychological assessment with regard to modification of
outcomes after stroke, whereas another early study involving an inception cohort
found that neuropsychological variables could predict such outcomes. A clear majority
of later cohort and case series studies provided support for the value of neuropsycho-
logical variables in the prediction of patient engagement and functional outcomes
after stroke, whereas the support for the incremental value in prediction of mortality
after cerebrovascular disease was inconsistent.
Epilepsy
Although neuropsychological data may not necessarily add much to determination of
lateralization of seizure focus, beyond what can be discerned from EEG and MRI
(Moser et al., 2000), there is evidence that such data can provide incremental value in
the subsequent management of patients with epilepsy. This evidence was based on
two studies with adults, as no studies of children with epilepsy were retrieved that
met the inclusion criteria for this review.
Elshorst et al. (2009) retrospectively evaluated predictors of post-operative cogni-
tive decline in patients with intractable left temporal lobe epilepsy who had under-
gone a left temporal lobectomy. They found that degree of mesial temporal
sclerosis and measures of pre-operative verbal learning were both independent pre-
dictors of cognitive outcome after surgery. In contrast, results from pre-operative
intracarotid amytal procedures did not add to this prediction. These findings
expanded on an earlier study by Stroup et al. (2003) who had included cases of
both dominant and non-dominant temporal lobectomy. They had found that side of
surgery (i.e. left hemisphere) and MRI findings had the highest predictive value with
regard to risk of post-operative decline in verbal memory, but that the proportion
of information retained over time delay on a test of verbal learning further
improved the accuracy of this prediction. It should be noted that in both studies,
the authors used the 90% reliable change index to define clinically significant
change between pre-surgical and post-surgical neuropsychological test scores. This
was a strength because standard error and normal variability over time were rarely
considered in studies in other domains. In summary, there was moderate evidence
of the incremental value of neuropsychological assessment in this domain. Both of
the supportive investigations were retrospective cohort studies. There were no
randomized-controlled trials.
Multiple sclerosis
The outcomes of persons with multiple sclerosis (MS) is another area where some
studies have yielded discrepant results regarding the incremental value of neuro-
psychological assessment. All of these studies involved adults with MS, as no studies
of children with MS were retrieved that met criteria for inclusion in this review.
On the one hand, Lincoln et al. (2002) failed to find any benefit of either neuro-
psychological assessment or a cognitive intervention program, compared to standard
care, in a randomized-controlled trial of patients with MS. On the other hand,
Portaccio et al. (2009) found that neuropsychological variables, along with other varia-
bles (i.e. gender and MRI findings) did contribute to the prediction of change from
benign (i.e. still functionally independent at least 10 years after disease onset) to no
longer benign clinical status in a prospective cohort study.
Again, the discrepant results may have been related to differences in study purpose
as well as methods. Lincoln et al. evaluated whether neuropsychological evaluations
eventually led to a change in quality of outcomes, whereas Portaccio et al. simply
focused on whether outcomes could be predicted (not necessarily modified) on the
basis of neuropsychological data. Lincoln et al. used a control group who did not
receive a detailed neuropsychological evaluation and focused on whether the out-
comes of that group differed from those who had undergone such an evaluation. In
contrast, Portaccio et al. had neuropsychological evaluations on all of their participants
and used these data in combination with other variables to predict disease evolution
over time. It should also be noted that all the participants in the Portaccio et al. study
had the benign form of MS at baseline, whereas the Lincoln et al.’s study included a
large proportion of patients with either primary or secondary progressive MS at the
beginning of their study. It was the study with patients with the relatively more
advanced form of the disease that had the non-supportive results. Another important
difference involved the follow-up interval, which was eight months in the Lincoln
et al. study, and five years in the Portaccio et al. study. There were no major problems
with attrition rates in either of the investigations but the findings from the multivari-
ate analyses in the Portaccio et al. study may have been affected by small sample size
and associated suboptimal event-to-variable ratio.
Several other studies have supported the incremental validity of neuropsychological
assessment with regard to prediction of various economical outcomes. Benedict and
colleagues (2005) found that three neuropsychological tests (Symbol Digit Modalities
Test [SDMT], Wisconsin Card Sorting Test, and Judgment of Line Orientation) were
20 J. DONDERS
able to discriminate persons with MS who were employed from those who were dis-
abled from employment. Disease duration also made an independent contribution to
this distinction but other variables such as fatigue or level of physical disability did
not. Honan, Brown, and Batchelor (2015) also found that the SDMT, along with meas-
ures of delayed verbal recall, was a strong correlate of unemployment and diminished
work hours. However, they did not control for variables such as disease characteristics
and physical disability status, so their work did not truly demonstrate incremental
value of neuropsychological variables.
Other studies included longer-term follow-up. Morrow and colleagues (2010) fol-
lowed for an average of 3.5 years a group of persons with MS who were initially
employed. After accounting for disease course (i.e. relapsing v. progressive) and base-
line demographics as well as physical disability status, declines over time on the SDMT
and California Verbal Learning Test–II were predictive of change in vocational status;
i.e. from employment to work disability. Similar findings were reported by Ruet and
colleagues (2013) who reported that baseline performance on measures of processing
speed (including SDMT and Paced Auditory Serial Addition Test) predicted vocational
outcomes seven years later, above and beyond baseline vocational and physical varia-
bles. A relative weakness of that study was small sample size with associated concerns
about participant-to-variable ratio in the analyses. Finally, a recent study by Kavaliunas
and colleagues (2018) found that cognitive impairment, as measured by the SDMT,
was associated with statistically significant decreases in yearly earnings, even after
adjusting for variables such as disease duration, physical disability status, and demo-
graphic background.
In summary, a high-quality randomized-controlled trial did not find evidence of
value of neuropsychological variables with regard to modification of outcomes after
MS, whereas a relatively lower quality prospective cohort study found that such varia-
bles could still predict such outcomes. The latter findings have been confirmed in
other studies, using case series or cohort designs. On balance, the findings supported
the incremental value of neuropsychological assessment in patients with MS.
Attention-deficit/hyperactivity disorder
Several studies have addressed the utility of neuropsychological assessment in the
care of children and adolescents with attention-deficit/hyperactivity disorder (ADHD).
No studies pertaining to ADHD in adults were found that met the selection criteria.
Pritchard, Koriakin, Jacobson, and Mahone (2014) surveyed parents of youth ages 3–17
years twice over the course of five months, including 37 whose children had under-
gone a neuropsychological evaluation at baseline and 82 whose children had not.
They reported that the former group had greater initiation of a range of services than
the latter group. However, these analyses were based upon within-group change, and
they did not account for the fact that there were considerable group differences at
baseline, such as in medication management, which was already high in the group
without neuropsychological evaluation (82%) compared to the other group (43%). It is
also important to appreciate that group membership was based on convenience, not
random assignment.
Breaux, Griffith, and Harvey (2016) followed a cohort of children prospectively from
age 3 to age 6. They determined which variables predicted eventual diagnosis of
ADHD at age 6. They found that, even after accounting for family income and baseline
symptoms of inattention and hyperactivity or impulsivity at age 3, specific neuro-
psychological measures obtained at age 4 still improved in a statistically significant
manner the prediction of ADHD status at age 6. Although this provided evidence for
the incremental validity of the neuropsychological measures, the authors also appro-
priately cautioned that those same measures would have misclassified about a third of
their sample if used in isolation.
In summary, there is moderate support from one inception cohort study and one
high-quality prospective cohort study that neuropsychological assessment may be of
incremental benefit in the prediction of development of ADHD and the associated out-
comes. There were no randomized-controlled trials in this area.
Cost savings
There are several studies that have provided evidence that participation in neuro-
psychological evaluations can be associated with health care savings. Van Kirk, Horner,
Turner, Dismuke, and Muzzy (2013) did a retrospective chart review of adults who
completed a neuropsychological evaluation at a VA medical center. They found that,
during the year following such an evaluation, there was a significant decrease in num-
ber and duration of hospitalizations, compared to the preceding year. This decrease
was not attributable to variables such as age, time, or changes in policy regarding pro-
vision of care. Although the exploratory and cross-sectional nature of the study pro-
hibited causal inferences, it was impressive that this decrease in service utilization was
associated with an average of $1869 in estimated cost savings per veteran served. In a
secondary analysis of the same sample, the same authors also found that those who
had failed performance validity measures during the neuropsychological evaluation
subsequently had more and longer hospitalizations during the next year than those
who had put forth adequate effort (Horner, Van Kirk, Dismuke, Turner, & Muzzy, 2014).
Estimated cost increases associated with inadequate effort were $466 for emergency
room visits and $649 for inpatient days. A similar conclusion was reached by Denning
and Shura (2018) who looked more specifically at the amount of disability payments
received by veterans who claimed cognitive deficits as the result of mild TBI. They
reported that the average disability payout to those who failed one or more perform-
ance validity tests during neuropsychological evaluations was $6391–$7063 per year,
compared to $4721 who did not fail any such measures.
Finally, Sieg, Mai, Mosti, and Brook (2018) performed a retrospective review of adult
inpatients who had been referred for bedside neuropsychological consultation
because of suspected cognitive impairment. They found that, compared to patients
who completed the assessment, those who refused it subsequently had higher
inpatient health care costs over a one-year follow-up period. Although the cross-sec-
tional nature of the study did not allow for causal inferences, the average patient who
refused a neuropsychological evaluation incurred about $11,622 more in inpatient hos-
pitalization costs over the subsequent year than those patients with the highest post-
22 J. DONDERS
discharge needs who did complete such an evaluation. Their findings also indicated
that refusal to participate in a neuropsychological evaluation was not just a result of
greater cognitive impairment to start with, so that was not confound.
In summary, there is evidence from several inception cohort studies using retro-
spective analyses that participation in neuropsychological assessments is associated
with reduced cost of health care over a one-year follow-up period. The lack of
randomized-controlled studies precludes causal inferences in this regard.
Miscellaneous
There have been a number of additional studies with various diagnostic groups that
have addressed how neuropsychological assessment variables are associated with
improvements in prediction of outcomes. MacNeill, Lichtenberg, and LaBuda (2000)
evaluated predictors of return to living alone after medical rehabilitation in a mixed
geriatric sample. They reported that cognition as assessed by the Mattis Dementia
Rating Scale was a statistically significant predictor of returning to living alone, after
accounting for baseline demographic and chronic illness and self-care status. However,
inspection of the details of the data suggested that in terms of improvement of over-
all classification accuracy with regard to the binary outcome of home discharge, this
was only a statistically non-significant trend.
Alosco et al. (2012) examined predictors of instrumental activities of daily living in
adults with heart failure. They found that cognition, as assessed by a combination of
the Mini Mental State Exam (MMSE) and the TMT, statistically significantly improved
the prediction of driving and medication management (although not financial man-
agement) beyond what could be accounted for on the basis of demographic back-
ground, physical fitness, and depression. With regard to the issue of driving in
particular, Ferreira, Simo ~es, and Maro ^co (2012) reported that the Addenbrooke
Cognitive Examination—Revised was superior to the MMSE alone in discriminating
safe from unsafe elderly drivers, as identified by a road test. However, this was only a
statistically non-significant trend. In addition, when the authors attempted to add
more variables to their models, they may have obtained unreliable findings due to
limitations in event-to-variable ratio with small subgroup sizes (n ¼ 25). In contrast,
Emerson and colleagues (2012) found that in a sufficiently large sample of commu-
nity-dwelling older drivers, measures from the Auditory Verbal Learning Test and Trail
Making Test contributed statistically significantly to multivariate predictor models of
moving violations and motor vehicle collisions after controlling for variables such as
level of education and baseline weekly mileage.
Finally, Devore and colleagues (2017) demonstrated the value of neuropsychological
variables in the prediction of long-term outcome of older adults who had experienced
post-operative delirium. A composite cognitive performance score, based on various
neuropsychological tests covering several specific domains, explained most of the vari-
ance in rate of cognitive decline up to three years later, after controlling for variables
such as living status and informant ratings of cognitive functioning at baseline.
Discussion
The purpose of this critical review was to evaluate the incremental value of neuro-
psychological assessment with regard to the prediction and/or management of out-
comes in patients with various neurological and other medical conditions who are at
risk for cognitive dysfunction. Overall, the findings were strongly supportive in MCI/
dementia (e.g. Bondi et al., 2014; Gomar et al., 2011) and TBI (e.g. Miller & Donders,
2003; Williams et al., 2013). They were moderately supportive for stroke (e.g. Wagle
et al., 2011), epilepsy (e.g. Elshorst et al., 2009), MS (e.g. Morrow et al., 2010), and
ADHD (e.g. Breaux et al., 2016). There were not sufficient papers with regard to other
conditions to draw any firm conclusions. However, there were several studies that
established an association between completion of a neuropsychological evaluation
and health care cost savings in mixed clinical samples (e.g. Sieg et al., 2018; Van Kirk
et al., 2013). The designs of those studies did not allow for causal inferences.
The supportive studies exemplified a wide range of potential benefits of neuro-
psychological assessment, including diagnostic clarification as well as improvement of
prediction of various longer-term outcomes. In pediatric samples, there was evidence
for improvement of prediction of educational outcomes in school-age children and
adolescents with TBI, and for future development of ADHD in younger children. With
regard to adults, the preponderance of the evidence supported the added value of
neuropsychological assessment in predicting level of independence with instrumental
activities of daily living after TBI, risk for post-surgical cognitive decline in persons
with intractable epilepsy, morbidity and (to a lesser extent) mortality after stroke, and
economic outcomes in persons with MS. Finally, there was substantial evidence that
neuropsychological assessment improves the differentiation of MCI from normal aging
as well as the prediction of conversion from MCI to dementia in the elderly. None of
this is to say that neuropsychological assessment should be used in isolation. As Watt
and Crowe (2018) have suggested, it should be combined with other techniques to
optimize both differential diagnosis and patient outcomes. Such other techniques
include those that are within the purview of neuropsychologists (e.g. comprehensive
history, inclusion of measures of mood and adjustment) as well as those for which col-
laboration with other health care professionals is important (e.g. neuroimaging,
serum biomarkers).
The majority of the studies that supported the incremental value of neuropsycho-
logical assessment in this review were Level III evidence and included primarily retro-
spective cohort and inception cohort studies. There were only two randomized-
controlled clinical trials that specifically evaluated the impact of neuropsychological
assessment; one in MS (Lincoln et al., 2002) and one in stroke (McKinney et al., 2002).
Both failed to support the incremental value of neuropsychological assessment on
modification of outcomes. It is, therefore, crucial that future research provides evi-
dence from at least high-quality cohort studies and preferably randomized-controlled
trials with regard to the benefit of neuropsychological assessment in the management
of patients with MCI, TBI, and other neurological conditions. Some ways to increase
quality of cohort studies in particular would be to include sufficiently large and repre-
sentative patient samples, in order to reduce inconsistency and indirectness of evi-
dence, and to conduct the studies prospectively in order to reduce the risk of bias. It
24 J. DONDERS
will also be important to gather and analyze more information about how neuro-
psychological data are actually used by providers who have more direct control over
the (modification of) outcomes of patients.
The quality ratings of many of the reviewed studies in this review were often
affected by methodological limitations, particularly with regard to sample size and/or
event-to-variable ratios in logistic regression models. These are limitations that can
and must be avoided in future studies. Effect sizes and confidence intervals should be
reported more consistently, in addition to p values. This would allow more formal
meta-analysis. Furthermore, the consideration of more sophisticated methods of deter-
mining the incremental validity of predictors (e.g. Sackett, Dahlke, Shewach, & Kuncel,
2017) is desirable.
Future studies should also make more concerted efforts at translating psychometric
findings regarding levels of statistical significance into dollar values. There have been
only a few studies that have shown associations between information from neuro-
psychological assessment and cost savings, and all were observational in nature (e.g.
Denning & Shura, 2018; Horner et al., 2014). In the near future, it will become increas-
ingly important for neuropsychologists to demonstrate not only that their work can
improve (prediction of) patient outcomes but also that this can be done in a cost-effi-
cient manner. Starting with contrasting the financial cost of one neuropsychological
evaluation with the savings in health care expenditures over one year would be an
example of that. Combining this type of information with data about referral preferen-
ces (e.g. Mahoney et al., 2017; Temple, Carvalho, & Tremont, 2006) as well as patient
and family satisfaction (e.g. Kirkwood, Peterson, Baker, & Connery, 2017; Rosado et al.,
2018) is also desirable.
There have been encouraging recent developments. In the field of TBI, comprehen-
sive consensus recommendations have been developed for outcome measures in
research with both children (McCauley et al., 2012; Wearne et al., 2018) and adults
(Wilde et al., 2010). Benedict and Walton (2012) have described statistical and anchor-
based methods for interpreting clinically meaningful change in MS. Goverover,
Chiaravalloti, O’Brien, and DeLuca (2018) have reported on improvements in the field
of cognitive rehabilitation with the same condition. In addition, Kalb et al. (2018)
have provided updated recommendations for screening for cognitive impairment in
MS. Clark and colleagues (2016) introduced the relatively new method of machine
learning strategies in the context of using neuropsychological data in the prediction
of conversion of MCI to dementia. Mateen and colleagues (2018) used the same
method in the prediction of falls in persons with neurological impairment. Future
research will need to address the replicability and external validity of such machine
learning strategies. Future studies may also explore further the potential benefits of
using fMRI activation, based on neuropsychological test performance, in prediction of
cognitive and functional outcomes in various neurological conditions (e.g. Hantke
et al., 2013).
Limitations of this review must also be considered. No studies prior to 2000 and no
studies in languages other than English were retrieved. There was no exhaustive litera-
ture by test name because this was considered impractical given the large number of
measures that exist. Due to the heterogeneity of the studies as well as the fact that
details about effect sizes, classification accuracy and confidence intervals were not
consistently reported, a meta-analysis was not possible and the review did not adhere
to all PRISMA guidelines (e.g. no formal measures such as risk ratio or consistency).
Furthermore, many of the studies (with the notable exception of those in epilepsy)
that reported on change in functioning did not explicitly consider base rates or reli-
ability of change, which are important considerations (Duff, 2012; Hinton-Bayre &
Kwapil, 2017). In addition, the vast majority of the studies did not include measures of
performance validity, which is really considered a standard in neuropsychological
assessment (Bush et al., 2005; Heilbronner, Sweet, Morgan, Larrabee& Millis, 2009).
It was also striking that several of the reviewed studies demonstrated bias in the
respective literature reviews. For example in the area of stroke, neither Nys et al.
(2005) nor Wagle et al. (2011), who both reported an incremental benefit of neuro-
psychological assessment with regard to prediction of outcomes in stroke, referred to
the prior study by McKinney et al. (2002) that failed to find such support. In future
studies, it will be incumbent on neuropsychologists to be more balanced in their
review of the literature and presentation of any purportedly new findings.
With these limitations in mind, a reasonable conclusion from this critical review is
that the preponderance of the evidence provides fairly encouraging but not com-
pletely conclusive support for the incremental value of neuropsychological assessment.
It needs to be realized that the degree of this support is not consistent across diag-
nostic groups. The current findings allow neuropsychologists to demonstrate to their
patients, referral sources, and insurance carriers some objective data to support the
reasonableness and necessity of their services.
Acknowledgments
There was no external funding for this work. The author gratefully acknowledges the advice and
support from the following individuals with regard to the literature search: William Barr, Ralph
Benedict, Marc Bondi, John DeLuca, and David Nyenhuis.
Disclosure statement
No potential conflict of interest was reported by the author.
References
Alosco, M. L., Spitznagel, M. B., Cohen, R., Sweet, L. H., Colbert, L. H., Josephson, R., … Gunstad,
J. (2012). Cognitive impairment is independently associated with reduced instrumental ADLs
in persons with heart failure. Journal of Cardiovascular Nursing, 27(1), 44–50. doi:10.1097/
JCN.0b013e318216a6cd
Alves de Moraes, S., Szklo, M., Tilling, K., Sato, R., & Knopman, D. (2003). Cognitive functioning as
a predictor of ischemic stroke incidence. Epidemiology, 14(6), 673–679. doi:10.1097/
01.ede.0000083262.58396.a3
Aretouli, E., Tsilidis, K. K., & Brandt, J. (2013). Four-year outcome of mild cognitive impairment:
The contribution of executive dysfunction. Neuropsychology, 27(1), 95–106. doi:10.1037/
a0030481
26 J. DONDERS
Arnett, A. B., Peterson, R. L., Kirkwood, M. W., Taylor, H. G., Stancin, T., Brown, T. M., … Wade,
S. L. (2013). Behavioral and cognitive predictors of educational outcomes in pediatric
traumatic brain injury. Journal of the International Neuropsychological Society, 19, 1–9.
Benedict, R. H. B., Wahlig, E., Bakshi, R., Fishman, I., Munschauer, F., Zivadinov, R., & Weinstock-
Guttman, B. (2005). Predicting quality of life in multiple sclerosis: Accounting for physical
disability, fatigue, cognition, mood disorder, personality, and behavior change. Journal
of the Neurological Sciences, 231(1–2), 29–34. doi:10.1016/j.jns.2004.12.009
Benedict, R. H. B., & Walton, M. K. (2012). Evaluating cognitive outcome measures for MS clinical
trials: What is clinically meaningful change? Multiple Sclerosis Journal, 18(12), 1673–1679. doi:
10.1177/1352458512454774
Bertolin, M., Van Patten, R., Greif, T., & Fucetola, R. (2018). Predicting cognitive functioning,
activities of daily living, and participation 6 months after mild to moderate stroke. Archives
of Clinical Neuropsychology, 33(5), 562–576. doi:10.1093/arclin/acx096
Bondi, M. W., Edmonds, E. C., Jak, A. J., Clark, L. R., Delano-Wood, L., McDonald, C. R., …
Salmon, D. P. (2014). Neuropsychological criteria for mild cognitive impairment improves
diagnostic precision, biomarker associations, and progression rates. Journal of Alzheimer’s
Disease, 42(1), 275–289. doi:10.3233/JAD-140276
Braun, M., Tupper, D., Kaufmann, P., McCrea, M., Postal, K., Westerveld, M., … Deer, T. (2011).
Neuropsychological assessment: A valuable tool in the diagnosis and management of neuro-
logical, neurodevelopmental, medical, and psychiatric disorders. Cognitive and Behavioral
Neurology, 24(3), 107–114. doi:10.1097/WNN.0b013e3182351289
Breaux, R. P., Griffith, S. F., & Harvey, E. A. (2016). Preschool neuropsychological measures
as predictors of later attention deficit hyperactivity disorder. Journal of Abnormal Child
Psychology, 44(8), 1455–1471. doi:10.1007/s10802-016-0140-1
Brewster, P. W. H., McDowell, I., Moineddin, R., & Tierney, M. C. (2012). Differential prediction
of vascular dementia and Alzheimer’s disease in nondemented older adults within 5 years of
initial testing. Alzheimer’s Disease and Dementia, 8(6), 528–535. doi:10.1016/j.jalz.2011.09.233
Burns, P. B., Rohrich, R. J., & Chung, K. C. (2011). The levels of evidence and their role
in evidence-based medicine. Plastic and Reconstructive Surgery, 128(1), 305–310. doi:10.1097/
PRS.0b013e318219c171
€ster, A. I., Barth, J. T., Koffler, S. P., Pliskin, N. H., … Silver, C. H. (2005).
Bush, S. S., Ruff, R. M., Tro
Symptom validity assessment: Practice issues and medical necessity NAN policy & planning
committee. Archives of Clinical Neuropsychology, 20, 419–426. doi:10.1016/j.acn.2005.02.002
Callahan, B. L., Ramirez, J., Berezuk, C., Duchesne, S., & Black, S. E. (2015). Predicting Alzheimer’s
disease development: A comparison of cognitive criteria and associated neuroimaging
biomarkers. Alzheimer’s Research and Therapy, 7, 68. doi:10.1186/s13195-015-0152-z
Cervilla, J., Prince, M., Joels, S., Lovestone, S., & Mann, A. (2004). Premorbid cognitive testing
predicts the onset of dementia and Alzheimer’s disease better and independently of APOE
genotype. Journal of Neurology, Neurosurgery and Psychiatry, 75, 1100–1106. doi:10.1136/jnnp.
2003.028076
Chelune, G. J. (2010). Evidence-based research and practice in clinical neuropsychology.
The Clinical Neuropsychologist, 24(3), 454–467. doi:10.1080/13854040802360574
Clark, D. G., McLaughlin, P. M., Woo, E., Hwang, K., Hurtz, S., Ramirez, L., … Apostolova, L. G.
(2016). Novel verbal fluency scores and structural brain imaging for prediction of cognitive
outcome in mild cognitive impairment. Alzheimer’s disease and dementia: Diagnosis,
assessment and disease monitoring, 2, 113–122. doi:10.1016/j.dadm.2016.02.001
Cohen, O. S., Vakil, E., Tanne, D., Molshatzki, N., Nitsan, Z., & Hassin-Baer, S. (2012). The Frontal
Assessment Battery as a tool for evaluation of frontal lobe dysfunction in patients
with Parkinson disease. Journal of Geriatric Psychiatry and Neurology, 25(2), 71–77. doi:
10.1177/0891988712445087
De Lepeleire, J., Heyrman, J., Baro, F., & Buntinx, F. (2005). A combination of tests for the
diagnosis of dementia had a significant diagnostic value. Journal of Clinical Epidemiology,
58(3), 217–225. doi:10.1016/j.jclinepi.2004.07.005
Denning, J. H., & Shura, R. D. (2018). Cost of malingering mild traumatic brain injury-related
cognitive deficits during compensation and pension evaluations in the veterans benefits
administration. Applied Neuropsychology: Adult. doi:10.1080/23279095.2017.1350684
Denti, I., Agosti, M., & Franceschini, M. (2008). Outcome predictors of rehabilitation for first
stroke in the elderly. European Journal of Physical Medicine and Rehabilitation, 44, 3–11.
Devanand, D. P., Pradhaban, G., Liu, X., Khandji, A., De Santi, S., Segal, S., … de Leon, M. J.
(2007). Hippocampal and entorhinal atrophy in mild cognitive impairment: Prediction of
Alzheimer disease. Neurology, 68(11), 828–836. doi:10.1212/01.wnl.0000256697.20968.d7
Devitt, R., Colantonio, A., Dawson, D., Teare, G., Ratcliff, G., & Chase, S. (2006). Prediction
of long-term occupational performance outcomes for adults after moderate to severe traumatic
brain injury. Disability and Rehabilitation, 28(9), 547–559. doi:10.1080/00222930500219258
Devore, E. E., Fong, T. G., Marcantonio, E. R., Schmitt, E. M., Travison, T. G., Jones, R. N., …
Inouye, S. K. (2017). Prediction of long-term cognitive decline following postoperative delirium
in older adults. The Journals of Gerontology: Series A, 72(12), 1697–1702. doi:10.1093/gerona/
glx030
Donders, J., & Strong, C. A. H. (2015). Clinical utility of the Wechsler Adult Intelligence
Scale—Fourth Edition after traumatic brain injury. Assessment, 22(1), 17–22. doi:10.1177/
1073191114530776
Duff, K. (2012). Evidence-based indicators of neuropsychological change in the individual
patient: Relevant concepts and methods. Archives of Clinical Neuropsychology, 27(3), 248–261.
doi:10.1093/arclin/acr120
Edmonds, E. C., Delano-Wood, L., Clark, L. R., Jak, A. J., Nation, D. A., McDonald, C. R., … Bondi,
M. W. (2015). Susceptibility of the conventional criteria for MCI to false positive diagnostic
errors. Alzheimer’s Dementia, 11(4), 415–424. doi:10.1016/j.jalz.2014.03.005
Elkins, J. S., Knopman, D. S., Yaffe, K., & Johnston, S. C. (2005). Cognitive function predicts first-
time stroke and heart disease. Neurology, 64(10), 1750–1755. doi:10.1212/01.WNL.0000161850.
01792.77
Elshorst, N., Pohlmann-Eden, B., Horstmann, S., Schulz, R., Woermann, F., & McAndrews, M. P.
(2009). Postoperative memory prediction in left temporal epilepsy: The Wada test is
of no added value to preoperative neuropsychological assessment and MRI. Epilepsy and
Behavior, 16(2), 335–340. doi:10.1016/j.yebeh.2009.08.003
Emerson, J. L., Johnson, A. M., Dawson, J. D., Uc, E. Y., Anderson, S. W., & Rizzo, M. (2012).
Predictors of driving outcomes in advancing age. Psychology and Aging, 27(3), 550–559. doi:
10.1037/a0026359
Eppig, J. S., Edmonds, E. C., Campbell, L., Sanderson, M., Delano-Wood, L., & Bondi, M. W. (2017).
Statistically-derived subtypes and associations with cerebrospinal fluid and genetic biomarkers
in mild cognitive impairment: A latent profile analysis. Journal of the International
Neuropsychological Society, 23(07), 564–576. doi:10.1017/S135561771700039X
Ferreira, I. S., Simo~es, M. R., & Maro ^co, J. (2012). The Addenbrooke Cognitive Examination—
Revised as a potential screening test for elderly drivers. Accident Analysis and Prevention,
49, 278–286. doi:10.1016/j.aap.2012.03.036
Ganguli, M., Lee, C. W., Snitz, B., Hughes, T., McDade, E. M., & Chang, C. C. H. (2014). How well
do MCI criteria predict progression to severe cognitive impairment and dementia? Alzheimer
Disease and Associated Disorders, 28(2), 113–121. doi:10.1097/WAD.0000000000000018
Gansler, D. A., Huey, E. D., Pan, J. J., Wasserman, E., & Grafman, J. H. (2017). Assessing
the dysexecutive syndrome in dementia. Journal of Neurology, Neurosurgery & Psychiatry,
88, 254–261. doi:10.1136/jnnp-2016-313576
Geroldi, C., Canu, E., Bruni, A. C., Forno, G. D., Ferri, R., Gabelli, C., … Frisoni, G. B. (2008).
The added value of neuropsychologic tests and structural imaging for the etiologic diagnosis
of dementia in Italian expert centers. Alzheimer Disease and Associated Disorders,
22(4), 309–320. doi:10.1097/WAD.0b013e3181871a47
Gomar, J. J., Bobes-Bascaran, M. T., Conejero-Goldberg, C., Davies, P., & Goldberg, T. E. (2011).
Utility of combinations of biomarkers, cognitive markers, and risk factors to predict conver-
sion from mild cognitive impairment to Alzheimer disease in patients in the Alzheimer’s
28 J. DONDERS
Disease Neuroimaging Initiative. Archives of General Psychiatry, 68(9), 961–969. doi:10.1001/

archgenpsychiatry.2011.96
Goverover, Y., Chiaravalloti, N. D., O’Brien, A. R., & DeLuca, J. (2018). Evidence-based cognitive
rehabilitation for persons with multiple sclerosis: An updated review of the literature from
2007 to 2016. Archives of Physical Medicine and Rehabilitation, 99(2), 390–407. doi:10.1016/
j.apmr.2017.07.021
Hanks, R. A., Jackson, A. M., & Crisanti, L. K. (2016). Predictive validity of a brief outpatient neuro-
psychological battery in individuals 1-25 years post traumatic brain injury. The Clinical
Neuropsychologist, 30(7), 1074–1086. doi:10.1080/13854046.2016.1194479
Hanks, R. A., Millis, S. R., Ricker, J. H., Giacino, J. T., Nakese-Richardson, R., Frol, A. B., … Gordon,
W. A. (2008). The predictive validity of a brief inpatient neuropsychologic battery for persons
with traumatic brain injury. Archives of Physical Medicine and Rehabilitation, 89(5), 950–957.
doi:10.1016/j.apmr.2008.01.011
Hantke, N., Nielson, K. A., Woodard, J. L., Breting, L. M. G., Butts, A., Seidenberg, M., … Rao,
S. M. (2013). Comparison of semantic and episodic memory BOLD fMRI activation in predict-
ing cognitive decline in older adults. Journal of the International Neuropsychological Society,
19(01), 11–21. doi:10.1017/S1355617712000951
Heilbronner, R. L., Sweet, J. J., Morgan, J. E., Larrabee, G. J., & Millis, S. R. (2009). American
Academy of Clinical Neuropsychology consensus conference statement on the neuropsycho-
logical assessment of effort, response bias, and malingering. The Clinical Neuropsychologist, 23,
1093–1129. doi:10.1080/13854040903155063
Hinton-Bayre, A. D., & Kwapil, K. J. (2017). Best practice approaches for evaluating significant
change for individuals. In S. C. Bowden (Ed.), Neuropsychological assessment in the age of evi-
dence-based practice: Diagnostic and treatment evaluations (pp. 121–154). New York, NY:
Oxford.
Holtzer, R., Verghese, J., Wang, C., Hall, C. B., & Lipton, R. B. (2008). Within-person across-neuro-
psychological test variability and incident dementia. Journal of the American Medical
Association, 300(7), 823–830. doi:10.1001/jama.300.7.823
Honan, C. A., Brown, R. F., & Batchelor, J. (2015). Perceived cognitive difficulties and cognitive
test performance as predictors of employment outcomes in people with multiple sclerosis.
Journal of the International Neuropsychological Society, 21, 1–13.
Horner, M. D., Van Kirk, K. K., Dismuke, C. E., Turner, T. H., & Muzzy, W. (2014). Inadequate effort
on neuropsychological evaluation is associated with increased healthcare utilization. The
Clinical Neuropsychologist, 28(5), 703–713. doi:10.1080/13854046.2014.925143
Ito, K., Fukuyama, H., Senda, M., Ishii, K., Maeda, K., Yamamoto, Y., … Ikeda, M. (2015).
Prediction of outcomes in mild cognitive impairment by using 18F-FDG-PET: A multicenter
study. ). Journal of Alzheimer’s Disease: JAD, 45(2), 543–552. doi:10.3233/JAD-141338
Jak, A. J., Bondi, M. W., Delano-Wood, L., Wierenga, C., Corey-Bloom, J., Salmon, D. P., … . Delis,
D. C. (2009). Quantification of five neuropsychological approaches to defining mild cognitive
impairment. American Journal of Geriatric Psychiatry, 17(5), 368–375. doi:10.1097/
JGP.0b013e31819431d5
Jansen, W. J., Handels, R. L. H., Visser, P. J., Aalten, P., Bouwman, F., Claassen, J., … Ramakers, I.
H. G. B. (2017). The diagnostic and prognostic value of neuropsychological assessment in
memory clinic patients. Journal of Alzheimer’s Disease, 55(2), 679–689. doi:10.3233/JAD-160126
Jedynak, B. M., Lang, A., Liu, B., Katz, E., Zhang, Y., Wyman, B. T., … Prince, J. L. (2012). A com-
putational neurodegenerative disease progression score: Method and results with the
Alzheimer’s disease neuroimaging initiative cohort. NeuroImage, 63(3), 1478–1486. doi:
10.1016/j.neuroimage.2012.07.059
Kalb, R., Beier, M., Benedict, R. H. B., Charvet, L., Costello, K., Feinstein, A., … DeLuca, J. (2018).
Recommendations for cognitive screening and management in multiple sclerosis care.
Multiple Sclerosis Journal. doi:10.1177/1352458518803785
Kavaliunas, A., Karrenbauer, V. D., Gyllensten, H., Manouchehrina, A., Glaser, A., Olsson, T., …
Hillert, J. (2018). Cognitive function is a major determinant of income among multiple
sclerosis patients in Sweden acting independently from physical disability. Multiple Sclerosis
Journal, doi:10.1177/1352458517740212
Keary, T., Frazier, T., Busch, R., Kubu, C., & Iampietro, M. (2007). Multivariate neuropsychological
prediction of seizure lateralization in temporal lobe epilepsy surgery cases. Epilepsia, 48(8),
1438–1446. doi:10.1111/j.1528-1167.2007.01098.x
Kirkwood, M. W., Peterson, R. L., Baker, D. A., & Connery, A. K. (2017). Parent satisfaction with
neuropsychological consultation after pediatric mild traumatic brain injury. Child
Neuropsychology, 23(3), 273–283. doi:10.1080/09297049.2015.1130219
Kringle, E. A., Terhorst, L., Butters, M. A., & Skidmore, E. R. (2018). Clinical predictors of engage-
ment in inpatient rehabilitation among stroke survivors with cognitive deficits: An exploratory
study. Journal of the International Neuropsychological Society, 24(6), 572–583. doi:10.1017/
S1355617718000085
Kubu, C. S., Ready, R. E., Festa, J. R., Roper, B. L., & Pliskin, N. H. (2016). The times are a-changin’:
Neuropsychology and integrated care teams. The Clinical Neuropsychologist, 30(1), 51–65. doi:
10.1080/13854046.2015.1134670
Lange, C., Suppa, P., Pietrzyk, U., Makowski, M. R., Spies, L., Peters, O., … Buchert, R. (2018).
Prediction of Alzheimer’s dementia in patients with amnestic mild cognitive impairment in
clinical routine: Incremental value of biomarkers of neurodegeneration and brain amyloidosis
added stepwise to cognitive status. Journal of Alzheimer’s Disease, 61(1), 373–388. doi:
10.3233/JAD-170705
Lee, D. Y., Youn, J. C., Choo, I. H., Kim, K. W., Jhoo, J. H., Pak, Y. S., … Woo, J. I. (2006).
Combination of clinical and neuropsychologic information as a better predictor of the pro-
gression to Alzheimer disease in questionable dementia individuals. American Journal of
Geriatric Psychiatry, 14(2), 130–138. doi:10.1097/01.JGP.0000192487.58426.d2
Leininger, S., Strong, C. A. H., & Donders, J. (2014). Predictors of outcome after treatment for
mild traumatic brain injury: A pilot study. Journal of Head Trauma Rehabilitation, 29, 209–116.
Leitner, D., Miller, H., & Libben, M. (2018). Assessing the predictive value of a neuropsychological
model on concurrent function in acute stroke recovery and rehabilitation. The Clinical
Neuropsychologist, 1. doi:10.1080/13854046.2018.1487586
Lincoln, N. B., Dent, A., Harding, J., Weyman, N., Nicholl, C., Blumhardt, L. D., … Playford, E. D.
(2002). Evaluation of cognitive assessment and cognitive intervention for people with multiple
sclerosis. Journal of Neurology, Neurosurgery & Psychiatry, 72(1), 93–98. doi:10.1136/
jnnp.72.1.93
MacNeill, S. E., Lichtenberg, P. A., & LaBuda, J. (2000). Factors affecting return to living alone
after medical rehabilitation: A cross-validation study. Rehabilitation Psychology, 45(4), 356–364.
doi:10.1037/0090-5550.45.4.356
Mahoney, J. J., Bajo, S. D., De Marco, A. P., Arredondo, B. C., Hilsabeck, R. C., & Broshek, D. K.
(2017). Referring providers’ preferences and satisfaction with neuropsychological services.
Archives of Clinical Neuropsychology, 32(4), 427–436. doi:10.1093/arclin/acx007
Mateen, B. A., Bussas, M., Doogan, C., Waller, D., Saverino, A., Kiraly, F. J., … Playford, E. D.
(2018). The Trail Making Test: A study of its ability to predict falls in the acute neurological
inpatient population. Clinical Rehabilitation, 32(10), 1396–1405. doi:10.1177/
0269215518771127
McCauley, S. R., Wilde, E. A., Anderson, V. A., Bedell, G., Beers, S. R., Campbell, T. F., … Yeates,
K. O. (2012). Recommendations for the use of common outcome measures in pediatric trau-
matic brain injury research. Journal of Neurotrauma, 29(4), 678–705. doi:10.1089/
neu.2011.1838
McKinney, M., Blake, H., Treece, K. A., Lincoln, N. B., Playford, E. D., & Gladman, J. R. F. (2002).
Evaluation of cognitive assessment in stroke rehabilitation. Clinical Rehabilitation, 16(2),
129–136. doi:10.1191/0269215502cr479oa
Miller, L. J., & Donders, J. (2003). Prediction of educational outcome after pediatric traumatic
brain injury. Rehabilitation Psychology, 48(4), 237–241. doi:10.1037/0090-5550.48.4.237
30 J. DONDERS
Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred reporting items for systematic
reviews and meta-analyses: The PRISMA Statement. British Medical Journal, 339, b2535. doi:
10.1136/bmj.b2535
Mohs, R. C., Knopman, D., Petersen, R. C., Ferris, S. H., Ernesto, C., Grundman, M., … Thai, L. J.
(1997). Development of cognitive instruments for use in clinical trials of antidementia drugs:
Additions to the Alzheimer’s Disease Assessment Scale that broaden its scope. Alzheimer
Disease and Associated Disorders, 11, S13–S21. doi:10.1097/00002093-199700112-00003
Morrow, S. A., Drake, A., Zivadinov, R., Munschauer, F., Weinstock-Guttman, B., & Benedict,
R. H. B. (2010). Predicting loss of employment over three years in multiple sclerosis: Clinically
meaningful cognitive decline. The Clinical Neuropsychologist, 24(7), 1131–1145. doi:10.1080/
13854046.2010.511272
Moser, D. J., Bauer, R. M., Gilmore, R. L., Dede, D. E., Fennell, E. B., Algina, J. J., … Cohen, R. A.
(2000). Electroencephalographic, volumetric and neuropsychological indicators of seizure
focus lateralization in temporal lobe epilepsy. Archives of Neurology, 57(5), 707–712. doi:
10.1001/archneur.57.5.707
National Multiple Sclerosis Society. (2006). Assessment and management of cognitive impair-
ment in multiple sclerosis. Retrieved from www.nationalmssociety.org/NationalMSSociety/
media/MSNationalFiles/Brochures/Clinical_Bulletin_Assessment-and-Management-of-Cognitive-
Impairment-in-MS.pdf
National Stroke Foundation (2010). Clinical guidelines for stroke management: A quick guide for
psychology. Retrieved from www.strokefoundation.com.au
Nys, G. M. S., van Zandvoort, M. J. E., de Kort, P. L. M., van der Worp, H. B., Jansen, B. P. W.,
Algra, A., … Kappelle, L. J. (2005). The prognostic value of domain-specific cognitive abilities
in acute first-ever stroke. Neurology, 64(5), 821–827. doi:10.1212/01.WNL.0000152984.28420.5A
€ Mancardi, G., & Roccatagliata, L. (2014). Isolated
Pardini, M., Uccelli, A., Grafman, J., Yaldizli, O.,
cognitive relapses in multiple sclerosis. Journal of Neurology, Neurosurgery and Psychiatry,
85(9), 1035–1037. doi:10.1136/jnnp-2013-307275
Park, S. H., Sohn, M. K., Jee, S., & Yang, S. S. (2017). The characteristics of cognitive impairment
and their effects on functional outcome after inpatient rehabilitation in subacute stroke
patients. Annals of Rehabilitation Medicine, 41(5), 734–742. doi:10.5535/arm.2017.41.5.734
Pastorek, N. J., Hannay, H. J., & Contant, C. S. (2004). Prediction of global outcome with acute
neuropsychological testing following closed-head injury. Journal of the International
Neuropsychological Society, 10, 807–817.
Portaccio, E., Stromillo, M. L., Goretti, B., Zipoli, V., Siracusa, G., Battaglini, M., … De Stefano, N.
(2009). Neuropsychological an MRI measures predict short-term evolution in benign multiple
sclerosis. Neurology, 73(7), 498–503. doi:10.1212/WNL.0b013e3181b351fd
Prigatano, G. P., & Morrone-Strupinsky, J. (2010). Advancing the profession of clinical neuro-
psychology with appropriate outcome studies and demonstrated clinical skills. The Clinical
Neuropsychologist, 24(3), 468–480. doi:10.1080/13854040802585048
Prigatano, G. P., & Pliskin, N. H. (2003). Clinical neuropsychology and cost outcome research: A
beginning. New York: Psychology Press.
Pritchard, A. E., Koriakin, T., Jacobson, L. A., & Mahone, E. M. (2014). Incremental validity of
neuropsychological assessment in the identification and treatment of youth with ADHD. The
Clinical Neuropsychologist, 28(1), 26–48. doi:10.1080/13854046.2013.863978
Richard, E., Schmand, B. A., Eikelenboom, P., & Van Gool, W. (2013). MRI and cerebrospinal fluid
biomarkers for predicting progression to Alzheimer’s disease in patients with mild cognitive
impairment: A diagnostic accuracy study. BMJ Open, 3(6), e002541. doi:10.1136/bmjopen-
2012-002541
Rosado, D. L., Buehler, S., Botbol-Berman, E., Feigon, M., Leo n, A., Luu, H., … Pliskin, N. H.
(2018). Neuropsychological feedback services improve quality of life and social adjustment.
The Clinical Neuropsychologist, 32(3), 422–435. doi:10.1080/13854046.2017.1400105
Ruet, A., Deloire, M., Hamel, D., Ouallet, J. C., Petry, K., & Brochet, B. (2013). Cognitive impair-
ment, health related quality of life and vocational status at early stages of multiple sclerosis:
A 7-year longitudinal study. Journal of Neurology, 260(3), 776–784. doi:10.1007/s00415-012-

6705-1
Ryan, R., & Hill, S. (2016). How to GRADE the quality of the evidence. Retrieved from http://cccrg.
cochrane.org/author-resources.
Sackett, P. R., Dahlke, J. A., Shewach, O. R., & Kuncel, N. R. (2017). Effect of predictor weighting
methods on incremental validity. Journal of Applied Psychology, 102(10), 1421–1434. doi:
10.1037/apl0000235
Schmand, B., Rienstra, A., Tamminga, H., Richard, E., van Gool, W. A., Caan, M. W. A., … Majoie,
C. B. (2014). Responsiveness of magnetic resonance imaging and neuropsychological assess-
ment in memory clinic patients. Journal of Alzheimer’s Disease, 40(2), 409–418. doi:10.3233/
JAD-131484
Scott, K. L., Strong, C. A. H., Gorter, B., & Donders, J. (2016). Predictors of post-concussion
rehabilitation outcomes at three-month follow-up. The Clinical Neuropsychologist, 30(1), 66–81.
doi:10.1080/13854046.2015.1127427
Sieg, E., Mai, Q., Mosti, C., & Brook, M. (2018). The utility of neuropsychological consultation in
identifying medical inpatients with suspected cognitive impairment at risk for greater hospital
utilization. The Clinical Neuropsychologist, 1. doi:10.1080/13854046.2018.1465124
Stiles-Shields, C., Corden, M. E., Kwasny, M. J., Schueller, S. M., & Mohr, D. C. (2015). Predictors of
outcome for telephone and face-to-face administered cognitive behavioral therapy for depres-
sion. Psychological Medicine, 45(15), 3205–3215. doi:10.1017/S0033291715001208
Stroup, E., Langfitt, J., Berg, M., McDermott, M., Pilcher, W., & Como, P. (2003). Predicting verbal
memory decline following anterior temporal lobectomy (ATL). Neurology, 60(8), 1266–1273.
doi:10.1212/01.WNL.0000058765.33878.0D
Struchen, M. A., Clark, A. N., Sander, A. M., Mills, M. R., Evans, G., & Kurtz, D. (2008). Relation of
executive functioning and social communication measures to functional outcomes following
traumatic brain injury. NeuroRehabilitation, 23, 185–198.
Temple, R., Carvalho, J., & Tremont, G. (2006). A national survey of physicians’ use of and satis-
faction with neuropsychological services. Archives of Clinical Neuropsychology, 21(5), 371–382.
doi:10.1016/j.acn.2006.05.002
Thomas, K. R., Eppig, J., Edmonds, E. C., Jacobs, D. M., Libon, D. J., Au, R., … Bondi, M. W.
(2018). Word-list intrusion errors predict progression to mild cognitive impairment.
Neuropsychology, 32(2), 235–245. doi:10.1037/neu0000413
van der Holst, H. M., van Uden, I. W. M., Tuladhar, A. M., de Laat, K. F., van Norden, A. G. W.,
Norris, D. G., … de Leeuw, F.-E. (2016). Factors associated with 8-year mortality in older
patients with cerebral small vessel disease. JAMA Neurology, 73(4), 402–409. doi:10.1001/
jamaneurol.2015.4560
Van Kampen, D. A., Lovell, M. R., Pardini, J. E., Collins, M. W., & Fu, F. H. (2006). The “value
added” of neurocognitive testing after sports-related concussion. The American Journal of
Sports Medicine, 34(10), 1630–1635. doi:10.1177/0363546506288677
Van Kirk, K. K., Horner, M. D., Turner, T. H., Dismuke, C. E., & Muzzy, W. (2013). Hospital service
utilization is reduced following neuropsychological evaluation in a sample of U.S. veterans.
The Clinical Neuropsychologist, 27, 750–761. doi:10.1080/13854046.2013.783122
Wagle, J., Farner, L., Flekkøy, K., Bruun Wyller, T., Sandvik, L., Fure, B., … Engedal, K. (2011).
Early post-stroke cognitive in stroke rehabilitation patients predicts functional outcomes at 13
months. Dementia and Geriatric Cognitive Disorders, 31(5), 379–387. doi:10.1159/000328970
Watt, S., & Crowe, S. F. (2018). Examining the beneficial effect of neuropsychological assessment
on adult patient outcomes: A systematic review. The Clinical Neuropsychologist, 32(3),
368–390. doi:10.1080/13854046.2017.1414885
Wearne, T., Anderson, V., Catroppa, C., Morgan, A., Ponsford, J., Tate, R., … McDonald, S. (2018).
Psychosocial functioning following moderate-to-sever pediatric traumatic brain injury:
Recommended outcome instruments for research and remediation studies. Neuropsychological
Rehabilitation, 1. doi:10.1080/09602011.2018.1531768
32 J. DONDERS
Wiberg, B., Kilander, L., Sundstro €m, J., Byberg, L., & Lind, L. (2012). The relationship between
executive dysfunction and post-stroke mortality: A population-based cohort study. BMJ Open,
2(3), e000458. doi:10.1136/bmjopen-2011-000458
Wiberg, B., Lind, L., Kilander, L., Zethelius, B., Sundelo €f, J. E., & Sundstro
€m, J. (2010). Cognitive
function and risk of stroke in elderly men. Neurology, 74(5), 379–385. doi:10.1212/
WNL.0b013e3181ccc516
Wilde, E. A., Whiteneck, G. G., Bogner, J., Bushnik, T., Cifu, D. X., Dikmen, S., … von
Steinbuechel, N. (2010). Recommendations for the use of common outcome measures in trau-
matic brain injury research. Archives of Physical Medicine and Rehabilitation, 91(11), 1650–1660.
doi:10.1016/j.apmr.2010.06.033
Williams, M. W., Rapport, L. J., Hanks, R. A., Millis, S. R., & Greene, H. A. (2013). Incremental value
of neuropsychological evaluations to computed tomography in predicting long-term out-
comes after traumatic brain injury. The Clinical Neuropsychologist, 27(3), 356–375. doi:10.1080/
13854046.2013.765507
Wyman-Chick, K. A., Martin, P. K., Barrett, M. J., Manning, C. A., & Sperling, S. A. (2017).
Diagnostic accuracy and confidence in the clinical detection of cognitive impairment in early-
stage Parkinson disease. Journal of Geriatric Psychiatry and Neurology, 30(3), 178–183. doi:
10.1177/0891988717701001

The Incremental Value of Neuropsychological Assessment: A Critical Review

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

The Incremental Value of Neuropsychological Assessment: A Critical Review

Uploaded by

Copyright:

Available Formats

The Clinical Neuropsychologist

ISSN: 1385-4046 (Print) 1744-4144 (Online) Journal homepage: https://www.tandfonline.com/loi/ntcn20

The incremental value of neuropsychological

To cite this article: Jacobus Donders (2019): The incremental value of

To link to this article: https://doi.org/10.1080/13854046.2019.1575471

Published online: 23 Apr 2019.

Submit your article to this journal

View Crossmark data

Full Terms & Conditions of access and use can be found at

The incremental value of neuropsychological assessment:

ABSTRACT ARTICLE HISTORY

CONTACT Jacobus Donders Jacobus.Donders@maryfreebed.com Department of Psychology, Mary Free Bed

Table 1. Inclusion criteria.

Figure 1. Flow diagram of literature search and study selection.

of neuropsychological assessment when considered in isolation in the context

Table 2. Papers reporting on the incremental value of neuropsychological assessment.

Traumatic brain injury

outcomes statistically significantly better than a model based on baseline functional

Disease Neuroimaging Initiative. Archives of General Psychiatry, 68(9), 961–969. doi:10.1001/

A 7-year longitudinal study. Journal of Neurology, 260(3), 776–784. doi:10.1007/s00415-012-

You might also like