INNOVATIONS

IN ASSESSMENT
(SORT OF …)
 Michael Harnadek, PhD,
CPsych
Neuropsychologist
Psychology, University Hospital
London Health Sciences Centre

Adjunct Clinical Professor

Dept of Clinical Neurological Sciences, Schulich
School of Medicine & Dentistry
Dept. of Psychology, Faculty of Social Sciences
Western University
 INNOVATIVE

ADJECTIVE

(of a product, idea, etc.) featuring new methods;

advanced and original
“There’s no good idea that cannot be
improved on” —MICHAEL EISNER
WHY THESE
IDEAS?
 MINIMIZING CONFUSION :
STANDARDIZED QUALITATIVE
DESCRIPTORS
Schoenberg’s Proposal
STANDARDIZED

THE PROBLEM …
DESCRIPTORS

• Qualitative descriptors are used to communicate test findings

• There is no consensus on what those descriptors are, or how to apply them
• Resulting ambiguity can lead to confusion
STANDARDIZED

AN ILLUSTRATION …
DESCRIPTORS

• Guilmette et al. surveyed AACN members

• Asked participants to classify 12 standard scores
130, 120, 115, 110, 95, 90, 85, 80, 75, 70, 60, 50
STANDARDIZED

AN ILLUSTRATION …
DESCRIPTORS

SS n

SS n
STANDARDIZED
DESCRIPTORS
WHAT ABOUT WECHSLER ?
STANDARDIZED

THE PROBLEM … PART 2

DESCRIPTORS

• Qualitative descriptors are used to communicate test findings

• There is no consensus on what those descriptors are, or how to apply them
• Resulting ambiguity can lead to confusion
• Schoenberg – lack of consensus in standard score/percentile that
correspond to a particular qualitative descriptor
• Descriptors that sound similar (low average, low normal, below average,
borderline ) do not overlap in the rarity of the score
STANDARDIZED SCHOENBERG – No concensus
DESCRIPTORS
STANDARDIZED SCHOENBERG – No concensus
DESCRIPTORS
 MY CONCERN - Psychometric vs. Functional Impairment

DSM-5 Mild vs Major NCD

STANDARDIZED
DESCRIPTORS
MILD - The cognitive deficits do not interfere with capacity
for independence in everyday activities (i.e., complex
instrumental activities of daily living such as paying bills or
managing medications are preserved, but greater effort,
compensatory strategies, or accommodation may be required).

MAJOR - The cognitive deficits interfere with independence

in everyday activities (i.e., at a minimum, requiring assistance
with complex instrumental activities of daily living such as
paying bills or managing medications)
 MY CONCERN - Psychometric vs. Functional Impairment

Mild Functional Impairment

STANDARDIZED
DESCRIPTORS
A “mild” functional impairment is one that may be evident to
the patient, and others; which interferes with their normal
daily activities, performance at work, and learning within an
educational environment, but which does not pose a complete
barrier to their functioning in those areas.
 MY CONCERN - Psychometric vs. Functional Impairment

Mild Functional Impairment

STANDARDIZED
DESCRIPTORS
A “mild” functional impairment is one that may be evident to
the patient, and others; which interferes with their normal
daily activities, performance at work, and learning within an
educational environment, but which does not pose a complete
barrier to their functioning in those areas.

DSB – mild
DSSeq – mild
LNS – moderate
CTT9 – low average
CTT18 – mild
CTT36 - mild
Arith – mild

“Overall a mild functional impairment of auditory

WM is seen”
STANDARDIZED
DESCRIPTORS
MY CONCERN - baserates
STANDARDIZED SCHOENBERG –
DESCRIPTORS

Borderline (6th – 15th %ile)

can be clinically meaningful for
some patients, but also seen in
baserates of non-neurological
cases. Interpretation based upon
individual case
STANDARDIZED SCHOENBERG –
DESCRIPTORS

“Impaired “ leaves room for

clinical interpretation

≤5th %ile is statistically rare

STANDARDIZED
DESCRIPTORS
SCHOENBERG –
… can reduce the likelihood of communication errors
by minimizing overlapping and difficult to
distinguish terms that have been variously used to
describe neuropsychological test scores
—MIKE R. SCHOENBERG
DISCUSSION

CREDITS: This presentation template was created by Slidesgo,

including icons by Flaticon, and infographics & images by
Freepik.
Please keep this slide for attribution.
 ABILITY
FOCUSED
BATTERY
Larrabee Approach
TRADITIONAL APPROACHES

FIXED BATTERY (e.g., HRNB, E-HRNB aka “Heaton” Battery)

• Constant compliment of tests (“all the tests – all the time”)

• Highly standardized administration and scoring
• Co-normed tests
• Long & inflexible to meet needs of specific diagnostic groups

FLEXIBLE BATTERY (e.g., Lezak approach, our typically approach)

• More patient focused
• More demands upon subjective clinical judgment of clinician
 BAUER (2000) – “Population-specific battery”
ALTERNATIVES ?

• Flexible battery
• Specific to a particular clinical population (e.g., Alzheimer’s disease;
epilepsy)
• Distinguish those with and those without (or temporal vs. non-temporal
epilepsy)
ABILITY — FOCUSED BATTERY

LARRABEE (2014)

FEATURES:
• Assesses core domains of ability
- using “relatively pure measures”
• Criterion validity
- sensitive to brain dysfunction
- sensitive to severity of impairment
- predictors of competence in iADLs
• PVTs for each area of ability
• Specific to individual clinical groups (MS, TBI, epilepsy, DAT)
ABILITY — FOCUSED BATTERY

CORE DOMAINS OF ABILITY

- Based upon FA studies – 6 domains (+ 1 extra)
1. Verbal symbolic abilities (Vocab; Fluency; Simil; Info)

2. Visualperceptual , visualspatial judgment & problem solving

(BD, VP, MR, JLO, Benton Faces)

3. Attention/working memory (Dspan, arith, LNS, Symbol Span)

4. Processing speed (Trails, Coding, Symbol Search)

5. Learning and memory – verbal and visual (LM, CVLT, VR, CVLT)

6. Sensorimotor function (FTT, grip, grooved pegs)

… (Executive functioning) (WCST, CT)

 SENSITIVE TO DYSFUNCTION
01
Test measures are chosen that have
demonstrated sensitivity to
ABILITY — FOCUSED

common forms of brain

dysfunction: TBI, stroke, DAT,
CRITERION
VALIDITY SENSITIVE TO SEVERITY
BATTERY

02
Measures chosen that can
differentiate degrees of severity of
injury (e.g., “Time to Follow
Command” in TBI)

PREDICTION OF iADLs
03
Measures chosen that have proven
helpful in predicting competence in
real world functioning (e.g., NAB
Driving Scene is correlated .55 with
on-road driving evaluation)
ABILITY — FOCUSED

PERFORMANCE VALIDITY
BATTERY

- Stand-alone & embedded PVTs

- Independent measures
- Specific to each ability domain
DISCUSSION
 Larrabee GJ. Test validity and performance validity: Considerations in providing
REFERENCES

a framework for development of an ability-focused neuropsychological test

battery. Arch Clin Neuropsy. 2014;29:695-714

Schoenberg MR, Rum RS. Towards reporting standards for neuropsychological

study results: A proposal to minimize communication errors with standardized
qualitative descriptors for normalized test scores. Clin Neurol Neurosurg.
2017;162:72-79

Guilmette TJ, Hagan LD, Giuliano AJ. Assigning qualitative descriptors to test
scores in neuropsychology: Forensic implications. Clin Neuropsychol.
2008;22(1):122-139.