American Academy of Neurology

Clinical Skills Examination

Training Program
(date)
(speaker)
(association)

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Goals of this session
• Understand the American Board of Psychiatry and
Neurology (ABPN) performance requirements for the
five Clinical Skills Evaluations (NEX) neurology
residents must pass to achieve board certification.
• Appreciate the difference between a grade of
ACCEPTABLE and BORDERLINE BUT
UNACCEPTABLE.
• Improve the inter-rater reliability of faculty evaluators
for acceptable versus unacceptable distinctions.

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 What is your role within your department?

1. Department chair
2. Residency program
director
3. Adult neurology faculty
4. Child neurology faculty
5. Resident
6. Fellow
7. Other 0% 0% 0% 0% 0% 0% 0%

1 2 3 4 5 6 7

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 For how many years have you been
practicing as a neurologist?
(Only faculty with a clinical appointment should answer.)

1. 0-4
2. 5-9
3. 10-14
4. 15-19
5. 20-24
0% 0% 0% 0% 0% 0%
6. 25 or more 1 2 3 4 5 6

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Have you ever served as an ABPN
oral board examiner?
(Only faculty with a clinical appointment should answer.)

1. Yes
0% 0%
2. No
1 2

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 For how many NEX examinations have
you served as a faculty evaluator?
(Only faculty with a clinical appointment should answer.)

1. 0
2. 1-4
3. 5-9
4. 10-14
5. 15-19
0% 0% 0% 0% 0% 0%

6. 20 or more 1 2 3 4 5 6

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 How confident are you that you understand the ABPN
criteria for determining whether a resident's performance on
an NEX merits a passing grade?
1. 1 – Not at all confident
2. 2
3. 3
4. 4
5. 5
6. 6
7. 7
8. 8
9. 9
0% 0% 0% 0% 0% 0% 0% 0% 0% 0%

10. 10 - Extremely confident 1 2 3 4 5 6 7 8 9 10

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Neurology Clinical Evaluation Exercise
(NEX)
• Every resident must pass an examination in
each of 5 categories during residency to sit
for the neurology written board exam.
 Ambulatory
 Critical care
 Neuromuscular
 Neurodegenerative
 Child neurology (for adult neuro
residents) or adult neurology (for child
neuro residents)

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Why train faculty evaluators?
•ABPN oral board examiners
underwent training
•Poor inter-rater reliability between
former ABPN oral board examiners
and local faculty. (Schuh, et al 2009)

© 2011 AMERICAN ACADEMY OF NEUROLOGY

© 2011 AMERICAN ACADEMY OF NEUROLOGY
 What constitutes a pass or fail?
5 (marginal pass)
• Deficiencies or errors in history or exam, but
enough information is obtained to formulate
the case.

4 (marginal fail)
• Deficiencies or errors in history or exam, which
result in the resident not being able to
successfully formulate the case.

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Clinical Skills Examination Training
(CSET) Program

Designed by a workgroup through the

Consortium of Neurology Program Directors.
Objectives:
Create a simple NEX training program that
can be utilized by faculty at neurology
training programs
Focus on pass-fail distinction

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Clinical Skills Examination Training Program
- Instructions
• We will review a series of 9 brief vignettes. For each,
there will be an example of an omission on the part of
the resident.
• Assume that the resident has performed a basic history
and physical examination that is otherwise adequate
• Select PASS or FAIL based on whether the omission is
egregious enough to warrant an overall failing grade.
• Review audience votes, as well as the voting results and
talking points from previous participants.
• Open discussion (Up to 1 minute for each side.)
• Re-vote

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Objectives
• Distinguish between those mistakes and
omissions that interfere with the formulation
of the case.
• Training through reflection and discussion
• Data gathering

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Criteria for assigning a failing grade
• Deficiencies or errors in history or exam
which result in the resident not being able
to successfully formulate the case.

© 2011 AMERICAN ACADEMY OF NEUROLOGY

© 2011 AMERICAN ACADEMY OF NEUROLOGY
 Critical Care Scenario: (PRE)
A 55 year old woman with known myasthenia gravis is
admitted to the intensive care unit with a chief
complaint of shortness of breath and generalized
weakness.

Omission: 0% 0%
The resident does not check neck flexion strength.
1 2

1. Pass
2. Fail

Historical Pass Rate: 76.2% (n=101)

Talking points:
Pass: Helpful, but not crucial. Respiratory parameters are crucial.
Fail: Might help me determine whether or not to intubate.

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Critical Care Scenario: (POST)
A 55 year old woman with known myasthenia gravis is
admitted to the intensive care unit with a chief
complaint of shortness of breath and generalized
weakness.

Omission: 0% 0%
The resident does not check neck flexion strength.
1 2

1. Pass
2. Fail

Historical Pass Rate: 76.2% (n=101)

Talking points:
Pass: Helpful, but not crucial. Respiratory parameters are crucial.
Fail: Might help me determine whether or not to intubate.

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Pediatrics Scenario: (PRE)
A 6 year old boy with normal birth and
development develops progressive difficulty
walking and climbing stairs. His calves are
large.

Omission: 0% 0%
The resident does not ask about family history.
1 2

1. Pass
2. Fail

Historical Pass Rate: 7.7% (n=104)

Talking points:
Pass: Won’t affect workup.
Fail: Family history can direct DNA testing

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Pediatrics Scenario: (POST)
A 6 year old boy with normal birth and development
develops progressive difficulty walking and climbing
stairs. His calves are large.

Omission:
The resident does not ask about family history. 0% 0%

1 2
1. Pass
2. Fail

Historical Pass Rate: 7.7% (n=104)

Talking points:
Pass: Won’t affect workup.
Fail: Family history can direct DNA testing

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Neuromuscular Scenario: (PRE)
A 37 year old woman, previously healthy, presents to
your office complaining of progressive muscle
weakness and intermittent double vision.

Omission:
The resident does not stabilize the elbow joint when 0% 0%
checking triceps strength.
1 2
1. Pass
2. Fail

Historical Pass Rate: 96.2% (n=104)

Talking points:
Pass: This is a good teaching point, but doesn’t affect formulation.
Fail: Poor exam suggests resident would not be able to properly assess in
the future.

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Neuromuscular Scenario: (POST)
A 37 year old woman, previously healthy, presents to
your office complaining of progressive muscle
weakness and intermittent double vision.

Omission:
The resident does not stabilize the elbow joint when
0% 0%
checking triceps strength.
1 2
1. Pass
2. Fail

Historical Pass Rate: 96.2% (n=104)

Talking points:
Pass: This is a good teaching point, but doesn’t affect formulation.
Fail: Poor exam suggests resident would not be able to properly assess in
the future.

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Neurodegenerative Scenario: (PRE)
A 75 year old woman is brought to your office by her
daughter for evaluation of mild memory loss.

Omission:
The resident does not ask about educational
background.
0% 0%

1. Pass 1 2

2. Fail

Historical Pass Rate: 75.7% (n=103)

Talking points:
Pass: If cognitive function is declining, level of education doesn’t matter.
Fail: This is important to give the physician a frame of reference.

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Neurodegenerative Scenario: (POST)
A 75 year old woman is brought to your office by her
daughter for evaluation of mild memory loss.

Omission:
The resident does not ask about educational
background.
0% 0%
1. Pass 1 2

2. Fail

Historical Pass Rate: 75.7% (n=103)

Talking points:
Pass: If cognitive function is declining, level of education doesn’t matter.
Fail: This is important to give the physician a frame of reference.

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Ambulatory Scenario: (PRE)
A 26 year old pregnant woman presents with a 3 week
history of bilateral hand paresthesias.

Omission:
The resident does not check lower extremity reflexes.
0% 0%
1. Pass
2. Fail 1 2

Historical Pass Rate: 15.8% (n=101)

Talking points:
Pass: This is probably carpal tunnel syndrome.
Fail: Although unlikely, a cervical myelopathy needs to be excluded.

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Ambulatory Scenario: (POST)
A 26 year old pregnant woman presents with a 3 week
history of bilateral hand paresthesias.

Omission:
The resident does not check lower extremity reflexes.
0% 0%
1. Pass
2. Fail 1 2

Historical Pass Rate: 15.8% (n=101)

Talking points:
Pass: This is probably carpal tunnel syndrome.
Fail: Although unlikely, a cervical myelopathy needs to be excluded.

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Critical Care Scenario: (PRE)
A 75 year old woman presents to the emergency
department with sudden onset of right sided weakness
and slurred speech.

Omission:
The resident does not check for a Babinski sign.

1. Pass 0% 0%
2. Fail
1 2

Historical Pass Rate: 85.9% (n=78)

Talking points:
Pass: Not part of the NIH stroke scale.
Fail: This is necessary to localize the lesion.

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Critical Care Scenario: (POST)
A 75 year old woman presents to the emergency
department with sudden onset of right sided weakness
and slurred speech.

Omission:
The resident does not check for a Babinski sign.

1. Pass 0% 0%
2. Fail
1 2

Historical Pass Rate: 85.9% (n=78)

Talking points:
Pass: Not part of the NIH stroke scale.
Fail: This is necessary to localize the lesion.

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Neuromuscular Scenario: (PRE)
Hospital consultation to see a 21 year old man with a
history of progressive ascending weakness.

Omission:
The resident does not ask about recent vaccinations.

1. Pass
2. Fail 0% 0%

1 2

Historical Pass Rate: 87.2% (n=78)

Talking points:
Pass: Wouldn’t affect diagnosis or management.
Fail: A recent vaccine could be an immunologic trigger for this.

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Neuromuscular Scenario: (POST)
Hospital consultation to see a 21 year old man with a
history of progressive ascending weakness.

Omission:
The resident does not ask about recent vaccinations.

1. Pass
2. Fail 0% 0%

1 2

Historical Pass Rate: 87.2% (n=78)

Talking points:
Pass: Wouldn’t affect diagnosis or management.
Fail: A recent vaccine could be an immunologic trigger for this.

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Pediatrics Scenario: (PRE)
A 12 year old boy presents with repetitive movements
of one hand.

Omission:
The resident does not take a medication history.

1. Pass
2. Fail 0% 0%

1 2

Historical Pass Rate: 8.1% (n=74)

Talking points:
Pass: Not likely to be relevant in a child.
Fail: Medication-induced movement disorder is on the differential.

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Pediatrics Scenario: (POST)
A 12 year old boy presents with repetitive movements
of one hand.

Omission:
The resident does not take a medication history.

1. Pass
2. Fail 0% 0%

1 2

Historical Pass Rate: 8.1% (n=74)

Talking points:
Pass: Not likely to be relevant in a child.
Fail: Medication-induced movement disorder is on the differential.

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Ambulatory Scenario: (PRE)
A patient presents with chronic, episodic vertigo.

Omission:
The resident does not ask about hearing loss.
1. Pass
2. Fail

0% 0%

1 2

Historical Pass Rate: 18.2% (n=77)

Talking points:
Pass: Wouldn’t affect work up or treatment.
Fail: Vestibular tumor or Ménière’s disease are considerations

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Ambulatory Scenario: (POST)
A patient presents with chronic, episodic vertigo.

Omission:
The resident does not ask about hearing loss.
1. Pass
2. Fail
0% 0%

1 2

Historical Pass Rate: 18.2% (n=77)

Talking points:
Pass: Wouldn’t affect work up or treatment.
Fail: Vestibular tumor or Ménière’s disease are considerations

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Session complete

© 2011 AMERICAN ACADEMY OF NEUROLOGY

How confident are you that you understand the ABPN
criteria for determining whether a resident's
performance on an NEX merits a passing grade?
1. 1 – not at all confident
2. 2
3. 3
4. 4
5. 5
6. 6
7. 7
8. 8
9. 9
10. 10 – Extremely confident 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%

1 2 3 4 5 6 7 8 9 10

© 2011 AMERICAN ACADEMY OF NEUROLOGY

 Developed by the AAN Clinical Skills
Examination Training Workgroup
Zach London, MD
Joe Kass, MD
Vicki Shanker, MD
Sonja Potrebic, MD, PhD
Yousef Mohammed, MD
Gauri Pawar, MD
Rachel Ditrapani, MD
Ryan Walsh, MD

© 2011 AMERICAN ACADEMY OF NEUROLOGY