NCLEX Computerized

Adaptive Testing
(CAT)
Shu-chuan Kao, PhD
Senior Manager, Measurement and
Testing, Examinations
Objectives
At the end of the session the participants will be able to:

• Discuss how Computerized Adaptive Testing (CAT) works

• Describe the variable length exam

What is “Adaptive” testing?
Adaptive exams offer customized testing. The difficulty of a test is
tailored to the person taking it.
• High-ability
o Candidates will receive more difficult items because
easy items provide less information about a high
performer’s ability

• Low-ability
o Candidates will receive fewer difficult items.
o Reduce guessing on items that are too difficult
What is “Adaptive” testing? (Cont.)
• The goal is to estimate your true ability
with great precision.
• If you answer an easy item correctly, the
CAT algorithms will select a more difficult
item for you to try.
• If you answer a difficult item incorrectly,
the CAT will select a less difficult item.
• Every time you answer an item, CAT re-
estimates your ability based on all
questions you have responded to.
• The more questions you answer, the more
precise your ability estimate becomes.
Updates for NCLEX® CAT
• Dichotomous and Polytomous scoring
oPartial knowledge is rewarded
oScoring rules vary depending on the item type
• Clinical judgment unfolding case study/scenarios and stand-
alone items
oEach case study scenario is selected to match your current
ability
oThe six items within a clinical judgment unfolding case
study are static, not adaptive
 Example NCLEX CAT
Item administered … Item score
Ability estimate 95% Confidence interval
Hard High
4
Case study scenario
3
RC, AC, PH, GS, TA, EO
2
Candidate ability
Item difficulty

1
Logit

0 Passing Standard

-1

-2

-3

-4
Easy Low 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Number of operational items
Important components of CAT

Variable Content
Length Balancin
Exam g

Exam
Stopping
Rules
Variable Length Exam
The number of questions depends upon a candidate's ability:
• Minimum: 85 items (70 operational items + 15 pretest items)
Shorter length exams – when a candidate's ability is “far”
from the passing standard

• Maximum: 150 items (135 operational items + 15 pretest items)

Longer length exams - when a candidate's ability is “close” to
the passing standard
Variable Length Exam
2023 NCLEX-RN and NCLEX-PN Test
Plans
https://www.nclex.com/test-plans.page
Content Balancing

Identify items in the

Determine the content
content area and
area that differs the
select an item with
most from the test
difficulty that matches
plan specifications
the ability estimate

Note: item type is not part of the selection algorithms

Stopping Rules and Pass/Fail Decisions
• Candidate performance is reported only as a pass-fail
classification.
• At least 70 operational items have been answered within the 5-
hour exam time.
• Three stopping rules are used to terminate an exam and make
the pass/fail decision.
• 95% Confidence Interval
• Maximum Length
• Run Out Of Time
Stopping Rule #1: 95% Confidence
Interval
• Respond to at least 70 operational items.
• The exam continues when the confidence interval straddles the
passing standard.
• Exam stops when the confidence interval is clearly above or
below the passing standard.
Ø At or ABOVE the passing standard à Pass
Ø Below the passing standard à Fail
Sample NCLEX Minimum Item Fail Chart
4
Begin Evaluation

3 95% Confidence Interval is below

the passing standard

2 Fail

1
Logit

-1

-2

-3

-4
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75
Number of operational items
Sample NCLEX Medium Length Pass Chart
4

3
95% Confidence Interval is above
the passing standard
2
Begin Evaluation
Pass
1
Logit

-1

-2

-3

-4
0 10 20 30 40 50 60 70 80 90 100 110 120 130

Number of operational items

Stopping Rule #2: Maximum Length
• Candidates with abilities very close to the passing standard may
receive maximum length examinations.
• When a candidate answers 135 operational items, computer
disregards the 95% certainty requirement.
• Outcome depends on the final ability.
• At or ABOVE the passing standardà Pass
• BELOW the passing standard à Fail
Sample NCLEX Maximum Length Pass
Chart
4

Begin Evaluation Maximum length exam

2
Evaluate the last ability estimate

1 Pass
Logit

-1

-2

-3

-4
0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150

Number of operational items

Stopping Rule #3: Run-Out-Of-Time
A candidate spends 5 hours on the exam, but the exam has not
been terminated by either of the previous two stopping rules.
• Responded to at least 70 OP items AND the final ability
estimate is above or at the passing standard à Pass
• Responded fewer than 70 OP items OR the final ability
estimate is below the passing standard à Fail
Sample NCLEX Run-Out-Of-Time Pass
Chart
4

3
Run out of time
2
Evaluate the last ability estimate
Begin Evaluation
Pass
1

-1

-2

-3

-4
0 10 20 30 40 50 60 70 80 90 100 110 120 130

Number of operational items

Sample NCLEX Run-Out-Of-Time Fail
Chart
4

3 Begin Evaluation
Run out of time
2
Fail

-1

-2

-3

-4
0 10 20 30 40 50 60 70 80 90 100 110 120 130

Number of operational items

Sample NCLEX Run-Out-Of-Time Fail
Chart
4

3
Run out of time
2
Evaluate the last ability estimate
Begin Evaluation
1 Fail

-1

-2

-3

-4
0 10 20 30 40 50 60 70 80 90 100 110 120 130

Number of operational items

Summary
• Computerized adaptive testing selects items for the person
taking the exam.
o Including clinical judgment standalone items and case study
Candidate
scenario itemsA who passed with 60 operational
items.
o Including items
Candidate thatpassed
B who can bewith
scored
130dichotomously
operational and
polytomously
items.
• PeopleCandidate C whoabilities
with different passed will
withreceive
100 operational
a different number of
items. items and the exam ran out of time.
• All exams comply with the test plan regardless of the test length
and candidate’s ability.
• Three stopping rules are used to make the pass/fail decision.
 Thank you!
More information can be located here or
please contact Customer Experience at
customerexpereience@ncsbn.org.