Archives of Physical Medicine and Rehabilitation

journal homepage: www.archives-pmr.org

Archives of Physical Medicine and Rehabilitation 2019;100:899-907

ORIGINAL RESEARCH

Development of a Computerized Adaptive Testing

System for Assessing 5 Functions in Patients with
Stroke: A Simulation and Validation Study
Gong-Hong Lin, PhD,a Yi-Jing Huang, PhD,a Ya-Chen Lee, PhD,b Shih-Chieh Lee, BS,a
Chia-Yeh Chou, MS,a,c Ching-Lin Hsieh, PhDa,d,e
From the aSchool of Occupational Therapy, College of Medicine, National Taiwan University, Taipei; bDepartment of Occupational Therapy,
College of Medical and Health Science, Asia University, Taichung; cDepartment of Occupational Therapy, College of Medicine, Fu-Jen Catholic
University, New Taipei; dDepartment of Physical Medicine and Rehabilitation, National Taiwan University Hospital, Taipei; and eDepartment of
Occupational Therapy, College of Medical and Health Science, Asia University, Taichung, Taiwan.

Abstract
Objective: The authors aimed to develop and validate the Computerized Adaptive Testing System for Assessing 5 Functions in Patients with
Stroke (CAT-5F) based on the Barthel Index (BI), Postural Assessment Scale for Stroke patients (PASS), and Stroke Rehabilitation Assessment of
Movement (STREAM) to improve the efficiency of assessment. The purposes of the CAT-5F assessment are to describe patients’ levels of
impairments or disabilities in the 5 functions and to serve as an outcome measure in patients with stroke.
Design: This is a data-mining study based on data from a previous study using simulation analysis to develop and validate the CAT-5F.
Setting: One rehabilitation unit in a medical center in Taiwan served as the setting for this study.
Participants: Data were retrieved from totals of 540 (initial assessment) and 309 (discharge assessment) participants with stroke assessed in a
previous study. The assessment data (NZ540) were from the BI, PASS, and STREAM.
Interventions: Not applicable.
Main Outcome Measures: The outcome measures for this study were from BI, PASS, and STREAM.
Results: The CAT-5F using the optimal stopping rule (limited reliability increased <0.010) had good Rasch reliability across the 5 functions
(0.86-0.96) and needed 12.7 items, on average, for the whole administration. The concurrent validity (Pearson product-moment correlation
coefficient, rZ0.91-0.96) and responsiveness (standardized response meanZ0.33-0.91) of the CAT-5F were sufficient in the patients.
Conclusion: The CAT-5F has sufficient administrative efficiency, reliability, concurrent validity, and responsiveness to simultaneously assess
basic activities of daily living, postural control, upper extremity/lower extremity motor functions, and mobility in patients with stroke.
Archives of Physical Medicine and Rehabilitation 2019;100:899-907
ª 2018 by the American Congress of Rehabilitation Medicine

The Barthel Index (BI), Postural Assessment Scale for Stroke pa-
tients (PASS), and Stroke Rehabilitation Assessment of Movement
Measure (STREAM) are public-domain, commonly used outcome
measures in clinical trials for assessing patients with stroke.1-3 The
BI and PASS are used to assess basic activities of daily living
(BADL) and postural control, respectively.4,5 The STREAM
Supported by the Ministry of Science and Technology, Taiwan (grant no. MOST 105-2314-B-
002-015-MY3, 106-2314-B-468-005, and MOST 107-2314-B-468-004-MY2).
Disclosures: None.
contains 3 tests for assessing upper extremity (UE) motor function
(STREAM-UE), lower extremity (LE) motor function (STREAM-
LE), and mobility (STREAM-mobility).6 According to the guide-
lines of stroke management proposed by the American Heart As-
sociation/American Stroke Association,7,8 the 5 functions (BADL,
postural control, UE/LE motor functions, and mobility) are the
primary rehabilitation targets in patients with acute and subacute
stroke. The 5 tests (BI, PASS, STREAM-UE, STREAM-LE,
STREAM-mobility) have good psychometric properties (eg, reli-
ability, validity, and responsiveness) in patients with acute and
subacute stroke.9-11 However, the inefficiency of administering

0003-9993/18/$36 - see front matter ª 2018 by the American Congress of Rehabilitation Medicine
https://doi.org/10.1016/j.apmr.2018.09.122
900 G.-H. Lin et al

these 5 tests is a concern if a clinician assesses all 5 tests in a time-

Table 1 Definition of terms related to CAT
limited therapeutic session. The total of 52 items of the 5 tests
imposes time and physical burdens on both patients and clinicians. Term Definition
Therefore, an efficient testing method or short forms of the 5 tests CAT A computer-based testing method in
are warranted. which the items for administration
To achieve efficient and precise assessment, computerized are adaptively selected according to
adaptive testing (CAT) is suggested.12-14 CAT is a computer-based an examinee’s level of function.12 For
testing method in which the items selected for administration are example, only difficult items will be
matched to an examinee’s level of function (Table 1 shows the selected for administration if an
definition of terms related to CAT). Item response theory is the examinee has a high level of function.
statistical basis of CAT for examining data model fitting and Item response theory Item response theory is a group of
separately estimating item difficulties and examinee abilities,15 mathematical models (eg, Rasch
which are used to select the items for administering CAT. model) for estimating an examinee’s
Because CAT does not administer items that are too difficult or too response on one or a set of items (eg,
easy for an examinee, CAT can use only a few items (ie, high pass or fail an item) by considering
efficiency) to precisely estimate an examinee’s level of function. both the examinee’s ability and item
For example, the CAT version of the Fugl-Meyer Motor Scale uses parameters (eg, item difficulty).15
only 20% of the items of the Fugl-Meyer Motor Scale while still Item response theory is the statistical
retaining good Rasch reliability (an index of precision).14 Addi- basis of CAT for examining data model
tionally, CAT can be administered online with a website browser fitting, selecting the items to be
on various devices (eg, a smartphone, tablet, personal computer, or administered, and estimating
laptop), and does not require the installation of additional com- examinees’ abilities/scores.
puter programs.16 Thus, CAT can be used to improve the Item bank A set of items in which all items fit a
administrative efficiency and reduce the assessment burden on model of item response theory (eg,
patients and clinicians. Rasch model).15 From the item bank,
The authors aimed to develop and validate a Computerized CAT selects items for administration.
Adaptive Testing System for Assessing 5 Functions in Patients Rasch reliability An index of reliability based on a Rasch
with Stroke (CAT-5F). The purposes of the CAT-5F assessment are model. Rasch reliability ranges from 0
to describe patients’ impairments or disabilities in the 5 functions to 1, and higher Rasch reliability
and to serve as an outcome measure in patients with stroke. Two indicates that the scores of a measure
specific research questions of this study were: (1) Can the CAT-5F are more precise (less standard
improve the efficiency of simultaneously assessing 5 functions error).24 Rasch reliability can be
(BADL, postural control, UE motor function, LE motor function, calculated for scores of an individual
and mobility) in patients with stroke? (2) What are the Rasch (eg, individual-level Rasch reliability)
reliability, concurrent validity and responsiveness of the CAT-5F or a group of examinees (eg, group-
in patients with stroke? level Rasch reliability).
Stopping rules Algorithms to determine when
administration of CAT is terminated
Methods (eg, when “Rasch reliability is high
enough” or “amount of Rasch
reliability increase is limited”).15
Participants
This was a data-mining study. All data used in this study were
retrieved from a previous study,17 which was conducted in a
medical center in Taiwan from January 2009 to January 2012. All
data were collected by an occupational therapist (assessing the BI) and a research nurse (assessing the PASS and STREAM), who
were trained and familiar with the tests in the study. The inclusion
criteria of the study from which the authors retrieved the data were
List of abbreviations:
as follows: (1) stroke onset within 10 days before hospital
ADL activities of daily living
admission; (2) diagnosis of stroke (cerebral hemorrhage or
BADL basic activities of daily living
infarction); and (3) ability to follow instructions. Patients were
BI Barthel Index
CAT computerized adaptive testing excluded if they met the following criteria: (1) unwillingness to
CAT-5F Computerized Adaptive Testing System for participate; (2) another stroke or other major disease during hos-
Assessing 5 Functions in Patients with Stroke pitalization, which could affect patients’ motor function, postural
FIM-M Functional Independence Measure-motor control, or BADL; (3) dysfunction of vision, hearing, or
scale communication, which could cause patients difficulty in following
LE lower-extremity the instructions of the assessments; and (4) a stay in the rehabil-
PASS Postural Assessment Scale for Stroke patients itation ward of less than 7 days. All the participants were assessed
SRM standardized response mean twice (at both admission to and discharge from a medical center)
STREAM Stroke Rehabilitation Assessment of
using the BI, PASS, and STREAM. The study had Institutional
Movement
Review Board approval, and all participants signed the informed
UE upper-extremity
consent forms before starting the study.

www.archives-pmr.org
Development and validation of the CAT-5F
Two sets of data (including the STREAM, PASS, and BI) were
retrieved from the previous data set and used for developing and
validating the CAT-5F, respectively. For developing the CAT-5F,
the authors randomly retrieved each participant’s data of either
admission or discharge to generate a set of data. This retrieval
method yielded a diversity of scores in the data set because the
data set contained lower scores from the data of admission and
higher scores from the data of discharge. These diverse scores
could contribute to precisely estimating the item properties of the
CAT-5F.12,13,18 In addition, the exclusion criterion of data retrieval
in this study was participant data with a missing rate of >50% in
one or more measures (the BI, PASS, and STREAM).
For validating the concurrent validity and responsiveness of the
CAT-5F, the authors retrieved the complete data of the BI, PASS,
and STREAM at admission and discharge.
Procedure
The authors developed the CAT-5F in 4 steps: (1) establishment of
the item bank of the CAT-5F; (2) simulation of the reliability and
administrative efficiency of the CAT-5F with 10 sets of stopping
rules; (3) selection of the final sets of stopping rules of the CAT-
5F; and (4) programming of the administrative system of the
CAT-5F.
Step 1
The item bank of the CAT-5F contained 5 domains/functions:
BADL, postural control, UE/LE motor functions, and mobility. The
authors used 5 sets of items with validated unidimensionality to
compose the 5-domain item bank of the CAT-5F.18-20 For the first 2
domains (BADL and postural control), the authors adopted the items
of the BI (9 items) and PASS (12 items).19,20 For the other 3 domains
(UE/LE motor functions and mobility), the authors adopted the items
of UE movement, LE movement, and mobility subsets (8, 9, and 10
items, respectively) of the 27-Item STREAM.18 All 48 items were
included in the 5-domain item bank of the CAT-5F. Next, the authors
estimated the item parameters (eg, item difficulty and step difficulty)
of the item bank, which the CAT-5F used to select items with tailored
difficulties for examinees. In addition, the authors examined the
unidimensionality of each domain of the item bank of the CAT-5F
using principal component analysis on standardized residuals.21 If
an eigenvalue of the first factor in principal component analysis was
<3.0, the unidimensionality of each domain was supported.21
Step 2
The authors conducted 10 simulation analyses to simulate the
reliability and administrative efficiency (number of items used) of
the CAT-5F with each of the 10 different sets of stopping rules.
The reliability of the CAT-5F included individual-level reliability
and group-level reliability. The individual-level reliability was the
Rasch reliability of the CAT-5F score for each participant, and the
group-level reliability was the average of the individual-level
reliability of the participants.
The 10 sets of stopping rules can be categorized into 2 groups:
(1) considering only limited individual-level reliability increase and
(2) considering both limited reliability increase and threshold of
individual-level reliability.14,22 Within each group of stopping rules,
5 different levels of limited reliability increase (<0.001, <0.005,
<0.010, <0.015, and <0.020) were proposed to explore the reli-
ability and administrative efficiency of the CAT-5F. For the group
considering only limited reliability increase, the administration of
the CAT-5F was terminated if assessing more items increased
www.archives-pmr.org
901
individual-level reliability by only a limited amount (eg, <0.001,
<0.005, <0.010, <0.015, or <0.020) in all domains. For the group
considering both limited reliability increase and threshold of reli-
ability, the CAT-5F administration was ended if the individual-level
reliability of the CAT-5F achieved the threshold (eg, 0.90) or limited
reliability increase (eg, <0.001, <0.005, <0.010, <0.015, or
<0.020) in all domains.14,22
Step 3
The sets with sufficient administrative efficiency and reliability
(eg, average number of items <15 and the group-level reliability
of each domain
0.85) were selected as the final sets of stopping
rules of the CAT-5F.
Measures
The BI5 is a BADL test. The BI contains 10 items, and its total
score ranges from 0 to 100 points. A higher total score represents
better BADL function. The reliability (eg, interrater reliability:
intraclass correlation coefficientZ0.94),10 validity (eg, concurrent
validity: correlation coefficientZ0.92-0.94),23 and responsiveness
(eg, standardized response meanZ1.20)23 of the BI are sufficient
in patients with stroke.10,23
The PASS4 is a stroke-specific measure of postural control. The
PASS is comprised of 12 items, and its total score ranges from 0 to
36 points. A higher total score indicates better postural control. The
PASS has good reliability (eg, interrater reliability: intraclass cor-
relation coefficientZ0.97),11 validity (eg, concurrent validity:
correlation efficientZ0.92-0.97),11 and responsiveness (eg, stan-
dardized response meanZ1.12)11 in patients with stroke.4,11
The STREAM6 contains 3 tests to assess UE motor function, LE
motor function, and mobility, respectively. Each test has 10 items, and
the ranges of the total scores of the 3 tests are 0-20, 0-20, and 0-30,
respectively. Good reliability (eg, test-retest reliability: intraclass
correlation coefficientZ0.98),9 validity (concurrent validity: corre-
lation efficientZ0.91-0.99),9 and responsiveness of the STREAM
(eg, standardized response meanZ0.95)9 have been found in patients
with stroke.9,24
Data analysis
Development of the CAT-5F
To estimate the item parameters of the item bank of the CAT-5F, the
authors applied the 5-dimensional polytomous Rasch partial credit
model analysis, which was conducted in ConQuest software.25,a In
the step of simulation of the CAT-5F, we adopted the Fisher infor-
mation function to select items for tailored testing and the maximum
a posteriori estimation with the Newton-Raphson iterative proced-
ure to estimate the participants’ scores.26 In addition, the first 5
items of the CAT-5F were selected from the 5 domains of the CAT-
5F, respectively, and the participants had to complete at least the 5
items before the administration of the CAT-5F was ended (ie, the
lowest number of items needed for administration was 5 items). This
setup ensured that each participant completed at least one item of
each domain of the CAT-5F.
Validation of the CAT-5F
To examine whether reducing the number of items for adminis-
tering the CAT-5F sacrificed precision of assessment, the authors
compared the group-level reliability of the CAT-5F and that of the
item bank.27
902 G.-H. Lin et al

Table 2 Characteristics of the participants for the CAT-5F development and validation, respectively
Development of the Validation of the
Characteristics CAT-5F (nZ540) CAT-5F (nZ309)
Mean age  SD (y) 65.513.7 65.213.8
Sex, men/women, n (%) 326 (60)/214 (40) 191 (62)/118 (38)
Stroke type, hemorrhage/infarction, n (%) 161 (30)/356 (66)* 87 (28)/212 (69)*
Side of hemiplegia, right/left/bilateral, n (%) 141 (26)/112 (21)/31 (6)* 76 (25)/63 (20)/16 (5)*
Days between onset and initial evaluation, median (min-max) 20 (4-567) 19 (4-95)
Days of rehabilitation ward stay, median (min-max) 32 (3-112) 32 (3-112)
Admission, mean  SD
BI 30.722.5 33.922.9
PASS 17.211.8 19.211.3
STREAM-UE 8.97.8 9.77.9
STREAM-LE 8.37.1 9.47.2
STREAM-mobility 11.58.9 13.08.8
Discharge, mean  SD
BI 63.627.5 68.624.4
PASS 19.214.2 26.99.2
STREAM-UE 9.08.3 12.77.2
STREAM-LE 9.07.9 12.76.6
STREAM-mobility 14.311.4 20.48.6
* The total number of the participants does not match the corresponding sample size because of missing data.

To validate the concurrent validity of the CAT-5F with the final
sets of stopping rules, the authors analyzed the Pearson product-
moment correlation coefficient (r) to examine the extent of as-
sociation between the scores of the CAT-5F and those of the
corresponding original tests (eg, BADL domain scores of the CAT-
5F vs BI scores) at both admission and discharge. Pearson r
0.75
indicated good concurrent validity.28
Responsiveness is the extent to which longitudinal differences
are obtained on repeated administrations of the same measure
when a real change in specific function has occurred.29 The
responsiveness of the original tests and the CAT-5F with the final
sets of stopping rules was investigated using the standardized
response mean (SRM) and paired t test. The P value of the paired
t test <0.05 represented that the CAT-5F could detect the par-
ticipants’ functional improvement during the period of hospitali-
zation in the rehabilitation ward. SRM shows the extent of
difference between the scores at admission and those at discharge.
The evaluation criteria of the SRM for the 5 functions were
different because different amounts of improvement were ex-
pected in the 5 functions during the early period after stroke
onset.30-34 For the UE and LE scores of the STREAM and CAT-
5F, an SRM value
0.20 indicated sufficient responsive-
ness.30-32,34 For the BADL, postural control, and mobility scores
of the original tests and CAT-5F, an SRM value
0.50 indicated
sufficient responsiveness.30,33,34
Results
The original study contained the data of a total of 590 partici-
pants.17 Of these participants, 540 participants completed the as-
sessments at admission, and 309 participants completed the
assessments at both admission and discharge. Thus, the data of the
540 and 309 participants were retrieved from the original study to
develop and validate the CAT-5F, respectively. The average ages
(in years) of the 540 and 309 participants were 65.9 and 65.2,
respectively. On average, the patients had severe BADL disability
at admission and moderate BADL disability at discharge accord-
ing to their scores on the BI. Table 2 shows further characteristics
of the participants.
The item bank of the CAT-5F contained 48 items, and the item
difficulties of the items were varied (-4.7 to 5.2, appendix 1). In each
domain of the item bank of the CAT-5F, the results of principal
component analysis showed that the eigenvalues of the first factor
were <3.0 (the eigenvalues of the 5 domains were 1.9-2.3), which
support the unidimensionality of each domain. The group-level
reliability of the item bank was high (0.92, 0.98, 0.89, 0.94, and
0.98 for the BADL, postural control, UE motor, LE motor, and
mobility domains, respectively; appendix 2).
Among the 10 candidate sets of stopping rules, only 3 sets of
stopping rules had sufficient administrative efficiency and reli-
ability to be selected as the final sets of stopping rules of the CAT-
5F (see appendix 2). The 3 final sets of stopping rules were
limited reliability increase <0.010 (final set A), limited reliability
increase <0.015 (final set B), and limited reliability increase
<0.001 or individual-level reliability
0.90 (final set C). In
general, among the 3 final sets of stopping rules, final set A had
the highest reliability (group-level reliability of the BADL,
postural control, UE motor, LE motor, and mobility domains
Z0.88, 0.96, 0.86, 0.91, and 0.96, respectively) but low admin-
istrative efficiency (average number of itemsZ12.7). In contrast,
final set B had the highest administrative efficiency (average
number of itemsZ11.0) but the lowest reliability (group-level
reliabilities of the 5 domains of the CAT-5FZ0.87, 0.95, 0.85,
0.91, and 0.96, respectively).
The examination of concurrent validity displayed very high
Pearson r (0.91-0.96, appendix 3) between the CAT-5F with the
final sets of stopping rules and the corresponding original tests.
For the responsiveness of the CAT-5F with the 3 final sets of
stopping rules, all the paired t tests showed significant differences
between the scores at admission and those at discharge (P<0.001).
Comparing SRMs among the 3 final sets of stopping rules of the

www.archives-pmr.org
Development and validation of the CAT-5F 903

Table 3 Responsiveness of the CAT-5F and the original tests (nZ309)

Change Between
CAT-5F and Original Tests Admission Discharge Admission and Discharge P SRM
CAT-5F with final set A of stopping rules
BADL -1.71.9 1.02.1 2.73.0 <.001 0.91
Postural control 0.33.0 2.82.8 2.64.2 <.001 0.61
UE motor function -0.44.7 1.84.4 2.36.7 <.001 0.34
LE motor function -0.54.2 1.64.0 2.16.0 <.001 0.36
Mobility -0.73.1 2.13.0 2.74.4 <.001 0.61
CAT-5F with final set B of stopping rules
BADL -1.71.9 0.92.1 2.72.9 <.001 0.91
Postural control 0.22.9 2.82.8 2.64.2 <.001 0.61
UE motor function -0.44.7 1.84.4 2.26.6 <.001 0.34
LE motor function -0.54.1 1.74.0 2.16.0 <.001 0.35
Mobility -0.73.0 2.12.9 2.74.4 <.001 0.62
CAT-5F with final set C of stopping rules
BADL -1.71.9 0.92.1 2.72.9 <.001 0.91
Postural control 0.22.9 2.82.8 2.64.2 <.001 0.62
UE motor function -0.54.7 1.74.4 2.26.7 <.001 0.33
LE motor function -0.64.2 1.64.0 2.16.0 <.001 0.36
Mobility -0.73.0 2.02.9 2.74.4 <.001 0.63
Original tests
BI 5.64.0 12.04.6 6.46.5 <.001 0.98
PASS 19.211.3 26.99.2 7.815.3 <.001 0.51
STREAM-UE 7.96.4 10.25.8 2.49.0 <.001 0.26
STREAM-LE 8.76.5 11.46.1 2.79.4 <.001 0.29
STREAM-mobility 13.08.8 20.48.6 7.412.9 <.001 0.57
NOTE. Values are mean  SD or as otherwise indicated.

CAT-5F, the SRMs of the domains were similar (BADL: 0.91,
postural control: 0.61-0.62, UE motor function: 0.33-0.34, LE
motor function: 0.35-0.36, and mobility: 0.61-0.63; Table 3).
Discussion
The authors developed the CAT-5F to efficiently and simulta-
neously assess BADL, postural control, UE motor function, LE
motor function, and mobility in patients with stroke. Our results
showed that the CAT-5F had high administrative efficiency (average
number of administration itemsZ11.0-12.7). The high adminis-
trative efficiency of the CAT-5F can be attributed to the feature of
CAT, which selects items tailored to an examinee’s level of function.
These results imply that the administration time could be reduced by
about 75% (52 items of the original tests vs 11.0-12.7 items of the
CAT-5F) if researchers and clinicians use the CAT-5F to replace the
BI, PASS, and STREAM. Assessment burdens on both patients and
examiners would be largely decreased.
The scores of the CAT-5F had sufficient reliability (group-
level reliabilityZ0.85-0.96), which was similar to the reliability
of the item bank (0.89-0.98). The sufficient reliability of the CAT-
5F could be explained by the feature of the CAT. The CAT-5F
administered tailored items, which matched a participant’s levels
of function. A participant’s responses on the tailored items pro-
vided useful information to precisely estimate levels of function
and thus yield high reliability. Therefore, although the CAT-5F
administered only selected items in the item bank, the reliability
of the CAT-5F was comparable to that of the item bank. The high
reliability of the CAT-5F indicates that the scores of the CAT-5F
have low random measurement error. Specifically, high group-
and individual-level reliabilities indicate that the CAT-5F can be
used to compare different levels of the 5 functions between
groups (eg, control and experimental groups in research) and in-
dividuals (eg, individual patients with stroke in clinical settings),
respectively.35
High correlations were found between the 5 domain scores of
the CAT-5F and those of the corresponding original tests. The
findings indicate good concurrent validity of the CAT-5F, which
supports that the CAT-5F is a valid measure of BADL, postural
control, UE/LE motor functions and mobility.
For the UE/LE motor functions/domains of the CAT-5F, the
SRMs were >0.30. For the other domains of the CAT-5F, the
SRMs were >0.60. These findings support the good responsive-
ness of the CAT-5F in patients with stroke, implying that the CAT-
5F can detect subtle improvements in the patients’ BADL,
postural control, UE/LE motor functions, and mobility. Thus, the
CAT-5F can be used as an outcome measure in both research and
clinical settings.
The CAT-5F had 3 final sets of stopping rules (final sets A-C).
In general, the administrative efficiency and psychometric prop-
erties (reliability, concurrent validity, and responsiveness) were
similar and sufficient when the CAT-5F adopted the 3 sets of
stopping rules. Specifically, among the 3 final sets of stopping
rules, the CAT-5F with the final set A (limited reliability increase
<0.010) had the highest reliability but low administrative effi-
ciency. In contrast, final set B (limited reliability increase <0.015)
had the lowest reliability and the highest administrative efficiency.
Therefore, the authors suggest that prospective users use final set
A of the stopping rules for the CAT-5F.

www.archives-pmr.org
904 G.-H. Lin et al

In addition to the BI, the Functional Independence Measure-

Appendix 1 Item and item difficulty of the CAT-5F
motor scale (FIM-M) is another commonly-used BADL test, and
the items of the FIM-M could be considered for use as the BADL Item
items for the CAT-5F. Previous studies have found the scores of Item of the CAT-5F Difficulty
the BI and the FIM-M to be highly correlated (Pearson r or BADL
Spearman rZ0.90-0.95).23,36,37 Additionally, this study found Bathing -0.3
that the scores of the BI and the CAT-5F’s BADL domain were Grooming -1.5
highly correlated (Pearson rZ0.90-0.94). These findings indicate Dressing 0.1
that the scores of the CAT-5F’s BADL domain are likely to be Bowels 4.0
highly correlated with those of the FIM-M. If so, the scores of the Bladder 0.4
CAT-5F’s BADL domain and those of the FIM-M are comparable, Toilet use 2.0
and even interchangeable using linear transformation. However, Transfers 0.3
the relationships between the CAT-5F’s BADL domain and the Mobility -2.8
FIM-M will need to be validated in future studies. Stairs -2.3
Postural control
Study limitations Sitting without support -2.6
Standing with support -1.7
The limitation of this study is that it was a simulation study. Standing without support -3.6
Further studies will be needed to cross-validate the methodology Standing on unaffected leg -1.5
and findings of this study, further validate other important psy- Standing on paretic leg -0.7
chometric properties (eg, test-retest reliability and minimal Supine to affected side lateral 0.3
detectable change), and examine the administration time (effi- Supine to unaffected side lateral -0.9
ciency) of the CAT-5F. In addition, 2 limitations of the CAT-5F Supine to sitting up on the edge of the table 0.4
should be noted. First, the CAT-5F did not assess patients’ Sitting on the edge of the table to supine 3.4
instrumental ADL, which is an important function for patients Sitting to standing up 5.2
with stroke. However, patients with acute or subacute stroke, most Standing up to sitting down 1.5
of whom stay in hospitals, are largely unlikely to perform Standing, picking up a pencil from floor 0.3
instrumental ADL (eg, washing clothes, housework, or traveling). UE motor function
Thus, the instrumental ADL domain was not included in the CAT- Protracts scapula in supine -0.2
5F. Future studies may include a measure of instrumental ADL to Extends elbow in supine (starting with elbow fully flexed) -0.9
further expand the utility of the CAT-5F for patients living in the Raises hand to touch top of the head (sitting) -0.1
community. Second, the 5 domains of the CAT-5F cannot be Places hand on sacrum (sitting) 0.8
administered separately. However, the CAT-5F has sufficient Raises arm overhead to fullest elevation (sitting) 0.0
administrative efficiency (needing only about 13 items) and can Supinates and pronates forearm (sitting and elbow -0.1
provide valuable information on the 5 important functions for flexed at 90 )
patients with stroke. Therefore, the limitations of the CAT-5F may Closes hand from fully opened position (sitting) 0.2
be acceptable for some users. Opens hand from fully closed position (sitting) 0.4
LE motor function
Flexes hip and knee in supine (attains half crook -2.1
Conclusions lying)
Flexes hip in sitting -1.1
The authors developed the CAT-5F to simultaneously assess
Extends knee in sitting -0.9
BADL, postural control, UE and LE motor functions, and mobility
Extends knee and dorsiflexes ankle in sitting -0.2
in patients with stroke. Our results indicate that the CAT-5F has
Dorsiflexes ankle in sitting 0.6
sufficient administrative efficiency, reliability, concurrent validity,
Flexes knee in sitting -0.8
and responsiveness to simultaneously assess BADL, postural
Plantar flexes ankle in sitting 0.8
control, UE/LE motor functions, and mobility in patients
Dorsiflexes affected ankle with knee extended 1.9
with stroke.
(standing)
Flexes affected knee with hip extended (standing) 1.9
Keywords Mobility
Rolls onto side (starting from supine) -2.7
Activities of daily living; Motor skills; Patient outcome Raises hips off bed in crook lying (bridging) -4.7
assessment; Postural balance; Rehabilitation; Stroke Moves from lying supine to sitting (with feet on the floor) -0.7
Rises to standing from sitting -0.5
Maintains standing for 20 counts -1.3
Corresponding author Places affected foot onto first step (or stool 18 cm high) 1.2
Walks 10 meters indoors 1.5
Ching-Lin Hsieh, PhD, School of Occupational Therapy, College Walks down 3 stairs alternating feet 3.6
of Medicine, National Taiwan University, F4, No.17, Xuzhou Rd., Takes 3 steps sideways to affected side 1.4
Zhongzheng Dist., Taipei City 100, Taiwan. E-mail address: Takes 3 steps backwards 2.2
clhsieh@ntu.edu.tw.

www.archives-pmr.org
www.archives-pmr.org

Development and validation of the CAT-5F

Appendix 2 Reliability and efficiency of the 10 sets of stopping rules for the CAT-5F
Sets of Rules
Measure* A B C D E F G H I J Item Bank
Precision (Rasch reliability)
BADL
Average 0.92 0.90 0.88 0.87 0.85 0.86 0.85 0.84 0.83 0.82 0.92
Maximal 0.95 0.95 0.95 0.93 0.93 0.95 0.93 0.93 0.93 0.92 0.95
Minimal 0.69 0.69 0.69 0.69 0.69 0.69 0.69 0.69 0.69 0.69 0.69
% of the patients with reliability
0.70 99.3 99.3 99.3 99.3 99.3 99.3 99.3 99.3 99.3 99.3 99.3
% of the patients with reliability
0.90 84.4 67.0 42.6 28.7 17.2 31.7 16.9 11.3 5.9 3.3 84.4
Postural control
Average 0.97 0.96 0.96 0.95 0.95 0.95 0.94 0.94 0.94 0.93 0.98
Maximal 0.99 0.98 0.98 0.98 0.97 0.99 0.98 0.98 0.97 0.97 0.99
Minimal 0.79 0.78 0.78 0.79 0.79 0.79 0.78 0.78 0.79 0.79 0.80
% of the patients with reliability
0.70 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0
% of the patients with reliability
0.90 97.8 97.8 97.4 97.4 97.2 97.4 95.7 94.8 94.8 94.6 98.1
UE motor function
Average 0.89 0.87 0.86 0.85 0.84 0.86 0.84 0.83 0.82 0.82 0.89
Maximal 0.98 0.98 0.97 0.97 0.97 0.97 0.97 0.96 0.94 0.94 0.98
Minimal 0.37 0.35 0.35 0.35 0.37 0.37 0.35 0.35 0.35 0.37 0.39
% of the patients with reliability
0.70 88.1 85.9 83.1 81.7 81.9 86.1 83.3 81.9 82.6 84.6 89.4
% of the patients with reliability
0.90 64.6 59.6 57.2 55.4 48.7 59.1 49.8 38.7 32.4 28.0 66.7
LE motor function
Average 0.93 0.92 0.91 0.90 0.89 0.90 0.88 0.88 0.87 0.87 0.94
Maximal 0.98 0.98 0.97 0.96 0.96 0.97 0.96 0.96 0.95 0.94 0.98
Minimal 0.57 0.54 0.54 0.54 0.58 0.57 0.54 0.54 0.54 0.58 0.59
% of the patients with reliability
0.70 97.2 96.5 95.9 95.9 95.9 97.0 96.1 95.9 95.9 94.8 97.8
% of the patients with reliability
0.90 82.8 79.6 76.9 74.8 66.3 78.3 64.1 53.0 44.3 39.3 83.3
Mobility
Average 0.98 0.97 0.96 0.96 0.96 0.96 0.95 0.95 0.95 0.94 0.98
Maximal 0.99 0.98 0.98 0.98 0.98 0.99 0.98 0.98 0.98 0.98 0.99
Minimal 0.81 0.81 0.80 0.81 0.81 0.81 0.81 0.80 0.81 0.81 0.82
% of the patients with reliability
0.70 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0
% of the patients with reliability
0.90 98.3 98.3 97.8 97.8 97.4 98.3 97.8 97.0 97.0 96.7 98.7
Efficiency (number of items)
Average 29.3 16.3 12.7 11.0 9.5 12.5 9.0 8.1 7.4 6.9 47.9
Median (1st quartile-3rd quartile) 29 (27-32) 16 (15-18) 13 (12-14) 11 (10-12) 10 (9-11) 10 (8-16) 8 (6-11) 7 (6-10) 7 (6-9) 6 (6-8) 48 (48-48)
Range 15-39 9-22 5-18 5-16 5-15 5-35 5-20 5-17 5-14 5-12 42y-48
% of the patients using 5-10 items 0.0 2.6 9.1 31.3 62.2 52.4 67.4 73.7 77.4 78.7 0.0
% of the patients using 11-15 items 1.3 33.3 83.1 65.4 31.3 19.6 24.4 18.3 9.6 3.5 0.0
% of the patients using >15 items 98.7 64.1 6.7 0.4 0.0 26.1 5.0 0.2 0.0 0.0 100.0
* Sets of stopping rules A-E: limited reliability increase (A: <0.001, B: <0.005, C: <0.010, D: <0.015, E: <0.020).
y
Sets of stopping rules F-J: Rasch reliability
.90 or limited reliability increase (F: <0.001, G: <0.005, H: <0.010, I: <0.015, J: <0.020).

905
906
Appendix 3 The concurrent validity of the CAT-5F with the 3 final sets of stopping rules (Pearson r)
Time of Assessment BI PASS
Admission
Final set A 0.92 0.96
Final set B 0.91 0.96
Final set C 0.91 0.96
Discharge
Final set A 0.94 0.93
Final set B 0.93 0.93
Final set C 0.94 0.94
NOTE. Final sets A-C are the sets of stopping rules “limited reliability increase <0.010”, “limited reliability increase <0.015”, and “Rasch reliability

.90 or limited reliability increase <0.001”, respectively.
Supplier
a. ConQuest software; ACER.
References
1. Sorrentino G, Sale P, Solaro C, Rabini A, Cerri CG, Ferriero G.
Clinical measurement tools to assess trunk performance after stroke: a
systematic review. Eur J Phys Rehabil Med 2018;54:772-84.
2. Sangha H, Lipson D, Foley N, et al. A comparison of the Barthel Index
and the Functional Independence Measure as outcome measures in
stroke rehabilitation: patterns of disability scale usage in clinical trials.
Int J Rehabil Res 2005;28:135-9.
3. Gor-Garcia-Fogeda MD, Molina-Rueda F, Cuesta-Gomez A, Carra-
tala-Tejada M, Alguacil-Diego IM, Miangolarra-Page JC. Scales to
assess gross motor function in stroke patients: a systematic review.
Arch Phys Med Rehabil 2014;95:1174-83.
4. Benaim C, Perennou DA, Villy J, Rousseaux M, Pelissier JY. Vali-
dation of a standardized assessment of postural control in stroke pa-
tients: the Postural Assessment Scale for Stroke Patients (PASS).
Stroke 1999;30:1862-8.
5. Mahoney FI, Barthel DW. Functional evaluation: the Barthel Index.
Md State Med J 1965;14:61-5.
6. Daley K, Mayo N, Danys I, et al. The Stroke Rehabilitation Assess-
ment of Movement (STREAM): refining and validating the content.
Physiother Can 1997;49:269-78.
7. Powers WJ, Rabinstein AA, Ackerson T, et al. 2018 guidelines for the
early management of patients with acute ischemic stroke: a guideline
for healthcare professionals from the American Heart Associatio-
n/American Stroke Association. Stroke 2018;49:e46-110.
8. Winstein CJ, Stein J, Arena R, et al. Guidelines for adult stroke
rehabilitation and recovery. Stroke 2016;47:e98-169.
9. Hsueh I-P, Hsu M-J, Sheu C-F, Lee S, Hsieh C-L, Lin J-H. Psycho-
metric comparisons of 2 versions of the Fugl-Meyer Motor Scale and 2
versions of the Stroke Rehabilitation Assessment of Movement.
Neurorehabil Neural Repair 2008;22:737-44.
10. Hsueh I-P, Lee M-M, Hsieh C-L. Psychometric characteristics of the
Barthel activities of daily living index in stroke patients. J Formos
Med Assoc 2001;100:526-32.
11. Mao H-F, Hsueh I-P, Tang P-F, Sheu C-F, Hsieh C-L. Analysis and
comparison of the psychometric properties of three balance measures
for stroke patients. Stroke 2002;33:1022-7.
12. Hsueh I-P, Chen J-H, Wang C-H, et al. Development of a computer-
ized adaptive test for assessing balance function in patients with
stroke. Phys Ther 2016;90:1336-44.
13. Hsueh I-P, Chen J-H, Wang C-H, Hou W-H, Hsieh C-L. Development
of a computerized adaptive test for assessing activities of daily living
in outpatients with stroke. Phys Ther 2013;93:681-93.
G.-H. Lin et al
STREAM-UE STREAM-LE STREAM-Mobility
0.96 0.96 0.95
0.96 0.95 0.95
0.95 0.95 0.95
0.95 0.96 0.96
0.95 0.96 0.95
0.95 0.96 0.95
14. Hou W-H, Shih C-L, Chou Y-T, et al. Development of a computerized
adaptive testing system of the Fugl-Meyer motor scale in stroke pa-
tients. Arch Phys Med Rehabil 2012;93:1014-20.
15. Weiss DJ, Kingsbury GG. Application of computerized adaptive
testing to educational problems. JEM 1984;21:361-75.
16. Papuga M, Dasilva C, McIntyre A, Mitten D, Kates S, Baumhauer J.
Large-scale clinical implementation of PROMIS computer adaptive
testing with direct incorporation into the electronic medical record.
Health Systems 2017;7:1-12.
17. Koh C-L, Pan S-L, Jeng J-S, et al. Predicting recovery of voluntary
upper extremity movement in subacute stroke patients with severe
upper extremity paresis. PLoS One 2015;10:e0126857.
18. Hsueh I-P, Wang W-C, Wang C-H, et al. A simplified stroke reha-
bilitation assessment of movement instrument. Phys Ther 2006;86:
936-43.
19. Wang Y-L, Lin G-H, Huang Y-J, Chen M-H, Hsieh C-L. Refining 3
measures to construct an efficient Functional Assessment of Stroke.
Stroke 2017;48:1630-5.
20. Hsueh I-P, Wang W-C, Sheu C-F, Hsieh C-L. Rasch analysis of
combining two indices to assess comprehensive ADL function in
stroke patients. Stroke 2004;35:721-6.
21. Linacre J. A user’s guide to Winstep. Chicago: MESA Press; 2009.
22. Lin G-H, Huang Y-J, Lee S-C, Huang S-L, Hsieh C-L. Development
of a computerized adaptive testing system of the Functional Assess-
ment of Stroke. Arch Phys Med Rehabil 2018;99:676-83.
23. Hsueh I-P, Lin J-H, Jeng J-S, Hsieh C-L. Comparison of the psy-
chometric characteristics of the functional independence measure, 5
item Barthel index, and 10 item Barthel index in patients with stroke. J
Neurol Neurosurg Psychiatry 2002;73:188-90.
24. Wang C-H, Hsieh C-L, Dai M-H, Chen C-H, Lai Y-F. Inter-rater
reliability and validity of the Stroke Rehabilitation Assessment of
Movement (STREAM) instrument. J Rehabil Med 2002;34:20-4.
25. Wu M, Adams R, Wilson M, Haldane S. ACER ConQuest 2.0: general
item response modelling software [computer program manual].
Camberwell, Australia: Australian Council for Educational Research
Ltd; 2007.
26. Reckase M. Multidimensional item response theory. New York, NY:
Springer; 2009.
27. Gibbons LE, Fredericksen R, Batey DS, et al. Validity assessment of
the PROMIS fatigue domain among people living with HIV. AIDS Res
Ther 2017;14:21.
28. Salter K, Jutai J, Teasell R, Foley N, Bitensky J, Bayley M. Issues for
selection of outcome measures in stroke rehabilitation: ICF activity.
Disabil Rehabil 2005;27:315-40.
29. De Vet HC, Bouter LM, Bezemer PD, Beurskens AJ. Reproducibility
and responsiveness of evaluative outcome measures: theoretical con-
siderations illustrated by an empirical example. Int J Technol Assess
Health Care 2001;17:479-87.
30. Bartolo M, De Nunzio AM, Sebastiano F, et al. Arm weight support
training improves functional motor outcome and movement smooth-
ness after stroke. Funct Neurol 2014;29:15-21.
www.archives-pmr.org
Development and validation of the CAT-5F
31. El-Helow MR, Zamzam ML, Fathalla MM, et al. Efficacy of modified
constraint-induced movement therapy in acute stroke. Eur J Phys
Rehabil Med 2015;51:371-9.
32. Kojovic J, Djuric-Jovicic M, Dosen S, Popovic MB, Popovic DB.
Sensor-driven four-channel stimulation of paretic leg: functional
electrical walking therapy. J Neurosci Methods 2009;181:100-5.
33. Mohan U, Babu SK, Kumar KV, Suresh B, Misri Z,
Chakrapani M. Effectiveness of mirror therapy on lower extremity
motor recovery, balance and mobility in patients with acute
stroke: a randomized sham-controlled pilot trial. Ann Indian Acad
Neurol 2013;16:634-9.
www.archives-pmr.org
907
34. Rao N, Zielke D, Keller S, et al. Pregait balance rehabilitation in acute
stroke patients. Int J Rehabil Res 2013;36:112-7.
35. Linacre JM. A user’s guide to WINSTEPS MINISTEP Rasch-model
computer programs. Chicago: MESA Press; 2006.
36. Kwon S, Hartzema AG, Duncan PW, Min-Lai S. Disability measures
in stroke: relationship among the Barthel Index, the Functional Inde-
pendence Measure, and the Modified Rankin Scale. Stroke 2004;35:
918-23.
37. Hachisuka K, Okazaki T, Ogata H. Self-rating Barthel index
compatible with the original Barthel index and the Functional Inde-
pendence Measure motor score. JUOEH 1997;19:107-21.