Professional Documents
Culture Documents
1 Health & Disability Research Institute, School of Public Health, Boston University, Boston, MA. 2 Department of Occupational Therapy, Sargent College of Health &
Rehabilitation Sciences, Boston University, Boston, MA. 3 Research Center for Children with Special Health Care Needs, Franciscan Hospital for Children, Boston, MA.
4 CREcare, LLC, Newburyport, MA. 5 Department of Educational Research, Measurement and Evaluation, Boston College Lynch School of Education, Chestnut Hill, MA, USA.
Correspondence to Dr Wendy Coster at Department of Occupational Therapy, Boston University, Sargent College of Health and Rehabilitation Sciences, 635 Commonwealth Avenue, Boston MA 02215,
USA. E-mail: wjcoster@bu.edu
PUBLICATION DATA AIM The aims of the study were to: (1) build new item banks for a revised version of the Pediatric
Accepted for publication 13th July 2011. Evaluation of Disability Inventory (PEDI) with four content domains: daily activities, mobility,
social ⁄ cognitive, and responsibility; and (2) use post-hoc simulations based on the combined
ABBREVIATIONS normative and disability calibration samples to assess the accuracy and precision of the PEDI
IRT Item response theory computer-adaptive tests (PEDI-CAT) compared with the administration of all items.
PEDI-CAT Pediatric Evaluation of Disability METHOD Parents of typically developing children (n=2205) and parents of children and adoles-
Inventory computer-adaptive tests cents with disabilities (n=703) between the ages of 0 and 21 years, stratified by age and sex,
participated by responding to PEDI-CAT surveys through an existing Internet opt-in survey panel
in the USA and by computer tablets in clinical sites.
RESULTS Confirmatory factor analyses supported four unidimensional content domains. Scores
using the real data post hoc demonstrated excellent accuracy (intraclass correlation coefficients
‡0.95) with the full item banks. Simulations using item parameter estimates demonstrated rela-
tively small bias in the 10-item and 15-item CAT versions; error was generally higher at the scale
extremes.
INTERPRETATION These results suggest the PEDI-CAT can be an accurate and precise assessment
of children’s daily performance at all functional levels.
The Pediatric Evaluation of Disability Inventory (PEDI) was known relation between item responses and positions on an
originally developed to provide a parent- or clinician-reported underlying functional continuum. Using this approach, proba-
functional test that was normed on children up to the age of bilities of children scoring a particular response on an item at
7½ years.1 The PEDI has a long history of application in various functional ability levels can be modeled. These proba-
developmental medicine, serving as a functional outcome mea- bility estimates are used to determine the child’s most likely
sure for clinical research and practice.2 The original PEDI position along the functional dimension. When assumptions
was a fixed-format test in which all items needed to be admin- of a particular IRT model are met, estimates of a child’s func-
istered to derive a score. In addition to some finding the tional ability do not strictly depend on a particular fixed set of
administration of all PEDI items potentially burdensome, items. This scaling feature allows one to compare persons
the restriction of norms only to young children also limited its along a functional outcome dimension even if they have not
use.3 The aim of this study is to report on the revision of the completed the identical set of functional items. CAT also opti-
PEDI into a series of computer-adaptive tests (CATs) that mizes the items administered, providing items most likely to
cover the 0 to 21-year age range. yield the greatest information for score estimation. CATs will
Computer-adaptive tests are being promoted as the next generally require fewer items than a comparable length fixed
logical step to more efficient assessments of health outcome.4 form instrument to achieve similar precision, although the
To use a CAT, item response theory (IRT) modeling is used gains in efficiency may not be uniform throughout the full
to express the association between an individual’s response to scale.5
an item and the outcome domain. IRT measurement models Early work has highlighted the possible benefits of using
are a class of statistical procedures used to develop measure- CAT in assessing children’s functioning, including improved
ment scales. The measurement scales comprise items with a efficiency with minimal loss of precision.6–9 We have shown
1100 DOI: 10.1111/j.1469-8749.2011.04107.x ª The Authors. Developmental Medicine & Child Neurology ª 2011 Mac Keith Press
that the original PEDI can be successfully engineered into What this paper adds
efficient CAT programs.8,10 The North American Shriners • The Pediatric Evaluation of Disability Inventory computer-adaptive tests mea-
Orthopedic Hospitals have recently developed a series of par- sure four domains (daily activities, mobility, cognitive ⁄ social, and responsibil-
ent- and child-reported CAT measures of physical functioning ity) in children and young people aged 0 to 21 years.
and participation for their network with promising results.11–15 • Psychometric properties (accuracy and precision) of the 10-item and 15-item
CAT are very good.
The aims of this project were to build and test new PEDI- • Precision varies with position on the functional continuum; extreme scores are
CAT item banks for an age range of 0 to 21 years and use less precise than scores in the midrange.
simulations based on the combined normative and disability
calibration samples to assess the accuracy and precision of the racial ⁄ ethnic and socioeconomic distribution of the US popu-
PEDI-CATs. lation according to the Year 2000 Census Bureau data (see
Table I for sample characteristics). Eligibility for participation
METHOD was determined by a series of parent-reported screening items
Selection of participants to ensure the child was developing typically. For example, any
The targeted normative population of interest for the PEDI- child with a disability, chronic condition, or receiving special-
CAT was civilian households in the contiguous United States ized services was considered ineligible for the normative
with children under 21 years of age. Normative data were col- sample. Once eligibility was determined, participation and
lected through the Internet. An online panel from YouGov consent were obtained.
(http://corp.yougov.com; Palo Alto, CA, USA) was the sample Data for most of the disability sample (n=617) were also col-
source for a nationally representative sample of 2205 parents. lected through the Internet by YouGov. Eligibility for the dis-
YouGov operates a panel of over one million respondents ability sample was determined by screening questions which
who have provided their names, street addresses, e-mail identified children and young people with a physical, cogni-
addresses, and other information, and who regularly partici- tive, or behavioral disability. We supplemented the disability
pate in online surveys. Panel members receive modest com- sample by collecting parent-reported mobility domain data at
pensation when they participate in online surveys. Panel two clinical sites that served a wide age range (n=86) in order
members are restricted to respondents with fluency in Eng- to add children with more severe physical disabilities
lish. (Table II). Both clinical sites collected parent-reported data
Quota sampling based on age was used to ensure that suffi- from an Internet system. The study received approval from
cient cases of typically developing children and adolescents each site’s governing institutional review board, and each par-
were collected within each of the PEDI age-based strata (100 ent respondent provided informed consent as well as Health
children and adolescents in each of the 21-age strata starting Insurance Portability and Accountability Act authorization
with 0–1 and stopping with 20–21). Within each age group, before participation.
equal proportions of sex were selected and efforts were made
to ensure that participants were representative of the Procedures
Item development
Table I: Demographics of participants The initial item pools for the PEDI-CATs were developed
through a comprehensive review of existing pediatric mea-
Normative Disability sures, the published literature on the functional outcomes of
Mean age (y:mo) (SD) 10:1 (6.1) 11:8 (4.7)
children and young people in hospital-based and community
Sex (M ⁄ F) 1126 ⁄ 1079 349 ⁄ 268 settings, and user feedback since the original PEDI’s publica-
Race and ethnicity, n (%) tion in 1992.16 Feedback about content coverage, content rele-
White 1438 (65.2) 491 (69.8)
Black 241 (10.9) 63 (9.0)
vance, and item clarity was compiled through focus groups,
Hispanic 207 (9.4) 63 (9.0)
Asian 30 (1.4) 13 (1.8)
Native American 13 (0.6) 7 (1.0) Table II: Conditions reported by disability sample, n (% yes)
Mixed 222 (10.1) 57 (8.1)
Other 49 (2.2) 7 (1.0)
Middle Eastern 4 (0.2) 2 (0.3) Developmental delay 192 (31.1)
School placement, n (%) Intellectual disability 50 (8.1)
Preschool ⁄ early childhood 294 (13.3) 132 (13.8) Hearing impairment 30 (4.9)
program ⁄ kindergarten Speech ⁄ language impairment 171 (27.7)
Elementary ⁄ middle ⁄ high school 1236 (56.1) 475 (67.6) Vision impairment 46 (7.5)
Ungraded 20 (0.9) 20 (2.8) Serious emotional disturbance 50 (8.1)
Undergraduate ⁄ college 196 (8.9) 20 (2.8) Orthopedic ⁄ movement impairment 23 (3.7)
Not in school 459 (20.8) 55 (7.8) Autism spectrum disorder 108 (17.5)
Parent ⁄ respondent education Attention deficit disorder 232 (37.6)
level, n (%) Traumatic brain injury 9 (1.5)
No high school 47 (2.1) 58 (8.3) Specific learning disability 78 (12.6)
High school graduate 392 (17.8) 141 (20.1) Health impairment 58 (9.4)
Some college 846 (38.4) 260 (37.0) Multiple disabilities 28 (4.5)
College graduate 573 (26) 142 (20.2) Other impairments ⁄ problem 96 (15.6)
Postgraduate 346 (15.7) 97 (13.8)
Note: more than one choice was possible.
Ratio of first
Number Variance and second Comparative Tucker–Lewis Root mean square
Scale of items (%) eigenvalues fit index index error approximation
Daily activities
5-CAT 0.97 2.44 0.88 1.46 0.88
10-CAT 0.99 2.15 )0.1 0.39 0.1
15-CAT 0.99 1.95 0 0.2 0
Mobility
5-CAT 0.95 2.03 0.68 1.35 0.87
10-CAT 0.98 1.84 0.08 0.39 0.19
15-CAT 0.99 1.64 )0.01 0.19 0.1
Social ⁄ cognitive
5-CAT 0.96 2.3 1.72 1.72 1.53
10-CAT 0.98 2.11 )0.1 0.57 0.38
15-CAT 0.99 1.82 0 0.29 0.1
Responsibility
5-CAT 0.98 3.48 0.67 2.13 2.36
10-CAT 0.99 3.14 )0.11 0.45 0.34
15-CAT 0.99 2.92 )0.11 0.34 0.11
DIFF, differential item functioning; RMSE, root mean square error; CAT, computer-adaptive tests.
RMSE
3
RMSE
4
2 3
2
1
1
0 0
40 45 50 55 60 65 70 75 80 30 35 40 45 50 55 60 65 70 75
CAT-5 CAT-10 CAT-15 Full item bank CAT-5 CAT-10 CAT-15 Full item bank
Social/cognitive
Responsibility
10 16
14
8 12
10
RMSE
RMSE
6
8
4 6
4
2
2
0 0
35 40 45 50 55 60 65 70 75 80 20 25 30 35 40 45 50 55 60 65 70 75 80
CAT-5 CAT-10 CAT-15 Full item bank CAT-5 CAT-10 CAT-15 Full item bank
Figure 1: Root mean square error (RMSE) for full item bank compared to scores from Pediatric Evaluation of Disability Inventory computer-adaptive tests
(CAT).
REFERENCES
1. Haley SM, Coster WJ, Ludlow LH, Haltiwanger JT, puter adaptive testing version of the Pediatric Evaluation of ity in children with cerebral palsy. Phys Ther 2009; 89: 589–
Andrellos PA. Pediatric Evaluation of Disability Inventory: Disability Inventory (PEDI). Arch Phys Med Rehabil 2005; 600.
Development, Standardization and Administration Manual. 86: 932–9. 16. Haley S, Coster W. PEDI-CAT: Development, Standardiza-
Boston, MA: Trustees of Boston University, 1992. 9. Jacobusse G, van Buuren S. Computerized adaptive testing tion and Administration Manual. Boston, MA: CRECare,
2. Haley SM, Coster WI, Kao YC, et al. Lessons from use of the for measuring development of young children. Stat Med LLC, 2010.
Pediatric Evaluation of Disability Inventory: where do we go 2007; 26: 2629–38. 17. Dumas H, Fragala-Pinkham M, Haley S, et al. Item bank
from here? Pediatr Phys Ther 2010; 22: 69–75. 10. Coster WJ, Haley SM, Ni P, Dumas HM, Fragala-Pinkham development for a revised Pediatric Evaluation of Disability
3. Ziviani J, Ottenbacher K, Shephard K, Foreman S, Astbury MA. Assessing self-care and social function using a com- Inventory (PEDI). Phys Occup Ther Pediatr 2010; 30: 168–
W, Ireland P. Concurrent validity of the Functional Inde- puter adaptive testing version of the pediatric evaluation of 84.
pendence Measure for children (WeeFIM) and the Pediatric disability inventory. Arch Phys Med Rehabil 2008; 89: 18. Reeve BB, Hays RD, Bjorner JB, et al. Psychometric evalua-
Evaluation of Disabilities Inventory in children with devel- 622–9. tion and calibration of health-related quality of life item
opmental disabilities and acquired brain injuries. Phys Occup 11. Haley SM, Chafetz RS, Tian F, et al. Validity and reliability banks: plans for the Patient-Reported Outcomes Measure-
Ther Pediatr 2001; 21: 91–101. of physical functioning computer adaptive tests for children ment Information System (PROMIS). Med Care 2007; 45: (5
4. Cella D, Gershon R, Lai J, Choi S. The future of outcomes with cerebral palsy. J Pediatr Orthop 2010; 30: 71–5. Suppl. 1) S22–31.
measurement: item banking, tailored short-forms, and com- 12. Tucker CA, Montpetit K, Bilodeau N, et al. Assessment of 19. Muthén BO, Muthén LK. Mplus User’s Guide. Los Angeles:
puterized adaptive assessment. Qual Life Res 2007; 16: 133–41. children with cerebral palsy using a parent-report computer Muthén & Muthén, 2001.
5. Choi SW, Reise SP, Pilkonis PA, Hays RD, Cella D. Effi- adaptive test II: upper extremity skills. Dev Med Child Neurol 20. Muraki E, Bock RD. PARSCALE: IRT Item Analysis and
ciency of static and computer adaptive short forms compared 2009; 51: 725–31. Test Scoring for Rating – Scale Data. Chicago: Scientific
to full-length measures of depressive symptoms. Qual Life 13. Tucker CA, Gorton GE, Watson K, et al. Assessment of Software International, 1997.
Res 2010; 19: 125–36. children with cerebral palsy using a parent–report computer 21. Liang T, Han K, Hambleton R. User’s Guide for Resid-
6. Gardner W, Shear K, Kelleher K, et al. Computerized adap- adaptive test I: lower extremity and mobility skills. Dev Med Plots-2: Computer Software for IRT Graphical Residual
tive measurement of depression: a simulation study. BMC Child Neurol 2009; 51: 717–24. Analyses, Version 2.0. Amherst, MA: University of Massa-
Psychiatry 2004; 4: 13. 14. Haley SM, Ni P, Dumas HD, Fragala-Pinkham MA, Tucker chusetts, Center for Educational Assessment, 2008.
7. Haley SM, Ni P, Fragala-Pinkham MA, Skrinar AM, Corzo CA. Measuring global physical health in children with 22. Swanson DB, Clauser BE, Case SM, Nungester RJ,
D. A computer adaptive testing approach for assessing physi- cerebral palsy: illustration of a bi-factor model and computer- Featherman C. Analysis of differential item functioning
cal function in children and adolescents. Dev Med Child Neu- ized adaptive testing. Qual Life Res 2009; 18: 359–65. (DIF) using hierarchical logistic regression models. J Educ
rol 2005; 47: 113–20. 15. Haley SM, Fragala-Pinkham MA, Dumas HM, et al. Evalua- Behav Stat 2002; 27: 53–75.
8. Haley SM, Raczek AE, Coster WJ, Dumas HM, Fragala- tion of an item bank for a computerized adaptive test of activ- 23. Haley SM, Ni P, Hambleton RK, Slavin MD, Jette AM.
Pinkham MA. Assessing mobility in children using a com- Computer adaptive testing improves accuracy and precision
APPENDIX I
Sample Pediatric Evaluation of Disability Inventory computer-adaptive tests items.
Daily activities Eating and mealtime Inserts a straw into a juice box. Please choose which response below best describes
(68 items) Getting dressed Puts on winter, sport, or work gloves. your child’s ability in the following:
Keeping clean Puts toothpaste on brush and brushes teeth Unable = can’t do, doesn’t know how or is too
thoroughly. young.
Home tasks Opens door lock using key. Hard = does with a lot of help, extra time, or effort.
Mobility Basic movement and When lying on back, turns head to both sides. A little hard = does with a little help, extra time
(97 items) transfers or effort.
Standing and walking Walks while wearing a light backpack. Easy = does with no help, extra time or effort,
Steps and inclines Goes up and down an escalator. or child’s skills are past this level.
Running and playing Pulls self out of swimming pool not using ladder. I don’t know.
Wheelchair Goes up and down ramp with wheelchair.
Social ⁄ Interaction Greets new people appropriately when introduced.
cognitive Communication Writes short notes or sends text messages or email.
(60 items) Everyday cognition Recognizes their printed name.
Self management When upset, responds without punching, hitting,
or biting.
Responsibility Organization and Keeping personal electronic devices in working How much responsibility does your child take for
(51 items) planning order (e.g. cell phone, computer). the following activities?
Includes having devices charged and available Adult ⁄ caregiver has full responsibility; the child
when needed; updating software. does not take any responsibility.
Adult ⁄ caregiver has most responsibility and child
takes a little responsibility.
Adult ⁄ caregiver and child share responsibility
about equally.
Child has most responsibility with a little direction,
supervision, or guidance from an adult ⁄ caregiver.
Child takes full responsibility without any direction,
supervision, or guidance from an adult ⁄ caregiver.
I don’t know.
Taking care of daily Buying clothing at a store, from a catalog or online.
needs Includes purchasing clothing, including outerwear
and undergarments.
Health management Following health and medical treatment
requirements.
Includes taking prescribed medication as directed;
following dietary restrictions; adhering to exercise
or other treatment routines.
Staying safe Using the Internet safely.
Includes recognizing scams and inappropriate
approaches from strangers; avoiding posting
inappropriate images; evaluating safety of files
before downloading.