Clinical Prediction Rules for Children: A Systematic Review Jonathon L. Maguire, Dina M.

Kulik, Andreas Laupacis, Nathan Kuppermann, Elizabeth M. Uleryk and Patricia C. Parkin Pediatrics 2011;128;e666; originally published online August 22, 2011; DOI: 10.1542/peds.2011-0043

The online version of this article, along with updated information and services, is located on the World Wide Web at:
http://pediatrics.aappublications.org/content/128/3/e666.full.html

PEDIATRICS is the official journal of the American Academy of Pediatrics. A monthly publication, it has been published continuously since 1948. PEDIATRICS is owned, published, and trademarked by the American Academy of Pediatrics, 141 Northwest Point Boulevard, Elk Grove Village, Illinois, 60007. Copyright 2011 by the American Academy of Pediatrics. All rights reserved. Print ISSN: 0031-4005. Online ISSN: 1098-4275.

Downloaded from pediatrics.aappublications.org at Indonesia:AAP Sponsored on May 30, 2013

Clinical Prediction Rules for Children: A Systematic Review abstract

CONTEXT: The degree to which clinical prediction rules (CPRs) for children meet published standards is unclear. OBJECTIVE: To systematically review the quality, performance, and validation of published CPRs for children, compare them with adult CPRs, and suggest pediatric-specic changes to CPR methodology. METHODS: Medline was searched from 1950 to 2011. Studies were selected if they included the development of a CPR involving children younger than 18 years. Two investigators assessed study quality, rule performance, and rule validation as methodologic standards. RESULTS: Of 7298 titles and abstracts assessed, 137 eligible studies were identied. They describe the development of 101 CPRs addressing 36 pediatric conditions. Quality standards met in fewer than half of the studies were blind assessment of predictors (47%), reproducibility of predictors (18%), blind assessment of outcomes (42%), adequate follow-up of outcomes (36%), adequate power (43%), adequate reporting of results (49%), and 95% condence intervals reported (36%). For rule performance, 48% had a sensitivity greater than 0.95, and 43% had a negative likelihood ratio less than 0.1. For rule validation, 76% had no validation, 17% had narrow validation, 8% had broad validation, and none had impact analysis performed. Compared with CPRs for adult health conditions, quality and rule validation seem to be lower. CONCLUSIONS: Many CPRs have been derived for children, but few have been validated. Relative to adult CPRs, several quality indicators demonstrated weaknesses. Existing performance standards may prove elusive for CPRs that involve children. CPRs for children that are more assistive and less directive and include patients values and preferences in decision-making may be helpful. Pediatrics 2011;128: e666e677
AUTHORS: Jonathon L. Maguire, MSc, MD, FRCPC,a,b,c,d,e,f Dina M. Kulik, MD,g Andreas Laupacis, MD, MSc, FRCPC,b,e,h Nathan Kuppermann, MD, MPH,i Elizabeth M. Uleryk, BA, MLS,j and Patricia C. Parkin, MD, FRCPCc,d,e,f
Department of Pediatrics and bKeenan Research Centre, Li Ka Shing Knowledge Institute, St Michaels Hospital, Toronto, Ontario, Canada; cDivision of Pediatric Medicine and the Pediatric Outcomes Research Team, gDivision of Pediatric Emergency Medicine, and jHospital Library, Hospital for Sick Children, Toronto, Ontario, Canada; dChild Health Evaluative Sciences, Hospital for Sick Children Research Institute, Toronto, Ontario, Canada; Departments of eHealth Policy Management and Evaluation, fPediatrics, and hMedicine, University of Toronto, Toronto, Ontario, Canada; and iDepartments of Emergency Medicine and Pediatrics, University of California, Davis School of Medicine, Davis, California KEY WORDS review, clinical prediction rule, child, preschool child, adolescent, decision trees, predictive value of tests, models, multivariate analysis ABBREVIATIONS CPRclinical prediction rule LRnegative likelihood ratio CIcondence interval EBMWGEvidence-Based Medicine Working Group Drs Maguire and Parkin provided the study concept and design, drafted the manuscript, performed statistical analysis, and supervised the study; Ms Uleryk developed and performed the electronic literature search; Drs Maguire and Kulik acquired the data; Drs Maguire, Kulik, Laupacis, Kuppermann, and Parkin performed analysis and interpretation of data; Drs Laupacis, Kuppermann, Kulik, and Parkin critically revised the manuscript for important intellectual content; and Dr Parkin provided administrative, technical, and material support. Dr Maguire had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. www.pediatrics.org/cgi/doi/10.1542/peds.2011-0043 doi:10.1542/peds.2011-0043 Accepted for publication May 27, 2011 Address correspondence to Jonathon L. Maguire, MSc, MD, FRCPC, Pediatric Ambulatory Clinic, St Michaels Hospital, 61 Queen St East, 2nd Floor, Toronto, Ontario, Canada M5C 2T2. E-mail: jonathon.maguire@utoronto.ca PEDIATRICS (ISSN Numbers: Print, 0031-4005; Online, 1098-4275). Copyright 2011 by the American Academy of Pediatrics FINANCIAL DISCLOSURE: The authors have indicated they have no nancial relationships relevant to this article to disclose.
a

e666

MAGUIRE et al

Downloaded from pediatrics.aappublications.org at Indonesia:AAP Sponsored on May 30, 2013

REVIEW ARTICLES

Clinical prediction rules (CPRs) have been dened as clinical decisionmaking tools that use 3 or more variables from history, physical examination, or simple tests to provide the probability of an outcome or suggest a diagnostic or therapeutic course of action for an individual patient.13 They are potentially powerful tools for reducing uncertainty and improving accuracy in medical decision-making by standardizing the collection and interpretation of clinical data.2 In some instances, they may minimize the use of potentially harmful or costly diagnostic testing. CPRs differ from decision analyses, which quantify the value of specied outcomes and use data from the literature to formulate health care policy; decision-support tools, which are designed to prevent errors when implementing decisions that have already been made; and practice guidelines, which reect a summation of the existing data on a particular topic and represent a consensus of expert opinion to address several patient care issues within a particular syndrome.4 To meet their objectives and be routinely incorporated into patient care, CPRs must be rigorously developed and meet the performance expectations of clinicians who use them.1,5 Methodologic standards for the development of CPRs have been described previously.1,3,4,6,7 These standards include several steps in rule development: creating the rule (derivation); testing the rule (validation); translating the results of the validated rule into practice (knowledge translation); and assessing the impact of the rule on physician behavior and clinical outcomes (impact analysis). The authors of 3 reviews have examined the methodologic quality of CPRs largely for adult health conditions and suggested methods for improving their quality.1,35 To our knowledge, a review focused on CPRs for child
PEDIATRICS Volume 128, Number 3, September 2011

health conditions has not yet been undertaken. We hypothesized that methodologic standards may need to be modied to meet the needs of child health practitioners, researchers, children, and parents because (1) severe outcomes for many pediatric conditions are uncommon, (2) uncertainty is inherent in communicating with and examining young children, and (3) parents and physicians place a particularly high value on not missing important diagnoses in children.811 The objectives of this study were to systematically identify existing studies of the derivation, validation, or impact of CPRs for health conditions of childhood; to evaluate their methodologic quality, performance, and rule validation by using current methodologic standards; to compare child CPRs with adult CPRs; and to suggest potential modications to those standards for CPRs developed for health conditions of childhood.

provides the probability of an out-

come or suggests a diagnostic or therapeutic course of action for an individual patient; and
is not a decision analysis, decision

support tool, or practice guideline. Only studies that involved children (from term birth to 18 years of age) were included. Studies that involved both adults and children were included if a separate analysis was performed for children. Studies that required the use of articial neural networks were not included. Selection of Studies Two reviewers (Drs Maguire and Kulik) independently assessed the inclusion of potentially relevant articles by using a 2-step process. First, the title and abstract from each article identied by the electronic search were assessed for inclusion. Second, when publications were identied as potentially relevant according to title and abstract or when uncertainty existed, the publications were reviewed manually. When there was discrepancy between the 2 reviewers, the studies were discussed and decisions were made by Drs Maguire and Kulik by consensus. Blinding of journal, institution, and author was not performed. Assessment of Methodologic Quality The quality of included studies was assessed by using 17 items from published guidelines for use in the development of CPRs.15 Each item was recorded as present (score of 1) or absent (score of 0), and the maximum total score was 17 (Table 1). Differences in opinion between reviewers (Drs Maguire and Kulik) were resolved by consensus. Given the importance of rule validation, hierarchy of rule validation was assessed separately (see next section).
e667

METHODS
Search Strategy We searched Medline and the EvidenceBased Medicine Reviews up to April 2011. Because there is no medical subject heading that species CPRs, an electronic search strategy was developed by an expert librarian (Ms Uleryk) on the basis of a previously validated strategy that was modied to exclude studies of only adults (see Appendix).12,13 In addition, the reference lists of identied CPRs were searched manually. There was no restriction on language. Inclusion Criteria Only prospective or retrospective studies that derived, validated, or assessed the impact of CPRs were included. A CPR was dened as a clinical decisionmaking tool that1,2,5:
includes 3 or more predictive vari-

ables obtained from the history, physical examination, or simple diagnostic tests;

Downloaded from pediatrics.aappublications.org at Indonesia:AAP Sponsored on May 30, 2013

TABLE 1 Assessment of Methodologic Quality

Quality Item Reports That Met Quality Item, % (n) Childrens Studies Adult Studiesa

All Recent Level 2 Studies Wasson et al3 Laupacis et al1 (19822011) (N 137) (20052011) (N 61) (19822011) (N 24) (19811984) (N 33) (19911994) (N 29) Prospective Study site well described Population well described Rule applied to all patients at risk Predictive variables Denition Blind assessment Reproducible Outcome variable Denition Blind assessment Adequate follow-up Sensibility Clinically sensible Easy to use Course of action Statistical analysis Mathematical technique reported Adequate power Adequate reporting of results 95% CIs reported on rule properties
a b

72 (98) 52 (71) 81 (11) 55 (76) 53 (72) 47 (65) 18 (24) 84 (115) 42 (58) 36 (49) 99 (135) 69 (94) 57 (78) 89 (122) 43 (59) 49 (67) 36 (49)

67 (41) 55 (33) 80 (49) 43 (26) 57 (35) 39 (24) 18 (11) 79 (48) 28 (17) 39 (24) 98 (60) 64 (39) 59 (36) 89 (54) 50 (30) 57 (35) 51 (31)

100 (23) 71 (17) 92 (22) 75 (18) 54 (13) 62 (15) 25 (6) 96 (23) 46 (11) 42 (10) 100 (24) 79 (19) 79 (19) 96 (23) 46 (11) 63 (15) 46 (11)

94 (32) 76 (25)
b

66 (19) 79 (23)
b

97 (32) 27 (3)
b

59 (17) 79 (23) 3 (1) 83 (24) 41 (12)

b

85 (28) 25 (3)
b

b b b

97 (28) 41 (12)
b

82 (23)
b b b

100 (29) 100 (29)

b

did not publish a detailed assessment of CPR methodologic quality. Reilly and Data not available.

Evans4

Assessment of Rule Performance We evaluated CPR performance by using sensitivity and negative likelihood ratio (LR), which have been used by others for evaluating CPRs.1,2,5,13,14 Sensitivity and LR are also independent of disease prevalence, which makes them useful measures for comparing CPRs from different populations with different outcomes.13,14 Data were extracted from each publication to construct a 2 2 table to calculate sensitivity, specicity, and LR and their 95% condence intervals (CIs).1,13 The following 4 rule-performance cutoffs were used to identify high-performing CPRs: sensitivity 0.95; lower limit of sensitivity 95% CI 0.95; LR 0.1; and upper limit of the LR 95% CI 0.1.1,2,5,13,1517 A sensitivity cutoff of 0.95 was chosen because it has been argued that few physicians would tolerate missing 5% of outcomes,5,6,17 and an LR of 0.1 was chosen because it is generally considered indicative of a clinically useful test.13,16
e668 MAGUIRE et al

Assessment of Hierarchy of Rule Validation Rule validation for CPRs that met inclusion criteria was assessed according to the hierarchy of evidence for CPRs published by the Evidence-Based Medicine Working Group (EBMWG).2 In this hierarchy, prediction rules that have been derived but not validated are the lowest level of evidence (level 4), rules that have been prospectively validated in only 1 sample are level 3, rules that have been broadly validated in multiple settings are level 2, and rules that have had impact analysis performed and demonstrated a change in clinician behavior with benecial consequences are level 1. Comparison With CPRs for Adult Health Conditions To compare CPRs for childhood health conditions with those for adults, methodologic quality indicators and hierarchy of rule validation abstracted through this review were compared

with those abstracted from adult CPRs published between 1981 and 2003 in 3 published reviews.1,3,4 Data Extraction Two reviewers (Drs Maguire and Kulik) independently used a standardized data-collection form to record methodologic quality indicators, performance, and hierarchy of rule validation for each study. Discrepancies between the reviewers were discussed and resolved by consensus. Summary statistics were calculated by using SAS 9.0 (SAS Institute, Inc Cary, NC).

RESULTS
Study Selection The electronic search strategy identied 7298 citations, which were screened by title and abstract to yield 392 potentially relevant studies. Review of the full text of these studies revealed 137 studies that fullled all inclusion criteria (Fig 1).11,18152

Downloaded from pediatrics.aappublications.org at Indonesia:AAP Sponsored on May 30, 2013

REVIEW ARTICLES

Titles and abstracts retrieved from electronic and bibliographical searches (N = 7298) Titles and abstracts that were not relevant, excluded (n = 6906) Titles and abstracts that were potentially relevant, extracted as full-text articles (n = 392) Full-text articles excluded (n = 255) Prognostic score (n = 108) <3 variables (n = 41) No separate analysis for children (n = 18) Not a CPR (n = 41) Review (n = 16) Other (n = 31) CPR studies included in the review (n = 137)

FIGURE 1
Flowchart for the inclusion of CPRs for health conditions of childhood.

quality item that was met most frequently was the clinical sensibility of the rule (99%). Quality items met in fewer than half of the studies included reproducibility of predictor variable assessment (18%), adequate follow-up to assess outcomes (36%), CIs reported on rule properties (36%), adequate blinding of outcomes (42%), sufcient study power (43%), adequate blinding of predictor variables (47%), and adequate reporting of results (49%). Of the 8 studies that met 16 of the 17 quality items,* 3 were level 2 studies (validated in multiple settings), and the remainder were level 4 studies (derived but not validated). Quality was higher for level 2 studies compared with level 3 and 4 studies (median: 12 vs 10 vs 9 quality items, respectively; P for trend .01). Assessment of Rule Performance The median sensitivity of the rules was 0.97 (range: 0.171.0), and the median lower 95% condence limit was 0.85 (range: 0.10 0.99). The median LR was 0.1 (range: 0.001 0.89), and the median upper 95% condence limit was 0.4 (range: 0.015.0). Sixty studies (48%) had a sensitivity of 0.95, and 13 of them (11%) had a lower 95% condence limit of 0.95. Fifty studies (43%) had an LR of 0.1, and 7 of them (6%) had an upper 95% condence limit of 0.1. Forty-four studies (35%) had both sensitivities of 95% and an LR of 0.1. Three studies (3%) had both a lower 95% condence limit for sensitivities of 0.95 and an upper 95% CI for LR of 0.1. Assessment of Hierarchy of Rule Validation Using the hierarchy described by the EBMWG2 to assess the degree of validation for the 101 CPRs, 76 rules (76%) were derived but not validated (level 4 evidence), 17 (17%) were prospec*Refs 11, 59, 60, 96, 98, 121, 124, and 134.

FIGURE 2
Number of CPRs for health conditions of childhood from 1982 through April 2011.

Characteristics of Included Studies Included studies were published between 1982 and 2011 (Fig 2) and involved a total of 285 404 children. No studies were identied before 1982. The reports on these studies described the development of 101 unique CPRs that address 36 pediatric conditions (Table 2). The most common conditions were occult serious bacterial infection in infants, streptococcal pharyngitis, bacterial meningitis, appendicitis, intracranial injury, extremity fractures, and malaria. The articles were published in a wide variety of journals including 4 general medical, 6 general pediatric, 6 emergency mediPEDIATRICS Volume 128, Number 3, September 2011

cine, 6 surgical, 2 family medicine, and 22 subspecialty journals. The median number of children enrolled for the derivation of each CPR was 324 (range: 37140 661). The median number of predictors assessed for possible inclusion was 10 (range: 3 84), and the median number of predictors included in the rules was 5 (range: 317). The median prevalence of the outcome being predicted by the rule was 0.25 (range: 0.006 [intracranial injury] to 0.75 [appendicitis]). Assessment of Methodologic Quality Studies met between 2 and 16 of the 17 quality items (Fig 3 and Table 1). The

Downloaded from pediatrics.aappublications.org at Indonesia:AAP Sponsored on May 30, 2013

e669

TABLE 2 Clinical Conditions for Which CPRs Have Been Developed for Children
Outcome Occult serious bacterial infection Streptococcal pharyngitis Bacterial meningitis Appendicitis Intracranial injury Extremity fracture Malaria Chest radiograph inltrate Septic joint Vesicoureteric reux Intra-abdominal injury Lyme meningitis Urinary tract infection Normal chest radiograph Inuenza Safe discharge from the emergency department Dehydration Uneventful course Pathologic diagnosis Pneumocystis pneumonia Persistent disease Undervaccination False-positive blood culture Emergency operative management Intrathoracic injury Cervical spine injury Difcult intravenous access Cervical infection Active rickets Tumor lysis syndrome Cervical infection HIV infection Pulmonary embolism Tuberculosis Pyloric stenosis Esophageal varices Population of Children Febrile infants Febrile neutropenia Sore throat Children at risk of meningitis Abdominal pain Head trauma Blunt ankle injury Fever in malaria-endemic region Suspected pneumonia Irritable joint Urinary tract infection Blunt abdominal trauma Meningitis Young girls with fever Respiratory syncytial virus infection Inuenza-like illness Bronchiolitis Vomiting or diarrhea Idiopathic thrombocytopenia Back pain HIV infection and pneumonia Graves disease Emergency department patients Children in the emergency department with blood culture taken Trauma Blunt torso trauma Trauma Children who require an intravenous line Adolescents who require pelvic exam Third-world children with leg deformity Leukemia Suspected pelvic inammatory disease Suspected HIV infection Suspected pulmonary embolism Suspected tuberculosis Suspected pyloric stenosis Chronic liver disease No. of Studies (N 137) 21 4 13 12 11 11 11 6 4 4 3 3 2 2 2 3 2 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

tively validated in 1 sample (level 3 evidence), 8 (8%) were validated in more than 1 setting (level 2 evidence), and no rule had undergone impact analysis (see Table 3 and Fig 4). Of the 8 level 2 rules, one was prospectively validated in 6 settings,39,66,73,84,98,124 one was prospectively validated in 5 settings,29,30,43,52,65 3 were prospectively validated in 4 settings, and 3 were prospectively validated in 2 settings.28,37,52,87,102,124 Comparison With CPRs for Adult Health Conditions Compared with CPRs for adult health conditions, the methodologic quality (Table 1) and hierarchy of rule validation (see Table 3 and Fig 4) qualitatively seem to be lower for CPRs for child health conditions.

DISCUSSION
We performed a systematic review to identify published CPRs for health conditions of childhood. One hundred thirty-seven study reports that described the development of 101 unique CPRs that addressed 36 childhood conditions were identied. The most intensively investigated conditions were acute infections and trauma, which are attractive candidates for improving patient care with a CPR because they are common, have the potential for poor outcomes, are prone to considerable clinical diagnostic uncertainty, and frequently lead to diagnostic testing or treatment that may be unnecessary or harmful. To evaluate the current state of CPRs for child health conditions, we described their methodologic quality, performance, and hierarchy of rule validation by using previously described guidelines for CPRs. The most important quality deciencies that affected the majority of studies were inadequate blinding of predictor vari Refs 34, 54, 77, 8183, 87, 95, 103, 107, and 121.

FIGURE 3
Number of CPRs for health conditions of childhood that included each number of quality items.

e670

MAGUIRE et al

Downloaded from pediatrics.aappublications.org at Indonesia:AAP Sponsored on May 30, 2013

REVIEW ARTICLES

ables and outcomes, limited assessment of the reproducibility of predictor variables, inadequate follow-up to assess outcomes, insufcient study power, inadequate reporting of results such as sensitivity and specicity, and lack of 95% CI reporting on rule properties. The methodologic quality measures used in the current study have been used previously to evaluate CPRs1,35,17 and are quite similar to validated items used for evaluating the quality of diagnostic tests.153155 Furthermore, there is considerable evidence that CPRs that fail to meet these methodologic standards are likely to be biased. Deciencies in blinding may result in an overestimation of diagnostic performance.16,153,155 Assessment of the interobserver reliability of predictor variables is necessary to determine if a rule will perform similarly when used by other physicians.1,5 Insufcient power for statistical modeling caused by an inadequate number of outcomes for a given number of predictor variables increases the possibility of spurious results through random effects.14,156,157 Incomplete follow-up for important outcomes can result in missed outcomes that can lead to an overestimation of diagnostic performance,5 which is particularly important for studies in which the reference standard for assessing outcomes, such as cranial computed-tomographic scanning for pediatric minor head injury, is not or cannot be applied to all patients, and clinical follow-up is used to capture outcomes.17 Lastly, inadequate reporting of CPR results makes it impossible for clinicians to know if the performance of a rule is adequate to meet their needs.1,3 We nd it concerning that 51% of the studies did not report sensitivity and specicity. These quality deciencies should be improved when developing CPRs for child health conditions in the future.
PEDIATRICS Volume 128, Number 3, September 2011

TABLE 3 Hierarchy of Rule Validation

Hierarchy of Rule Validation Rules, % (n) Child Rules, Present Study (19822011) (N 101)a 76 (76) 17 (17) 8 (8) 0 (0) Adult Rules Wasson et al (19811984)a (N 33) 56 (20) 28 (10) 11 (4) 6 (2)
3

Laupacis et al1 (19911994)a (N 29) 47 (15) 13 (4) 34 (11) 6 (2)

Reilly and Evans4 (20002003)a (N 41) 24 (10) 24 (10) 39 (16) 12 (5)

Level 4: derivation Level 3: narrow validation Level 2: broad validation Level 1: impact analysis
a

Year of publication of CPRs included in each review.

FIGURE 4
Percentage of CPRs for children and adults that met each level of the EBMWG hierarchy.

Thirty-ve percent of identied studies had both a CPR sensitivity of 0.95 and an LR of 0.1, but only 3% of the studies had 95% condence in both of these performance indicators. Although CPRs that are almost perfect may be desirable, it may not be attainable for pediatric clinical scenarios in which the history and physical examination are prone to interobserver variability (particularly common with very young children), the number of children with the disorder of interest is small, and the outcomes are rare. It may simply be impossible to recruit the tens of thousands of patients required to develop and then prospectively validate CPRs to this degree of precision for many child health conditions. Furthermore, attempts to achieve 95% sensitivity may come at a cost of overexposing children to harmful effects of diagnostic testing such as computed-tomographic scanning, which is associated with measurable

lifetime risk of lethal malignancy.158,159 Therefore, the challenges of attaining ideal rule performance must be reconciled with the realities of pediatric practice and the expectations of physicians and parents. Seeking 95% sensitivity for CPRs for child health conditions may be an elusive and counterproductive goal, especially when the sensitivity of a less-than-perfect CPR is superior to a clinicians judgment alone. As the eld of evidencebased medicine moves toward involving patients in decision-making,160,161 there may be opportunities to increase the interface between 2 areas of research: CPRs (aimed at assisting clinicians) and decision aids (aimed at assisting patients/parents in making choices that t with their values and preferences).162 We propose that, moving forward, CPRs for childhood health conditions be considered aids to clinical decision-making and not rigid rules. We suggest that CPRs for child
e671

Downloaded from pediatrics.aappublications.org at Indonesia:AAP Sponsored on May 30, 2013

health conditions aim to empower clinicians with data to augment clinical judgment and incorporate the values and preferences of parents and children in the decision-making process.11 CPRs for childhood health conditions seem to lag behind their adult counterparts in terms of the EBMWG hierarchy of rule validation.2 However, close inspection of similar rules for children and adults reveals that there are unique challenges to undertaking the derivation, validation, and impact analysis of CPRs for children. Two examples are illustrative. In the rst example, investigators studied CPRs for ankle injuries in both children and adults.39,73,98,163 Although the authors of these studies reported similar mechanisms, outcome rates, number of study sites, sample sizes, and ruleperformance characteristics, the studies in children took 2 to 3 times as long to complete as those for adults. In the second example, investigators studied CPRs for traumatic brain injury in both children and adults.11,164 Although the authors of those studies reported similar rule-performance characteristics, the study in children demonstrated 10-fold fewer outcomes, and required twice the number of study sites and study subjects as the study in adults. Of the 101 rules assessed by using the EBMWG hierarchy, only 8 rules for health conditions of childhood would be considered rules that can be used in various settings with condence in their accuracy (level 2 evidence) because of prospective validation in broad or multiple settings. The remainder would be considered rules that clinicians may use with caution (level 3 evidence) because of validation in only 1 narrow prospective sample or rules that require further evaluation (level 4 evidence) because they are not validated or validated only in split samples, large retrospective dae672 MAGUIRE et al

tabases, or by statistical techniques. However, close inspection of the 70 rules that the EBMWG has categorized as level 4 reveals a spectrum of validation methods, some of which may provide a level of evidence higher than level 4. Examples of these methods include (1) statistical techniques such as cross-validation, bootstrapping, classication, and regression-tree techniques,25,36,38,60 (2) split-set validation using a percentage of the data to derive the rule and the remainder to validate it,88 and (3) prospective validation using an extension of the derivation cohort in which the derivation set is closed and the rule is derived after enrolling an a priori number of outcomes followed by prospective enrollment of a validation cohort using the full set of predictor variables and clinicians blinded to the derived rule.11 Each of these techniques provides progressively superior validation but may all be considered level 4 evidence, equivalent to rules with no validation, according to the EBMWG hierarchy. Given the challenges involved with achieving the sample sizes needed to provide level 1 and 2 CPR validation for child health conditions, we suggest that these efcient approaches to validation be differentiated in future modications of the evidence-based medicine validation hierarchy for CPRs. It is important to acknowledge the limitations of this systematic review. First, our electronic search strategy may not have identied all CPRs for children. However, examination of the reference lists of all identied prediction-rule publications failed to reveal any additional studies. Second, the proposed quality and performance metrics we used treated each item with equal weight, and certain components may be more important than others. Third, although most CPRs maximized sensitivity at the expense of

specicity, this was not true for all rules. For the few that prioritized specicity over sensitivity, our sensitivity and LR performance benchmarks are not appropriate. Finally, although the items we used to assess methodologic quality and performance have been well described in the literature, they have not been rigorously developed or validated.15,13

CONCLUSIONS
High-performing, rigorously developed, and well-validated CPRs have the potential for improving child health outcomes and limiting resource use but are uncommonly developed and rarely used in pediatric practice. We have identied several important issues related to the quality, performance, and validation of CPRs for childhood health conditions that are barriers to their development and implementation. We have made several recommendations including modifying existing methodologic standards to include more efcient approaches to validation and considering CPRs for children to be more assistive and less directive. We also suggest that pediatric CPRs attempt to incorporate patients and parents values and preferences in the decision-making process, especially when the performance of high-quality rules is less than ideal but considerably better than a clinicians judgment alone. We hope that this review will assist developers and users of pediatric CPRs in overcoming these barriers and increase the use of CPRs in pediatric practice.

APPENDIX: ELECTRONIC SEARCH STRATEGY

With the assistance of Ms Uleryk (director of the Hospital for Sick Children Library), a comprehensive literature search was run by using the OVID search platform in Medline and the Evidence-Based Medicine Reviews from the beginning of the database un-

Downloaded from pediatrics.aappublications.org at Indonesia:AAP Sponsored on May 30, 2013

REVIEW ARTICLES

til April 2011. The following terms were searched by using specic database indexing and text-word equivalents to identify articles for review: (models, statistical/ or Monte Carlo method/ or probability/ or regression analysis/ or multivariate analysis/ or predict*.mp.) and (Decision Trees/ or predictive value of tests/ or ((decision: or predict:) adj5 (rule: or model: or algorithm: or aid or score:)).ti,ab.) and (limit to age groups birth to 18 years of age or pediatrics/) and (cohort studies/ or longitudinal studies/ or REFERENCES
1. Laupacis A, Sekar N, Stiell IG. Clinical prediction rules: a review and suggested modications of methodological standards. JAMA. 1997;277(6):488 494 2. McGinn TG, Guyatt GH, Wyer PC, Naylor CD, Stiell IG, Richardson WS. Users guides to the medical literature: XXII how to use articles about clinical decision rules. Evidence-Based Medicine Working Group. JAMA. 2000;284(1):79 84 3. Wasson JH, Sox HC, Neff RK, Goldman L. Clinical prediction rules: applications and methodological standards. N Engl J Med. 1985;313(13):793799 4. Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med. 2006;144(3):201209 5. Stiell IG, Wells GA. Methodologic standards for the development of clinical decision rules in emergency medicine. Ann Emerg Med. 1999;33(4):437 447 6. Stiell IG, Lesiuk H, Wells GA, et al; Canadian CT Head and C-Spine Study Group. The Canadian CT Head Rule Study for patients with minor head injury: rationale, objectives, and methodology for phase I (derivation). Ann Emerg Med. 2001;38(2):160 169 7. Stiell IG, Lesiuk H, Wells GA, et al; Canadian CT Head and C-Spine Study Group. Canadian CT Head Rule Study for patients with minor head injury: methodology for phase II (validation and economic analysis). Ann Emerg Med. 2001;38(3):317322 8. Bennett JE, Sumner W 2nd, Downs SM, Jaffe DM. Parents utilities for outcomes of occult bacteremia. Arch Pediatr Adolesc Med. 2000;154(1):43 48 9. Apkon M. Decision analysis and clinical uncertainty. Curr Opin Pediatr. 2003;15(3): 272277

follow-up studies/ or prospective studies/ or prognosis/ or disease-free survival/ or treatment outcome/ or treatment failure/ or disease progression/ or morbidity/ or incidence/ or prevalence/ or mortality/ or cause of death/ or fatal outcome/ or hospital mortality/ or infant mortality/ or maternal mortality/ or survival rate/ or survival analysis/ or disease-free survival/ or natural history.tw. or evaluation studies.pt. or evaluation studies as topic/ or validation studies.pt. or validation studies as topic/ sensitivity and spec-

icity/ or predictive value of tests/ or ROC curve/ or diagnostic errors/ or false negative reactions/ or false positive reactions/ or observer variation/ or likelihood functions/ or (likelihood or likelihood ratio:).tw.

ACKNOWLEDGMENTS
The Pediatric Outcomes Research Team is supported by a grant from the Hospital for Sick Children Foundation. Dr Maguire was supported by a Canadian Institutes of Health Research fellowship.

10. Brennan CA, Somerset M, Granier SK, Fahey TP, Heyderman RS. Management of diagnostic uncertainty in children with possible meningitis: a qualitative study. Br J Gen Pract. 2003;53(493):626 631 11. Kuppermann N, Holmes JF, Dayan PS, et al; Pediatric Emergency Care Applied Research Network (PECARN). Identication of children at very low risk of clinicallyimportant brain injuries after head trauma: a prospective cohort study. Lancet. 2009;374(9696):1160 1170 12. Ingui BJ, Rogers MA. Searching for clinical prediction rules in Medline. J Am Med Inform Assoc. 2001;8(4):391397 13. Hess EP, Thiruganasambandamoorthy V, Wells GA, et al. Diagnostic accuracy of clinical prediction rules to exclude acute coronary syndrome in the emergency department setting: a systematic review. CJEM. 2008;10(4):373382 14. Concato J, Feinstein AR, Holford TR. The risk of determining risk with multivariable models. Ann Intern Med . 1993;118(3): 201210 15. Stiell IG, McKnight RD, Greenberg GH, et al. Implementation of the Ottawa ankle rules. JAMA. 1994;271(11):827 832 16. Jaeschke R, Guyatt G, Lijmer J. Diagnostic tests. In: Guyatt G, Rennie D, Meade MO, Cook DJ, eds. Users Guides to the Medical Literature. Chicago, IL: AMA Press; 2002: 121140 17. Maguire JL, Boutis K, Uleryk EM, Laupacis A, Parkin PC. Should a head-injured child receive a head CT scan? A systematic review of clinical prediction rules. Pediatrics. 2009;124(1). Available at: www.pediatrics.org/cgi/content/full/ 124/1/e145 18. Haydel MJ, Shembekar AD. Prediction of intracranial injury in children aged ve

19.

20.

21.

22.

23.

24.

25.

26.

years and older with loss of consciousness after minor head injury due to nontrivial mechanisms. Ann Emerg Med. 2003; 42(4):507514 Friedman JN, Goldman RD, Srivastava R, Parkin PC. Development of a clinical dehydration scale for use in children between 1 and 36 months of age. J Pediatr. 2004; 145(2):201207 Da Dalt L, Marchi AG, Laudizi L, et al. Predictors of intracranial injuries in children after blunt head trauma. Eur J Pediatr. 2006;165(3):142148 Chandramohan D, Carneiro I, Kavishwar A, Brugha R, Desai V, Greenwood B. A clinical algorithm for the diagnosis of malaria: results of an evaluation in an area of low endemicity. Trop Med Int Health. 2001;6(7): 505510 Attia M, Zaoutis T, Eppes S, Klein J, Meier F. Multivariate predictive models for group A beta-hemolytic streptococcal pharyngitis in children. Acad Emerg Med. 1999;6(1): 8 13 Attia MW, Zaoutis T, Klein JD, Meier FA. Performance of a predictive model for streptococcal pharyngitis in children [published correction appears in Arch Pediatr Adolesc Med. 2001;155(10):1179]. Arch Pediatr Adolesc Med. 2001;155(6):687 691 Ammann RA, Hirt A, Luthy AR, Aebi C. Identication of children presenting with fever in chemotherapy-induced neutropenia at low risk for severe bacterial infection. Med Pediatr Oncol. 2003;41(5):436 443 Atabaki S, Steill IG, Bazarian JJ, et al. A clinical decision rule for cranial computed tomography in minor pediatric head trauma. Arch Pediatr Adolesc Med. 2008; 162(5):439 445 Avery RA, Frank G, Glutting JJ, et al. Predic-

PEDIATRICS Volume 128, Number 3, September 2011

Downloaded from pediatrics.aappublications.org at Indonesia:AAP Sponsored on May 30, 2013

e673

27.

28.

29.

30.

31.

32.

33.

34.

35.

36.

37.

tion of Lyme meningitis in children from a Lyme disease endemic region: a logisticregression model using history, physical, and laboratory ndings. Pediatrics. 2006; 117(1). Available at: www.pediatrics.org/ cgi/content/full/117/1/e1 Bachur RG, Harper MB. Predictive model for serious bacterial infections among infants younger than 3 months of age. Pediatrics. 2001;108(2):311316 Baker MD, Bell LM, Avner JR. Outpatient management without antibiotics of fever in selected infants. N Engl J Med. 1993; 329(20):14371441 Baskin MN, ORourke EJ, Fleisher GR. Outpatient treatment of febrile infants 28 to 89 days of age with intramuscular administration of ceftriaxone. J Pediatr. 1992; 120(1):2227 Berger RM, Berger MY, van Steensel-Moll HA, Dzoljic-Danilovic G, Derksen-Lubsen G. A predictive model to estimate the risk of serious bacterial infections in febrile infants. Eur J Pediatr. 1996;155(6):468 473 Bleeker SE, Moons KG, Derksen-Lubsen G, Grobbee DE, Moll HA. Predicting serious bacterial infection in young children with fever without apparent source. Acta Paediatr. 2001;90(11):1226 1232 Bleeker SE, Moll HA, Steyerberg EW, et al. External validation is necessary in prediction research: a clinical example. J Clin Epidemiol. 2003;56(9):826 832 Bleeker SE, Derksen-Lubsen G, Grobbee DE, et al. Validating and updating a prediction rule for serious bacterial infection in patients with fever without source. Acta Paediatr. 2007;96(1):100 104 Bond GR, Tully SB, Chan LS, Bradley RL. Use of the MANTRELS score in childhood appendicitis: a prospective study of 187 children with abdominal pain. Ann Emerg Med. 1990;19(9):1014 1018 Bonsu BK, Harper MB. Differentiating acute bacterial meningitis from acute viral meningitis among children with cerebrospinal uid pleocytosis: a multivariable regression model. Pediatr Infect Dis J. 2004;23(6):511517 Bonsu BK, Ortega HW, Marcon MJ, Harper MB. A decision rule for predicting bacterial meningitis in children with cerebrospinal uid pleocytosis when Gram stain is negative or unavailable. Acad Emerg Med. 2008;15(5):437 444 Boutis K, Komar L, Jaramillo D, et al. Sensitivity of a clinical examination to predict need for radiography in children with ankle injuries: a prospective study. Lancet. 2001;358(9299):2118 2121

38. Chavanet P, Schaller C, Levy C, et al. Performance of a predictive rule to distinguish bacterial and viral meningitis. J Infect. 2007;54(4):328 336 39. Clark KD, Tanner S. Evaluation of the Ottawa ankle rules in children. Pediatr Emerg Care. 2003;19(2):7378 40. Cotton BA, Beckert BW, Smith MK, Burd RS. The utility of clinical and laboratory data for predicting intraabdominal injury among children. J Trauma. 2004;56(5): 1068 1074; discussion 1074 1065 41. Crain EF, Bulas D, Bijur PE, Goldman HS. Is a chest radiograph necessary in the evaluation of every febrile infant less than 8 weeks of age? Pediatrics . 1991;88(4): 821 824 42. Dado G, Anania G, Baccarani U, et al. Application of a clinical score for the diagnosis of acute appendicitis in childhood: a retrospective analysis of 197 patients. J Pediatr Surg. 2000;35(9):1320 1322 43. Dagan R, Sofer S, Phillip M, Shachak E. Ambulatory care of febrile infants younger than 2 months of age classied as being at low risk for having serious bacterial infections. J Pediatr. 1988;112(3):355360 44. Dayan PS, Vitale M, Langsam DJ, et al. Derivation of clinical prediction rules to identify children with fractures after twisting injuries of the ankle. Acad Emerg Med. 2004;11(7):736 743 45. Dubos F, Lamotte B, Bibi-Triki F, et al. Clinical decision rules to distinguish between bacterial and aseptic meningitis. Arch Dis Child. 2006;91(8):647 650 46. Dunning J, Daly JP, Lomas JP, et al. Derivation of the childrens head injury algorithm for the prediction of important clinical events decision rule for head injury in children. Arch Dis Child . 2006;91(11): 885 891 47. Edslev PW, Rosthoj S, Treutiger I, et al. A clinical score predicting a brief and uneventful course of newly diagnosed idiopathic thrombocytopenic purpura in children. Br J Haematol. 2007;138(4):513516 48. Fatti GL, Zar HJ, Swingler GH, Fatti GL, Zar HJ, Swingler GH. Clinical indicators of Pneumocystis jiroveci pneumonia (PCP) in South African children infected with the human immunodeciency virus. Int J Infect Dis. 2006;10(4):282285 49. Feldman DS, Straight JJ, Badra MI, Mohaideen A, Madan SS. Evaluation of an algorithmic approach to pediatric back pain. J Pediatr Orthop. 2006;26(3):353357 50. Fine AM, Nigrovic LE, Reis BY, Cook EF, Mandl KD. Linking surveillance to action: incorporation of real-time regional data

51.

52.

53.

54.

55.

56.

57.

58.

59.

60.

61.

62.

into a medical decision rule. J Am Med Inform Assoc. 2007;14(2):206 211 Steinhoff MC, Walker CF, Rimoin AW, Hamza HS. A clinical decision rule for management of streptococcal pharyngitis in lowresource settings. Acta Paediatr. 2005; 94(8):1038 1042 Garra G, Cunningham SJ, Crain EF. Reappraisal of criteria used to predict serious bacterial illness in febrile infants less than 8 weeks of age. Acad Emerg Med. 2005; 12(10):921925 Glaser NS, Styne DM; Organization of Pediatric Endocrinologists of Northern California Collaborative Graves Disease Study Group. Predicting the likelihood of remission in children with Graves disease: a prospective, multicenter study. Pediatrics. 2008;121(3). Available at: www.pediatrics. org/cgi/content/full/121/3/e481 Goldman RD, Carter S, Stephens D, Antoon R, Mounstephen W, Langer JC. Prospective validation of the pediatric appendicitis score. J Pediatr. 2008;153(2):278 282 Gorelick MH, Shaw KN, Murphy KO. Validity and reliability of clinical signs in the diagnosis of dehydration in children. Pediatrics. 1997;99(5). Available at: www. pediatrics.org/cgi/content/full/99/5/e6 Gorelick MH, Shaw KN. Clinical decision rule to identify febrile young girls at risk for urinary tract infection. Arch Pediatr Adolesc Med. 2000;154(4):386 390 Gorelick MH, Hoberman A, Kearney D, Wald E, Shaw KN. Validation of a decision rule identifying febrile young girls at high risk for urinary tract infection. Pediatr Emerg Care. 2003;19(3):162164 Greenes DS, Schutzman SA. Clinical significance of scalp abnormalities in asymptomatic head-injured infants. Pediatr Emerg Care. 2001;17(2):88 92 Holmes JF, Sokolove PE, Brant WE, Kuppermann N. A clinical decision rule for identifying children with thoracic injuries after blunt torso trauma. Ann Emerg Med. 2002; 39(5):492 499 Holmes JF, Sokolove PE, Brant WE, et al. Identication of children with intraabdominal injuries after blunt trauma. Ann Emerg Med. 2002;39(5):500 509 Lynch T, Platt R, Gouin S, Larson C, Patenaude Y. Can we predict which children with clinically suspected pneumonia will have the presence of focal inltrates on chest radiographs? Pediatrics. 2004;113(3 pt 1). Available at: www.pediatrics.org/ cgi/content/full/113/3/e186 Humiston SG, Rodewald LE, Szilagyi PG, et al. Decision rules for predicting vaccina-

e674

MAGUIRE et al

Downloaded from pediatrics.aappublications.org at Indonesia:AAP Sponsored on May 30, 2013

REVIEW ARTICLES

63.

64.

65.

66.

67.

68.

69.

70.

71.

72.

73.

74.

tion status of preschool-age emergency department patients. J Pediatr . 1993; 123(6):887 892 Isaacman DJ, Shults J, Gross TK, Davis PH, Harper M. Predictors of bacteremia in febrile children 3 to 36 months of age. Pediatrics. 2000;106(5):977982 Jaffe DM, Binns H, Radkowski MA. Developing a clinical algorithm for early management of cervical spine injury in child trauma victims. Ann Emerg Med. 1987; 16(3):270 276 Jaskiewicz JA, McCarthy CA, Richardson AC, et al; Febrile Infant Collaborative Study Group. Febrile infants at low risk for serious bacterial infection: an appraisal of the Rochester criteria and implications for management. Pediatrics . 1994;94(3): 390 396 Karpas A, Hennes H, Walsh-Kelly CM. Utilization of the Ottawa ankle rules by nurses in a pediatric emergency department. Acad Emerg Med. 2002;9(2):130 133 Kharbanda AB, Taylor GA, Fishman SJ, Bachur RG. A clinical decision rule to identify children at low risk for appendicitis. Pediatrics. 2005;116(3):709 716 Kneyber MC, Moons KG, de Groot R, Moll HA. Predictors of a normal chest x-ray in respiratory syncytial virus infection. Pediatr Pulmonol. 2001;31(4):277283 Kneyber MCJ, Moons KGM, de Groot R, Moll HA. Prediction of duration of hospitalization in respiratory syncytial virus infection. Pediatr Pulmonol . 2002;33(6): 453 457 Kocher MS, Zurakowski D, Kasser JR. Differentiating between septic arthritis and transient synovitis of the hip in children: an evidence-based clinical prediction algorithm. J Bone Joint Surg Am. 1999; 81(12):16621670 Kocher MS, Mandiga R, Zurakowski D, Barnewolt C, Kasser JR. Validation of a clinical prediction rule for the differentiation between septic arthritis and transient synovitis of the hip in children. J Bone Joint Surg Am. 2004;86-A(8):1629 1635 Leroy S, Marc E, Adamsbaum C, Gendrel D, Breart G, Chalumeau M. Prediction of vesicoureteral reux after a rst febrile urinary tract infection in children: validation of a clinical decision rule. Arch Dis Child. 2006;91(3):241244 Libetta C, Burke D, Brennan P, Yassa J. Validation of the Ottawa ankle rules in children. J Accid Emerg Med . 1999;16(5): 342344 Lintula H, Pesonen E, Kokki H, Vanamo K, Eskelinen M. A diagnostic score for chil-

75.

76.

77.

78.

79.

80.

81.

82.

83.

84.

85.

86.

87.

dren with suspected appendicitis. Langenbecks Arch Surg. 2005;390(2):164 170 Luhmann SJ, Jones A, Schootman M, et al. Differentiation between septic arthritis and transient synovitis of the hip in children with clinical prediction algorithms. J Bone Joint Surg Am. 2004;86-A(5):956 962 Biss TT, Brandao LR, Kahr WHA, Chan AKC, Williams S. Clinical probability score and D-dimer estimation lack utility in the diagnosis of childhood pulmonary embolism. J Thromb Haemost. 2009;7(10):16331638 Macklin CP, Radcliffe GS, Merei JM, Stringer MD. A prospective evaluation of the modied Alvarado score for acute appendicitis in children. Ann R Coll Surg Engl. 1997;79(3):203205 Mansbach JM, Clark S, Christopher NC, et al. Prospective multicenter study of bronchiolitis: predicting safe discharges from the emergency department. Pediatrics. 2008;121(4):680 688 McCarthy PL, Sharpe MR, Spiesel SZ, et al. Observation scales to identify serious illness in febrile children. Pediatrics. 1982; 70(5):802 809 McConnochie KM, Roghmann KJ, Pasternack J, Monroe DJ, Monaco LP. Prediction rules for selective radiographic assessment of extremity injuries in children and adolescents. Pediatrics. 1990;86(1):4557 McIsaac WJ, Goel V, To T, Low DE. The validity of a sore throat score in family practice. CMAJ. 2000;163(7):811 815 McIsaac WJ, Kellner JD, Aufricht P, et al. Empirical validation of guidelines for the management of pharyngitis in children and adults [published correction appears in JAMA. 2005;294(21):2700]. JAMA. 2004; 291(13):15871595 McIsaac WJ, White D, Tannenbaum D, Low DE. A clinical score to reduce unnecessary antibiotic use in patients with sore throat. CMAJ. 1998;158(1):75 83 McBride KL. Validation of the Ottawa ankle rules: experience at a community hospital. Can Fam Physician. 1997;43(PubMed did not provide):459 465 Meehan WP 3rd, Bachur RG. Predictors of cerebrospinal uid pleocytosis in febrile infants aged 0 to 90 days. Pediatr Emerg Care. 2008;24(5):287293 Moreno L, Krishnan JA, Duran P, Ferrero F. Development and validation of a clinical prediction rule to distinguish bacterial from viral pneumonia in children [published correction appears in Pediatr Pulmonol. 2006;41(5):494]. Pediatr Pulmonol. 2006;41(4):331337 Fischer Walker CL, Rimoin AW, Hamza HS,

88.

89.

90.

91.

92.

93.

94.

95.

96.

97.

98.

99.

Steinhoff MC. Comparison of clinical prediction rules for management of pharyngitis in settings with limited resources. J Pediatr. 2006;149(1):64 71 Nigrovic LE, Kuppermann N, Malley R. Development and validation of a multivariable predictive model to distinguish bacterial from aseptic meningitis in children in the postHaemophilus inuenzae era. Pediatrics. 2002;110(4):712719 Nigrovic LE, Kuppermann N, Macias CG, et al. Clinical prediction rule for identifying children with cerebrospinal uid pleocytosis at very low risk of bacterial meningitis. JAMA. 2007;297(1):52 60 Oman JA, Cooper RJ, Holmes JF, et al; NEXUS II Investigators. Performance of a decision rule to predict need for computed tomography among children with blunt head trauma. Pediatrics . 2006; 117(2). Available at: www.pediatrics.org/ cgi/content/full/117/2/e238 Oostenbrink R, van der Heijden AJ, Moons KG, Moll HA. Prediction of vesico-ureteric reux in childhood urinary tract infection: a multivariate approach. Acta Paediatr. 2000;89(7):806 810 Oostenbrink R, Moons KG, Donders AR, Grobbee DE, Moll HA. Prediction of bacterial meningitis in children with meningeal signs: reduction of lumbar punctures. Acta Paediatr. 2001;90(6):611 617 Oostenbrink R, Moons KG, Twijnstra MJ, Grobbee DE, Moll HA. Children with meningeal signs: predicting who needs empiric antibiotic treatment. Arch Pediatr Adolesc Med. 2002;156(12):1189 1194 Oostenbrink R, Moons KG, Derksen-Lubsen AG, Grobbee DE, Moll HA. A diagnostic decision rule for management of children with meningeal signs. Eur J Epidemiol. 2004; 19(2):109 116 Owen TD, Williams H, Stiff G, Jenkinson LR, Rees BI. Evaluation of the Alvarado score in acute appendicitis. J R Soc Med. 1992; 85(2):87 88 Palchak MJ, Holmes JF, Vance CW, et al. A decision rule for identifying children at low risk for brain injuries after blunt head trauma. Ann Emerg Med . 2003;42(4): 492506 Pantell RH, Newman TB, Bernzweig J, et al. Management and outcomes of care of fever in early infancy. JAMA. 2004;291(10): 12031212 Plint AC, Bulloch B, Osmond MH, et al. Validation of the Ottawa ankle rules in children with ankle injuries. Acad Emerg Med. 1999;6(10):10051009 Redd SC, Kazembe PN, Luby SP, et al. Clini-

PEDIATRICS Volume 128, Number 3, September 2011

Downloaded from pediatrics.aappublications.org at Indonesia:AAP Sponsored on May 30, 2013

e675

cal algorithm for treatment of Plasmodium falciparum malaria in children. Lancet. 1996;347(8996):223227 100. Reed JL, Mahabee-Gittens EM, Huppert JS. A decision rule to identify adolescent females with cervical infections. J Womens Health (Larchmt). 2007;16(2):272280 101. Rietveld E, Vergouwe Y, Steyerberg EW, et al. Hospitalization for respiratory syncytial virus infection in young children: development of a clinical prediction rule. Pediatr Infect Dis J. 2006;25(3):201207 102. Rimoin AW, Hamza HS, Vince A, et al. Evaluation of the WHO clinical decision rule for streptococcal pharyngitis. Arch Dis Child. 2005;90(10):1066 1070 103. Samuel M. Pediatric appendicitis score. J Pediatr Surg. 2002;37(6):877 881 104. Santolaya ME, Alvarez AM, Becker A, et al. Prospective, multicenter evaluation of risk factors associated with invasive bacterial infection in children with cancer, neutropenia, and fever. J Clin Oncol. 2001;19(14): 34153421 105. Santolaya ME, Alvarez AM, Aviles CL, et al. Prospective evaluation of a model of prediction of invasive bacterial infection risk among children with cancer, fever, and neutropenia. Clin Infect Dis. 2002;35(6): 678 683 106. Sard B, Bailey MC, Vinci R. An analysis of pediatric blood cultures in the postpneumococcal conjugate vaccine era in a community hospital emergency department. Pediatr Emerg Care. 2006;22(5):295300 107. Schneider C, Kharbanda A, Bachur R. Evaluating appendicitis scoring systems using a prospective pediatric cohort. Ann Emerg Med. 2007;49(6):778 784 108. Silva JMP, Diniz JSS, Lima EM, Vergara RM, Oliveira EA. Predictive factors of resolution of primary vesico-ureteric reux: a multivariate analysis. BJU Int . 2006;97(5): 10631068 109. Smeesters PR, Campos D Jr, Van Melderen L, de Aguiar E, Vanderpas J, Vergison A. Pharyngitis in low-resources settings: a pragmatic clinical approach to reduce unnecessary antibiotic use. Pediatrics. 2006; 118(6). Available at: www.pediatrics.org/ cgi/content/full/118/6/e1607 110. Steele R, Green SM, Gill M, Coba V, Oh B. Clinical decision rules for secondary trauma triage: predictors of emergency operative management. Ann Emerg Med. 2006;47(2):135 111. Steinhoff MC, Abd el Khalek MK, Khallaf N, et al. Effectiveness of clinical guidelines for the presumptive treatment of strepto-

coccal pharyngitis in Egyptian children. Lancet. 1997;350(9082):918 921 112. Sun BC, Hoffman JR, Mower WR. Evaluation of a modied prediction instrument to identify signicant pediatric intracranial injury after blunt head trauma. Ann Emerg Med. 2007;49(3):325332, 332.e321 113. Thacher TD, Fischer PR, Pettifor JM. The usefulness of clinical features to identify active rickets. Ann Trop Paediatr. 2002; 22(3):229 237 114. Van den Bruel A, Aertgeerts B, Bruyninckx R, Aerts M, Buntinx F. Signs and symptoms for diagnosis of serious infections in children: a prospective study in primary care. Br J Gen Pract . 2007;57(540): 538 546 115. van den Broek WT, van der Ende ED, Bijnen AB, Breslau PJ, Gouma DJ. Which children could benet from additional diagnostic tools in case of suspected appendicitis? J Pediatr Surg. 2004;39(4):570 574 116. Webster AP, Goodacre S, Walker D, Burke D. How do clinical features help identify paediatric patients with fractures following blunt wrist trauma? Emerg Med J. 2006; 23(5):354 357 117. Young Infants Clinical Signs Study Group. Clinical signs that predict severe illness in children under age 2 months: a multicentre study. Lancet. 2008;371(9607):135142 118. Yen K, Riegert A, Gorelick MH. Derivation of the DIVA score: a clinical prediction rule for the identication of children with difcult intravenous access. Pediatr Emerg Care. 2008;24(3):143147 119. Ayoola OO, Adeyemo AA, Osinusi K. Predictors of bacteraemia among febrile infants in Ibadan, Nigeria. J Health Popul Nutr. 2002;20(3):223229 120. Baqui AH, Arifeen SE, Rosen HE, et al. Community-based validation of assessment of newborn illnesses by trained community health workers in Sylhet district of Bangladesh. Trop Med Int Health. 2009; 14(12):1448 1456 121. Bhatt M, Joseph L, Ducharme FM, Dougherty G, McGillivray D. Prospective validation of the pediatric appendicitis score in a Canadian pediatric emergency department. Acad Emerg Med . 2009;16(7): 591596 122. Bonadio WA, Hagen E, Rucka J, Shallow K, Stommel P, Smith D. Efcacy of a protocol to distinguish risk of serious bacterial infection in the outpatient evaluation of febrile young infants. Clin Pediatr (Phila). 1993;32(7):401 404 123. Friedman MJ, Attia MW. Clinical predictors

of inuenza in children. Arch Pediatr Adolesc Med. 2004;158(4):391394 124. Gravel J, Hedrei P, Grimard G, Gouin S. Prospective validation and head-to-head comparison of 3 ankle rules in a pediatric population. Ann Emerg Med . 2009;54(4): 534 540.e531 125. Holmes JF, Mao A, Awasthi S, McGahan JP, Wisner DH, Kuppermann N. Validation of a prediction rule for the identication of children with intra-abdominal injuries after blunt torso trauma. Ann Emerg Med. 2009;54(4):528 533 126. Horwood C, Liebeschuetz S, Blaauw D, Cassol S, Qazi S. Diagnosis of paediatric HIV infection in a primary health care setting with a clinical algorithm. Bull World Health Org. 2003;81(12):858 866 127. Kuppermann N, Fleisher GR, Jaffe DM. Predictors of occult pneumococcal bacteremia in young febrile children. Ann Emerg Med. 1998;31(6):679 687 128. Lacour AG, Zamora SA, Gervaix A. A score identifying serious bacterial infections in children with fever without source. Pediatr Infect Dis J. 2008;27(7):654 656 129. Muhe L, Oljira B, Degefu H, Enquesellassie F, Weber MW. Clinical algorithm for malaria during low and high transmission seasons. Arch Dis Child . 1999;81(3): 216 220 130. Mwangi TW, Mohammed M, Dayo H, Snow RW, Marsh K. Clinical algorithms for malaria diagnosis lack utility among people of different age groups. Trop Med Int Health. 2005;10(6):530 536 131. Nandi S, Kumar R, Ray P, Vohra H, Ganguly NK. Clinical score card for diagnosis of group A streptococcal sore throat. Indian J Pediatr. 2002;69(6):471 475 132. Narayan S, Mahadevan S, Serane VT. Keith Edwards score for diagnosis of tuberculosis. Indian J Pediatr. 2003;70(6):467 469 133. Olaleye BO, Williams LA, DAlessandro U, et al. Clinical predictors of malaria in Gambian children with fever or a history of fever. Trans R Soc Trop Med Hyg. 1998;92(3): 300 304 134. Osmond MH, Klassen TP, Wells GA, et al; Pediatric Emergency Research Canada (PERC) Head Injury Study Group. CATCH: a clinical decision rule for the use of computed tomography in children with minor head injury. CMAJ. 182(4):341348 135. van der Schouw YT, van der Velden MT, Hitge-Boetes C, Verbeek AL, Ruijs SH. Diagnosis of hypertrophic pyloric stenosis: value of sonography when used in conjunction with clinical ndings and labora-

e676

MAGUIRE et al

Downloaded from pediatrics.aappublications.org at Indonesia:AAP Sponsored on May 30, 2013

REVIEW ARTICLES

136.

137.

138.

139.

140.

141.

142.

143.

144.

tory data. AJR Am J Roentgenol. 1994; 163(4):905909 Slap GB, Forke CM, Cnaan A, et al. Recognition of tubo-ovarian abscess in adolescents with pelvic inammatory disease. J Adolesc Health. 1996;18(6):397 403 Truong TH, Beyene J, Hitzler J, et al. Features at presentation predict children with acute lymphoblastic leukemia at low risk for tumor lysis syndrome. Cancer. 2007;110(8):18321839 Tuerlinckx D, Bodart E, Jamart J, Glupczynski Y. Prediction of Lyme meningitis based on a logistic regression model using clinical and cerebrospinal uid analysis: a European study. Pediatr Infect Dis J. 2009; 28(5):394 397 Viccellio P, Simon H, Pressman BD, et al; NEXUS Group. A prospective multicenter study of cervical spine injury in children. Pediatrics. 2001;108(2). Available at: www. pediatrics.org/cgi/content/full/108/2/e20 Wald ER, Green MD, Schwartz B, Barbadora K. A streptococcal score card revisited. Pediatr Emerg Care. 1998;14(2):109 111 Yacoub S, Mohammed MJ, Ramsan M, Albonico M. Clinical predictors of malaria and other febrile illnesses in children under ve on Pemba Island, Tanzania. Trop Doct. 2005;35(2):78 81 Yilmaz HL, Yildizdas RD, Alparslan N, Ozcan K, Yaman A, Kibar F. Screening tools for bacteraemia in a selected population of febrile children. Ann Acad Med Singapore. 2008;37(3):192199 Bailey B, Gravel J, Goldman RD, Friedman JN, Parkin PC. External validation of the clinical dehydration scale for children with acute gastroenteritis. Acad Emerg Med. 2010;17(6):583588 Joachim L, Campos D Jr, Smeesters PR. Pragmatic scoring system for pharyngitis in low-resource settings. Pediatrics. 2010; 126(3). Available at: www.pediatrics.org/ cgi/content/full/126/3/e608

145. Gana JC, Turner D, Roberts EA, Ling SC. Derivation of a clinical prediction rule for the noninvasive diagnosis of varices in children. J Pediatr Gastroenterol Nutr. 2010; 50(2):188 193 146. Bilkis MD, Gorgal N, Carbone M, et al. Validation and development of a clinical prediction rule in clinically suspected community-acquired pneumonia. Pediatr Emerg Care. 2010;26(6):399 405 147. Anwar M, Ahmad A, Ahmad F, Mazhar A. Modied Kenneth Jones criteria for diagnosing tuberculous meningitis in children. J Coll Physicians Surg Pak. 2010;20(4): 258 261 148. Bin SS, Schutzman SA, Greenes DS. Validation of a clinical score to predict skull fracture in head-injured infants. Pediatr Emerg Care. 2010;26(9):633 639 149. Dubos F, Korczowski B, Aygun DA, et al. Distinguishing between bacterial and aseptic meningitis in children: European comparison of two clinical decision rules. Arch Dis Child. 2010;95(12):963967 150. Prasarnphanich T. The accuracy of clinical diagnosis of inuenza in Thai children. J Pediatr Infect Dis. 2010;5(2):155159 151. Houben ML, Bont L, Wilbrink B, et al. Clinical prediction rule for RSV bronchiolitis in healthy newborns: prognostic birth cohort study. Pediatrics. 2011;127(1):35 41 152. Sultan J, Hughes PJ. Septic arthritis or transient synovitis of the hip in children: the value of clinical prediction algorithms. J Bone Joint Surg Br . 2010;92(9): 1289 1293 153. Whiting P, Rutjes AW, Reitsma JB, Bossuyt PM, Kleijnen J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol. 2003;3:25 154. Whiting PF, Weswood ME, Rutjes AW, Reitsma JB, Bossuyt PN, Kleijnen J. Evaluation of QUADAS, a tool for the quality assess-

ment of diagnostic accuracy studies. BMC Med Res Methodol. 2006;6:9 155. Bossuyt PM, Reitsma JB, Bruns DE, et al. Toward complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Acad Radiol. 2003;10(6): 664 669 156. Moss M, Wellman DA, Cotsonis GA. An appraisal of multivariable logistic models in the pulmonary and critical care literature. Chest. 2003;123(3):923928 157. Ciampi A, Chang CH, Hogg S, McKinney S. Recursive partitioning: a versatile method for exploratory data analysis in biostatistics. In: MacNeill I, Umphrey G, eds. Time Series and Econometric Modelling/ Biostatistics. Boston, MA: D. Reidel Publishing Co; 1987:2350 158. Brenner DJ, Hall EJ. Computed tomography: an increasing source of radiation exposure. N Engl J Med. 2007;357(22):2277 2284 159. Smith-Bindman R. Is computed tomography safe? N Engl J Med. 2010;363(1):1 4 160. Guyatt G, Cook D, Haynes B. Evidence based medicine has come a long way. BMJ. 2004; 329(7473):990 991 161. Kon A. The shared decision-making continuum. JAMA. 2010;304(8):903 162. OConnor AM, Legare F, Stacey D. Risk communication in practice: the contribution of decision aids. BMJ . 2003;327(7417): 736 740 163. Stiell IG, Greenberg GH, McKnight RD, Nair RC, McDowell I, Worthington JR. A study to develop clinical decision rules for the use of radiography in acute ankle injuries. Ann Emerg Med. 1992;21(4):384 390 164. Stiell IG, Wells GA, Vandemheen K, et al. The Canadian CT head rule for patients with minor head injury. Lancet. 2001;357(9266): 13911396

PEDIATRICS Volume 128, Number 3, September 2011

Downloaded from pediatrics.aappublications.org at Indonesia:AAP Sponsored on May 30, 2013

e677

Clinical Prediction Rules for Children: A Systematic Review Jonathon L. Maguire, Dina M. Kulik, Andreas Laupacis, Nathan Kuppermann, Elizabeth M. Uleryk and Patricia C. Parkin Pediatrics 2011;128;e666; originally published online August 22, 2011; DOI: 10.1542/peds.2011-0043
Updated Information & Services References including high resolution figures, can be found at: http://pediatrics.aappublications.org/content/128/3/e666.full.h tml This article cites 152 articles, 30 of which can be accessed free at: http://pediatrics.aappublications.org/content/128/3/e666.full.h tml#ref-list-1 This article has been cited by 5 HighWire-hosted articles: http://pediatrics.aappublications.org/content/128/3/e666.full.h tml#related-urls This article, along with others on similar topics, appears in the following collection(s): Office Practice http://pediatrics.aappublications.org/cgi/collection/office_pra ctice Information about reproducing this article in parts (figures, tables) or in its entirety can be found online at: http://pediatrics.aappublications.org/site/misc/Permissions.xht ml Information about ordering reprints can be found online: http://pediatrics.aappublications.org/site/misc/reprints.xhtml

Citations

Subspecialty Collections

Permissions & Licensing

Reprints

PEDIATRICS is the official journal of the American Academy of Pediatrics. A monthly publication, it has been published continuously since 1948. PEDIATRICS is owned, published, and trademarked by the American Academy of Pediatrics, 141 Northwest Point Boulevard, Elk Grove Village, Illinois, 60007. Copyright 2011 by the American Academy of Pediatrics. All rights reserved. Print ISSN: 0031-4005. Online ISSN: 1098-4275.

Downloaded from pediatrics.aappublications.org at Indonesia:AAP Sponsored on May 30, 2013