JOURNAL OF TROPICAL PEDIATRICS, VOL. 57, NO.

1, 2011

Validation of Pediatric Index of Mortality-2 Scoring System

in a Pediatric Intensive Care Unit, Barbados
by Seetharaman Hariharan,1 Kandamaran Krishnamurthy,2 and Dionne Grannum3
1
Department of Clinical Surgical Sciences, The University of the West Indies, St. Augustine, Trinidad & Tobago
2
Department of Paediatrics, The University of the West Indies, Cave Hill Campus, Barbados
3
Department of Paediatrics, Queen Elizabeth Hospital, Barbados

Downloaded from https://academic.oup.com/tropej/article-abstract/57/1/9/1723622 by guest on 17 October 2019

Correspondence: Seetharaman Hariharan, Department of Clinical Surgical Sciences, The University of the West Indies,
St. Augustine, Trinidad & Tobago, West Indies. Tel/Fax: þ1 868 662 4030; E-mail: <uwi.hariharan@gmail.com>.

Summary
This study evaluated the outcome of patients in a pediatric intensive care unit (ICU) of a developing
country applying Pediatric Index of Mortality (PIM) version-2 scoring system. A total of 163 consecu-
tive patients were prospectively studied. Data included demographics, diagnoses at admission, PIM-2
score, the duration of ICU stay and hospital outcome. Predicted mortality and standardized mortality
ratio (SMR) were calculated. Respiratory and neurological illnesses were the main admission diagnoses.
The mean length of stay was 5.4 [95% Confidence Intervals (CI): 4–6.9] days. The mean predicted
mortality was 6.2% (95% CI: 4.3–8.1); the observed mortality rate was 5.5%, the SMR being 0.89.
Hosmer–Lemeshow analysis calibrated PIM-2 for the case mix [2 ¼ 5.64 (df ¼ 7), p ¼ 0.58]. The area
under the ROC curve was 0.82 (95% CI: 0.72–0.92) showing a good discriminant function.
Performance of the pediatric ICU in Barbados is comparable to that of developed world by
risk-adjusted outcome evaluation.

Key words: intensive care unit, performance, developing country.

Introduction score implying that the scoring system is usually

Many illness severity scoring systems are being used
for predicting the outcome of patients admitted to
intensive care units (ICUs) in terms of morbidity
and mortality [1]. Although it is difficult to predict
individual outcome of ICU patients accurately, there
have been attempts to codify and validate models,
which may prognosticate groups of patients having
similar presentation of the illness [2]. Scoring systems
are primarily being used to predict the general prog-
nosis of patients, but are additionally being used as
performance indicators of ICUs [3]. This is usually
accomplished by calculating the risk-adjusted mortal-
ity for a particular unit and comparing it with that of
another [4]. Pediatric Risk of Mortality-version III
(PRISM III), is one of the popular scoring systems
used in pediatric ICUs throughout the world [5–7].
Pediatric Logistic Organ Dysfunction (PELOD) is a
scoring system specifically used in patients with
multi-organ system failure in ICUs [8].
Pediatric Index of Mortality (PIM) was introduced
by Shann et al. [9] in 1997 to predict outcome in
children admitted to ICUs. Like most other prognos-
tic scoring systems, this system was also updated to
PIM-2, which has been better than the earlier version
in outcome predictability [10]. PIM-2 is an admission
applied during the time of admission, before any
ICU intervention is undertaken. This is different to
other scoring systems, which could be applied at dif-
ferent time-intervals throughout the ICU stay.
International comparisons of intensive care out-
comes are important because critical care delivery
patterns and resource consumption may vary consid-
erably in various regions [11]. This is especially true
in many developing nations. There has been a many
studies comparing the performance of prognostic
scoring systems in the pediatric ICUs of different re-
gions [12–14]; but there is still a need for more work
comparing different regions of the world [15].
With this background, we conducted this prospect-
ive study in a pediatric ICU, to determine the value
of PIM-2 scoring system in predicting the outcome of
our case-mix, and also assessing the performance of
the unit to compare it with international reports.
Methods
Hospital and ICU setting
Barbados is an island nation of the English-speaking
Caribbean, with a population of 250 000. The Queen
Elizabeth Hospital, Barbados, is a 500-bed tertiary

ß The Author [2010]. Published by Oxford University Press. All rights reserved. For Permissions, please email: journals.permissions@oup.com 9
doi:10.1093/tropej/fmq031 Advance Access published on 12 May 2010
 S. HARIHARAN ET AL.

care centre, affiliated to the University of the West 35

Indies. The pediatric ICU in QEH is a 4-bed open
unit, admitting pediatric patients predominantly be- 30
longing to medical specialty but also a few surgical
25

Percentage (%)
cases. Patients get admitted from Emergency
Department directly and from the general wards. A 20
Senior Registrar in pediatric medicine takes care of
the unit around the clock under the supervision of a 15
Consultant. The nurse–patient ratio is 1:1. The
Radiology department and the Pathology, 10
Microbiology laboratories of the hospital have

Downloaded from https://academic.oup.com/tropej/article-abstract/57/1/9/1723622 by guest on 17 October 2019

state-of-the-art equipment facilitating a wide variety 5
of investigations. Majority of the patients admitted
0
to the unit have invasive lines, which include arterial

e p ers
es

ts

s
di rs
es
es

es

er
ien
ss
and central venous lines. The unit has facilities for

e
ss
ss

as
d

th
io ord
Po ogic llne

ra isor
ne
ne

at

se

O
blood gas analyses, portable radiograph and

ill
ill

s
ol ar i

di
d
l
ry

us
iv
ca

al

ic
ultrasound.

ul
to

ol
gi

sc

ct
ra

ab
o

pe
Approval of Ethics Committee was obtained prior

va
l

fe
ro
pi

et
sto

In
o

at
es

eu

M
di
to conducting the study. All patients consecutively

m
R

ar

ae
C
admitted to the pediatric ICU over a period of

1 year from January 2008 to December 2008 were
enrolled for prospective collection of data. The
demographic data recorded were the age and
gender of the patients. The diagnoses on admission
were noted. PIM-2 scoring system was applied on the
day of admission to all the patients before any thera-
peutic intervention was undertaken. Patients were
followed up throughout their stay in ICU and
during the entire hospital stay to record their final
outcome. The length of stay (LOS) in ICU was re-
corded. The hospital outcome was classified into
either ‘Discharged’ or ‘Died’.
The regression equation published with PIM-2
scoring system was used to calculate the predicted
mortality in the ICU patients.
Mann–Whitney U-test was used to compare
PIM-2 scores and LOS between survivors and
non-survivors. Hosmer–Lemeshow goodness-of-fit
analysis was done to calibrate the scoring system.
Receiver Operating Characteristic curve (ROC) ana-
lyses were done to analyze the discriminant function
of the system. Standardized mortality ratio (SMR)
(ratio of the observed to the predicted mortality
rate) was obtained for the case mix. The statistical
significance was fixed at p < 0.05 level. Statistical
analyses were done using the Statistical Package for
Social Sciences (SPSSÕ ), (version 12) (Chicago, IL,
USA) software.
Results
During the period of study, 163 patients were
admitted to the ICU. The median age of the patients
was 5 years, 1–8 [Interquartile ranges, (IQR)]. Of
these 163 patients, 94 were males (57.7%).
Respiratory illnesses such as bronchiolitis and pneu-
monia were the most common diagnoses on admis-
sion followed by neurological and cardiovascular
H
FIG. 1. Diagnostic categories.
illnesses. The denominations of the diagnostic cate-
gories are shown in Fig. 1.
Overall, the patients stayed in the ICU ranging
from 1 to 92 days and the mean LOS was 5.4 [95%
Confidence Intervals (CI): 4–6.9] days. 84% of the
patients had mechanical ventilatory support during
their stay in ICU.
The predicted mortality according to the PIM2
score ranged from 0 to 87.8% with a mean of
6.2% (95% CI: 4.3–8.1). The observed mortality
rate was 5.5%, the SMR being 0.89. Table 1 shows
the age, LOS and the outcomes in all the patients
studied. Hosmer–Lemeshow Chi-square analyses
for goodness-of-fit for PIM-2 system showed a
good calibration of the model to our case-mix [H–L
2 ¼ 5.64 (df ¼ 7), p ¼ 0.58]. The distribution of the
deciles of risk in our case-mix has been depicted in
Table 2.
Figure 2 shows the ROC curve for PIM-2 scoring
system. The area under the curve (AUC) for the
ROC curve for PIM-2 system was 0.82 (95% CI:
0.72–0.92).
Discussion
The major finding of the present study is the reason-
ably good performance of the PIM-2 scoring system
in a pediatric ICU of a Caribbean developing
country.
There have been many published reports compar-
ing several different versions of pediatric prognostic
scoring systems such as PIM, PRISM and PELOD
[5, 7, 8]. Some reports found PIM to be performing
well in pediatric ICUs compared to other systems

10 Journal of Tropical Pediatrics Vol. 57, No. 1

S. HARIHARAN ET AL.

TABLE 1
Comparison of survivors and non-survivors

Variable Overall Survivors Non-survivors Significance*,

(n ¼ 163) (n ¼ 154) (n ¼ 9) p-value

Age, mean  SD (years) 4.9  4.4 4.9  4.3 4.7  5.0 0.43
Gender (n) (%)
Male 94 (57.7) 90 (58.4) 4 (44.4)
Female 69 (42.3) 64 (41.6) 5 (55.6)
LOS, mean  SD 5.4  9.2 5.2  8.7 9.1  14.9 0.32
Predicted mortality, mean  SD (%) 6.2  12.1 5.4  10.3 20  26.5 0.001

Downloaded from https://academic.oup.com/tropej/article-abstract/57/1/9/1723622 by guest on 17 October 2019

*By Mann–Whitney U-test.

TABLE 2
Hosmer-Lemeshow goodness-of-fit analysis for PIM 2

Groups Survivors Non-survivors Total (n ¼ 163)

(Predicted mortality)
Observed (n ¼ 154) Expected Observed (n ¼ 9) Expected

1 17 16.4 0 0.6 17
2 29 27.9 0 1.1 29
3 12 11.6 0 0.4 12
4 19 18.3 0 0.7 19
5 15 15.4 1 0.6 16
6 15 15.3 1 0.7 16
7 14 15.2 2 0.8 16
8 15 15.1 1 0.9 16
9 18 18.7 4 3.3 22

Hosmer–Lemeshow 2 ¼ 5.64 (df ¼ 7), p ¼ 0.58.

1.0 The overall age range, diagnostic categories and

observed mortality in the present study was mostly
comparable to those reported from many parts of the
0.8 world [16–18]. A study from Trinidad, a neighboring
Caribbean country reported that neonates were also
admitted in the multidisciplinary ICU because of the
policy of the neonatal ICUs, which do not usually
Sensitivity

0.6
admit neonates who had been delivered elsewhere
and/or transferred from other neonatal units [18].
0.4 However, the pediatric ICU in Barbados does not
admit neonates.
The LOS of patients varied widely, although there
0.2 was no statistical significant difference between sur-
vivors and non-survivors. LOS in pediatric ICU has
been found to be an important predictor of quality,
0.0 enabling the comparison of resource utilization
0.0 0.2 0.4 0.6 0.8 1.0 among different institutions [19]. Organizational fac-
1 - Specificity tors known to foster team-oriented care are asso-
ciated with shorter LOS. Also the size of the
FIG. 2. ROC curve for PIM-2. pediatric ICU is an important factor in that a rela-
tively larger size pediatric ICU may keep pediatric
ICU beds occupied longer [19]. Our pediatric ICU
[12, 13, 16, 17]. PIM is also better in terms of its is 4 bedded that would have contributed to the rela-
simplicity, validity, discriminatory power, etc. tively shorter mean LOS. Despite this, there were
Hence, we applied the current version of PIM in patients who stayed for months, due to the unavail-
our unit. ability of a high-dependency unit in the hospital.

Journal of Tropical Pediatrics Vol. 57, No. 1 11


Calibration of a prognostic model is done by test-
ing ‘the degree of correspondence between the prob-
abilities of the outcome as predicted by the score and
the observed frequency of the outcome, at different
levels of probability’. A good calibration is tradition-
ally represented by a p-value > 0.1 [20]. Calibration
of a prognostic model is usually done by Hosmer–
Lemeshow goodness-of-fit test, which compares the
predicted and observed outcomes in subgroups of
patients belonging to deciles of risk. The PIM-2 scor-
ing system when introduced, calibrated well for risk
deciles but poorly for diagnostic groups. The authors
who designed the PIM-2 scoring system have
acknowledged the fact that the system may not per-
form well in different environments. Although there
may be a need of changing the coefficients used for
the regression equation to suit individual needs, the
authors have opined against this alteration, since
they feel it may defeat the purpose of the model [6].
There have been many reports from different parts
of the world calibrating PIM2. A report from
Netherlands calibrated the model and found to be
acceptable [2 ¼ 4.92 (df ¼ 8), p ¼ 0.77] [12]. A study
from Pakistan applied PIM2 and found that it cali-
brated well (2 ¼ 9.65, p ¼ 0.29) [14]. The study from
Trinidad had 2 ¼ 5.61 (df ¼ 8), p ¼ 0.69, pointing to
good calibration [18]. Also in Spain, the model cali-
brated well [2 ¼ 4.87 (df ¼ 8), p ¼ 0.85] [21]. The pre-
sent study also found that PIM2 calibrated well to
the case-mix of Barbados.
Discriminant analysis of a model determines its
ability to categorize patients into two outcome
groups such as survivors and non-survivors. ROC
curve analysis is the most commonly applied meth-
odology to determine the discriminatory ability of a
prognostic scoring system. Conventionally, an AUC
of >0.7 is considered acceptable, >0.8 is good and
>0.9 is excellent. The AUC for the ROC for PIM2
has been reported from many countries. It was 0.9
from Australia and New Zealand [8], 0.88 from
Pakistan, another developing country, which was
similar to a report from Spain (0.87) [14, 21]; our
AUC is similar to that reported from India (0.81)
[10]. The AUC of the ROC curve reported from
Trinidad was quite low (0.62) [18].
The SMR is a well-recognized and valid measure
for comparing the risk-adjusted mortality between
different centers, although it may vary between re-
gions according to the case-mix as well as the care
offered. Additionally, the SMR may be useful as a
comparative measure between different scoring sys-
tems applied in the same setting. The study from
Australia and New Zealand reported an SMR of
0.97 [8] and the SMRs reported from developing
countries such as India and Pakistan were 1.57 and
1.4, respectively [10, 14], implying underprediction of
mortality. The SMR of the present study is 0.89 was
similar to that reported from Trinidad (0.86), which
could be considered better than that reported from
12
S. HARIHARAN ET AL.
developed countries [18]. Although this may be inter-
preted that the unit is performing well, it may be dif-
ficult to arrive at such conclusion because like the
Trinidad study, there could have been overprediction
of mortality. There is an opinion that performance
evaluation of ICUs by prognostic models may have
errors since many units evaluated this way qualify
with ‘honors’ due to ‘grade inflation’ [22].
Our study has few limitations. The number of pa-
tients studied may be relatively smaller to calibrate
applying goodness-of fit analysis, although this has
Downloaded from https://academic.oup.com/tropej/article-abstract/57/1/9/1723622 by guest on 17 October 2019
been done in many other studies with similar sample
sizes. Outcome evaluation in critically ill pediatric
patients is a challenging task. It has been well-
recognized that this area has not been addressed ad-
equately, which needs plenty of research in the future
[23]. Rating the quality of ICUs applying a
severity-of-illness scoring system is also highly con-
troversial because performance of a unit is multidi-
mensional [24, 25].
Nevertheless, the present study found that
PIM2 scoring system calibrated well and had a rea-
sonable discriminatory ability when applied to the
case-mix in a PICU in Barbados. Thus, it could be
used as a beneficial tool for evaluation of risk-
adjusted mortality and comparison of units within
the region.
References
1. Hariharan S, Zbar A. Risk scoring and prediction of
outcomes in perioperative and surgical intensive care
patients – a review. Curr Surg 2006;63:226–36.
2. Hariharan S. Prognostication of critically ill patients –
The ‘science’, the ‘art’ and the creeping commercialism.
West Indian Med J 2007;56:467–9.
3. Zimmerman JE. Measuring intensive care performance:
a way to move forward. Crit Care Med 2002;30:
2149–50.
4. Becker RB, Zimmerman JE. ICU Scoring systems allow
prediction of patient outcomes or comparison of ICU
performance. Crit Care Clin 1996;12:503–14.
5. Gemke RJ, van Vught J. Scoring systems in pediatric
intensive care: PRISM III versus PIM. Intensive Care
Med 2002;28:204–7.
6. Thukral A, Lodha R, Irshad M, et al. Performance of
Pediatric Risk of Mortality (PRISM), Pediatric Index
of Mortality (PIM), and PIM2 in a pediatric intensive
care unit in a developing country. Pediatr Crit Care
Med 2006;7:356–61.
7. Bilan N, Galehgolab BA, Emadaddin A, et al. Risk of
mortality in pediatric intensive care unit, assessed by
PRISM-III. Pak J Biol Sci 2009;12:480–5.
8. Qureshi AU, Ali AS, Ahmad TM. Comparison of
three prognostic scores (PRISM, PELOD and
PIM 2) at pediatric intensive care unit under Pakistani
circumstances. J Ayub Med Coll Abbottabad 2007;19:
49–53.
9. Shann F, Pearson G, Slater A, et al. Paediatric index of
mortality (PIM): a mortality prediction model for chil-
dren in intensive care. Intensive Care Med 1997;23:
201–7.
Journal of Tropical Pediatrics Vol. 57, No. 1
S. HARIHARAN ET AL.
10. Slater A, Shann F, Pearson G. Paediatric Index of
Mortality (PIM) Study Group. PIM 2: a revised version
of the Paediatric Index of Mortality. Intensive Care
Med 2003;29:278–85.
11. Angus DC, Sirio CA, Clermont G, et al. International
comparisons of critical care outcome and resource con-
sumption. Crit Care Clin 1997;13:389–407.
12. Slater A, Shann F. ANZICS Paediatric Study Group.
The suitability of the Pediatric Index of Mortality
(PIM), PIM2, the Pediatric Risk of Mortality
(PRISM), and PRISM III for monitoring the quality
of pediatric intensive care in Australia and New
Zealand. Pediatr Crit Care Med 2004;5:447–54.
13. Lacroix J, Cotting J. for the Pediatric Acute Lung
Injury and Sepsis Investigators (PALISI) Network.
Severity of illness and organ dysfunction scoring in chil-
dren. Pediatr Crit Care Med 2005;6:S126–34.
14. Sachdeva RC, Guntupalli KK. International compari-
sons of outcomes in intensive care units. Crit Care Med
1999;27:2032–3.
15. van Keulen JG, Polderman KH, Gemke RJ. Reliability
of PRISM and PIM scores in paediatric intensive care.
Arch Dis Child 2005;90:211–4.
16. Tibby SM, Taylor D, Festa M, et al. A comparison of
three scoring systems for mortality risk among retrieved
intensive care patients. Arch Dis Child 2002;87:421–5.
17. Ozer EA, Kizilgunesler A, Sarioglu B, et al. The com-
parison of PRISM and PIM scoring systems for
Journal of Tropical Pediatrics Vol. 57, No. 1
mortality risk in infantile intensive care. J Trop
Pediatr 2004;50:334–8.
18. Hariharan S, Chen D, Merritt-Charles L. Risk-adjusted
outcome evaluation in a multidisciplinary intensive care
unit. West Indian Med J 2007;56:228–33.
19. Ruttimann UE, Pollack MM. Variability in duration of
stay in pediatric intensive care units: a multi-institu-
tional study. J Pediatr 1996;128:35–44.
20. Tibby SM, Murdoch IA. Calibration of the paediatric
index of mortality score for UK paediatric intensive
care. Arch Dis Child 2002;86:65.
21. Prieto Espuñes S, López-Herce Cid J, Rey Galán C,
Downloaded from https://academic.oup.com/tropej/article-abstract/57/1/9/1723622 by guest on 17 October 2019
et al. Prognostic indexes of mortality in pediatric inten-
sive care units. An Pediatr (Barc) 2007;66:345–50.
22. Popovich MJ. If most intensive care units are graduat-
ing with honors, is it genuine quality or grade inflation?
Crit Care Med 2002;30:2145–6.
23. Forrest CB, Shipman SA, Dougherty D, et al.
Outcomes research in pediatric settings: recent trends
and future directions. Pediatrics 2003;111:171–8.
24. Glance LG, Osler TM, Dick A. Rating the quality of
intensive care units: is it a function of the intensive care
scoring system? Crit Care Med 2002;30:1976–82.
25. Hariharan S, Dey PK, Chen D, et al. Analytic hierarchy
process for measuring and comparing the global perfor-
mance of intensive care units. J Crit Care 2005;20:
117–25.
13