. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
RESEARCH Original article
Q
The effect of ICU telemedicine on mortality
and length of stay
Benjamin A Kohl*, Margaret Fortino-Mullen

, Amy Praestgaard

,
C William Hanson*, Joseph DiMartino

and E Andrew Ochroch*

*Department of Anesthesiology and Critical Care, Perelman School of Medicine, University of Pennsylvania, Philadelphia, USA;

Penn eLert
Telemedicine Program, University of Pennsylvania Health System, Philadelphia, USA;

Department of Biostatistics and Epidemiology,
Perelman School of Medicine, University of Pennsylvania, Philadelphia, USA
Summary
We conducted a retrospective, observational study of patient outcomes in two intensive care units in the same hospital.
The surgical ICU (SICU) implemented telemedicine and electronic medical records, while the medical ICU (MICU) did
not. Medical charts were reviewed for a one-year period before telemedicine and a one-year period afterwards. In the
SICU, records were obtained for 246 patients before and 1499 patients after implementation; in the MICU, records
were obtained for 220 patients and 285 patients in the same periods. The outcomes of interest were ICU length of
stay and mortality, and hospital length of stay and mortality. Outcome variables were severity-adjusted using APACHE
scoring. A bootstrap method, with 1000 replicates, was used to assess stability of the ndings. The adjusted ICU length
of stay, ICU mortality, and hospital mortality for the SICU patients all decreased signicantly after the implementation
of telemedicine. There was no change in adjusted outcome variables in the MICU patients. Implementation of
telemedicine and electronic records in the surgical ICU was associated with a profound reduction in severity-adjusted
ICU length of stay, ICU mortality, and hospital mortality. However, it is not possible to conclude denitively that the
observed associations seen in the SICU were due to the intervention.
Introduction
Telemedicine has been practised in intensive care units
(ICUs) in the US for at least 20 years and there has been an
increase in the number of centres using telemedicine in the
last 10 years.
1
Unfortunately, studies evaluating the impact
of remote ICU care on morbidity and mortality have not
been conclusive.
25
Most ICU telemedicine studies have
employed a single-centre historical control design,
comparing outcomes after implementing telemedicine with
outcomes from the same ICU before the use of
telemedicine.
68
Such a design cannot control for potential
confounding variables such as changes in the healthcare
system. As a result, it is difcult to know whether observed
changes are due to random variation, patient selection,
changes in stafng structures, new medicines or technology,
or the implementation of novel quality/safety initiatives
2,9
While it is difcult, although not impossible, to conduct
randomized, double-blind, placebo-controlled studies with
a telemedicine intervention, there are a number of
techniques which can reduce confounding effects, such as
patient matching, stratication and/or propensity
analysis.
10,11
In a previous study, a comparison of pre-and post
implementation data in our surgical ICU (SICU), suggested
that the introduction of telemedicine was associated with
reductions in mortality and length of stay, in addition to
major cost savings for the health system.
12,13
However, the
telemedicine programme was implemented in tandem with
other hospital-wide quality improvement initiatives, such
as a hand hygiene campaign and the use of care bundles to
prevent deep vein thrombosis, ventilator associated
pneumonia and central line infections. The aim of the
present study was to compare mortality and length of stay
changes over time between the SICU (a unit with
telemedicine services) and a medical intensive care unit
(MICU), a unit without telemedicine services at the same
hospital.
Telemedicine
The telemedicine system was installed in the SICU in
November 2004. The software included an electronic
medical record system and videoconferencing (VISICU
Accepted 21 March 2012
Correspondence: Dr Benjamin A Kohl, Department of Anesthesiology and Critical
Care, Perelman School of Medicine, University of Pennsylvania 3400 Spruce
Street, Founders 5, SICU Administration, Philadelphia PA 19104, USA
(Fax: 1 215 614 0350; Email: Benjamin.Kohl@uphs.upenn.edu)
Journal of Telemedicine and Telecare 2012; 18: 282286 DOI: 10.1258/jtt.2012.120208
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
eICU remote monitoring system, Phillips Electronics,
Amsterdam, The Netherlands). This enabled the provision
of critical care services from an offsite central monitoring
facility.
14
The system includes two-way audio conferencing,
one-way video conferencing (i.e. the telemedicine team can
view activity in the patients room by means of a remotely
controlled camera), an electronic medical record available
to both the telemedicine and bedside clinicians, and
continuous physiological monitoring that can detect trends
in vital signs and laboratory values as well as alert the
telemedicine staff if these numbers deviate from pre-dened
limits. Using this system, a small number of physicians and
critical care nurses assist the bedside care team from a
remote location, known as the Clinical Operations Room.
The remote ICU team consults on critical issues, monitors
patients for physiological deterioration and facilitates
communication between care providers. In addition,
telemedicine physicians have access to all radiology
examinations and continuous telemetry data. Details of all
ICU admissions are entered into the system by either the
telemedicine nurse or a trained data coordinator. At the
time of the present study, the telemedicine programme was
responsible for covering a total of ve intensive care units
(69 beds) in three hospitals. The SICU analysed for the
present study was the largest (24 beds) unit covered by
telemedicine.
Two care providers in the Clinical Operations Room
monitor ICU patients 24 h/day, seven days a week. During
daytime hours (07:0019:00) there are two ICU nurses, and
in the evening (19:0007:00) there is one physician
(intensivist) and one ICU nurse. The telemedicine nurses
perform audits for benchmarking of outcomes, review
patient proles for updates in the plans of care and respond
to clinical alarms and enquiries. In addition, they evaluate
and intervene on patient safety measures (e.g. redirecting a
delirious patient who is attempting to get out of bed) and
ensure compliance with best-care practices. Rounds involve
the evaluation of all new patient data and
videoconferencing into the patient room, and are
completed every 14 h based on need. Updates are
communicated by the day telemedicine nurse to the
incoming physician and nurse. The physicians
communicate frequently with the bedside team and are able
to assist as necessary. A button in each SICU room can be
pressed to alert the remote intensivist that there is a request
to assist with an emergency and they should activate the
camera. In addition, the tele-ICU team frequently initiate
contact with the ICU staff if there is a particular concern
regarding patient status or if they believe there should be a
change in management. Either party may call the other by
telephone to discuss any matters of concern more privately.
All telemedicine physicians are credentialled, but their
non-telemedicine clinical duties are entirely at the Hospital
of the University of Pennsylvania.
The stafng paradigms (i.e. standard of care) within the
MICU and SICU are similar but not identical. The MICU
follows a closed intensivist stafng model that is covered
overnight solely by residents. Both the attending
intensivist and the pulmonary fellow (training in critical
care) are available on-call at home. In-house coverage
overnight is provided by residents, in addition to a fellow
(training in critical care). The attending SICU intensivist is
available on-call at home. Implementation of the
telemedicine programme did not affect either stafng
paradigm. The MICU does not use an electronic medical
record.
Methods
We performed a retrospective, observational study using
medical chart review of patients in the SICU and MICU at
the Hospital of the University of Pennsylvania. The study
was approved by the appropriate ethics committee.
Specially trained critical care nurses conducted chart
reviews and extracted ICU admission day information.
APACHE scoring was performed using data from the rst
24 h of ICU admission.
15
The range for APACHE scores is
0299, higher scores indicating more severe illness. For the
pre-implementation phase, a list of all patients admitted to
the MICU and SICU between April 2003 and March 2004
was obtained. In general, the MICU patients were older and
had a greater number of co-morbidities when compared
with SICU patients (thus explaining their greater APACHE
scores). A minimum of 65 consecutive charts for each
quarter were reviewed for data abstraction to ensure the
availability of information needed to calculate the APACHE
scores. Pre-implementation records were selected at the
same starting point and were chosen as consecutive
admissions over a period of one year.
People trained in quality assurance performed audits of
abstracted data on 10% of the records to ensure accurate
data collection. Records that did not provide the necessary
data to permit APACHE calculation were excluded from the
study and the next admission was then reviewed. This
process resulted in 246 SICU patients and 220 MICU
patients for analysis in the pre-implementation phase.
Implementation of telemedicine and electronic medical
records occurred in the SICU in November 2004.
Post-implementation data were collected for the two units
from July 2005 to June 2006 and were collected in the
MICU with the same methodology described above,
resulting in 285 patients for analysis. Post-implementation
data were collected on all SICU patients admitted during the
12-month period via the newly implemented electronic
medical record (n 2100). Readmissions and off-service
patients (primary neurosurgical or primary cardiac surgical
patients) were excluded from analysis. In addition, records
that were either incomplete or otherwise ineligible for
APACHE calculation were excluded. This resulted in a total
of 1499 SICU patients for the post-implementation phase.
Four outcomes were considered in the statistical analysis:
hospital and ICU length of stay, and hospital and ICU
mortality.
B A Kohl et al. ICU telemedicine
Journal of Telemedicine and Telecare Volume 18 Number 5 2012 283
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Standard packages were used for the statistical analysis
(STATA 11, StataCorp, TX, USA and SAS 9.1, SAS Institute
Inc. Cary, NC, USA). Mean APACHE III scores were
compared between ICUs by analysis of variance (ANOVA).
Analyses of covariance (ANCOVA) were used to compare pre
and post tele-ICU implementation with hospital length of
stay, using the APACHE III score as a covariate. A similar
ANCOVA model was tted for the ICU length of stay
outcome. Pre- to post-implementation changes in hospital
and ICU mortality outcomes were analysed with logistic
regression. Finally, logistic regression was used to compare
the pre- and post-telemedicine implementation with
changes in ICU mortality. A bootstrap method, with 1000
replicates, was used to assess stability of the ndings.
Results
Severity of illness
On average, MICU patients were more ill than SICU patients
(P , 0.0001), as indicated by higher APACHE scores, see
Table 1. While the mean MICU APACHE score decreased
from pre- to post-implementation (indicating that the
severity of illness decreased), the mean SICU APACHE score
increased from pre- to post-implementation (indicating that
the severity of illness increased). The ANOVA indicated
that both of these changes were signicant (P , 0.0001 and
P 0.005 for MICU and SICU, respectively), and that the
pre- to post-rates of change in the two units were
signicantly different from each other (ANOVA interaction
P , 0.0001).
Hospital length of stay and mortality
The unadjusted and severity-adjusted hospital length of stay
and mortality results for both ICUs pre- and post-
telemedicine implementation are summarised in Table 1.
Unadjusted and severity-adjusted hospital length of stay
decreased for both the MICU and SICU, although neither
change was signicant. Similarly, the MICU and SICU pre-
to post-rates of change were not signicantly different from
each other. Unadjusted hospital mortality in the MICU
population decreased signicantly. However this difference
was not signicant after adjusting for severity of illness
(P 0.24). Hospital mortality in the SICU, decreased
signicantly after telemedicine was implemented, both in
the unadjusted analysis as well as when adjusted for severity
of illness (0.13 to 0.04, OR 0.30, P 0.023). The MICU and
SICU model-adjusted rates of change were not signicantly
different from each other.
ICU length of stay and mortality
The unadjusted and severity-adjusted length of stay and
mortality results for both ICUs pre- and post- telemedicine
implementation are summarised in Table 2. Unadjusted
(4.9 to 5.9 d, P 0.08) and severity adjusted (5.3 to 6.1 d,
P 0.62) ICU length of stay both increased in the MICU
after telemedicine; however neither of these ndings were
signicant. In contrast, both unadjusted (5.0 to 3.3d,
P , 0.001) and severity adjusted (6.3 to 3.9 d, P , 0.001)
ICU length of stay decreased signicantly in the SICU after
implementation of telemedicine. The rates of change
between the two units over time were signicantly different
from each other (ANCOVA interaction P 0.005). While
unadjusted ICU mortality decreased signicantly in the
MICU after the telemedicine intervention (0.80 to 0.57,
P , 0.001), there was no signicant change in the MICU
mortality after adjusting for severity of illness. Both
unadjusted and severity adjusted ICU mortality for SICU
patients decreased signicantly after implementation of
telemedicine (0.09 to 0.01, OR 0.15, P 0.003).
Discussion
A randomized, double-blind, placebo-controlled study
would provide the best evidence about whether or not
telemedicine affects outcomes. However, it would be
difcult to organize. As a result many centres, including our
own, have chosen to conduct observational studies by
comparing historical control data with
post-implementation data.
4,5,6,8,12,16,17,18
Unfortunately, it
is not possible to control for all potential confounding
variables. To help mitigate some of these confounders, we
evaluated data from two ICUs in the same health system.
The results indicate an association between the
implementation of telemedicine and a decrease in ICU
length of stay, ICU mortality and hospital mortality. No
such associations were seen in the medical ICU not exposed
Table 1 Hospital length of stay and mortality
Control Telemedicine
MICU (Pre) MICU (Post) P-value SICU (Pre) SICU (Post) P-value
No of patients 220 285 246 1499
APACHE score mean (SEM) 100.7 (2.5) 80.1 (2.5) ,0.001 46.2 (1.5) 54.1 (0.6) 0.005
Unadjusted hospital length of stay mean d (SEM) 13.2 (1.0) 11.3 (0.7) NS 15.6 (0.9) 15.1 (0.6) NS
Severity adjusted hospital length of stay mean d (SEM) 12.5 (1.1) 10.9 (0.8) NS 19.0 (1.0) 16.7 (0.8) NS
Unadjusted hospital mortality mean (SEM) 0.88 (0.02) 0.65 (0.03) ,0.001 0.11 (0.02) 0.06 (0.01) 0.003
Severity-adjusted hospital mortality mean (SEM) 0.74 (0.05) 0.56 (0.04) NS 0.13 (0.03) 0.04 (0.01) 0.023
NS denotes P 0.05
B A Kohl et al. ICU telemedicine
284 Journal of Telemedicine and Telecare Volume 18 Number 5 2012
to telemedicine. Despite an increase in the severity of illness
scores (i.e. patients were more ill) after telemedicine
implementation, there was a profound decrease in ICU
length of stay and ICU mortality in the SICU. In the
non-intervention ICU, however, there was a decrease in
severity of illness scores (i.e. patients were less ill), with no
change in ICU length of stay. These results may be partly
explained by the initiation of the telemedicine service and
the ability to provide additional oversight with best
practices. Compliance with best care processes has been
shown to reduce ICU morbidity.
19,20
The present study had certain limitations. The principal
drawback of all observational studies is that causal relations
cannot be established from observed associations. In
addition, the medical chart abstraction was
non-randomized and involved selecting consecutive charts
within each quarter. This could bias results if the start of
each quarter coincided with other confounding inuences.
During the period of study, working time restrictions were
instituted for all house staff (residents and fellows). While it
is possible that these regulations may have had a differential
effect on mortality in the two ICUs, a comparison in
different medical specialties did not support this
contention.
21
One important difference in the overnight stafng,
however, was that an advanced trainee (critical care fellow)
was present in the SICU and not the MICU. It is therefore
possible that the SICU had greater oversight during evening
hours, which may have contributed to the improved
outcomes. During the study period, there were no
signicant changes in faculty stafng or in the level of
training for the residents who were providing patient care in
the two ICUs. Another limitation of the study was that
severity of illness in the populations being compared was
very different. In particular, the MICU patients were not
only signicantly more ill (as indicated by their higher
APACHE scores) than the SICU patients, but their baseline
APACHE scores in the pre-intervention period were
signicantly greater than most similar MICU
populations.
19,22,23,24
Given this nding, however, one
would have expected the ICU length of stay to have
signicantly decreased as the severity of illness scores
decreased. This did not occur.
In the present study there was a dramatic disparity in the
sizes of the post-implementation populations compared.
This was a result of the electronic medical record that was
implemented at the same time as telemedicine. Since the
software automatically calculates the APACHE score for the
rst 24 h of ICU admission, we chose to include all patients
admitted to the SICU during the study period because
manual chart abstraction was no longer needed. Thus,
manual chart abstraction for the purposes of APACHE
scoring was undertaken for both pre-implementation
groups in addition to the post-implementation MICU
group. We used a bootstrap calculation to conrm the
stability of our ndings.
A nal limitation arises from the fact that our
telemedicine programme, from the beginning, consisted of
two key elements: (1) additional oversight by remote nurses
and physicians via bi-directional communication links and
(2) an electronic medical record. Thus it is not possible to
know whether the observed outcomes were due solely to the
additional oversight provided and what, if any, was the
independent effect of installing the electronic medical
record. Indeed, the marriage between most telemedicine
technologies and electronic medical records is a
confounding effect that must be considered in all such
studies.
2
In conclusion, implementation of telemedicine in the
surgical ICU was associated with signicant reductions in
severity-adjusted ICU length of stay and mortality, as well as
hospital mortality. Over the same period, and within the
same hospital, a medical ICU not using telemedicine had
no signicant change in any of the measured outcomes
after adjusting for severity of illness. However, it is not
possible to conclude denitively that the observed
associations seen in the SICU were due to the intervention.
Further work is thus required to quantify the effect of
telemedicine on ICU outcomes.
References
1 Berenson RA, Grossman JM, November EA. Does telemonitoring of
patients the eICU improve intensive care? Health Aff (Millwood)
2009;28:w93747
2 Kahn JM, Hill NS, Lilly CM, et al. The research agenda in ICU
telemedicine: a statement from the Critical Care Societies Collaborative.
Chest 2011;140:2308
3 Young LB, Chan PS, Lu X, Nallamothu BK, Sasson C, Cram PM. Impact of
telemedicine intensive care unit coverage on patient outcomes: a
systematic review and meta-analysis. Arch Intern Med 2011;171:498506
4 Lilly CM, Cody S, Zhao H, et al. Hospital mortality, length of stay, and
preventable complications among critically ill patients before and after
tele-ICU reengineering of critical care processes. JAMA 2011;305:217583
5 Thomas EJ, Lucke JF, Wueste L, Weavind L, Patel B. Association of
telemedicine for remote monitoring of intensive care patients with
mortality, complications, and length of stay. JAMA 2009;302:26718
Table 2 ICU length of stay and mortality
Control Telemedicine
MICU (Pre) MICU (Post) P-value SICU (Pre) SICU (Post) P-value
No of patients 220 285 246 1,499
APACHE score mean (SEM) 100.7 (2.5) 80.1 (2.5) ,0.001 46.2 (1.5) 54.1 (0.63) 0.005
Unadjusted ICU length of stay mean d (SEM) 4.87 (0.42) 5.89 (0.40) NS 5.04 (0.48) 3.29 (0.13) ,0.001
Severity-adjusted ICU length of stay mean d (SEM) 5.27 (0.52) 6.09 (0.43) NS 6.25 (0.50) 3.86 (0.17) ,0.001
Unadjusted ICU mortality mean (SEM) 0.80 (0.03) 0.57 (0.03) ,0.001 0.08 (0.02) 0.03 (0.004) ,0.001
Severity-adjusted ICU mortality mean (SEM) 0.54 (0.06) 0.42 (0.04) NS 0.09 (0.02) 0.01 (0.003) 0.003
NS denotes P 0.05
B A Kohl et al. ICU telemedicine
Journal of Telemedicine and Telecare Volume 18 Number 5 2012 285
6 Breslow MJ, Rosenfeld BA, Doerer M, et al. Effect of a multiple-site
intensive care unit telemedicine program on clinical and economic
outcomes: an alternative paradigm for intensivist stafng. Crit Care Med
2004;32:318
7 Marcin JP, Nesbitt TS, Kallas HJ, Struve SN, Traugott CA, Dimand RJ. Use
of telemedicine to provide pediatric critical care inpatient consultations
to underserved rural Northern California. J Pediatr 2004;144:37580
8 Rosenfeld BA, Dorman T, Breslow MJ, et al. Intensive care unit
telemedicine: alternate paradigm for providing continuous intensivist
care. Crit Care Med 2000;28:392531
9 Grigsby J, Bennett RE. Alternatives to randomized controlled trials in
telemedicine. J Telemed Telecare 2006;12(Suppl. 2):7784
10 Wunsch H, Linde-Zwirble WT, Angus DC. Methods to adjust for bias and
confounding in critical care health services research involving
observational data. J Crit Care 2006;21:17
11 Ho PM, Peterson PN, Masoudi FA. Evaluating the evidence: is there a rigid
hierarchy? Circulation 2008;118:167584
12 Kohl BA, Gutsche JT, Kim P, Sites FD, Ochroch EA. Effect of telemedicine
on mortality and length of stay in a university ICU. Crit Care Med
2007;35(Suppl. 12):A22
13 Kohl BA, Sites FD, Gutsche JT, Kim P. Economic impact of eICU
implementation in an academic surgical ICU. Crit Care Med
2007;35(Suppl. 12):A26
14 Breslow MJ. Remote ICU care programs: current status. J Crit Care
2007;22:6676
15 Knaus WA, Wagner DP, Draper EA, et al. The APACHE III prognostic
system. Risk prediction of hospital mortality for critically ill hospitalized
adults. Chest 1991;100:161936
16 Zawada ET Jr, Kapaska D, Herr P, et al. Prognostic outcomes after the
initiation of an electronic telemedicine intensive care unit (eICU) in a
rural health system. S D Med 2006;59:3913
17 Morrison JL, Cai Q, Davis N, et al. Clinical and economic outcomes of the
electronic intensive care unit: results from two community hospitals. Crit
Care Med 2010;38:28
18 McCambridge M, Jones K, Paxton H, Baker K, Sussman EJ, Etchason J.
Association of health information technology and teleintensivist
coverage with decreased mortality and ventilator use in critically ill
patients. Arch Intern Med 2010;170:64853
19 Gajic O, Afessa B, Hanson AC, et al. Effect of 24-hour mandatory versus
on-demand critical care specialist presence on quality of care and family
and provider satisfaction in the intensive care unit of a teaching hospital.
Crit Care Med 2008;36:3644
20 Hasibeder WR. Does standardization of critical care work? Curr Opin Crit
Care 2010;16:4938
21 Prasad M, Iwashyna TJ, Christie JD, et al. Effect of work-hours regulations
on intensive care unit mortality in United States teaching hospitals.
Crit Care Med 2009;37:25649
22 Digiovine B, Chenoweth C, Watts C, Higgins M. The attributable
mortality and costs of primary nosocomial bloodstream infections in the
intensive care unit. Am J Respir Crit Care Med 1999;160:97681
23 Khan H, Belsher J, Yilmaz M, et al. Fresh-frozen plasma and platelet
transfusions are associated with development of acute lung injury in
critically ill medical patients. Chest 2007;131:130814
24 Rosenberg AL, Hofer TP, Strachan C, Watts CM, Hayward RA. Accepting
critically ill transfer patients: adverse effect on a referral centers outcome
and benchmark measures. Ann Intern Med 2003;138:88290
B A Kohl et al. ICU telemedicine
286 Journal of Telemedicine and Telecare Volume 18 Number 5 2012
Copyright of Journal of Telemedicine & Telecare is the property of Royal Society of Medicine Press Limited
and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright
holder's express written permission. However, users may print, download, or email articles for individual use.