Telehealth

By Dana R. Sax, David R. Vinson, Cyrus K. Yamin, Jie Huang, Troy M. Falck, Reena Bhargava,
Debra J. Amaral, and Mary E. Reed doi: 10.1377/hlthaff.2018.05079
HEALTH AFFAIRS 37,
NO. 12 (2018): 1997–2004

Tele-Triage Outcomes For Patients ©2018 Project HOPE—

The People-to-People Health
Foundation, Inc.

With Chest Pain: Comparing

Physicians And Registered Nurses

Dana R. Sax
ABSTRACT We took advantage of a change in protocol in an integrated (danakindermann@gmail.com)
is an emergency physician at
delivery system’s telephone consultation service—routing callers Kaiser Permanente Oakland
complaining of chest pain to physicians instead of registered nurses, Medical Center, in California.

whenever feasible—to explore whether tele-triage outcomes differed by

David R. Vinson is an
staffing type. Comparing outcomes of 11,315 physician-directed calls to emergency physician at Kaiser
Permanente Sacramento
those of an equal number of nurse-directed calls in 2013, we found that Medical Center, in California.
the physician-directed calls were briefer (eight minutes versus thirteen
minutes), produced fewer ED referrals (10 percent versus 16 percent), Cyrus K. Yamin is an
emergency physician at Kaiser
and resulted in higher patient adherence to the providers’ site-of-care Permanente Oakland Medical
Center.
recommendation (86 percent versus 82 percent). Mortality rates at seven
days were low for both physician- and nurse-directed calls (0.1 percent). Jie Huang is a programmer/
We suspect that providers’ immediate access to callers’ comprehensive analyst in the Division of
Research, Kaiser Permanente,
electronic health records and patients’ rapid access to outpatient care in Oakland.
likely contributed to the program’s success. Our findings suggest that
Troy M. Falck is regional
tele-triage can be used to safely and effectively manage an emergent assistant clinical director at
complaint, and that physicians’ expertise may bring additional efficiency the Appointment and Advice
Call Center, Permanente
to the process. Medical Group, in Oakland.

Reena Bhargava is an
internist in the Department of
Internal Medicine, Kaiser
Permanente Santa Clara

D
emand for acute care services has
increased over the past decade,
with the rate of emergency de-
partment (ED) visits per year out-
pacing population growth.1 The
increasing volume has led to ED overcrowding,
which has been associated with increased mor-
tality and costs.2,3 Chest pain is the second lead-
ing cause of ED visits, with over eight million
visits annually.4 Of the $80 billion per year at-
tributed to ED care nationally in 2011, chest pain
topped the list, accounting for $5 billion.5 Sever-
al strategies have been proposed to manage de-
mand and reduce crowding in emergency care,
including a surge in alternative sites of care,
changes in patient financial incentives and in-
surance coverage, the use of advanced practi-
tioners in EDs, operational changes to promote
efficiency, and increasing access to primary care
providers, but results have been mixed.6–8
Tele-triage may offer an innovative strategy to Medical Center, in California.
help contain the increase in ED costs and volume
for chest pain. Telehealth technologies (includ- Debra J. Amaral is a leader
ing telephone, email, video, and remote moni- of strategic innovation
at the Appointment and
toring devices) are becoming an integral part of
Advice Call Center,
the health care system and can enable delivery in Permanente Medical Group.
a timely, cost-effective way.9,10 Telephone advice
and triage services have been studied in the Mary E. Reed is a research
outpatient setting, primarily in Europe, and a scientist in the Division of
Research, Kaiser Permanente.
systematic review found no increased rate of ad-
verse events resulting from inappropriate tri-
age.11 Pediatric studies have suggested that using
telephone consultation services for urgent com-
plaints is safe and cost-effective, and the services
help route patients to the most appropriate ven-
ue of care.12–16 Little is known about the use of
tele-triage for adult patients who might other-
wise seek emergency care or about its implica-
tions on cost, quality of care, and patient expe-
rience.

D e c em b e r 2 0 1 8 37:12 H ea lt h A f fai r s 1997

Downloaded from HealthAffairs.org on January 22, 2024.
Copyright Project HOPE—The People-to-People Health Foundation, Inc.
For personal use only. All rights reserved. Reuse permissions at HealthAffairs.org.
Telehealth
Decisions on clinician staffing for telehealth
services in potentially emergent situations may
be key. Most evidence of tele-triage and advice
lines has been focused on nurse-directed ad-
vice.16–18 A systematic review of European studies
comparing physician- and nurse-directed tele-
triage consultation services showed mixed re-
sults with respect to downstream ED use and
costs.19 As far as we know, no comparison of
physician- and nurse-directed consultation ser-
vices for urgent or emergent complaints in the
US has been conducted.
We examined data from the Appointment and
Advice Call Center of Kaiser Permanente North-
ern California (KPNC), a large integrated health
care delivery system, which adopted a novel pro-
tocol for forwarding calls from patients report-
ing chest pain. Instead of directing all such calls
to registered nurses assisted by physicians, the
center started forwarding those calls directly to
physicians during specified dual-staffing hours.
Outside those hours, or when a physician was
not readily available, the calls were routed
to nurses with physician assistance, as before.
We compared venue-of-care recommendations,
patients’ care-seeking behavior, ED use, and pa-
tient outcomes for calls directed by physicians
and nurses during the dual-staffing hours. We
also used standard costs to roughly translate
study findings into dollar amounts. The poten-
tial to triage a highly prevalent, high-risk com-
plaint such as chest pain over the telephone rep-
resents an innovative use of tele-triage. If found
to be safe and cost-effective, it could serve as a
model for other systems seeking to manage the
growing demand for care for this and possibly
other acute complaints.
Study Data And Methods
Study Setting And Source Population Kaiser
Permanente Northern California provides com-
prehensive inpatient, ED, and ambulatory care
for over four million people in Northern Califor-
nia. Its membership is highly representative of
the local and state population.20 KPNC includes
twenty-one medical centers and has more than
one million ED visits annually.
Call Center And Physician-Directed Con-
sultation Protocols The appointment and ad-
vice call center covers the entire KPNC patient
population and receives over a million calls per
month from members seeking advice for their
health problems or requesting appointments.
The service is open twenty-four hours a day, sev-
en days a week. All callers hear an initial record-
ing asking them to call 911 or go to the closest
hospital if they feel they are having an emergen-
cy. Despite this warning, many callers with po-
1998 Health A ffairs D e ce m b er 2 018 3 7: 1 2
Downloaded from HealthAffairs.org on January 22, 2024.
Copyright Project HOPE—The People-to-People Health Foundation, Inc.
For personal use only. All rights reserved. Reuse permissions at HealthAffairs.org.
tentially emergent complaints proceed to seek
advice over the phone.21
Responses to calls follow clear protocols to
maximize patient safety. Calls are initially an-
swered by nonlicensed teleservice representa-
tives who follow highly structured scripts to
book appointments, send messages to callers’
providers, provide information about facility
resources and locations, and forward any calls
seeking clinical advice to call center nurses. The
nurses then use complaint-based structured al-
gorithms developed by Permanente Medical
Group physicians to guide their questions dur-
ing the call. Lastly, any call that meets criteria
for a critical concern based on these algorithms
triggers the nurse to consult with a call center
physician. Nurses present the case to the physi-
cian, who—after reviewing the medical history—
provides advice and assists with venue-of-care
recommendations. Nurses then complete the
call with the patient. Thus, nurses do not provide
advice in isolation for any potentially emergent
problem; instead, detailed protocols dictate phy-
sician participation for these cases.
Call center physicians are board-certified or
board-eligible emergency physicians who work
remotely, exclusively on the phone, to triage ur-
gent and emergent symptoms; they are con-
nected to the call center through a software plat-
form. These physicians have full and immediate
access to the caller’s medical records through
KPNC’s integrated electronic health record
(EHR).22 In addition to speaking with nurses
about critical cases, call center physicians also
review and approve prescriptions for specific
protocol-driven, nurse-directed calls.
Possible call outcomes for emergent com-
plaints include the activation of emergency med-
ical services, referral to the ED by nonambulance
transport, booking a physician’s office visit ap-
pointment within a specified time period, book-
ing a telephone appointment, sending an urgent
message to the patient’s provider, and offering
home care advice.
All calls, including those for potentially emer-
gent cases, followed this flow until late 2012:
teleservice representative to nurse, with physi-
cian involvement if the call met certain criteria.
At that time, the flow of calls for chest pain was
modified as described below.
Revised Protocol For Chest Pain While the
flow described above was effective in safely han-
dling calls across a spectrum of patient concerns,
call center physician leadership sought ways to
improve efficiency and optimize resource use for
certain high-risk complaints through appropri-
ate triage decisions. Beginning in late 2012 a new
protocol was introduced according to which call-
ers ages thirty-six and older bypassed the nurse
and were instead forwarded directly to a physi-
cian if they complained to the teleservice repre-
sentative of chest pain. Physicians were staffed to
take chest pain calls during all weekday office
hours. For these calls, physicians did not use a
predetermined algorithm (unlike the nurse
calls) and made triage decisions based on their
clinical judgment after taking a detailed patient
history and reviewing the patient’s EHR in real
time. It was felt that Permanente Medical Group
emergency physicians’ extensive experience
handling chest pain patients in the ED, in com-
bination with access to the EHR during the call,
could streamline triage decisions and provide an
advantage over nursing protocols.
This call center model, with dual staffing by
physicians and nurses, allows for the handling of
a multitude of complaints, decreases idle time,
and offsets costs while physicians wait for chest
pain calls.
A Natural Experiment: Routing Calls To
Physician Or Nurse According To Current
Wait Time Even when the direct-to-physician
chest pain protocol was in place, calls were au-
tomatically routed according to the standard
protocol as necessary to reduce caller wait times.
Specifically, if the wait time to speak with a phy-
sician was more than a minute, teleservice rep-
resentatives routed chest pain calls to a nurse
instead of a physician. As with all other critical
calls, if the chest pain was deemed potentially
cardiac in nature, based on clear endpoints in the
nurse’s algorithm, the nurse put the patient on
hold to review management decisions with a call
center physician.
This routing of calls based on wait time created
a natural experiment for evaluating physician- or
nurse-directed tele-triage for chest pain calls.
Importantly, most such calls routed to a nurse
still involved physician consultation. This study
compared nurse with physician consultation as
needed versus physician-only calls. The initial
assignment of calls to physicians or nurses
was based on the concurrent volume of referral
calls, independent of other call and patient char-
acteristics.
Data Processing And Analysis We con-
ducted a retrospective population-based cohort
study of a natural experiment. We identified all
calls to the call center with an initial complaint of
chest pain in the period January 1–December 31,
2013. We excluded calls that were ineligible for
the direct-to-physician protocol for chest pain
(including patients younger than age thirty-
six, pregnant women, and those who com-
plained of upper-respiratory infection or trau-
ma). We also excluded calls that were handled
during center hours with nurse-only instead of
dual coverage. (See online appendix exhibit A1
Copyright Project HOPE—The People-to-People Health Foundation, Inc.
For personal use only. All rights reserved. Reuse permissions at HealthAffairs.org.
for a more detailed discussion of data and meth-
ods.)23
Using automated data, we collected call char-
acteristics and linked each call with the patient’s
EHR data by medical record number to collect
data on patient characteristics, ED use, and out-
comes. Deaths were ascertained using hospital
and billing claims databases, administrative
health plan databases, state death certificate reg-
istries, and Social Security Administration files
as available at each site.
Although the natural experiment design, with
assignment of calls to a physician or a nurse
based on wait time, already offered comparable
patient characteristics between groups, we used
propensity scores to further account for minimal
differences in patient and call characteristics be-
tween calls routed to a nurse and those routed to
a physician (see appendix exhibit A2).23 The pri-
mary outcome was the recommendation of the
tele-triage provider (physician or nurse) on the
venue of care (ED visit, clinic appointment, tele-
phone appointment, or message to the patient’s
primary care provider). Secondary outcomes
were the concordance between the provider’s
recommendation and the patient’s subsequent
care-seeking behavior, and seven- and thirty-
day all-cause mortality. We compared the differ-
ences in the rates of these outcomes between
physician- and nurse-directed calls in the
matched sample.
Lastly, we used standard unit costs to illustrate
potential cost differences between physician-
and nurse-directed calls. Using average ED costs
from the Medical Expenditure Panel Survey and
average salaries of California nurses and emer-
gency physicians provided by the Bureau of
Labor Statistics and the American College of
Emergency Physicians, we estimated potential
health system savings through the use of a di-
rect-to-physician protocol for chest pain calls.
Limitations Our study had several limita-
tions. First, the results may be less generalizable
to other settings with less access to or integra-
tion of services across venues or lack of a
comprehensive EHR, or to systems with fee-for-
service payment models that incentivize in-
creased service use.
Second, other health care systems might not
have sufficient staffing or demand to support a
tele-triage system.
Third, we provide a simple cost translation of
our findings to illustrate the potential cost sav-
ings of implementing a direct-to-physician pro-
tocol, but we acknowledge that there are many
other variables in a full cost analysis (such as the
costs of implementing a tele-triage system and
clinicians’ work time when not addressing chest
pain calls).
December 2018 3 7:12 Health Affa irs 1999
Downloaded from HealthAffairs.org on January 22, 2024.
 Telehealth

Study Results pensity-matching analysis to mitigate these dif-

Among all 29,673 calls in our study data, 12,064 ferences, and the propensity-matched cohorts
were initially directed to a physician and 17,609 were very similar (exhibit 1). (See appendix ex-
to a nurse. After propensity matching, the study hibits A3a and A3b for the distribution of pro-
population was 22,630, with 11,315 calls in each pensity scores by call type before and after
group. The patient and call characteristics in the matching.)23 Physician-directed calls were brief-
two groups were similar, although there were er than nurse-directed calls (eight minutes ver-
small differences in demographic and comorbid- sus thirteen minutes).
ity profiles for reasons we could not explain We found that 87 percent of patients were
(see appendix exhibit A2).23 We employed pro- ultimately directed to venues outside of the ED.
Although both groups infrequently recom-
mended the ED, physicians had fewer ED refer-
Exhibit 1
rals than nurses (10 percent versus 16 percent).
Baseline characteristics of patients who used a telephone consultation service for chest Most calls led to a recommendation of an
pain, by type of clinician, 2013 appointment—the result in 75 percent of physi-
cian-directed calls and in 65 percent of nurse-
Patient characteristic Physician-directed calls Nurse-directed calls
directed calls. In 9 percent and 14 percent of
Age (years)
calls, respectively, physicians and nurses sent
36–49 32.60% 32.90%
a message to the caller’s primary care provider
50–64 36.43 36.36
65–74 17.17 16.98 in lieu of an in-person visit.
75 or more 13.80 13.76 Nurse- and physician-directed calls both had
Sex high rates of adherence to the provider’s recom-
Female 62.73% 62.67% mendation. The concordance rate between call
Race/ethnicity recommendation and patient action was higher
White 55.42% 55.28%
among physician-directed calls than among
Black 12.66 12.53 nurse-directed calls (86 percent versus 82 per-
Hispanic 15.32 15.68 cent; p < 0:05) across all call outcomes; among
Asian 14.98 14.94 calls with an ED recommendation, physician-
Other 1.61 1.57 directed calls had a nonsignificantly higher rate
Emergency department copayment (85 percent versus 82 percent) (exhibit 2). Of
$0–$50 44.89% 44.81% note, approximately 80 percent of patients who
$55–$100 40.31 40.53 received a recommendation for a clinic visit were
$125–$250 6.71 6.58 seen within twenty-four hours (data not shown),
10–30% 8.10 8.08
which highlights the high access to timely out-
Any event in prior 12 months
patient care in this system.
ED visit 27.17% 27.06%
Approximately 80 percent of chest pain
Hospitalization 7.33 7.12
Office visit 92.63 92.81
patients who went to the ED after using the
Call center call 12.18 12.22 call center service were discharged after an ED
Chronic conditions evaluation, and there was no significant differ-
High cholesterol 45.78% 45.39% ence in the hospital admission rate between phy-
Hypertension 41.66 41.71 sician- and nurse-directed calls (18.4 percent and
Diabetes 16.00 16.08 19.3 percent, respectively; p ¼ 0:35). The seven-
Prediabetes 13.08 13.16 and thirty-day mortality rates were very low for
Coronary artery disease 8.95 8.73 both physician- and nurse-directed calls, and the
Atherosclerotic disease 7.97 7.76 differences were not significant (exhibit 3).
Anxiety 10.46 10.22
Among the 9,506 patients with nurse-directed
Prior ischemic events
calls referred to settings other than the ED (mes-
Myocardial infarction 5.37% 5.30%
sage to the doctor or an appointment), the mor-
Coronary revascularization 5.19 4.96
Stroke or transient ischemic attack 4.91 4.55
tality rate was 0.22 percent at thirty days; among
Risk factors
the 10,138 patients with physician-directed calls
referred outside the ED, the rate was 0.19 per-
Premature family history of
myocardial infarction 1.51% 1.50% cent. For all outcomes, sensitivity analyses with-
Obesity 21.83 22.02 out propensity matching produced consistent
Smoking 11.45 11.45 results (appendix exhibit A4).23
Nationally, the mean costs of an ED visit and
office visit in 2013 were $1,50024 and $228,25
SOURCE Authors’ analysis of data from the Kaiser Permanente Northern California Appointment and
respectively; the average hourly rates of an ED
Advice Call Center and administrative health plan databases. NOTES The data are for calls received
during dual staffing hours only. There were 11,315 calls to physicians and the same number of calls to physician and ED nurse in California were $220
nurses. None of these differences was significant (p > 0:05). ED is emergency department. and $49, respectively.26,27 We applied these stan-

2000 H e a lt h A f fai r s D e c em b e r 2 0 1 8 37:12

Downloaded from HealthAffairs.org on January 22, 2024.
Copyright Project HOPE—The People-to-People Health Foundation, Inc.
For personal use only. All rights reserved. Reuse permissions at HealthAffairs.org.
dard costs to our study findings of minutes per Exhibit 2
call, ED referral rates, and office visit referral
Concordance between the clinician’s recommended venue of care and the patient’s choice of
rates for physician- and nurse-led calls. For ex- care for chest pain for all calls to a telephone consultation service and calls recommending
ample, for 100 calls for chest pain, physician-led an emergency department (ED) visit, by type of clinician, 2013
calls would cost $4,528 less than nurse-led calls.
Across one year with approximately 30,000
chest pain calls, as in our study setting, this could
amount to a $1,904,400 lower cost for physician-
led calls (see the appendix for details).23

Discussion
We analyzed a propensity-matched cohort of
22,630 calls for chest pain made to a telephone
consultation service in an integrated delivery
system. Despite the high-risk potential of a com-
plaint of chest pain, this tele-triage service re-
ferred 87 percent of calls to venues other than the
ED. Although physicians referred fewer patients
to the ED than nurses did, both call types were
effective in triaging patients to an appropriate
venue of care without incurring adverse patient
outcomes. In addition, the high concordance
rate between recommendation and patient
care-seeking behavior suggests that patients val-
ue the advice the service provides.
We found that the introduction of a direct-to-
physician protocol was associated with modest
but significant reductions in ED referrals and use
as well as improved concordance between call
center recommendation and patient action,
compared to nurse-led calls. Physician-led calls
were also briefer than nurse calls, without any
clear difference in outcomes—although our SOURCE Authors’ analysis of data from the Kaiser Permanente Northern California Appointment and
Advice Call Center and administrative health plan databases. NOTES The data are for calls received
study had limited power to detect differences. during dual staffing hours only. The whiskers represent 95% confidence intervals. Physician-directed
These findings suggest that for high-risk com- calls had 3.9 percent (95% CI: 2.9, 4.9) greater concordance among all calls. The difference in con-
plaints, physician expertise may bring additional cordance rate for all calls was significant (p ¼ 0:05).
efficiency to the triage of patients calling with
chest pain complaints. After the time period in-
cluded in our study, the direct-to-physician chest costs. Overall, the estimated reductions in ED
pain program was expanded to twenty-four visits through use of physician-led tele-triage
hours a day, seven days a week, and other com- may have potential to lower health care costs.
plaints have been added to the direct-to-physi- Prior studies have assessed the efficacy and
cian protocol. safety of phone triage and how it affects ED
Although a complete analysis of the costs of a use. Pediatric studies for urgent complaints have
physician- versus nurse-led tele-triage service is shown that telephone triage services are safe,
beyond the scope of our study, we used standard decrease ED use, and are associated with high
unit costs to estimate a nearly $2 million cost parental satisfaction.12–16 Chest pain–related
savings using physician-led calls during the one- phone calls have been studied in 911-equivalent
year study period. This translation of event rates or emergency medical services settings in the
into dollars is not an analysis of the costs to United Kingdom and France. In these acute
implement or staff nurse- or physician-led tele- emergent cases, physicians took calls (usually
triage services, and we cannot disaggregate the made by bystanders) and successfully triaged
percentage of time that nurses and doctors patients to an appropriate prehospital mode of
spend on non–chest pain calls to make a true transport.27,28 A Texas hospital found that a tele-
cost comparison on that basis. Still, this is likely phone triage service of patients considering an
a conservative estimate, given that we used the ED visit decreased ED use, but this was in con-
average US ED visit cost. Most ED visits for chest junction with an up-front ED visit fee if their
pain are high-acuity visits, which incur higher complaint after a medical screening exam was

December 2 018 3 7:12 Health A ffairs 2001

Downloaded from HealthAffairs.org on January 22, 2024.
Copyright Project HOPE—The People-to-People Health Foundation, Inc.
For personal use only. All rights reserved. Reuse permissions at HealthAffairs.org.
Telehealth
Exhibit 3
Seven- and thirty-day all-cause mortality rates among
patients using a telephone consultation service for chest
pain, by type of clinician, 2013
SOURCE Authors’ analysis of data from the Kaiser Permanente
Northern California Appointment and Advice Call Center, hospital
and billing claims databases, administrative health plan data-
bases, state death certificate registries, and Social Security Ad-
ministration files as available at each site. NOTES The data are for
calls received during dual staffing hours only. The whiskers rep-
resent 95% confidence intervals.
considered nonemergent.29
While most telephone consultation services
are nurse directed, there are few studies assess-
ing the effect of provider type (nurse versus
physician), and most found no significant differ-
ences in ED referrals or utilization. A large,
multicenter study in the United Kingdom that
randomly assigned callers to nurses or physi-
cians found no difference in ED referrals or
health system costs, although calls were specifi-
cally for same-day primary care consultation
requests.30 Similarly, a study that randomly as-
signed calls to nurses and doctors for urgent
pediatric complaints found no difference in ED
or urgent care referral rates; there was also no
difference in concordance between the provider-
recommended venue of care and patient care-
seeking behavior.16
Our study adds significantly to these findings
as the first published US-based analysis of tele-
phone consultation of adult patients with a po-
tentially emergent complaint. In contrast to pri-
or studies, ED referrals (and predicted overall
health system costs) were lower, and concor-
2002 H e a lt h A f fai r s D e c em b e r 2 0 1 8 37:12
Downloaded from HealthAffairs.org on January 22, 2024.
Copyright Project HOPE—The People-to-People Health Foundation, Inc.
For personal use only. All rights reserved. Reuse permissions at HealthAffairs.org.
dance between recommended venue and patient
care-seeking behavior was higher among physi-
cian- than nurse-directed calls. We suspect that
the difference in ED referrals may be related to
the nature of chest pain calls. Given the high-risk
nature of the complaint, and the likely increased
experience of physicians in risk-stratifying pa-
tients, we suspect that physicians felt more com-
fortable directing appropriate patients to venues
other than the ED. Nurses followed highly struc-
tured algorithms that may have led to more con-
servative recommendations (despite reviewing
their plans with a physician to confirm the rec-
ommendation), while physicians relied on re-
views of medical records and expertise in draw-
ing out subtleties in symptoms to form their
recommendations.
Telephone consultation for patients with chest
pain represents a particularly innovative and
potentially controversial use of telemedicine.
Missed diagnosis of acute myocardial infarction
accounts for the second-highest number of mal-
practice claims against emergency physicians.31
Although the vast majority of chest pain ED visits
do not receive a life-threatening diagnosis, an
exaggerated perceived risk on the part of both
clinicians and patients leads to particularly high
resource use among this population.32 This
might be expected to drive increased referrals
to the ED by telephone service providers or,
alternatively, lead to delayed diagnoses and in-
creased mortality among patients inappropriate-
ly referred away from the ED. On the contrary,
our findings suggest that a triage system based
on a thorough history of present illness and re-
view of pertinent medical history is able to iden-
tify higher-risk patients who need ED evaluation.
We suspect that integrated settings with com-
prehensive EHRs may provide the ideal environ-
ment for tele-triage for urgent and emergent
complaints, and that our findings might not be
fully generalizable to other settings. We feel that
several factors may enhance the success of tele-
triage for urgent or emergent complaints. To
properly identify high-risk patients, telehealth
providers should have seamless access to a
patient’s EHR for coordinated and connected
care. There must be reliable, convenient, and
same-day alternatives to ED care. The telephone
service should be well publicized and easy to use,
and it should present a value-added alternative to
ED care for patients. Both physicians and hospi-
tal managers should play key leadership roles in
developing and managing the service. Lastly, re-
imbursement for telemedicine providers in some
less integrated settings has proved to be a per-
sistent challenge.9 Appropriate compensation is
critical to maintaining quality and appropriate
resource use for potentially high-risk decisions.
Conclusion
In conclusion, our study shows that both physi-
cian- and nurse-directed tele-triage safely and
effectively handled patient calls for chest pain
in an integrated delivery system, but physicians’
expertise may have conferred additional efficien-
cy. Because of fewer ED referrals, higher adher-
ence to provider recommendations, and briefer
calls when handled by physicians, our study find-
ings suggest that a direct-to-physician protocol
for acute chest pain calls may offer a valuable
service to health systems by reducing ED visits
and costs for an emergent complaint. ▪

The authors acknowledge Maria Glymour

(University of California San Francisco)
for her help with contributions to the
original study design. This project was
funded by the Kaiser Permanente
Northern California Community Benefit
Program.

NOTES
1 Moore BJ, Stocks C, Owens PL.
Trends in emergency department
visits, 2006–2014 [Internet]. Rock-
ville (MD): Agency for Healthcare
Research and Quality; 2017 Sep
[cited 2018 Oct 18]. (HCUP Statisti-
cal Brief No. 227). Available from:
https://www.hcup-us.ahrq.gov/
reports/statbriefs/sb227-
Emergency-Department-Visit-
Trends.jsp
2 Sun BC, Hsia RY, Weiss RE,
Zingmond D, Liang LJ, Han W, et al.
Effect of emergency department
crowding on outcomes of admitted
patients. Ann Emerg Med. 2013;
61(6):605–611.e6.
3 Pines JM, Griffey RT. What we have
learned from a decade of ED
crowding research. Acad Emerg
Med. 2015;22(8):985–7.
4 Weiss AJ, Wier LM, Stocks C,
Blanchard J. Overview of emergency
department visits in the United
States, 2011 [Internet]. Rockville
(MD): Agency for Healthcare Re-
search and Quality; 2014 Jun [cited
2018 Oct 18]. ((HCUP Statistical
Brief No. 174). Available from:
https://www.hcup-us.ahrq.gov/
reports/statbriefs/sb174-
Emergency-Department-Visits-
Overview.jsp
5 Slutzman JE, Wilson M, Cutler D,
Schuur JD. 188 costs attributable to
emergency department care by di-
agnosis. Ann Emerg Med. 2015;
66(4, Suppl):S67–8.
6 Mason S, Mountain G, Turner J,
Arain M, Revue E, Weber EJ. Inno-
vations to reduce demand and
crowding in emergency care; a re-
view study. Scand J Trauma Resusc
Emerg Med. 2014;22:55.
7 Morgan SR, Chang AM, Alqatari M,
Pines JM. Non-emergency depart-
ment interventions to reduce ED
utilization: a systematic review. Acad
Emerg Med. 2013;20(10):969–85.
8 Hernandez-Boussard T, Burns CS,
Wang NE, Baker LC, Goldstein BA.
The Affordable Care Act reduces
emergency department use by young
adults: evidence from three states.
Health Aff (Millwood). 2014;33(9):
1648–54.
9 Tuckson RV, Edmunds M, Hodgkins
ML. Telehealth. N Engl J Med. 2017;
377(16):1585–92.
10 Dorsey ER, Topol EJ. State of
telehealth. N Engl J Med. 2016;
375(2):154–61.
11 Huibers L, Smits M, Renaud V,
Giesen P, Wensing M. Safety of
telephone triage in out-of-hours
care: a systematic review. Scand J
Prim Health Care. 2011;29(4):
198–209.
12 McConnochie KM, Wood NE,
Herendeen NE, Ng PK, Noyes K,
Wang H, et al. Acute illness care
patterns change with use of tele-
medicine. Pediatrics. 2009;123(6):
e989–95.
13 Bunik M, Glazner JE, Chandramouli
V, Emsermann CB, Hegarty T,
Kempe A. Pediatric telephone call
centers: how do they affect health
care use and costs? Pediatrics.
2007;119(2):e305–13.
14 Barber JW, King WD, Monroe KW,
Nichols MH. Evaluation of emer-
gency department referrals by tele-
phone triage. Pediatrics. 2000;
105(4 Pt 1):819–21.
15 Kempe A, Luberti AA, Hertz AR,
Sherman HB, Amin D, Dempsey C,
et al. Delivery of pediatric after-
hours care by call centers: a multi-
center study of parental perceptions
and compliance. Pediatrics. 2001;
108(6):E111.
16 Lee TJ, Baraff LJ, Guzy J, Johnson D,
Woo H. Does telephone triage delay
significant medical treatment?: Ad-
vice nurse service vs on-call pedia-
tricians. Arch Pediatr Adolesc Med.
2003;157(7):635–41.
17 Navratil-Strawn JL, Hawkins K, Wells
TS, Ozminkowski RJ, Hawkins-Koch
J, Chan H, et al. Listening to the
nurse pays off: an integrated Nurse
HealthLine programme was associ-
ated with significant cost savings.
J Nurs Manag. 2014;22(7):837–47.
18 O’Connell JM, Johnson DA,
Stallmeyer J, Cokingtin D. A satis-
faction and return-on-investment
study of a nurse triage service. Am J
Manag Care. 2001;7(2):159–69.
19 Bunn F, Byrne G, Kendall S. The
effects of telephone consultation and
triage on healthcare use and patient
satisfaction: a systematic review. Br J
Gen Pract. 2005;55(521):956–61.
20 Gordon N, Lin T. The Kaiser Per-
manente Northern California Adult
Member Health Survey. Perm J.
2016;20(4):34–42.
21 Bhargava R, Temkin TL, Fireman B,
Levin E, Amaral D. Outcomes of
chest pain calls to an advice and
appointment center. Poster pre-
sented at: American Heart Associa-
tion Scientific Sessions; 2012 Nov
3–7; Los Angeles, CA.
22 Bornstein S. An integrated EHR at
Northern California Kaiser Perma-
nente: pitfalls, challenges, and ben-
efits experienced in transitioning.
Appl Clin Inform. 2012;3(3):318–25.
23 To access the appendix, click on the
Details tab of the article online.
24 Agency for Healthcare Research and
Quality. Emergency room services—
mean and median expenses per
person with expense and distribu-
tion of expenses by source of pay-
ment: United States, 2014 [Inter-
net]. Rockville (MD): AHRQ; [cited
2018 Oct 26]. Available via query
from: https://meps.ahrq.gov/
mepsweb/data_stats/tables_
compendia_hh_interactive.jsp
25 Bureau of Labor Statistics. Occupa-
tional employment statistics, occu-
pational employment and wages,
May 2017 [Internet]. Washington
(DC): BLS [last modified 2018 Mar
30; cited 2018 Oct 19]. Available
from: https://www.bls.gov/oes/
current/oes291141.htm#st
26 Katz B. 2015 emergency physician
compensation report highlights re-
gional salary trends. ACEPNow [blog
on the Internet]. 2015 Oct 13 [cited

December 2 01 8 3 7:12 Health Affa irs 2003

Downloaded from HealthAffairs.org on January 22, 2024.
Copyright Project HOPE—The People-to-People Health Foundation, Inc.
For personal use only. All rights reserved. Reuse permissions at HealthAffairs.org.
Telehealth

2018 Oct 19]. Available from:
https://www.acepnow.com/article/
2015-emergency-medicine-
compensation-report-released/
27 Deakin CD, Sherwood DM, Smith A,
Cassidy M. Does telephone triage of
emergency (999) calls using Ad-
vanced Medical Priority Dispatch
(AMPDS) with Department of
Health (DH) call prioritisation ef-
fectively identify patients with an
acute coronary syndrome? An audit
of 42,657 emergency calls to
Hampshire Ambulance Service NHS
Trust. Emerg Med J. 2006;23(3):
232–5.
28 Manzo-Silberman S, Assez N, Vivien
B, Tazarourte K, Mokni T, Bounes V,
et al. Management of non-traumatic
chest pain by the French Emergency
Medical System: insights from the
DOLORES registry. Arch Cardiovasc
Dis. 2015;108(3):181–8.
29 Butcher L. Offering non-urgent pa-
tients alternatives to emergency
care. Westchester (IL): Healthcare
Financial Management Association;
2013 Jan.
30 Campbell JL, Fletcher E, Britten N,
Green C, Holt TA, Lattimer V, et al.
Telephone triage for management of
same-day consultation requests in
general practice (the ESTEEM trial):
a cluster-randomised controlled trial
and cost-consequence analysis. Lan-
cet. 2014;384(9957):1859–68.
31 Brown TW, McCarthy ML, Kelen GD,
Levy F. An epidemiologic study of
closed emergency department mal-
practice claims in a national data-
base of physician malpractice insur-
ers. Acad Emerg Med. 2010;17(5):
553–60.
32 Hsia RY, Hale Z, Tabas JA. A na-
tional study of the prevalence of life-
threatening diagnoses in patients
with chest pain. JAMA Intern Med.
2016;176(7):1029–32.

2004 H ea lt h A f fai r s D e c e mb e r 2 0 1 8 3 7 :1 2
Downloaded from HealthAffairs.org on January 22, 2024.
Copyright Project HOPE—The People-to-People Health Foundation, Inc.
For personal use only. All rights reserved. Reuse permissions at HealthAffairs.org.