Running head: TECHNOLOGY TO IMPROVE ED TRIAGE 1

Improving Emergency Department Triage Using Innovative Technology

Christie M. Sheffield

James Madison University

TECHNOLOGY TO IMPROVE ED TRIAGE 2

Improving Emergency Department Triage Using Innovative Technology

Emergency Department (ED) overcrowding is a near universal issue with multilayer

implications for providers and patients. Long wait times to see providers are coming; creating

safety issues for those that need the most expeditious care (Salway, Valenzuela, Shoenberger,

Mallon, & Viccellio, 2017). Triage is a prioritization process that sorts patients into categories

based on how quickly a patient should receive treatment. These categories help ED staff

determine who needs to be seen first (Gilboy, Tanabe, Travers, & Rosenau, 2011). The more

overcrowding is an issue, the more important it becomes to make accurate triage assessments.

Current triage systems are highly subjective and lack consistency. The most popular

triage algorithm in the United States, the Emergency Severity Index (ESI), has been studied at

length. It is generally accepted as an adequate way to determine how long a patient can wait to

be seen by a provider (Gilboy, Tanabe, Travers, & Rosenau, 2011). However, there are

controversies surrounding how often the algorithm is applied correctly. Garbez et al. postulates

that the ESI algorithm lacks explicit direction and is “open to interpretation” (2011, p. 527).

There is room for biases and contextual influences. One study found that nurses were influenced

by additional facts such as the smell of alcohol and presence of stress in patients that presented

with complaints suspicious for myocardial infarction and stroke. Patients presenting with these

characteristics were assigned less critical ESI levels than patients with comparable complaints

(Brannon & Carson, 2003). There is no consensus regarding experience level of triage nurses

and that influence on triage decisions. Studies have shown that more experienced nurses tend to

use additional clinical factors such as patient’s history and symptoms other than chief complaint

to make triage designations (Garbez, Carrieri-Kohlman, Stotts, Chan, & Neighbor, 2011). Other

studies have shown that newer nurses to tend adhere more closely to the algorithm while those
TECHNOLOGY TO IMPROVE ED TRIAGE 3

with experience tend to rely more heavily on that experience and case histories (Stone & Wolf,

2017). Stansfield (2015) identified four themes of influence in triage nurse assessments “(1)

clinical information; (2) education and experience; (3) characteristics, attitudes, and beliefs of

triage nurses; and (4) environment of care.” In 2017, the Emergency Nurses Association (ENA)

provided a position statement that recommends ongoing assessment and evaluation of triage

processes to mitigate some of the effects of subjectivity (Stone & Wolf, 2017).

Inaccurate triage designations can have deleterious consequences for patients. A study

that looked into reasons for delayed transfer from the ED to the intensive care unit (ICU)

concluded that, even though several factors contribute to delays, triage level designation was one

of the most significant (Yurkova & Wolf, 2011). Inappropriate triage may delay treatment

potentiating patient decompensation or adverse events; creating openings for questions of

liability. Under-triage can also lead to longer lengths of stay (LOS) for patients and is the reason

many patients leave without being seen (LWBS) by a provider. These are common metrics used

to determine overall quality of ED care than can influence compensation levels for care

provided.

Improving ED triage accuracy is an important quality initiative to consider. Measures

should be taken to decrease inconsistency amongst nurse assessments for triage. ESI has been

studied to have “good” interrater reliability but has wide individual case variability. This is

related to the clinical judgement components that are required to differentiate the urgent from the

emergent (Bergs, Verelst, Gillet, & Vandijck, 2014). By far, most patients fall into these two

classes: ESI levels 3 and 2. Another goal is to increase accuracy of triage compared to clinical

diagnoses to 90%. The ENA reports that as much as half of patients with myocardial infarction

are under-triaged (Stone & Wolf, 2017). Triage quality initiatives should aim to decrease the
TECHNOLOGY TO IMPROVE ED TRIAGE 4

time until seen by a provider for the sickest patients. ED providers specialize in stabilizations,

not definitive management. The quicker sick patients can be seen by ED providers, the sooner

they can be transferred to the appropriate level of care (Yurkova & Wolf, 2011).

For an accurate assessment of change initiatives, measures must be established to

determine the impact (“Institute for Healthcare Improvement,” n.d.). Some potential measures

are:

 Measure consistency of nurse triage assessments via case studies

 Compare nurse triage assessments to key data points known at time of disposition.

 Compare diagnoses to triage designation for patients that meet key criteria for

“sick”, in the amount of time until seen by an ED provider.

 Compare general nurse population triage assessments to those that have been

determined to be subject matter experts.

Comparing nurse triage assessments retrospectively to key data points that are known at the time

of disposition can indicate whether initial assessments can be considered accurate. Accuracy of

triage assessments can be determined by comparing the ESI level designation made by a nurse at

time of triage to key objective indicators of acuity. Examples of indicators are: abnormal vital

signs, certain diagnoses, required interventions (i.e. surgery), disposition (discharge versus

admit), level of care (med/surg versus ICU), etc. Indicators can be used to calculate an acuity

“score” that is assessed to determine the frequency that it matches the assigned ESI level.

Measuring consistency amongst nurses using case studies is a great way to measure

consistency, but does not mean anything in regards to accuracy. Comparing diagnoses in “sick”

patients to triage assessments would be cumbersome and time consuming. First, a definition of

“sick” needs to be determined. Also by using diagnoses, it limits the study to only those that fall
TECHNOLOGY TO IMPROVE ED TRIAGE 5

under the preselected categories. Using subject matter experts appears to be a good way to

determine how accurately nurses use a triage system. There is a lack of sufficient numbers and

access to “experts.” Furthermore, there is no way to measure if the tool itself is the most accurate

way to determine acuity.

There are at least four possible solutions to improve triage accuracy and consistency:

ongoing education of ED nurses in the use of the ESI triage algorithm, utilizing an electronic

triage system, implementing a combination system of electronic and nursing assessment and the

introduction of Artificial Intelligence (AI) and machine learning. A study published in 2017

states “Regardless of previous training or years of emergency department nursing experience,

triage refresher training has been shown to increase accuracy of triage categorization, thus

leading to a decreased risk of poor patient outcomes (Brosinski, Riddell, & Valdez).” This

would suggest that a simple method to increase accuracy of ESI triage is to have ongoing

education with staff. However, this method does not address the subjectivity of the tool. Dugas

et.al. (2016) studied the use of an electronic triage system that uses predictive analytics strictly

adhering to potential resource utilization which is similar to how ESI triage level is determined.

Results showed an increase in accuracy regarding resource usage compared to traditional nurse

designated ESI. The concern is whether or not resource utilization is the most reliable means to

determine when a patient should be seen by a provider. In 2012, Kunisch performed a study

utilizing a computerized clinical decision (CDS) system. CDS “scores” were assigned during

triage and were available to nurses. The CDS model “score” predicted resource utilization

accurately more often than nurses did. Nurse accuracy did not increase with having the CDS tool

available. Another study assessed the accuracy of using machine learning technology to develop

an electronic triage tool (E-triage) that did not rely so heavily on resource utilization. Their stated
TECHNOLOGY TO IMPROVE ED TRIAGE 6

conclusion was “E-triage more accurately classifies ESI level 3 patients and highlights

opportunities to use predictive analytics to support triage decision-making” (Levin et al., 2018).

This suggests that E-triage can be a viable alternative with the potential to eliminate the

subjectivity issue of the ESI tool utilized by ED triage nurses.

The best solution to fix both subjectivity and reliability of a tool for Emergency

Department triage is to use machine based learning technology with predictive analytics.

Machine learning technology has access to data from the medical record and can use this

information in addition to entered triage data to provide a quick analysis and calculate small

differences in patients. This has the potential to decrease inconsistency and improve accuracy.

Predictive analytics embraces the knowledge gained from historical data and applies it to future

probabilities. Traditional triage classifications are limited to designated categories of patients, 1-

5. Someone has to still decide which “level 2” patient needs care first. Technology has the ability

to “list” patients in order of acuity thus clarifying which patient needs to be seen next.

Current practices in ED triage function well to make quick decisions to estimate how

soon patients need to be seen by providers. These systems rely heavily on nursing judgement and

can be affected by biases. A more accurate triage system would combine assessment data,

objective data, all patient complaints and historical data to instantaneous assign a prediction.

Predictions can imply an order of acuity that is removed from biases Quality improvements in

triage have the potential to decrease the number of adverse events, improve the care patients

receive and mitigate some of the stress providers experience trying to make the best decision for

a large number of patients. Improved triage can decrease LWBS and LOS metrics which

improve compensation for care. ED triage using machine learning and predictive analytics can

quickly provide ED staff with accurate triage sorting without subjectivity.

TECHNOLOGY TO IMPROVE ED TRIAGE 7

References

Bergs, J., Verelst, S., Gillet, J.-B., & Vandijck, D. (2014). Evaluating Implementation of the

Emergency Severity Index in a Belgian Hospital. Journal of Emergency Nursing, 40(6),

592–597. https://doi.org/10.1016/j.jen.2014.01.006

Brannon, L. A., & Carson, K. L. (2003). The representativeness heuristic: influence on nurses’

decision making. Applied Nursing Research, 16(3), 201–204.

https://doi.org/10.1016/S0897-1897(03)00043-0

Brosinski, C. M., Riddell, A. J., & Valdez, S. (2017). Improving Triage Accuracy: A Staff

Development Approach. Clinical Nurse Specialist, 31(3), 145–148.

https://doi.org/10.1097/NUR.0000000000000291

Dugas, A. F., Kirsch, T. D., Toerper, M., Korley, F., Yenokyan, G., France, D., … Levin, S.

(2016). An Electronic Emergency Triage System to Improve Patient Distribution by

Critical Outcomes. The Journal of Emergency Medicine, 50(6), 910–918.

https://doi.org/10.1016/j.jemermed.2016.02.026

Garbez, R., Carrieri-Kohlman, V., Stotts, N., Chan, G., & Neighbor, M. (2011). Factors

Influencing Patient Assignment to Level 2 and Level 3 Within the 5-Level ESI Triage

System. Journal of Emergency Nursing, 37(6), 526–532.

https://doi.org/10.1016/j.jen.2010.07.010

Gilboy, N., Tanabe, P., Travers, D., & Rosenau, A. (2011). Emergency Severity Index (ESI): A

Triage Tool for Emergency Department Care, Version 4. Implementation handbook 2012

edition. Retrieved from https://www.ahrq.gov

Institute for Healthcare Improvement: Science of Improvement: Establishing Measures. (n.d.).

Retrieved April 17, 2019, from

TECHNOLOGY TO IMPROVE ED TRIAGE 8

http://www.ihi.org:80/resources/Pages/HowtoImprove/ScienceofImprovementEstablishin

gMeasures.aspx

Levin, S., Toerper, M., Hamrock, E., Hinson, J. S., Barnes, S., Gardner, H., … Kelen, G. (2018).

Machine-Learning-Based Electronic Triage More Accurately Differentiates Patients With

Respect to Clinical Outcomes Compared With the Emergency Severity Index. Annals of

Emergency Medicine, 71(5), 565-574.e2.

https://doi.org/10.1016/j.annemergmed.2017.08.005

Salway, R., Valenzuela, R., Shoenberger, J., Mallon, W., & Viccellio, A. (2017). EMERGENCY

DEPARTMENT (ED) OVERCROWDING: EVIDENCE-BASED ANSWERS TO

FREQUENTLY ASKED QUESTIONS. Revista Médica Clínica Las Condes, 28(2), 213–

219. https://doi.org/10.1016/j.rmclc.2017.04.008

Stanfield, L. M. (2015). Clinical Decision Making in Triage: An Integrative Review. Journal of

Emergency Nursing, 41(5), 396–403. https://doi.org/10.1016/j.jen.2015.02.003

Stone, E., & Wolf, L. (2017). ENA Position statement: Triage qualifications and competency.

Retrieved from https://www.ena.org

Yurkova, I., & Wolf, L. (2011). Under-triage as a Significant Factor Affecting Transfer Time

between the Emergency Department and the Intensive Care Unit. Journal of Emergency

Nursing, 37(5), 491–496. https://doi.org/10.1016/j.jen.2011.01.016