a Effects of Computerized Physician Order Entry and Clinical Decision Support Systems on Medication Safety A Systematic Review Rainu Kaushal, MD, MPH; Kaveh G. Shojania, MD; David W. Bates, MD, MSe Backgrounds Iatrogenic injuries related to medica- tionsare common, costly, and clinicaly significant. Com- puterized physician order entry (CPOE) and clinical de- Cision support systems (CDSs) may reduce medication Methods: We identified trials that evaluated the ef- fects of CPOE and CDSSs on medication safety by elec- tonically searching MEDLINE and the Cochrane Li brary and by manually searching the bibliographies of retrieved articles. Studies were included for systematic review ifthe design was a random nonrandomized controlled trial, oran observational study ‘with controls and if the measured outcomes were clini- cal (eg, adverse drug events) or surrogate (eg, medica tion errors) markers. Two reviewers extracted all the data Discussion resolved any disagreements, ‘controlled trial, a Results: Five trials assessing CPOE and 7 assessing iso- lated CDSSs met the criteria. Of the CPOE studies, 2 dem- onstrated a matked decrease in the serious medication, error rate, 1 an improvement in corollary orders, 1 an improvement in 5 prescribing behaviors, and 1 an im- provement in nephrotoxic drug dose and frequency. Of the 7 studies evaluating isolate statistically significant improvements in antibiotic- associated medication errors or adverse drug events and 1 an improvement in theophylline-associated medica- lion errots. The remaining 3 studies had nonsignificant results, Conelusions: Use of CPOE and isolated CDSS can sub- stantially reduce medication error rat ies have not been powered to detect differences in ad- >but most stud- verse drug events and have evaluated a small number of homegrown” systems. Research is needed to evaluate commercial systems, to compare the various applica tions, to identily key components of applications, and to identify factors related to successful implementation of these systems. Arch Intern Med. 2005;163:1409-1416 From the Division of General Internal Medicine, Brigham and Women’s Hospital, Partners Healthcare system, and Harvard Medical School Boston, Mass (Drs Kaushal and ates); and he Department of Medicine, University of California San Francisco (r Shojana). Drs Kaushal ‘and Shojania have no relevant financial interest this article ‘complete lst of Dr Bates honoraria and financial interests appears at the end of this ail EDICATION ERRORSand, adverse drug events (ADEs) are common, costly, and clinically important problems." Each year, an estimated 770000 people are Injured or die in hospitals from ADEs, which are injuries resulling from drug use.'° Adult hospital incidence rates of ADEs have ranged from 210 7 per 100 ad- missions,*"""” although determination of 4 precise national estimate is difficult be- cause studies have used varying defini- lions."! Approximately 28% of ADEs are associated with a medication error and. therefore are judged to be preventable? Of preventable ADEs, 56% occurred during drug ordering Two inpatient studies, 1 im adults! and 1 in children,’ found that medication er- rors occurred a rates of more than 59% and that approximately half ofall medication errors occurred atthe stage of drug order ing. The principal types of medication er- rors include missing a dose and incorrect medication doses, requencies, or routes? The frequency and type of medication e rors found depend on the method used to detect them, Other studies,’ which used direct observation method to assess how accurately orders are carried out, found high rates of drug administration errors. Analysis of medication errors sug- {gests thal prevention strategies targeting systems rather than individuals are most effective in reducing errors."* Computer physician order entry (CPOE) and clinical decision support systems (CDSSs) are promising interventions that target the ordering stage of medications, where most medication errors and preventable ADEs ‘occur. Despite growing public mandates and the obvious theoretical advantages of these systems, organizational adoption of CPOE and CDSs has been limited. The much publicized Institute of Medicine (©2003 American Medical Association, All rights reserved. ‘Downloaded From: http:/jamanetwork.com/pdfaccess.ashx?url=/data/journals/intemed/S447/ on 03/15/2017Table 4. Hlerarehy of Study Design tr uy Design 7 Fandoraed contaled ‘i 2 Nonandomiea one tl a (Observational studies "ath conto 4 (Observational studios “atau conto esenpton| Telus quasandarized processes sch as ‘iter allocation Includes prospectively planned tis with redtermined elt cara and outcome Ieasutes or prospective cohort studies that inelud trenton and conto groupe Includes retrospective, taupe tne series (achanga intend aut the interven), case-contal studs, cohort ‘tie ith conto, sd heath sens research a ines adjutant for hay confounding vale Includes cohort ste without contals ad systematic reves we signed the highest o aves eel of ty deign nuded inthe evi, fodowed by an” (eg, a sytematc review hat incuded 1 andomzed corre valand several tbeenatonal studee ith controls wae designated report To Err Is Human drew at- tention to this “digital divide” be- tween health care and other indus- trial sectors and called for more widespread adoption of informa- tion technology solutions to im prove medication safety. Results of recent surveys" of US hospitals in- dicate that only 4.3% to 15.0% of hospitals have an electronic medi- calion order-entry system in place I partially implemented systemsand institutions in the process of acu ing systems are inchided, this fig- ure may be as high as 30%!" The degree to which health care has lagged behind other sectors in at tomating complex and hazardous processes is striking. For example, adoption of information technol. ‘ogy interventions sich as bar cod- ingand automated drug delivery sys- tems to reduce drug administration errors has also been slow. There- fore, weundertook this study tosys- tematicaly review the cumulative evidence on the ellects of CPOE and CDSSs on medication safety Ls DEFINITION OF CPOE AND CDSS CComputesiced pyisan onderentry e- ferstoavarcy of computer based gs. tems that share the common features of omating ti meetin ovringpro- ese a that ensre standardized eg ible, and complete orders. Clintal de- Cat stipe ysteme ave but ino tlmowtall CPOE systems to varying de- gens tescchniel decking pe a) ‘ides computerized advice regarding tig dose retest oences and tore sophisticated CDSSecan perform drug allergy checks, drug-lsboratory value cheeks, and drug-deug intern tn cheela aden provide eindery about corollary orders (eg, prompting the user to ondergncose chesisafer or dering inculn) or drug gudelines = ‘linical decision support systems may also be implemented without CPOE Basic CDSE often asst aks such s drug lection, dosing, and di tation, and ioc refined CDSS can in Coppa pao pi view such advice and thet proceed with 2 conventional handwriten medica: tion onder. STUDY IDENTIFICATION, [AND SELECTION Studies were identified by searching the US Notional Library of Medicine MEDLINE electronic bibliographic dar tabase and the electronic Cochrane Li bray The MEDLINE searchstatgy wa performed using the following MeSH ferme: hospital information mst: de tito support systems cinta and dag therapy, compater-assstd. In addition wre searched for ky tle words related to computeitd ender ey und com bined the results ofthese searches with MeSH terms capturing adverse events ted medical errors: medial ero ar see dace, sentinel survellnc, ana Safety. The Cochrane Library was searched using similar key cmsand ile trod: Reference lis from ll relevant stcle, including 2 systematie re- views were eve to deny ad ‘ital pain orden Spill, we sought articles de- scribing compulstzel spss bx per forming general ofder entry or CDSSS {or guiding physicians in the order ‘writing process. Computerized pro- grams that screen for potential ADES were not included, unless they interact swith users during the order-weiting pro- cess," and neither were CDSS built {nto programmable intravenous inl sion pumps" Although both ofthese practices play arle in improving medi ation salety, they do not alec the stage of order writing, which is the focus of this review STUDY EVALUATION woofs (RK and KG) reviewedall theartiles to determine the evel of evi ther lor procter ellcivenes ing ftameworks developed by the Univers sity of California San Francisco Sanford Evidence-Based Practice Cen- te forthe evaluation of study design (Table 1) and measured osteomes (Fable 2) Ths clasiation scheme was developed Beense ofthe heterog- revs tate of te ston evan, {CPOE and CDSS Thescheme incorpor astenesol sisting tnmeworsted Feomnerction far ethamingard jar thesizng evidence" We included ar tices wih a minimum level study de sign (obseritonal studies with contos) ilove outcomes (ouropate cial Stcomes) Stes that reported a mix tute oflve2 and evel 3 ottcomes (ot Comes with an indirect or unestablished Connection tothe target safety out- ome) were icles had heen de- ed prspentvely Dingrements were sual by discon OUTCOME DEFINITIONS Medication errors ae errors inthe pro- ess of ordering. transerbing, dispens- {ng administering, or monitoring med Cations One exanpletean order wien for acetaminophen without a route of administration, Medication errors in- les mixture of errors with difering Potentials for patient injury Potential ADEs are mediation er rors with significant potential to harm 5 patient that may or may not actually Teacha patient an example ofan inter Cepted potential ADE tan order writ ten fora morphine overdose thats no- ticed and corrected by a pharmacist, before the drug is administered. Anex- simple of nonitterceped potential ADE ian administered overdose of mor- Phinetos patient who does not have any Fequelae, Medication erorsand poten- tal ADEs were considered surogateo comes (level 2). “Adverse drug events are injuries re- sullng rom druguse and therefore con- (wepnnytep) SRCHINTERN MEDIVOE IGE, JONES 00> (©2003 American Medical Association, All rights reserved. ‘Downloaded From: http:/jamanetwork.com/pdfaccess.ashx?url=/data/journals/intemed/S447/ on 03/15/2017stitute clinical outcomes (level 1). Ad= verse drug events associated with a medication error are considered pre- ventable, whereas those not associated with a medication error are considered nonpreventable. An example ofa pre- venlable ADE is the development of Arash alter the administration of peni= tillin toa known pentcillin-allergi pa- tient. In contrast, a nonpreventable ADE isthe development ofa pentcillin- associated rash in a patient with no known previous allergies, Nonintercepted serious medica- tion errors inchide nonintercepted po- tential ADEs and preventable ADEs (i, smedication errors that either have the potential to or actualy cause harm to & patient). Errors that are not inter cepted and have the potential to oF ac tually cause injury are the most impor tant from the perspective of patient safely DATA EXTRACTION AND ANALYSIS Sudies were grouped into 2 categories: those evaluating CPOE with CDSSs and those evalating CDSS alone. No stud ies were found assessing CPOE alone We didnot quantitatively score the qu Covel Oulome Measure 7 ined tome 2 Surogate outcome a Ober 4 None Table 2. Hlerarehy of Outcome Measures ‘ay maasre of maby or morally atuling ‘abst drug events 25 deed inthe “Outcome Dstintonesubeecton ofthe to (Observed errs, termediate tomes (9 laboratory test resus) with a wel-stablisod ‘connection othe lcal outcomes of rst (ustay advrse rents) (ther mesurble variables with an inet nestablebed concn tots apa sey outcome (og, pretst/pos test ate an edcational Intention and compance with optima” or commande” prascibig practice) No eutcomesrbvant to detasing media arore or ‘abet aves og, hasty dasebas an ‘approach to detecting eos but reports no measured outcomes) ity of the studies owing to the recog- nized difficulties i quality scoring i general” and especially for aheterog- fencous group of studies sch as those included in this review. Nonetheless, both reviewers abstracted each study for prospectively determined elements per- taining to methodological quality” In addition, the reviewers described the Study design, setting, outcomes, and re- sults Other extracted information in- chided data regarding potential harm from the practice and issues regarding cost and implementation —L ss STUDY DESIGN AND SETTINGS Twelvestudies met the incluston ext= teria for study design and mea- sured outcomes. The 5 studies listed in Table 3 evaluated CPOE with CDSSs.2% In the first study." in- vestigators at the Regenstriel Insti- tute for Health Care (affiliated with the Indiana University Sehool of Medicine, Indianapolis) conducted ‘randomized controlled trial of 2181 patients evaluating the effects of CPOE on corollary order preserib- ing. The remaining 4 studies" evaluated the CPOE system at Brigham and Women’s Hospital ‘Table 3, Studies of Computerized Physiolan Order Entry (CPOE) With Clinical Decision Support Systems (CDSSs) Bate tak" 1900 Surgical and itn care ad BI, tetany ate Central with Hanard nest POE with COSSe for 1817 adultnpatnts in medal ‘nasa BW ‘Source Stay Deseripton| way Design ‘sty outcome esa ‘verge tal pact acy and pyscan Lawl tT with Tevel2and3 arorsof_ 2s provement in odeng ‘07 reminders (using CPOE) on physicians misionincorolary of cooly macatios by Coro odes for 21 fandomizd to orders) faculty and resides Adlington ina general ——_recav raiders (Pett) ‘medal wad at 3 puie at) teaching hosp atid eth the nda Urey Schoo of Medine Bates POE ith CDSS fr 6771 Love 2and3 (2 study Level (ADE rata) and__5% Decrease in noniearcepted ad inpatints on maa, designs) Tove 2 sau serous mation ere Lael 3 (etrospectve time sees) ‘ich otal® 2000 CPOE with CDSSs forall adit Level 3 (rtrspecive Levels 2nd (changos:__ImprovenentinS posting ington at BW store aajsis) in prsciting Pracces (001 for echo pracess). 5 comparisons) Cherow etal CPOE witha COSSto aust Level (RCT witha Level 2 (appropriate 13% Decreasa inappropriate 201 ug dose and equany in erossoerdsign) ug dose and ose (P01) and 20% 7490 ad inpatients ith trqueney) acess in inappropriate ‘onal nstelncy a BWH medion ars) Lye (ADE) and eve 2 (maa autor mediation rss) (P= 37) and 17% decease inpraventable ADEs (P= 37) 1% Decrease in mdiatin ‘rors (P= 01) and 86% theres in onatercapted ‘rous mation ere (Pet frequency (P01) ‘Abrevatons: ADE, averse dug event, BUH, Bghar and Womeis Hospi A, andoried controled ral, ‘Downloaded From: http:/jamanetwork.com/pdfaccess.ashx?url=/data/journals/intemed/S447/ on 03/15/2017 (©2003 American Medical Association, All rights reserved.(BWH). The first BWH study” was ‘cross-sectional analysis of 6771 pa Lients comparing an intervention pe- riod of CPOE with CDSSswith a his- torical period, the next 2 BWH studies” were time series analy ses, and the final BH study® was randomized controlled trial with, 4 crossover design of 7490 pa- Table 4 lists 7 studies?™" that evaluated isolated CDSSs; 6 were randomized controlled trials and 1 ‘wasa prospective before-after analy sis, All of these studies were con- ducted in the inpatient setting, Bur- ton et ab” assessed the use of a computerized aminoglycoside dos- ing program for 75 patients at the Dallas Veterans Affairs Medical Cen- ter in Texas. Evans etal," at LDS Hospital, Salt Lake City. Utah, per- formed 2 studies on antibiotic CDSSs: 4 randomized controlled trial” of empiric antibiotic drug se- lection using CDSSs with 451 pa- jents and a cross-sectional analy- sis!” comparing an intervention period of a computer-assisted anti- infective drug management pro- ‘gram with a historical control pe- Hiod for 1136 patients in the intensive care unit. Casner etal!and Hurley et al® performed random- controlled trials of comput id theophylline dosing programs with 17 and 48 patients, respec- lively. Mungall et al” evaluated a heparin dosing program for 25 in- patients, and White et al" evalu- ated a warfarin dosing program for 39 inpatients. In summary, many of the CDSS studies had small sample sizesand consequently were under powered, Table 4, Studies of Clinical Decision Support Systems (CDSSs) Soe Stay Deseripton| ‘suey oesgn| Stuy Outcomes Hay etal 1086 Usa oa computed Taal RET) Teva (ial ‘Paes wth neal vey ‘heopyln dosing program maniesttons of ‘manta raup vs nonin {or 4 inpatnts at Preston theophyline toxicity: vel study ru ( ‘na lorthecte Commun 2 te serum Ler rte af oxen Hospital, Nethete, Veta theopyineees) intervention patents (18.9%) vs contol (37.8%) (P= 08) Useota computerized watain Level (RCT) None ofthe ierverion dosing program for 30 tompletons): vl 2 ates had bleeding Ingato at Veterans (overantcoaution) armpcations ws 3 conta amination Masel patiets (8) (P=.07) Canta Pala, Cal othe Fete intrvertan patents were nies Calor, ‘rr anticoagulated (vs Divi, Medi Caner 47%) (P= 11) Burton etal?"1901 Use a computing Lowel (RT) Level 2 tx serum Lower ae fox eel aminoglycoside dosing el) intervention pati (5.6%) ogra for 7 inpatients a vs contls (027%) (P= 40) ‘Dal Vterae Aare Medial Center a 680-68 tara care contr in Tras Coanereta’® 1009 Use of eampuisrand Lowel (RT) Love (ical No signiteat terenes inary ‘heap dosing program mniesttons of ‘utzon One patient (8) a for 1 inpatients atthe theophyine toxic): Sul group wxibied sans Thamsnean General Hosp, level? subterapauic or of testy Ws noe in conl Pac, Tex Supraberapetc serum group (P= 0) One patent theophyine lees) inach group had toi lel (= 0); propcion of patients wit ubtherapee fol as 238% or etuay group and 16.7% treo Grane stal" 1994 Use of acompuseznd antbiode Laval (RCT witha Level 2(1 ofS primary ‘7% Great pathogen ‘hg saacton consultant fr crossover ucomes was pathogen ssc aan antbiotc “Stipa at LOS esa) suscep rug ragimen suggested bya Hospital, 520-64 prescbed antbioie amputer eonsutant vs arama teaching hospital regimens) plysiians (P001) and teary rer eentarin Sat Lake Gy, tah Manga eal Use computerized heparin Level (RCT) Level (leding avers) Fewer intervention pats led “a osng program for 25 (42%) vs conta (7.7%) inpaents at MeLarn (P=8) login Media Carter in Fle Mi, na ind Regional hea Cnt, Medan, Meh rans etal 1098 Computerbasd ant-intcive Laval 2 (prospective Level (an primary 7 Decrease in ADES caso ug management progam beforeafer, outcome vas ADEs due by atv agar for 136 patents roma sal) toanvnocive agers) (P= 02) ‘Pbed ICU at LDS Hopital ‘Abrevatons: ADE, averse dug event OU, itansve ae unit AT, andomiaedcotalled al, (©2003 American Medical Association, All rights reserved. ‘Downloaded From: http:/jamanetwork.com/pdfaccess.ashx?url=/data/journals/intemed/S447/ on 03/15/2017STUDY OUTCOMES Most of the inchided CPOE studies primarily measured level 2and level 3 outcomes because level 1 out- comes are significantly less fre- quent Therefore, larger and longer Studies are necessary to measure the cllects of an intervention on ADE rates, and the costs of such studies are very high. The first 2 BWH stud- ies?” primarily measured nonin Lercepted serious medication er- rors (level 2) and medication errors {level 2) butalso included ADEs as secondary outcome (level 1). The ther 3 CPOE studies reported level and level3 outcomes (ie, preserib- ing practices,” corollary orders, and appropriate drug dose and frequency). Corollary orders level and level 3 outcomes) are orders needed to detect oF ameliorate po- tential effects ofa tigger order, for example, ordering regular labora- tory tests of coagulation status af- ler starting a patient on intrave- nous heparin therapy. Similarly, the studies evaluate ing CDSSs report level land level2 cutcomes, with level 1 outcomes of- ten a secondary end point Burton etal” reported rates of toxic serum aminoglycoside levels (level 2) Evans ef al determined rates of pathogen susceptibility to an anti- biotic drug regimen (level 2)* and rates of anti-infective drug- associated ADES (level 1)" Casnicr etal and Hurley et al" reported ratesof toxic scrum theophylline lev- els (level 2). Mungall etal and White etal® reported bleeding com- plications (level 1), and White eal also reported overanticoagulation rates (level 2). CPOE AND MEDICATION SAFETY The first BWH study"*assessing the impact of CPOE with CDSSs dem- onstrated 55% decrease in nonin- lercepted serious medication er- rors (P=.01). As a secondary ‘outcome, thisstudy found a 17% dk crease in the preventable ADE rate, which was not statistically signifi. cant (P2.37). The CPOE applica lion at the time of this stidy in- cluded only basic decision support, ‘with limited checking for allergies FINTERN MED VOLTS, NET aus WP and drug-drug interactions. The cond study," a time series analysts, evaluated medication error rates be- fore CPOE and in the 3 years sub- sequent {o its implementation. It demonstrated an 81% decrease in medication errors and an 86% de- crease in nonintercepted serious ‘medication errors (P<.001 for both). This study found a decrease in the rate of ADEs per 1000 patient-days from 14,7 to 9.6 during the study (P=.09) anda decrease in the num- ber of preventable ADEs [rom 5 to 2 (P=05) The remaining 3 studies as- sessed more specific types of medi cation errors, Overhage et aP* dem- onstrated a greater than 25% improvement in the rates of corol- lary orders with implementation of computerized reminders. Teich et al” demonstrated 5 prescribing im- provements in types, doses, and fre- quencies of drug tse with the imple- ‘mentation of computerized clinical decision support. Finally, Chertow et al demonstrated a 13% de- crease in inappropriate dose and a 24% decrease in inappropriate fre- quency for nephrotoxic drugs in pa- tients with renal insufficiency (P<.001 for both). CDSSs AND MEDICATION SAFETY Three of the studies assessing iso- lated CDSSs evaluated computer- ‘zed antibiotic drug advice and dem- onstrated lower rates of toxic levels, improved pathogen susceptibility anda decreased anti-infective drug associated ADE rate, Burton et ab” evaluated a computerized aminogly- coside dosing program and demon- strated lower rates of oxic levels in intervention patients, but the re: sults were not statistically signil cant (P40). Evans et al! demon- strated 4 17% greater pathogen susceptbility to an antibiotic drug, regimen suggested by a computer consultant vs a physician (P<001). In another study, Evans et al re- ported a 70% decrease in ADEs caused by anti-infective agents through use of a computer-based antiinfective drug management pro- gram (P=.02) Two other studies evaluated theophylline dosing. Casner et al demonstrated no difference in rates of toxic serum levels. In contrast, Hurley et al? demonstrated signifi- ceantly lower rates of toxic levels in Intervention patients (18.0%) than incontrol patients (37.8%) (P=.04) The final 2 studies evaluated anticoagulation agents. Evaluation ‘ofa heparin dosing system demon- strated lower rates of bleeding events in intervention patients (4.2%) vs control patients (7.7%), but with- ‘out statistical significance (P=.6). Similarly, evaluation of a warfarin (Coumadin) dosing program dem- onstrated lower rates of bleeding complications (0% vs 8%) and over anticoagulation rates (5% vs 179%), but neither result was statistically significant (P=.11)." ste) These studies provide evidence that the use of CPOE with CDSSs sig- nificantly decreases medication e1 ror and serious medication error rales at 2 institutions with “home: ‘grown’ systems. Howes fon ADE rates has not been ad- cequately tested because studies with sullicient power have not been per formed. There is a strong correla- lion between medication errors and ADEs, so such applications will al- most certainly reduce ADE rates. Nevertheless, medication errors have widely varying potential for harm, and it seems easiest to prevent those that rarely cause injury.” Some of the CDSS studies, particularly those evaluating anti- biotic drug-associated programs, demonstrated focal reductions in x the effect medication errors with statistical s nificance as well as some decreases in ADE rates, Although other CDSSs tended to have statistically insignifi ceant results, these studies"! were underpowered, with sample sizes of 17 to 75 patients, Compr hensive applications, including CPOE and sophisticated decision support, will likely have the great- cst effect. ‘One important question is whether the currently available data are uflicently compelling that CPOE should be widely adopted or whether further research is required. We be- lieve that further studies targeted at 4 few eritical questions are desirable (©2003 American Medical Association, All rights reserved. ‘Downloaded From: http:/jamanetwork.com/pdfaccess.ashx?url=/data/journals/intemed/S447/ on 03/15/2017but not requirement before wide- spread adoption. Forexample,amul- Ucenter std evaluating the impact of CPOE on ADE rates would cost tens of millions of dollars and would bbe hard to perform because CPOE is acomples application touching on 0 rmany parts ofthe clinical and infor- mation systems, Instead, research should focus on questions such asthe following: What are the differences among various CPOE systems? What ae barierstoadoption? What are the key decision support elements? How cllective are specific pieces of deci sion support? How should these ap- plications be implemented in com rmunity hospitals? Most studies of CPOE have as- sessed only 2 internally developed homegrown) systems. To date, ise semination ofthese ystems has been limited fora variety of reasons. Most hospitals use commercial systems Relatively few vendors have CPOE applications that have broad use at mote than a handful of hospitals, As with evaluations of therapeutic agents, there isa reasonable expec: tation of a class effect™ with many CPOE systems, but elases of CPOE remain to be established. For ex- ample, CPOE systems with no deci sion support wilalmost certainly de crease error rates less than systems with sophisticated decision sup- port. Thus, one area for further re- search consists of developing tools to assess the extent to which a specie ‘commercial CPOE application will redice the medication error rate of the preventable ADE rate. Compari- sons among such commercial pod ucts will likewise be informative Organizational adoption of CPOE has been limited. One sur- vey" of 668 hospitals indicated that 15% had at least partially imple mented CPOE. A more recent sur- 1? of pharmacy directorsat L091 acute care hospitals in the United States (49% response rate) re- ported that 4.3% of hospitals had an clectronic medication ordet-entry system in place. ‘Many barriers to CPOE adop- lion exist. Rogers" suggests that per- ceived attabutes ofan innovation and ‘organizational socal context strongly allect the innovation adaption rate. Perhaps most important, a health care institution must gerner financial and FINTERN MED VOLT, JUNE aus WP organizational support before intro- ducing CPOE with CDSSs, Comput- erized physician order entry re- quires large up-front capital investment with more remote, albeit substantial, returns. Such invest- ‘ment is especially challenging when organizations are losing money. In ad- dition to the financial obstacles, {implementing sophisticated new clini cal information systems presents sub- stantial organizational challenges ow- fing to the impact on institutional celture and clinical workflow" and the need to accommodate existing in- stitutional systems used for billing, laboratory, and pharmacy data In addition, the efficacy of in- dividual decision support elements warrants further investigation. Many of the CDSS studies included in this review produced nonsignificant re- sults. Yet, itis difficult to draw de- nitive conclusions because of the small sample sizes, Larger studies need to be performed, as do studies identifying key, successful deci- sion support elements, COSTS AND BENEFITS ‘OF CPOE AND CDSSs Purchasing commercial CPOE sys- {ems is generally more expensive than is internally developing systems. Brigham and Women's Hospital has reported costs of $1.9 million for de- veloping and implementing CPOE in 1002, with ongoing maintenance costs of $500000 per year, although this was incremental to what was al- ready ahighly developed clinical sys- tem.” Fewer data are available re- garding the cost of purchasing and {implementing large commercial sys- tems, but it may be on the order of tens of millions of dollars, espe- cially if related clinical applications stich asa.clinical data repository must be upgraded, Several studies!" re- port that only minimal resources are needed to introduce or maintain de- cision support programs into exist ing order-entry programs. The beneficial effects of CPOE systems extend beyond medication ssifety and include reduced costsand quality improvement, These ben- clits have been achieved by provid ing feedback about the appropriate- ness and costs of laboratory and radiologic tests, easy implementa- tion of clinical pathways, improved quality measurement, and im- proved coding and billing. Brigham and Women’s Hospital estimated net savings of $5 to $10 million per year forthe CPOE system.” Ina random- ized controlled clinical tial, Tie yet al® demonstrated that CPOE linked to comprehensive elec- tronte medical record system re- sulled in charges that were $887 (12.7%) lower per admission. Cost savings associated with averted ADEs may be considerable, For ex- ample, BWH a 720-bed academic stitution, estimated costs before CPOE implementation of $2.8 mi lion annually for preventable ADEs." Evans etal® reported a $100000 per yearcost avoidance with acomput assisted antibiotic drug dosing pro- gram attributable to decreased an- Ubiotic drug use and avoided ADEs. POTENTIAL FOR HARM As with any other technology, CPOE and CDSSs may introduce different types of medication errors. Incor- rect default dosing or route sugges lions may lead to potentially erro- neous orders. For example, the first lime series analysis” at BWH dem- ‘onstrated an initial increase in in lercepted potential ADEs attribut- able to the ordering screen structure for potassium chloride, which made iteasy to order large doses of intra- venous potassium. Once iden lied, this error was rectified, but this event underscores the importance of ‘ongoing close scrutiny of CPOE and. CDSS. In general, as users become accustomed to CPOE and CDSSs, they are likely to accept computer suggestions with minimal reflec lion,” emphasizing the importance of testing decision support default settings and suggestions When CPOE systems are not electronically linked to computer ized pharmacy systems, pharma- cists must manually reenter orders Into the pharmacy system, with are- sultant increase in chance of error. Pedersen et al” found that 25.7% of surveyed hospitals with electronic prescribing lacked information sys- tem linkages to pharmacy systems. The trigger level for comput rized warnings must be set to the appropriate sensitivity. In situa (©2003 American Medical Association, All rights reserved. ‘Downloaded From: http:/jamanetwork.com/pdfaccess.ashx?url=/data/journals/intemed/S447/ on 03/15/2017lions with a potential for signifi- ‘cant harm, itis important that pro- viders receive warnings without being overwhelmed by alarms of marginal value. Hardware outages and software instability pose far ther risks. In particular, the reliabil- ity needed for CPOE is much higher than that required for systems that simply report laboratory test re- sults. Finally, physicians can el tronically write an order in the wrong patients record, analogous to handwriting an order in the wrong patient’s medical chart. PRESENT LEGISLATION AND PUBLIC MANDATES In the meantime, public and pri- vate groups are increasingly de- manding implementation of CPOE and other information technolo- ses. The Leapfrog Group, a consor~ lium of companies that belong tothe Business Roundtable, has endorsed. CPOE inhospitals as 1 of 3 changes that would most improve patient safety in the United States." A Medi ceare Payment Advisory Commis sion report” suggested instituting fe nancial incentives for CPOE implementation, Legislation has also been introduced at the federal and state levels, United States Senators Bob Graham (D-Fla) and Olympia Snowe (R-Maine) recently intro- duced a bill, itled the "Medication Errors Reduction Act of 2001,” toes- lablish an informatics system grant program for hospitals and skilled nursing facilites. At the state level, California recently enacted legisla~ lUon stipulating that acute-care hos- pitals implement information tech- nology stich as CPOE to reduce medication-related errors: Inconclusion, use of CPOE and, isolated CDSSs significantly de- creases medication ertor rates and provides other important benefits re- lated to medication use. Funding constraints and public pressure 10 bridge the digital divide in health, care make a large tial comparing pa per ordering to sophisticated CPOE systems unlikely. However, a need. exists for research evaluating com- mercial systems, the relative ben- fits of different classes of systems, and factors related to successful implementation of these systems. (wepnnytep) SRCHINTERN MEDIVOE IGE, JONES 00> Accepted for publication Au 2002, Portions of this article are based ‘on work performed by the authors in collaboration with the University of California San Francisco-Stanford Evidence-Based Practice Center un- der contract 290-97-0013 to the “Agency for Healthcare Research and Quality, Rockville, Mel Dr Bates has received hono- aria for speaking from the Eclipsys Corp (Boca Raton, Fla), which has li- censed the rights to the BWH Clini- cal Information System. The hospital no longer has a financial relation- ship with Eclipsys, Dr Bates isa coin- ventor on patent No. 6029138 held by BWH on the use of decision support software for medical management, li- censed to Medicalis Corp (Boston, Mass). He holds a minority equity po- sition in the privatety held company Medicalis, whick develops Web- based decision support for radiology test ordering, and he serves asa con- sultant to Medicalis. Hei also acon- sultant and serves on the advisory board for McKesson MedManage- ment (Minneapolis, Minn), a com- pany that assists hospitals im prevent- ing adverse drug events We thank Robert M. Wachter, MD, Bradford W. Duncan, MD, Kathryn M. McDonald, MSc, and Amy J Markowitz, JD, for their careful review of early drafts of this manu- script. The authors are responsible for the contents of this article, including any clinical or treatment recomimen- dations. No statement in this article should be construed as an official po- sition of the Agency for Healtheare Research and Quality or of the US Department of Health and Human Services. Corresponding author and re- prints: Rainu Kaushal, MD, MPH, Di- vision of General Internal Medicine, Brigham and Women’s Hospital, PBBA3, 75 Francis St, Boston, MA 02115 (e-mail: rkaushal@partners org). Se 1. Bahar, Mila, orson Eg Mss HL Meezonarorinnursig hares 2d sat esp. Am J Hosp Pharm 19629867901 2. Bate OW, Cale, Ld, at Incines ot dese rg ovens and poten adrse ug 0 0. w * u 6 % m. 0 a (©2003 American Medical Association, All rights reserved. ‘Downloaded From: http:/jamanetwork.com/pdfaccess.ashx?url=/data/journals/intemed/S447/ on 03/15/2017 ons npleatonsor penton JANA 1985 m0 On 5, lan EL uber MD, tar HN. Ca putson of mdesion eros nan Asiana {Brish hosp Aah thar, 185 2250-2500 (sen DG Pasta SL, Ean BS, Uy oF. Buri. Avro coors in ospiaad p ‘hms: exes lenght ty, a coe, an. rinaable moe dANA 1997277201306 (Gulen D4 Swe Bates urd ‘mondson Leap LL Prevenabieaaiere dug vin spd ae: aconpratie sy fimenecae untae genera uns, Cet (are Md 10725 1280-1207, Lisi TS, Last BM, Pol H. Masesion escrting rosin aching hsp er perce. Ab Irn Med 0871571589 tars. Kaushal Rate DW, Landigan 6 ta Mod ation eros and adese dg avers in pei: e paar, JAMA 20012852114 270, (ule, ts, Sl 0, Compr 8, Ne lal AR, Lape LT nit paring ss famous ot et ade eat b. lem or quality improve. J Cam J Qual Inpro. 10652 41-508. Bats DW, Boyle, Vander Vist MB, Stier 4. Lane Ratoni betwen mediation tosand aes dug eves. Gen oer Me, ‘s0100-205. Jha AK, Koparman Tech JM ety ing atese dug eves elope ol cm ‘ter baee manor ard arprisan wth chat ‘eviews oun ear Am Med Ina Aso. 1065305214. US General Accouning Oc, Aaverse Drag vent Mashing, OC US Geer Acauing (fice: 200 Publ No, GADHEHS-0021 Lisp Late OW, ln De Systems nals of adverse dug wens ANA 1085 m6 Intute of Medi, To rf ame: iting 2 Safer Heath Sytem Washington 0: No ‘al easy ress 09. ah, Garman PL Heed WR. Physician adr ey in US hospi. roc AWA Syn. 086, 2is200 Prcirenn CA Sit P, Sane JP. ASHP a ‘a sane of pumacy pais inhospa s ‘ings: pec nd zseaing—2001 Ad eat yr Par 2001 5822512058. Bates DM, Coben Lape LL. Ovethage ‘Shabt Mi Sherr T. eduigh racuoay erosin medese sing itrmacn what (09) J An es no Asc, 2001 8290-06 Wat, ar), Dowy 5, Female Cn putesad adie an rug dosage twimpro pr Scrbg pace acane Danas Sst Re uonscoo0a04 Mure, Hees RB, Hann SE Sith Kes compte tase neal econ supp tans on phys romance and pte 0 comes: stmt evi JAMA 198280 os ache RA, Goa, nde TA, t ‘capt dest 0 preven om dors cig evn: delopment and eva Sonia commaniy acing spl ANA aga 7-120 dM, Stan Pps Stl Ect ofcon putar-based alerts onthe eaten ard ou aes thsi pts. rch nen Me fesse 51-847 Tal OR, Sane TE, Wote WO, ea Mia alam 2nd tetany ontwous intusin aes thesia ston a comparison con iterated verve manual son yen ‘Tadotore Vase Anesth 1037200208, Baie Brches O, Gone 8 Coral A new Sedition conde far ‘euln of arabes pressure Bimed I Stun Tech! 1003.27 248.251 ‘Ais IM, Rees 6, Gover A, eal. Comp ‘ssid conhuous ius any ng (are anstesa comparison wih 2 manual ‘nthe, Aestesclgy1085 8-0 ‘Ovens OX eae Ri A normative sate ‘rarer for delopment of practice que les especie poplin. Med Deis olin, 195717400426, Har Haan M, Woot Set Cet ‘ode the US Pennie Seve Tek ore revo af the process. Am. Prev Med 20 asp ‘ya 6, Sehunanan Cok Dea Grades tt teemmandatan ar anhambate agente ree 20014389875. Bucher HC, Gaya GH, Cook Dd, Holo & Med forthe secs Basa Meine Warn Sau. Uses gues tothe mei rate XC appoglncl val ests how ‘uuse nan mesug the tt of anit etn n sug ené pois. ANA 1088 Py Mdow Langhorne , Grimshaw J neg Ing htropeaos pce of edancs in syste fers Ano len Med. 187 127-080 ms, Jun P. Wis. Bloch Egger. The hazards sein the quality of cna as fo ma sts AU 106 262 1054-1060, ‘Mee Briton A. Bick, MePerson Sand ttron Bain iho in eh cave Sear irrig the eieex:chowsing be ‘wen rndmaed an eo-ndoniae hats fw opaa12315, ‘Soup OF, Bain JA Warton SO, eal Met sais of ebseresional suds pie on proposal for parting AWA 2000288 ‘ice 2012 Jun P Asn 6, Egger, Systems ravens ina cate aes the quay of cool lnc tN 200 204246, ‘Shoji Duan BW, MDonag Wack ML Mig Hal ae Suter A Cia Ana Sis of Pan Sate Practices. skill, Ma osc rete Resear and uly 20 ARO putin Mo. O1-058 vera JM, Teme WM Zhu XH, Medora (Gl Aron altel odes" tee (wepnnytep) SRCHINTERN MEDIVOE IGE, JONES 00> ra 2. a. Py 2. o. a 2 a 6 « a. a vant rr of aniston. JA ed itr A oe 007498478, Bes Leap Cullen La Etotcon- uterine physio tam er ena on preven of seus median tor AWA 0062609317316, ie D Teich Lal Theat com- utercn ysis rar ey on maces Forpovrton Jam ie It Asa. 0006: mat Toth Mera Sebi. Kaptan ‘Spur, BtesOW, Etec of compere Sen ce ry in rsebing paces, Ach Inn 2000318027427. Chartow GM, Lee Kapeman Git ued mein dosing forint wither su Fein sAl 200 285250 2848, Burton, Ae C, il OP J ny T,Shop- rd MD. Vato MR. Acordia oh cost eat of computed bayesian amineghon sie acninitaton, ln Phaze Ter 190 ‘oes. Frans BS, iaseen DC Petonk Landed HP Buia JP Improving empire amine sk lesan singcongser ein supon Arch tome 1-8. vane, Past SL Clean, tal Acon- putersisted maagenarproga oan esd team agen Ent Me {one 05250258. Caner P, Rely A, Ho HA randomize con oldie carputarzed phamacouratthe- thle dosing versus ame pecan do ing. Cn Parnaco Thr 18853684 300, ey $F, Oo Ll, MeN gn AU. A fandom ert nal il paraco- Ina opine doseg. Ar Rey esp Ds. ogg ier210- 128, Muga DR, abe D Faster PL, ta. A pro- spesive randomized compas afte acc raeyoleomputrassisd vas GUSTO nome- (ya: ced hapa herp. Cn Pharmacol ‘er. 00435581506 White, Hang Veron Pea nisin ot wai tray comparison psn ds ing wth computed sin. Gen nn ea e791, ter FA, Lays A, Wes GA, Scat Users gues tte mal trate, Xp ling cal il rents, Buds foro tering whathr atu isenrg (nae) cet ANG 1062021711877 Rogers EM itsion flown. Naw York, Tse re 18. Loves Rey TBA, Sou 6, Dison otc ofthe peopl nd organization 4 s (©2003 American Medical Association, All rights reserved. ‘Downloaded From: http:/jamanetwork.com/pdfaccess.ashx?url=/data/journals/intemed/S447/ on 03/15/2017 spec of medial informas ew oe (rae. J Ar Med nm sso 109747058, Coser, Hels ans. Elregtc tor reued or surest nglemenaion bear norma, Sd Hoth Tes n= farm 1981-46-18 Berg M.Pat ere iefomation ah care wrt soca ‘ied 198958710. ‘Maeno TA oucing physi area ‘igre medio cer, pat on (grind cites bb, Aad ed 1288 Gezo2s, ‘ik 0 Ras. crea trian th udgy i te rel wa. Heath A aio) snae.7:2038 lar Tec, apernan Impact oir ‘mason evn an mada eI Proeeinge 1 the 196 HIMUS a Contrance. Ch go: Heer Intmaton an Marae net Systm Soci 199519. ‘Shana, Yoke, Pa Fisk. Mau, nes OW. Reducing varconjin ian us Inga conputraed gle rest of aan lin eon tt J Ar led int Assoc ‘ng $54 560, Toi J, laser Beckley RF, ta Toward cos five, uy ee agar Ca ug System. Sten 8 Procedngs Fam fe Scot mul Mobos Does CP tegnten Symposia. DC Compa: ‘ed Part Racor eu: 1065-24 “Tey Wd ier ME OvehgeM, MeDonad CL Physician instr ora wring on mio computer werkstatons: fects on ecu Eaton, JA 005260 70.3, ‘nes OW, Sp, Cl, a Toe costs of sonra dgversnosinaedpaions ANA wararra07 a Enns PsonkSL Csen 0 ute. E Sonola caput ass abit dose far an Pharmac 1H023:1025- 1051, Nazam OM, Macks AM Promating pet Salty whl te son Comm aa) Ingor 2001274s0-435 Mise A Gain 5, Dabanc SF, Saber, ck (CR mprovingtsy ota cart ep ‘ro inate: lo Pact 20003313 316 Medias Payment Aviary Commision. Ae porto te Congress: Stet Medicare sues ‘Washington, OC: Madeare Payment disoy Commis: 190 Mean rr aducton Att 200, $824, 17 Cong, tt es (2001 Calioma Sr 125, Chapa B15 5153263 (Septanoer 28,200)