You are on page 1of 18

Hosp Pharm 2013;48(3 Suppl 2):S1S5

2013 Thomas Land Publishers, Inc.


www.thomasland.com
doi: 10.1310/hpj4803-S1

A Sociotechnical Model for Pharmacy


David C. Classen, MD, MSp; and Jeff Brown MEd

he landmark Institute of Medicine (IOM) report


To Err Is Human, published in 2000, described
the magnitude of medical errors and breaches in
patient safety in the US health care system, together with
a blueprint for building safety systems in health care
organizations.1 Yet, reports from the IOM issued 12
years later suggest that little progress has been made in
improving safety in inpatient or ambulatory settings.2,3
Initial optimism that improving patient safety
could be quickly addressed has faded, and a growing
appreciation of the tremendous difficulty in making
health care error free has developed. At the time that To
Err Is Human was published by the IOM, there was
much enthusiasm that technology was the universal
antidote to safety problems. Indeed, there was almost
magical thinking that technology in the form of electronic health records (EHRs) would erase medication
safety issues entirely, if only they were widely adopted.
By 2013, there has been wide adoption of EHRs in
hospitals and ambulatory health care centers as well as
broad use of medication safety interventions such as
computerized prescriber order entry.4 This has been
due largely to the HITECH Act mandates and criteria
for meaningful use of electronic data the level of
use that providers must attain to qualify for incentive
payments under the HITECH Act.5
MEANINGFUL USE INCENTIVE PROGRAM
In 2009, the HITECH Act (part of the American
Recovery and Reinvestment Act) authorized expenditures of about $20 billion (with estimates ranging
from $9 billion to $27 billion) over 5 years to promote
the adoption and use of EHR technologies that would
be connected through a national health information
network. As its centerpiece, the HITECH Act created
incentives for all hospitals and eligible physicians not
only those associated with large systems to adopt
and use electronic information.6
The legislation set forth a plan for the meaningful
use of health information technology (HIT) to im-

prove the quality of care and enable changes in delivery


systems essential to health care reform. Additionally
under the HITECH Act, hospitals and physicians who
make meaningful use of interoperable EHRs can
qualify for extra payments through Medicare and
Medicaid. Physicians who adopt electronic records by
2014 can qualify for Medicare bonus payments of up
to $44,000. Beginning in 2016, those who have not
adopted EHRs will be penalized in the form of reduced
Medicare reimbursements. Similarly, Medicaid providers can receive up to $63,750 over 5 years. These
payments and penalties depend on the provider meeting
the requirements for meaningful use.
The broad goal is to gradually acclimate providers
to work-flow changes and practice improvement opportunities that, ideally, will accompany the adoption
of technology. Thus, the meaningful use criteria have
been structured into 3 stages, each progressively more
rigorous:
 Stage 1 criteria, which took effect in 2011, requires
that providers be able to electronically transmit medication orders, record patient information and problem lists, demonstrate use of decision support tools,
test systems to exchange health information with
other providers, and submit a small number of clinical quality measures.
 Stage 2 criteria, to take effect in 2014, will require
more robust exchange of information and other
high-value uses of EHRs.
 Stage 3 criteria, to take effect in 2015, will require
providers to demonstrate greater use of decision
support tools, higher levels of information exchange, and actual improvement in care coordination and patient outcomes.
Despite the broad adoption of technology to improve medication safety, reports of adverse drug
events in hospital inpatients continue to occur at a high
rate.2 A recent report cites poor technology design

*The University of Utah School of Medicine, Salt Lake City, Utah and Pascal Metrics Washington DC; Applied Research
Associates, Fairborn, Ohio. Corresponding author: David C. Classen, MD, MS, University of Utah School of Medicine, 561
East Northmont Way, Salt Lake City, UT 84103; phone: 385-226-1557; e-mail: dcclassen@hotmail.com

Hospital Pharmacy

S1

Sociotechnical Model for Pharmacy

and poor integration into the clinical environment as


reasons for technologys lack of impact on medication
safety.7 Both of these reports suggest that the development of a sociotechnical model is a key factor in
successfully improving safety.2,7 This paper outlines
how such a sociotechnical model might be applied in
the hospital pharmacy setting.
PATIENT SAFETY AND THE PHARMACY
The safety of medications has been an issue since
ancient times; as has been observed by many, there are no
unsafe medications just unsafe doses. Most developed
countries have regulatory systems that require the extensive evaluation of the safety of new medications that
are brought to market. However, the safe use of medications that are already on the market is an entirely different matter; there is little regulatory oversight, and
accountability for medication safety falls primarily upon
health care providers such as physicians, pharmacists, and
nurses.2 Pharmacists often play the major role in medication safety, especially in the ambulatory setting of care.
HEALTH INFORMATION TECHNOLOGY IN
THE PHARMACY
Pharmacy was one of the first areas in health care to
adopt HIT to improve medication safety. The movement
began in hospital pharmacies and spread to ambulatory
pharmacies. Although the initial focus of HIT was on
billing, the development of pharmacist order entry programs allowed for various forms of safety interventions,
such as checking for allergies or drug-drug interactions,
and calculating dosage adjustments. These practices became standard care in both inpatient and outpatient
settings. However, many studies have shown that compliance with these safety practices is less than optimal and
often puts patients at risk for adverse drug events.2
MEDICATION USE MODEL
Because of persistent medication safety issues in the
inpatient setting, hospitals began to adopt a broader
range of technologies in the pharmacy to reduce errors.
These included robots for medication dispensing,
computerized prescriber order entry for medication
ordering, and barcoding for medication administration.
The goal was the virtual elimination of medication safety
hazards through technological interventions. A schematic of these various technologies and their application
in the medication use process is shown in Figure 1.
RESULT OF AUTOMATION IN THE PHARMACY
The US Veterans Administration (VA) hospitals
took the lead in implementing pharmacy technologies in

S2

Volume 48, Suppl 2, 2013

their inpatient facilities across the country. Automation


of the medication use process was a major step forward
in improving medication safety and served as a model
for adoption by other organizations. As such, there
was great interest in the impact of technology on actual medication safety. A group of researchers at a VA
hospital undertook a large study of the occurrence of
adverse drug events across all stages of the computerized medication use process.8 They found a high incidence of medication safety problems, with 483
clinically significant inpatient adverse drug events
among 937 hospital admissions in a 20-week period. It
was concluded that the major reason for these adverse
events was a failure to optimize the technology for the
sociotechnical environment in which it was used and a
failure to optimize the technology after implementation
(eg, development and implementation of effective
clinical decision support within the EHR system).
CHALLENGES WITH PHARMACY AUTOMATION
Many hospitals have emulated what the VA has
done by automating the entire medication use process.
However, these projects are complex, expensive, and
time consuming; additionally, they are highly dependent on the implementation and integration with an
EHR. In many hospitals, these projects have been
technology focused and often not conducted within
or even in consideration of their existing sociotechnical
environment.9 Consequently, the potential safety benefits of such technologies have not been fully realized.2
SOCIOTECHNICAL MODEL
The concept of sociotechnical systems arose from
studies of work organization, work systems, and
mechanization in the British coal mining industry beginning in the late 1940s.10 These studies were spurred
by the realization that increased mechanization had
generally not yielded increased coal production, as had
been anticipated. However, productivity had risen
in some mechanized mines, and research in these
helped to identify organizational and work-design
principles to optimize the fit of people and technology
as interrelated and complementary components of
work systems.10 Instead of designing technology and
dropping it abruptly in the workplace without
considering its affect on human work performance
and vice versa it became apparent that technology
designers needed to take an integrative perspective on
human and technical system roles and tasks.
Over the next 6 decades, safety investigation in
high-risk sociotechnical domains, such as aerospace
and air transportation, identified organizational and

Sociotechnical Model for Pharmacy

Figure 1. Technologies to reduce errors in the medication management process.


work-design principles that support not only productivity, but also risk mitigation and safety.11-15 In late
2011, the IOM introduced health professionals to
these principles, presenting a contemporary model of
sociotechnical systems in its report Health IT and
Patient Safety.2 This report highlighted that accidental
patient injury or death typically occurs as a result of
the unanticipated effects of the interactions among
sociotechnical system components namely, people,
technology, processes, organization, and environment.
For example, successful medication administration using a smart intravenous pump infusion system
(smart pump) is not simply the product of a nurses
interaction with the pumps computer interface at the
bedside. Delivery of medication to a patient via the
smart pump is the end stage of a more extensive
process, which includes the physician order and fulfillment process, and has many points of potential

interactive missteps. Following are some examples of


potential interactive misadventures that were identified after an organization-wide smart pump implementation (Brown, field notes).
 Physicians handwritten prescriptions were sometimes
misinterpreted, resulting in processing of incorrect
medication and dosage at the onset of the medication
preparation and administration process. These errors
were usually caught by pharmacy or at the bedside,
but sometimes they were not noticed before the incorrect medication/dose was administered to the patient.
 The nurses operation and management of the pump
occurred in a clinical environment characterized by
high levels of noise, which made alarm detection
difficult outside of the patients room.
 The pumps alarm had the same tone for all conditions, preventing auditory discrimination of criticality.

Hospital Pharmacy

S3

Sociotechnical Model for Pharmacy

 False upstream occlusion alarms were numerous,


lending to alarm fatigue and distrust of alarm
significance.
 In a dimly lit clinical setting, a small and non-illuminated operator screen was difficult to read and
operate. Fat fingering hitting incorrect keys
often resulted in mis-sets and re-sets.
 If the drug to be administered was not in the
pumps drug library, the nurse needed to operate
the pump in basic mode, overriding its safety features. This occurred frequently.
 Nurses were often unfamiliar with the names of
generic drug substitutions, which made it challenging to cross-check that they had the correct medication for their patient. Google searches were
heavily relied upon.
 The therapeutic dosing rate for small adult and
pediatric patients was sometimes lower than the
pump could achieve. Nurses tried to compensate
for this constraint by periodically interrupting the
infusion in an effort to limit the dose.
 Nurses were frequently confronted with the need to
calculate unit conversions, when the pharmacys
units of measure differed from those used in the
physicians order. Sometimes, the difference in units
was not identified, leading to incorrect dosing.
 Work organization for nurses provided little or no
timely opportunity for mutual support and crosschecking of unit conversion calculations and other
problem solving.
 Reach-back by bedside nurses to pharmacy, for
timely guidance regarding a medication, was not
supported by pharmacy organization, staffing, and
work processes.
 Competing demands on nurses attention regularly
interrupted pump set-up, resulting in prospective
memory challenges; failing to come back to the
appropriate point in the interrupted procedure
upon returning to the task.
The challenges that were observed with the clinical
use of smart pumps arose from interactions among
people, technology, processes, organization, and
environment not simply from a nurse interacting with
a smart pump. Although some of the challenges might
seem innocuous, they nonetheless harbored risk to
patients. All were implicated in instances of patients
receiving incorrect medications and/or incorrect dosages.
There were no instances of lasting patient harm in the
cases identified during this study, but the opportunity for
death or serious injury due to the administration of an
incorrect medication and/or dosage was clearly present.

S4

Volume 48, Suppl 2, 2013

The most common organizational response to


the potential misadventures was to retrain pharmacy
and nursing personnel on policy and procedure and,
in the case of nursing personnel, the operation of the
smart pump. This focus on the people closest to the
misadventure those who were attempting to safely
fulfill their tasks and patient care responsibilities
despite awkward technology, turbulent processes,
disruptive work environments, and unsupportive
work organization left conditions for failure essentially untouched.
NEED FOR HUMAN FACTORS RESEARCH AND PRACTICE
IN HEALTH CARE
Approaches to the improvement of patient safety
have thus far had transient and limited benefit.16 As
a result, there is a widespread call for increased application of human factors principles and methods
to improve the design and functionality of health
care systems, especially in areas where technology is
applied.2,17,18
The overarching goal of human factors professionals is to aid in designing tools, processes,
technologies, organizations, and environments that
support safe and effective human performance. Too
often, designs for health care technologies are based
on flawed assumptions about the domain and environment of use or on requirements derived from
preference surveys, focus groups, and market research
approaches that typically do not yield designs that
match the operators needs in the context of use.7
Despite recognition of the need, no human factors
toolkit is available from health care oversight entities
or technology vendors to achieve these purposes. It is
imperative that health care organizations depart from
the notion of safety management as something that
can be achieved through a succession of narrowly
defined safety projects that address surface manifestations of adverse component interactions that
is, the phenotype rather than the genotype, the component rather than its interactive context. This is especially important in areas of health care where
technology has been aggressively adopted, such as
medication safety. While there is need for research to
deepen understanding of safety management in complex sociotechnical systems, human factors principles
and methods that can help already exist, yet are rarely
applied to safety management in health care.19-21
Cognitive system engineering (CSE) is a branch of
human factors science and practice that has arisen
specifically to study and support risk mitigation and
safety improvement in sociotechnical systems.22 CSE

Sociotechnical Model for Pharmacy

specialists develop technology design requirements by


first researching the context for which a new technology is contemplated. They consider the interactive
effects of system components in developing design requirements and support implementation efforts that
are sensitive to unanticipated risks. They also conduct
advanced safety investigations, seeking an understanding of adverse events through a sociotechnical
lens. Benefits of CSE applied to development of design
requirements include technologies that best match the
needs of clinicians in the context of use, proactive
surveillance for emergent risk and the development
of corrective strategies, and more effective adverse
event investigation leading to more effective safety interventions. Had this approach been used for automation of the medication use process, more safety
benefits might have been achieved.2
Improving patient safety remains a major challenge for the US health care system. Early results suggest
that the broad adoption of technology to improve safety
has not achieved the intended result, especially in the
area of medication safety an area that has seen perhaps the most technology adoption.2 The IOM has
called for a new approach to patient safety: the sociotechnical model, which offers the potential for a leap
ahead in system-based safety management for health
care. To truly effect improvement in patient safety,
safety management must become autonomic. It is essential to the safety-health of the organization, underlying and guiding all activity.
REFERENCES
1. Institute of Medicine. To Err Is Human: Building a Safer
Health System. Washington, DC: National Academy Press; 2000.
2. Institute of Medicine. Health IT and Patient Safety: Building
Safer Systems for Better Care. 2011. http://www.iom.edu/
Reports/2011/Health-IT-and-Patient-Safety-Building-SaferSystems-for-Better-Care.aspx. Accessed January 22, 2013.
3. Wynia MK, Classen DC. Improving ambulatory patient
safety: learning from the last decade, moving ahead in the next.
JAMA. 2011;306:2504-2505.
4. Pedersen CA, Schneider PJ, Scheckelhoff DJ. ASHP national survey of pharmacy practice in hospital settings:
dispensing and administration2011. Am J Health Syst
Pharm. 2012;69:768-785.
5. Blumenthal D. Implementation of the Federal Health Information Technology Initiative. N Engl J Med. 2011;365:
2426-2431.
6. Presidents Council of Advisors on Science and Technology. Report to the President. Realizing the Full Potential of
Health Information Technology to Improve Healthcare for
Americans: The Path Forward. Office of Science and Technology Policy. December 2010. http://www.whitehouse.gov/

sites/default/files/microsites/ostp/pcast-nitrd-report-2010.pdf.
Accessed January 23, 2013.
7. Brown J, Tonkel J, Classen D. Using technology to enhance
safety. In: The Essential Guide for Patient Safety Officers. 2nd
ed. Oakbrook, IL: The Joint Commission; 2013.
8. Nebeker J, Hoffman J, Weir C, et al. High rates of adverse
drug events in highly computerized hospital. Arch Intern Med.
2005;165:1111-1116.
9. Troiano D, Morrison J, Federico F, Classen D. Safely automating the medication use process. Not as easy as it looks.
J Healthc Inf Manag. 2009;23(4):17-23.
10. Trist E. The evolution of socio-technical systems: a conceptual framework and an action research program. In: Van de
Ven A, Joyce W, eds. Perspectives on Organizational Design
and Behavior. NY: Wiley Interscience; 1981.
11. Rasmussen J. Human errors. A taxonomy for describing
human malfunction in industrial installations. J Occup Accident. 1982;4:311-333.
12. Rasmussen J, Duncan K, Leplat J, eds. New Technology and
Human Error. Vol. 4. New York: John Wiley & Sons; 1987.
13. Hollnagel E, Nemeth CP, Dekker S, eds. Resilience
Engineering Perspectives. Volume 1: Remaining Sensitive to
the Possibility of Failure. Farnham, UK: Ashgate Publishing
Company; 2008.
14. Weick KE, Sutcliffe KM. Managing the Unexpected: Resilient Performance in an Age of Uncertainty. 2nd ed. New
York: Jossey-Bass; 2007.
15. Patankar MS, Brown JP, Treadwell MD. Safety Ethics: Cases
from Aviation, Healthcare, and Occupational and Environmental
Health. Farnham, UK: Ashgate Publishing Limited; 2005.
16. Classen DC, Resar R, Griffin F, et al. Global trigger tool
shows that adverse events in hospitals may be ten times greater
than previously measured. Health Aff (Millwood). 2011;30:
581-589.
17. Gurses AP, Ozok AA, Pronovost PJ. Time to accelerate
integration of human factors and ergonomics in patient safety.
BMJ Qual Saf. 2012;21:347-351.
18. Brown JP. Achieving high reliability: other industries
can help health cares safety transformation. J Healthc Risk
Manag. 2004;24(2):15-25.
19. Nemeth CP, Wears RL, Woods DD, et al. Minding the
gaps: creating resilience in healthcare. In: Henriksen K, Battles JP,
Keyes MA, Grady ML, eds. Advances in Patient Safety: New
Directions and Alternative Approaches. Vol. 3. Performance
and Tools. Rockville, MD: AHRQ. Publication No. 08-0034-3.
20. Dekker S. Drift into Failure: From Hunting Broken
Components to Understanding Complex Systems. Farnham,
UK: Ashgate; 2011.
21. Woods DD, Hollnagel E. Joint Cognitive Systems: Patterns in
Cognitive Systems Engineering. Boca Raton, FL: CRC Press; 2006.
22. Hollnagel E, Woods DD. Joint Cognitive Systems: Foundations of Cognitive Systems Engineering. Boca Raton, FL:
CRC Press; 2005. g

Hospital Pharmacy

S5

Hosp Pharm 2013;48(3 Suppl 2):S6S9


2013 Thomas Land Publishers, Inc.
www.thomasland.com
doi: 10.1310/hpj4803-S6

Technology and Automation in Hospital Pharmacies:


Current and Future States
Mark H. Siska, BS Pharm, MBAp

he use of automation and information systems


in hospital pharmacy practice surfaced in the
1960s, primarily to manage the growing operational demands with limited resources. The professional practice model transitioned from a focus on
distribution, preparation, and dispensing in the 1970s
to patient-centered pharmaceutical care in the early
1990s and then to medication therapy management at
the turn of the millennium. As a result, the use of
automation and technology to augment and advance
hospital pharmacy practice became critical.
The publication of consecutive Institute of Medicine (IOM) reports To Err is Human: Building a
Safer Health System (1999) and Crossing the Quality
Chasm (2001) significantly changed the landscape
of medication management technology.1,2 Although
operational efficiency remained an important driver,
the IOM reports placed technology and automation
in the forefront for improving the quality and safety
of medication use. The 1999 report made it clear that,
to advance medication safety, health-system pharmacies must implement proven medication safety
practices to reduce reliance on memory; to standardize terminology; to use constraints and forcing
functions, protocols, and checklists; and to minimize
data handoffs.1 The report also referenced several
emerging technologies and called for automation of
patient-specific clinical information within the context
of an electronic health record (EHR) integrated with
computerized prescriber order entry (CPOE), drug
distribution, and medication administration systems.
These systems ideally work together within and
across organizational boundaries; share real-time,
patient-specific clinical information; provide timely,
relevant clinical-decision support at the point of care;
and create data-mining capabilities.1,2
In this article, I examine the current state of
hospital pharmacy technology, level of integration,
and security and privacy considerations. In addition,

I discuss the future state of technology in hospital


pharmacy practice and the role pharmacy informatics
will play to support next generation systems.
CURRENT STATE OF HIT: BUILDING THE CORE
Despite increased levels of awareness and the
development of a conceptual framework for an ideal
technology-enabled medication use process, adoption
was slow until the passage of the American Recovery
and Reinvestment Act and meaningful use measures
in 2009 and 2010.3,4 This legislation provided the
stimulus in the form of financial incentives and eventual penalties to aid health systems with implementing
core medication management supporting technologies.
More important, these measures demonstrated a commitment by health care and government leaders to
leverage health information technology (HIT) to improve the quality and safety of the medication use
process. Prior to the meaningful use ruling in 2010,
hospitals exhibited limited adoption of CPOE;
however, rates nearly tripled from 19% in 2010 to
54% in 2012 following its publication.5 Similar increases were observed with the implementation of
other core systems, including barcode-enabled medication administration, medication reconciliation systems, and electronic medical records (Figure 1).6 The
publication of the meaningful use timelines and corresponding financial incentives have kept health systems busy building and implementing applications
that support core medication management.
INTEGRATION
A fully integrated, technology-enabled closed-loop
medication management system is designed to feed
outcomes from medication processes back into the
system to facilitate the development and use of improved practices and to effectively monitor patient
care across the continuum. A closed-loop system
consists of a succession of applications that are

*Assistant Director, Informatics & Technology, Pharmacy Services, Mayo Clinic, 201 W. Center Street, Rochester, MN 55902;
e-mail: siska.mark@mayo.edu

S6

Volume 48, Suppl 2, 2013

Technology and Automation in Hospital Pharmacies

Figure 1. Percentage of US hospitals implementing core


medication use process systems 2009-2012.5,6 EMR 5
electronic medical record; CPOE-CDS 5 computerized
prescriber order entry and clinical decision support;
BCMA 5 barcode medication administration; Med Rec 5
medication reconciliation system.
interconnected and can share data fully and in real
time. The transition between each process is seamless;
information is computer readable and usable across
all systems that support the medication use process.
Closed-loop medication management effectively reduces many steps in the complex process, significantly
reducing opportunities for error.7
Despite a well-designed plan to build an ideal,
technology-enabled medication management infrastructure, achievement of a fully integrated state has
been difficult. Obstacles include the absence of welldefined processes, a lack of uniform technical and
semantic standards, as well as the desire to expeditiously address the safety and quality concerns identified by the IOM. Instead, a focus on single functional
areas and quick wins has produced a number of
single-threaded and ineffectively integrated solutions.
The first-generation EHR medication management
applications have offered good integration within
their proprietary suite of tools. However, the lack of
both incentives for vendors to connect and an overall
integration strategy within the medication management domain has made integration a significant
challenge, with many remaining opportunities for
improvement.
FUTURE STATE
Any conversations involving the future state of
automation and technology for medication management must center on connectivity and interoperability,
patient-centered care, remote and mobile capabilities,

technology-enabled technician distribution, automation, and analytics.8


The future practice model positions pharmacists in
the role of medication therapy managers. Thus, it
is clear that the supporting technologies will require
access to large amounts of patient-centered, real-time
data and information that are effectively filtered and
prioritized across the continuum of care. Integrated
information systems must allow for continuous monitoring and detection of medication-related problems
and prospectively alert the pharmacist of the need for
intervention. In addition, the information must be accessible remotely, when and where the user needs it and
independent of the computer or mobile device being
used.9
The operational systems must be built around the
clinical systems and created in a way that permits
product selection, dispensing, distribution, and preparation oversight. To support a technology-enabled
technician distribution model, it will be necessary for
the technologies to provide additional assurances that
technicians have performed distribution tasks correctly and in an auditable fashion.10
Despite offering a number of significant benefits,
currently available pharmacy operational systems will
require dramatic re-engineering in the emerging era of
integration and closed-loop medication management.
These systems must evolve and progress to pharmacy
patient care systems for the provision of medication
therapy management (Table 1). Additionally, they
must incorporate functionality to support pharmacist
documentation; notification of colleagues and other
disciplines of the need to intervene; surveillance, review, and analysis of clinical data for monitoring; and
education of patients and providers.11
ROLE OF PHARMACY INFORMATICS
The future state of technology to support medication management will depend on pharmacy informatics reform. This pharmacy subspecialty will
be called upon to develop systems that support
clinical practice, with drug distribution and preparation as end products of managing medication therapy and improving medication-related outcomes.10,11
Pharmacy informatics will play a significant role in
medication-related data quality management, security and privacy, monitoring, and ongoing measurement and evaluation of the effectiveness of
technology-related solutions. Informaticists will participate in development efforts and in maintaining
current technology and knowledge assets that include
the following12:

Hospital Pharmacy

S7

Technology and Automation in Hospital Pharmacies

Table 1. Pharmacy patient care system functional


requirements11
System domain
Documentation

Examples





Medication histories
Home medication lists
Allergies and intolerances
Pharmaceutical care plans

Responsibility-sharing






RPh-patient assignments
RPh work calendars
Facilitate RPh consults
Messaging between RPhs

Review and analysis






Holistic med profile review


Prioritization of care activity
Pharmacokinetic analyses
Pharmacodynamic analyses

Education management






Patient education content


Track education provided
Patient learning assessment
Provider education content

Note: RPh 5 Registered pharmacist.

 Corrective maintenance steps to correct production-related problems with the network, interface,
hardware, software, or utilization of information
systems.
 Customized maintenance steps to modify already
existing functionality to improve the user interface
or address new regulations or safety and quality
initiatives.
 Enhancement maintenance steps to improve the
system beyond already-existing capabilities, including hardware and software.
 Preventive maintenance steps to reduce the risk of
system problems, replace servers or hardware, and
upgrade security patches.
Pharmacy informatics requires the continuous study
of the human impact of information systems and
development of technologies and automation to optimize
safety and quality. New skills will be required, including13:
 A firm grasp of the fundamental principles of pharmacy practice and the medication use process,
 Comprehensive knowledge about the safe and effective use of medications,
 Expertise to effectively translate and seamlessly
communicate medication information across the
medication use process continuum of care,
 Ability to interpret and implement requirements to
ensure the safe and comprehensive use of medications across all disciplines and processes,

S8

Volume 48, Suppl 2, 2013

 Exploratory analysis coupled with deep knowledge


of clinical processes,
 Outcomes data analytical skills, and
 Understanding of processoutcome relationships.
SECURITY AND PRIVACY CONSIDERATIONS
As primary administrators of medication-related
data, those who practice pharmacy informatics must
maintain a firm grasp of their organizations security
and privacy strategies, policies, and procedures. Furthermore, they must understand the potential threats
and sources of security and privacy breaches and ensure that all needed administrative, physical, and
technical controls are in place to reduce privacy and
security risks.14
SUMMARY
An environmental scan of the current and future
state of medication management technologies reveals
that most health systems have moved beyond good
intentions to implementing the core applications and
making progress toward a technology-enabled closedloop medication system. The initiation of the meaningful use objectives and associated financial incentives
and penalties has prompted health systems to take
more determined steps to improve adoption. Current
pharmacy information management systems must
change from a focus on distribution and fulfillment to
that which supports patient-centric pharmaceutical
care across the medication use continuum.
REFERENCES
1. Kohn LT, Corrigan JM, Donaldson MS, eds. To Err Is
Human: Building a Safer Health System. Washington, DC:
National Academy Press; 1999.
2. Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st
Century. Washington, DC: National Academy Press; 2001.
3. 111th Congress of the United States of America. American
Recovery and Reinvestment Act of 2009Title XIII: Health
Information Technology. http://www.gpo.gov/fdsys/pkg/BILLS111hr1enr/pdf/BILLS-111hr1enr.pdf.bin/getdoc.cgidbname5
111cong_bills&docid5fih1enr.pdf. Accessed January 15, 2013.
4. Centers for Medicare and Medicaid Services. Final rule on
meaningful use. Fed Regist. 2010;75:4431444588.
5. Pedersen CA, Schneider PJ, Scheckelhoff DJ. 2012 ASHP
national survey results: implications and trends for todays
practice. Session presented at the 2012 ASHP Midyear Clinical
Meeting; December 5, 2012; Las Vegas, NV.
6. The 7th annual state of pharmacy automation. Pharm
Purch Prod. 2012;9(8):1-96. http://www.pppmag.com/article/

Technology and Automation in Hospital Pharmacies

pppv9n8s0/State_of_Pharmacy_Automation_2012. Accessed
January 15, 2013.
7. Bowles M. Closed loop electronic medication management. White paper. http://www.marcbowles.com/Publications/
CLEMM%20Whitepaper_Final_Extract.pdf. Accessed January 15, 2013.
8. Fortier CR. Safe, effective, and efficient medication distribution: challenging paradigms and bedrocks. Am J Health Syst
Pharm. 2011;68:1127.
9. Buchholz D, Dunlop J, Jimison E, Maxson G. Enabling
device-independent mobility with dynamic virtual clients. White
paper; Intel Corporation: November 2009. http://www.intel.
com/content/dam/doc/white-paper/intel-it-mobile-computingindependent-mobility-dynamic-virtual-clients-paper.pdf. Accessed January 15, 2013.
10. Technology-enabled practice: a vision statement by the
ASHP Section of Pharmacy Informatics. Am J Health Syst
Pharm. 2009; 66:15731577.

11. Flynn A. Technology innovation priorities to support a


relational, patient-side pharmacy practice. Presented at: University Hospital Pharmacy Executive Group. Considerations in
Advancing the Practice of Pharmacy in Academic Medical
Centers. University of Michigan, Ann Arbor; March 2-3, 2012.
12. Health Information Technology Toolkit. Ongoing System
Maintenance, Administration, and Data Quality Management.
Bloomington, MN: Stratis Health; 2009. www.stratishealth.
org/documents/HITToolkitNH/2.Utilize/2.1Implement/2.1.21
Ongoing_System_Maintenance.doc. Accessed January 24, 2013.
13. American Society of Health-System Pharmacists. ASHP
statement on the pharmacists role in informatics. Am J Health
Syst Pharm. 2007;64:200203.
14. Houlding D. Health information at risk: successful strategies for healthcare security and privacy. White paper. Intel
Corporation; 2011. http://www.intel.com/content/dam/www/
public/us/en/documents/white-papers/strategies-for-healthcaresecurity-and-privacy-paper.pdf. Accessed January 15, 2013. g

Hospital Pharmacy

S9

Hosp Pharm 2013;48(3 Suppl 2):S10S14


2013 Thomas Land Publishers, Inc.
www.thomasland.com
doi: 10.1310/hpj4803-S10

Implementation of a Clinical Decision Support System


Stacy Calloway, MS, PharmD, BCPSp

odays health care environment imposes many


challenges on health care organizations and
providers, including the increasing complexity
of patient care coupled with the dynamic nature of
medical information, clinical practice guidelines,
and mandatory reporting requirements. New federal
requirements have been put in place for electronic
medical records and clinical decision support, as well
as critical patient safety issues such as health care
associated infections (HAIs), antibiotic-resistant infectious diseases, and medication errors. A major
barrier to addressing these challenges may be the
fragmentation of medical information, with data
spread across various databases and systems within an
organization. This hinders hospital-based clinicians
from retrieving relevant and timely information
needed to provide quality patient care. Emerging
evidence suggests that clinical information technologies, such as electronic medical records, computerized
physician order entry, and electronic decision support,
can improve the quality of care within a hospital.1-3
Medical institutions are increasingly using clinical
decision support systems (CDSS) to assist clinicians
in maximizing the benefits of clinical surveillance
throughout their organizations.4,5 These systems integrate patient data from a variety of hospital information sources and offer tools for automated,
real-time surveillance, as well as alerting, analysis,
and reporting capabilities. A 2009 study found that
hospitals with automated notes and records, order
entry, and clinical decision support had fewer complications, lower mortality rates, and lower costs.6
Hospitals that had higher scores in decision support
were associated with a 16% decrease in adjusted odds
of complications, as well as a $538 lower cost for all
hospital admissions.
Although CDSS is a highly promising approach
for identifying patients who would most benefit from
interventions by pharmacy clinicians, there are few
reported examples of the effect of implementing CDSS
on clinical pharmacy measures. This article describes

the impact of this system on pharmacy clinical interventions and the costs associated with those interventions. In addition, the processes and lessons learned
from this initiative are discussed.
IMPLEMENTATION OF SOFTWARE
Good Shepherd Medical Center is a 425-bed
acute-care community hospital in eastern Texas. The
pharmacy department uses a decentralized model, with
14 staff pharmacists on site along with 3 remote orderverification pharmacists, 4 night-shift pharmacists, 3
clinical pharmacists, and 29 pharmacy technicians. The
pharmacists are decentralized to serve the intensive care
unit (ICU), neonatal ICU (NICU), telemetry units, medical/surgery units, central pharmacy, and the IV admixture room. The clinical pharmacists specialize in
oncology, critical care, and internal medicine; they address complex clinical questions and perform therapeutic
drug monitoring of dangerous/high-risk and high-cost
drugs. These pharmacists also serve in staffing roles when
needed, attend daily multidisciplinary rounds, and participate on hospital committees.
Good Shepherd implemented a CDSS in February
2011 to document clinical interventions, follow drug
consults, and document adverse drug reactions. This
data-mining software facilitates clinical decision
making, medication safety, and infection prevention
and management. The pharmacy department has
trained multiple departments in the hospital, including
case management, social work, and physicians, on the
use of the software to optimize their workflow. To
fully optimize the new system within the department of
pharmacy, in-service training was provided to all the
pharmacists; a computer-based learning module was
also required to document competency on the new
system. Pharmacists learned how to look up patients,
follow pharmacokinetic consults, and document interventions and adverse drug reactions. To manage daily
work flow, pharmacists learned how to run daily
alerts for intravenous to oral (IV to PO) conversions
and to adjust dosages for changes in renal function.

*Pharmacy Clinical Director, Good Shepherd Medical Center, 700 East Marshall Avenue, Longview, Texas 75601; e-mail:
scalloway@gsmc.org

S10

Volume 48, Suppl 2, 2013

Implementation of a Clinical Decision Support System

Department standards were revised to add clinical


intervention documentation. An average of 5 pharmacy interventions per shift was set as standard,
and an average of 8 documented interventions per shift
achieved above standard. Cost values for the
interventions were unchanged to maintain consistency
in reporting; however, new intervention categories
were assigned cost values based on previously published pharmacy intervention literature.7-11
MULTIDISCIPLINARY ROUNDS:
WORKFLOW REDUCTION
On a daily basis, the 3 clinical pharmacists meet
with a multidisciplinary care team (MDT) representing
social work, case management, dietary, respiratory,
nursing, and physical therapy on each unit. The MDTs
discuss individual patients to optimize care and expedite discharge.
Before the implementation of CDSS, pharmacists
who attended multidisciplinary rounds would perform
time-consuming pre-rounding functions, requiring
them to print out patients medication profiles and lists
of patients to follow. An entirely paper-based record
system was used to provide case summaries and to
document the MDTs discussion and decisions.
With the new clinical surveillance system, pharmacists bring their tablet computers to rounds. This
enables them to access patient profiles, identify and act
on any drug-related problems, and document interventions in real time. During rounds, the pharmacists
screen patients for IV to PO conversions, renal dosing,
antibiotic streamlining, and any drug-related problems. In addition, they review any alerts (discussed
below) that have fired for specific patients and determine whether to leave a note on the patients chart
or call the physician.
TARGETED CLINICAL ALERTS
The CDSS software allows the pharmacy to set up
alerts to identify patients for priority clinical interventions. The decentralized pharmacists can run alerts
on patients who may need renal dose adjustments or
are candidates for IV to PO switch. The pharmacy
department has a hospital-approved protocol for
pharmacists to follow in performing these services, in
addition to pharmacokinetic dosing, without physician approval. Other alerts were set up for situations
that required follow-up by a clinical pharmacist, such
as those with more chart work and interaction with the
patient and physician.
To realize some of our major clinical initiatives,
hospital administration identified antimicrobial

stewardship, adverse drug reactions, and core measures as target areas for CDSS alerts and pharmacy
interventions.
Antimicrobial Stewardship Alert
Improving antimicrobial use and preventing antimicrobial resistance are key priorities for hospitals
nationwide. Antimicrobial stewardship programs
ensure appropriate drug selection, dosing, and duration. According to the Centers for Disease Control
and Prevention, studies indicate that nearly 50% of antimicrobial use in hospitals is unnecessary or inappropriate.7
Furthermore, overuse of antibiotics contributes to the
growing challenges posed by Clostridium difficile and
other antibiotic-resistant bacteria.12
CDSS can enable pharmacists to compare and
leverage patient data from different systems for
example, pharmacy orders and microbiology results
in real time. It also offers tools for identifying potential
adverse drug events, IV to PO conversion opportunities, drug-bug mismatches, and discontinuation or
de-escalation opportunities in a timely manner. Another way the CDSS has aided antibiotic stewardship
is in its ability to compile an instant antibiogram.
Before the new software was implemented, antibiogram data were compiled and reported manually and
this usually took between 1 and 2 weeks. Antibiograms are created by the CDSS data-mining capability
to pull microbiology data directly from the laboratorys data system. The pharmacist can decide
which isolates and drugs to include in the antibiogram
and can specify dates and units of the hospital to
get the most accurate and up-to-date view of institutional resistance patterns. Instead of generating a
hospitalwide antibiogram every year, we are able to
generate instant snapshots and quarterly reports for
the units.
Once an institutions resistance patterns have been
identified, empiric therapy can be developed for various disease states. Information derived from the
antibiogram can also justify development of antibiotic
restrictions and use of pharmacy-preferred antibiotics.
In collaboration with the emergency department at
Good Shepherd, physician guidelines were developed
for the treatment of urinary tract infections (UTIs) in
the inpatient and outpatient settings based on unitspecific data generated by CDSS. Although many
institutions typically use a fluoroquinolone for UTI
therapy, by running a quick antibiogram we learned
that E. coli susceptibilities to this class are relatively
low (74%), whereas susceptibilities to ceftriaxone
exceed 95%.

Hospital Pharmacy

S11

Implementation of a Clinical Decision Support System

Antimicrobial stewardship efforts have been multifaceted at Good Shepherd, with a more formalized
program taking shape in recent years. An initial effort
was the development of criteria for the use of vancomycin. Patients who have received vancomycin for
more than 72 hours are screened via the CDSS alert.
Upon the determination of vancomycin trough goals,
the pharmacist assesses whether vancomycin is indicated and intervenes if the patient does not meet
criteria for use.
The CDSS antimicrobial monitoring alerts are
useful for identifying drug-bug mismatches and positive cultures with no antibiotic orders on the patients
profile. Pharmacists keep track of culture results for
the patients on their units, in addition to days of antibiotic therapy, to determine when to intervene for
drug de-escalation and discontinuation. Dose optimization and IV to PO conversion also play a large role
in stewardship efforts. Making sure that patients are
on the proper dose to optimally manage an infection
may require an increase in dosage of antibiotics, including cefepime, piperacillin-tazobactam, and other
antibiotics that require high-dose regimens for some
types of infections. A renal dosing protocol allows
pharmacists to automatically adjust medications, including certain antibiotics, to avoid adverse effects in
patients who have fluctuations in renal function.
Adverse Drug Reactions: Trigger Tools Alert
The use of CDSS has become instrumental in the
identification and documentation of adverse drug
events (ADEs). The concept of trigger tools involves
identification of abnormal lab values or the dispensing
of antidote-type medications to ascertain a possible
ADE. Lab values in the trigger tool alert function in-

5000
4500
4000
3500
3000
2500
2000
1500
1000
500
0

clude neutropenia (white blood cell count , 3,000/mm3),


thrombocytopenia (platelet count , 50,000/mm3),
hypoglycemia (blood glucose level , 60 mg/dL), international normalized ratio (INR) . 3, activated
partial thromboplastin time . 100 seconds, elevated
serum drug levels, and presence of C. difficile toxin. A
report can be generated daily to identify patients who
have these lab triggers. The pharmacist can review
a complete list of medications dispensed to those patients using the CDSS platform.
Antidote medications used to reverse unfavorable
physiological states include phytonadione for high
INR, digoxin immune Fab for digoxin toxicity, glucagon or 50% dextrose in water for low glucose levels,
or naloxone for oversedation. Patients receiving these
antidotes are screened for potential ADEs; this activity is documented in the patients medical record as a
clinical intervention. Documentation includes the type
of ADE and level of severity and whether treatment
was required. These data are compiled monthly with
other reported ADEs and presented to the hospitals
Pharmacy and Therapeutics Subcommittee and the
Patient Care Committee. The use of CDSS to identify
these triggers and efficiently compile data in a simple,
straightforward fashion makes this program invaluable to the pharmacist attempting to identify trends in
problems related to a certain class of drug or to a
particular unit of the hospital.
Core Measures Alerts
Alerts were also created within the CDSS to
identify patients who may be failing core measures
based on their disease state. For example, core measures alerts were created for patients with stroke, including those admitted with a diagnosis of cerebrovascular

CDSS Implementaon

Figure 1. Number of pharmacist clinical interventions accepted by physicians: before and during CDSS implementation. CDSS 5 clinical decision support system.

S12

Volume 48, Suppl 2, 2013

Implementation of a Clinical Decision Support System

$300,000
$250,000

CDSS Implementaon

$200,000
$150,000
$100,000
$50,000
$0

Figure 2. Accepted pharmacist clinical interventions translated into dollar savings: before and during CDSS implementation. CDSS 5 clinical decision support system.
event, stroke, or transient ischemic attack, or those
who had received alteplase therapy. Medication profiles for stroke patients are reviewed for a statin or
antithrombotic agent. Patients without the required
medications are forwarded to the hospitals core
measure specialists for follow-up. This type of alert
was also set up for patients admitted with congestive
heart failure or acute myocardial infarction.
RESULTS
Clinical interventions made by Good Shepherd
pharmacists using the CDSS were associated with
a very high physician acceptance rate (.99%), and the
average number of accepted interventions increased
remarkably 116% per month since implementation
of the system (Figure 1). The majority of interventions
have been completed during patient rounds, but interventions have also occurred in a variety of other
settings, such as order clarifications, therapeutic interchanges, and medical record reconciliation.
Cost savings related to pharmacist interventions
increased from an average of $118,068 per month to
$184,945 per month (Figure 2), representing an estimated annual savings increase of $802,524 and total
savings of $2,219,340.
DISCUSSION
The results described here have been well received
by Good Shepherds administration. CDSS implementation has helped the hospital to overcome an existing
struggle to obtain and document the data needed for
clinical initiatives. The department of pharmacy now
uses a multifaceted approach to encourage pharmacists

to document interventions. These interventions are


essentially the same activities the pharmacists were
already performing, but with training on the new
system, they can add documentation to get credit for
their work.
The ease of documentation on the mobile CDSS
platform was instrumental in the dramatic and sustained increase in pharmacist clinical interventions,
which in turn demonstrated significant value to the
institution. As a result, 2 additional full-time clinical
pharmacist positions have been approved, and
funds have been budgeted for tablet computers for
pharmacists to use on daily multidisciplinary rounds
to retrieve patient data and document interventions
with the CDSS.
Clinical information technologies can improve the
quality of care within the hospital environment by
integrating patient data from a variety of hospital
information sources and offering tools for automated,
real-time surveillance, alerts, analysis, and reporting.
Additionally, these systems can enhance antimicrobial
stewardship by ensuring appropriate drug selection,
dosing, and duration. The CDSS was shown to positively impact pharmacy clinical interventions, clinical
documentation, and institutional costs.
REFERENCES
1. Amarasingham R, Diener-West M, Plantinga L, et al.
Hospital characteristics associated with highly automated and
usable clinical information systems in Texas, United States.
BMC Med Inform Decis Mak. 2008;8:39-50.
2. Bates DW. The quality case for information technology in
healthcare. BMC Med Inform Decis Mak. 2002;2:7-10.

Hospital Pharmacy

S13

Implementation of a Clinical Decision Support System

3. Bates DW, Gawande AA. Improving safety with information technology. N Engl J Med. 2003;348(25):2526-2534.

stay, extra costs, and attributable mortality. JAMA. 1997;277:


301-306.

4. Coiera E, ed. Guide to Health Informatics. 2nd ed. London:


Arnold; 2003.

9. Bates DW, Cullen DJ, Laird N, et al. Incidence of adverse


drug events and potential adverse drug events. Implications for
prevention. ADE Prevention Study Group. JAMA. 1995;274:
29-34.

5. Kaushal R, Shojania K, Bates D. Effects of computerized


physician order entry and clinical decision support system on
medication safety: a systemic review. Arch Intern Med. 2003;
163:1409-1416.
6. Amarasingham R, Plantinga L, Diener-West M, et al.
Clinical information technologies and inpatient outcomes.
Arch Intern Med. 2009;169:108-114.
7. Lazarou J, Pomeranz BH, Corey PN. Incidence of adverse
drug reactions in hospitalized patients: a meta-analysis of
prospective studies. JAMA. 1998;279:1200-1205.
8. Classen DC, Pestotnik SL, Evans RS, Lloyd JF, Burke JP.
Adverse drug events in hospitalized patients. Excess length of

S14

Volume 48, Suppl 2, 2013

10. Bates DW, Spell N, Cullen DJ, et al. The costs of adverse
drug events in hospitalized patients. Adverse Drug Events
Prevention Study Group. JAMA. 1997;277:307-311.
11. Suh DC, Woodall BS, Shin SK, Hermes-De Santis ER.
Clinical and economic impact of adverse drug reactions in
hospitalized patients. Ann Pharmacother. 2000;34:13731379.
12. Centers for Disease Control and Prevention. Get smart
for healthcare. 2011. http://www.cdc.gov/getsmart/healthcare/.
Accessed January 16, 2013. g

Hosp Pharm 2013;48(3 Suppl 2):S15S16


2013 Thomas Land Publishers, Inc.
www.thomasland.com
doi: 10.1310/hpj4803-S15

Continuing Education Post-Test


1. The sociotechnical model is important because
health information technology (HIT) in its current
stage of implementation has not realized its true
potential to improve patient safety.
a. True
b. False
2. Meaningful use incentives apply to the use of electronic data by:
a. Large health care systems.
b. Physicians.
c. All hospitals.
d. All of the above.
3. Technological interventions in the medication use
process include all of the following, except:
a. Physician order entry.
b. Smart infusion pumps.
c. Medication order verification.
d. Automated surveillance for adverse events.
4. The potential safety benefits of automating the
medication use process have not been fully realized because:
a. Automation projects have not considered the
existing sociotechnical environment.
b. Technologies are not optimized after implementation.
c. Clinical decision support is not provided
within the electronic health record system.
d. All of the above.
5. Specialists in cognitive system engineering develop
technology design requirements by first conducting:
a. Proactive surveillance for emergent risk.
b. Interviews with hospital administrators.
c. Research on the context for new technology.
d. On-site inspections of the health care facility.
6. Core technologies to support medication management include all of the following, except:
a. Electronic medical record.
b. Smart pumps.
c. Computerized prescriber order entry.
d. Barcode medication administration.

7. The benefits of closed-loop medication management include all of the following, except:
a. Improved oversight of pharmacy compounding practices.
b. Deployment and development of improved
practices.
c. Ability to effectively monitor patient care
across the continuum.
d. Ability to reduce the opportunity for error.
8. A major obstacle to achievement of an integrated,
technology-enabled medication management infrastructure is:
a. A focus on single functional areas.
b. Uniform technical and semantic standards.
c. Incentives for vendors to connect.
d. An overall integration strategy.
9. The future state of automation and technology for
medication management:
a. Centers on remote and mobile capabilities.
b. Positions pharmacists in the role of medication
therapy managers.
c. Requires real-time data that are filtered and
prioritized.
d. All of the above.
10. Pharmacy informatics must play a significant
role in all of the following maintenance responsibilities, except:
a. Customized maintenance.
b. Enhancement maintenance.
c. Operational maintenance.
d. Preventive maintenance.
11. Data show that hospitals with automated notes
and records, order entry, and clinical decision
support had:
a. Fewer complications.
b. Lower costs.
c. Lower mortality rates.
d. All of the above.
12. Clinical decision support systems (CDSS) assist
with pharmacy workflow by providing alerts that
target patients who require clinical interventions.
a. True
b. False

Hospital Pharmacy

S15

Continuing Education Post-Test

13. According to the Centers of Disease Control and


Prevention, what percentage of antimicrobial use
in hospitals is considered unnecessary or inappropriate?
a. 15%
b. 30%
c. 50%
d. 75%
14. What are some ways that a CDSS improves antimicrobial stewardship efforts?
a. Generation of an electronic antibiogram
b. Alerts for patients on vancomycin .72 hours
c. Alerts for drug-bug mismatches
d. All of the above
15. Data-mining software facilitates all of the following, except:
a. Medication safety.
b. Clinical decision making.
c. Reduced need for documentation.
d. Infection prevention/management.

Accreditation
This CE activity is co-sponsored by
ProCE, Inc. and Hospital Pharmacy.
ProCE, Inc. is accredited by the Accreditation Council for Pharmacy Education as a provider of continuing
pharmacy education. ACPE Universal
Activity Number 0221-9999-13-016H04-P has been assigned to this
knowledge-based, home-study activity
(initial release date 03-05-13). This activity is approved for 1.5 contact hours (0.15 CEU) in
states that recognize ACPE providers. This CE activity is provided at no cost to participants. Statements
of credit will be issued online upon completion of the
evaluation and the post-test with a score of 70% or
higher. No partial credit will be given.
Release Date: March 5, 2013
Expiration Date: March 5, 2016
How to Obtain CE Credit
Continuing education for this activity is processed
through the ProCE online CE Center. To receive CE
credit, please go to:
 www.ProCE.com/HealthIT-HPJ
 Enroll in the activity and complete the evaluation
and post-test.
With a passing grade of 70% or greater on the
post-test, you will be able to print your CE statement
of credit online.
For questions related to registering for and
obtaining CE credit, contact ProCE at 630-540-2848
or info@ProCE.com.

S16

Volume 48, Suppl 2, 2013