Preventing pediatric medication errors: Testing the iDoseCheck

*Jacqueline Ellis, RN, PhD, Avi Parush, PhD, Régis Vaillancourt, Pharm.D, Catherine
Campbell, P. Eng, M.Des, Danica Irwin, BSc Pharm., Anthony Whitehead, PhD, Debby
VosKamp RN, BScN, Aileen Nutt, RN, MScN, Brenda Martelli, RN(EC) MEd., Pat Elliott-
Miller, RN, M.Sc.N, Ken Farion, MD FRCPS

*Contact Author

Jacqueline Ellis

jellis@uottawa.ca
Abstract

Objective: Dosing errors are the most common type of pediatric medication errors, with

morphine being the high alert drug most commonly given in error. The independent double-

check is used to avoid dosing errors but fails to detect between 5% and 20% of errors. The

iDoseCheck, a graphic computer application, was created to aid nurses in the calculation and

verification tasks of the independent double check. The purpose of this study was to test the

iDoseCheck in a pediatric emergency department and post-surgical care unit, by comparing

nurses’ perceptions on the use of the application before and after its deployment.

Methods: The study used a mixed-methods pre and post design and employed both qualitative

and quantitative measures.

Results: Pre and post system usability scores were 88.4 and 88.1 respectively, corresponding to

an A+ grade for usability. Pre and post scores regarding dose calculation confidence, preference

and helpfulness of the iDoseCheck showed significantly higher scores for the application than

for the conventional method. Focus groups participants noted that the iDoseCheck was easy to

use, visually appealing, and conveyed an appropriate amount of information. Nurses endorsed

iDoseCheck as essential for doing calculations in time sensitive and stressful situations such as

resuscitation.

Conclusions: This proof of concept research indicated that iDoseCheck is easy to learn, simple to

use and provides nurses with sound technologic support for the independent double check. This

support has the potential to decrease wrong dose medication errors and save lives.
Introduction

Medication errors are the most frequent type of errors in hospital settings and are

potentially more harmful and have a higher incidence rate in pediatric than in adult populations1.

Combined adverse events and medication errors can be three times higher in pediatric than in

adult populations2,3. The American Pediatric Association has identified incorrect dosing as the

most commonly reported medication error in pediatrics4. Calculation errors represent between

6% and 15% of reported pediatric drug errors, including 5 to 10 fold over-doses5, with over-dose

errors being more pervasive than under-dose errors. A recent review of the nursing literature on

factors contributing to medication errors6 identified deviations from procedures, including

distractions during administration, excessive workloads and limited knowledge of medications.

More specifically, a fast-paced health care environment can be the source of numerous

distractions and interruptions, demanding from nurses constant multitasking, both in action and

thought, during the process of medication preparation and administration7.

Although some reject the notion of a relationship between medication calculation errors

and nurses’ deficits in their mathematical skills arguing that research has focused on nursing

students rather than on practicing nurses in real life settings8, there is some evidence indicating

that such a relationship does exist9,10. As described by Shanks and Enlow11, in a study of Finish

nurses testing their mathematical and dosage calculation skills only 17% were able to make the

calculations without error12. Similarly, results from research analyzing the relationship between

nurses’ scores from a calculations test during their initial orientation and their later record of

medication errors over a three year period, identified that two-thirds of nurses who failed the test

made at least one medication error during the study period, compared to half of the nurses with

passing grades13.

1
 Medication type, especially in pediatric settings, can be an additional factor contributing

to medication errors. Drugs acting on the nervous system, opioids and sedatives in particular, are

those most frequently associated with medication errors in children14, with morphine being the

high alert drug most commonly related to pediatric drug errors15,16. Drug dosing for children is

weight-based and can be complex due to the calculations involved and the weight variability

which can range from 0.5 kg for a premature newborn to over 100 kg for an obese adolescent4.

The accuracy of the dose calculations are essential, particularly in the case of high alert drugs,

since they carry elevated error risk for causing significant harm17.

The independent double check has been used as a method that can effectively identify

and avoid medication errors at the point of care18. It is a process whereby two nurses

independently verify the patient, medication, drug concentration, drug, dose, time and route19.

However, it has limitations. Research indicates that it fails to detect between 5% to 10% of errors

under ideal conditions20, up to 20% under moderate stress circumstances21,22, in addition to being

subject to process variations and not always being carried out independently23,24. In light of such

limitations, iDoseCheck, a graphic computer application developed by an interprofessional

collaboration, was created to aid nurses in the calculation and verification tasks of the

independent double check for the administration of intravenous morphine to children25. When

tested in a simulation study by comparing it against the traditional paper/pencil and calculator

(PPC) dose calculation method, the iDoseCheck proved to be safe and effective. Its usability

score was significantly higher (91.1) than the comparison benchmark value (68) and participants

preferred it over and felt more confident with it than with the PPC procedure25. The purpose of

this study was to test the iDoseCheck in a pediatric emergency department and post-surgical care

2
unit, by comparing nurses’ perceptions on the use of the application before and after its

deployment.

Methods

Design

The study used a mixed-methods pre and post design and employed both qualitative and

quantitative measures.

Measures and Procedures

System Usability Scale (SUS)

The SUS is a 10-item scale that assesses subjective perception of usability26. Questions

address two key components: Usability and Learnability. Items are scored on a 5 point Likert

scale with descriptors strongly disagree and strongly agree. The scores for each question are

multiplied by 2.5 and final SUS score can range from 0 to 100, where higher scores indicate

better usability. The SUS has good face validity and correlates highly (Cronbach’s alpha 0.91)

with longer usability scales27,28.

Perceived helpfulness of iDoseCheck features

A subjective measure of perceived helpfulness of the visual aids, features and functions

of iDoseCheck was developed by the study authors. The questions addressed specific features

and functions of iDoseCheck on 5-point Likert scale that ranged from 1 (not helpful) to 5 (very

helpful). Participants were asked to respond to 5 questions about the helpfulness of visual aids

such as the height of the patient, standard dose scale, written calculation, error warnings, and

images of the vial(s) and syringe. Two questions about ease of error recovery and iDoseCheck as

an accurate method to prepare IV bolus morphine were also scored on a 1 to 5 scale.

Confidence

3
 Participants’ confidence in the accuracy of drug dose calculations and preparation was

measured with a 6-item questionnaire. Responses were on a 5-point Likert scale and ranged from

strongly disagree to strongly agree.

Preference

Preference for iDoseCheck or paper and pencil/calculator were ascertained with a 5 item

questionnaire that focused on support for dose calculations specifically and support for the

double check generally, speed, accuracy, and error detection. The response scale was coded 1

(preference for PPC), to 5 (preference for iDoseCheck).

Procedures

The study protocol received ethics approval from both the study hospital and the

associated university. During the baseline period nurses were introduced to the study aims and

trained to use iDoseCheck during group or individual sessions. Training for iDoseCheck

consisted of a series of hands-on demonstrations calculating simulated drug doses. Immediately

after the training session each nurse completed the evaluation questionnaires. When 80% of

nurses were trained to use iDoseCheck the intervention period began. iDoseCheck was

strategically placed on a desktop or laptop computer such that it was readily available to nurses

doing an independent double check prior to administering a dose of IV bolus morphine. During

the intervention period, the nurse checking the morphine dose used iDoseCheck and the nurse

administering the dose used the standard paper and pencil/calculator method (PPC). This ensured

that the conventional PPC method was available for each double check thus mitigating any risks

associated with iDoseCheck, which is considered to be a practice change. There was no change

to the other components of the double check and nurses worked independently as per usual

routine and policy. The study was stopped after 100 doses of morphine were administered on

4
each patient care unit (emergency department and post-surgical care unit) and nurses completed

the same questionnaires that were administered pre-intervention.

Five focus groups with nurses from both units where iDoseCheck was used took place

once the intervention had finished. They explored nurses’ views on what they perceived as

challenges and opportunities when using the application, on further refinements and

developments for iDoseCheck, and on whether it could potentially contribute towards risk

reduction in morphine administration in nursing practice.

Setting

The study took place in a 165-bed tertiary care pediatric hospital in eastern Ontario. Data

collection occurred on a 27-bed post-surgical care unit and a 40-bed Emergency Department,

reporting around 66,000 visits per year.

Results

Demographics. Ninety-three nurses answered the pre-questionnaire, 59 (63%) were full

time and 29 (31%) were part time employees. Twelve (13%) nurses were in their first year of

practice, 41 (44%) nurses had between two and 10 years of experience and 39 (42%) nurses had

more than 11 years of experience in pediatric nursing. Fifty nurses answered the post

questionnaire, 28 (56%) were fulltime and 21 (41%) were part-time employees. Seven (14%)

nurses were in their first year of practice, 25 (50%) nurses had between two and ten years of

experience, 17 (34%) nurses had more than 11 years of experience in pediatric nursing.

System Usability Scale (SUS). The pre-intervention mean SUS score was 88.4 (n = 93,

SD = 11.8) with a corresponding 95% confidence interval (CI) [86.0, 90.8]. This places the

usability score firmly in the A+ letter grade range designated by Sauro29. Furthermore, it means

5
the application has a higher SUS score than 98% to 99% of other applications. The post-

intervention SUS scores were similar. The mean was 88.1 (n = 49, SD = 11.4) with a

corresponding 95% CI [84.8, 91.4].

Confidence. Each question for the confidence rating was analyzed independently using a

one-sample t-test against a neutral value of 3. Both pre and post intervention, the questions

scored significantly higher than the neutral value of 3, p = < .01 (Table 1.). As such, participants

were confident (1) using the iDoseCheck to calculate the dose, (2) that the calculations obtained

were accurate given the patient weight, (3) that the correct morphine volume was calculated, (4)

that they were able to perform the double check with a colleague using the conventional method,

(5) that the dose was within a standard range for the patient, and (6) that they were able to

perform each calculation without hesitation.

When iDoseCheck was compared post-intervention to the paper/pencil/calculator

method, there was no significant difference for four out of the six confidence items (Table 2.).

For items on dose accuracy and on the dose being within range (not an over or under dose)

iDoseCheck had significantly higher confidence ratings than the conventional method, p = < .05.

Preference. For both the pre and post intervention there was a significant preference for

using iDoseCheck, p = < .01. (Table 3.) In each case, participants preferred iDoseCheck for (1)

the general support in dosage calculations, (2) the speed at which the calculations could be

performed, (3) the accuracy of the dose calculations, (4) the support in detecting dose errors, and

(5) for support while performing the double check.

Helpfulness Evaluation of Interface Components. Interface component evaluation

questions were analyzed against a benchmark value of 1 (not helpful). Pre and post intervention,

6
all questions scored significantly higher than the benchmark, p = < .01 (Table 4.) As such,

participants found all interface components helpful including the (1) patient image, (2) standard

dose scale, (3) dose calculation presented, (4) vial and syringe image, and (5) the under-

dosing/over-dosing warning messages. In addition, it was also found that the application (6) was

easy to recover from errors, and (7) it was perceived as an accurate method to prepare IV bolus

morphine.

Focus Groups. Focus group participants examined reactions to iDoseCheck, specifically, the

facilitators, barriers, and potential improvements to the tool. Focus groups were audio-recorded

and transcribed verbatim. An inductive-deductive approach30 was used to analyse the data.

Facilitators. Nurses indicated that iDoseCheck was easy to use, visually appealing, and

conveyed an appropriate amount of information. Nurses appreciated that the interface and

navigation was intuitive and all necessary information was on one screen. As one participant

indicated “It is not busy. It is very clean and easy to read, you are not bombarded with a thousand

extra things that you are trying to read through.” Participants expressed appreciation for the

graphics and visual elements of iDoseCheck user interface. Many indicated that the interface

visuals such as the image of the ampoule, syringe, the size image, the calculation range and the

actual calculation served as important cues for them in their practice. “I find the ampule is a very

good image because you can see how many to take out in order to decrease errors.” “I like

having the picture of the syringe and how much to pull up, it is like an extra set of eyes to show

it is right.”

Barriers. Frequently, nurses mentioned the location of iDoseCheck as a primary barrier to

consistent and efficient uptake. It was located in the medication room which was described as a

7
busy enclosed space and computers with iDoseCheck installed were hard to come by. Some

nurses indicated that there were many practice changes being implemented in their departments,

leaving the iDoseCheck to fight for staff attention and location. “Honestly, I think it is an

excellent program but with everything in that med room right now, the med dispensing cabinets

and everyone in line for that and people would have to be in line for iDoseCheck, I think there is

just too much going on in that one area.”

A small number of nurses indicated concern about relying on a computer to do calculations,

thus losing the ability to perform this skill without the computer. “For junior staff, a nurse with

little experience may rely on it too much,…and they would not gain the experience of doing their

own checks—that could be happening.” To a lesser extent, some nurses saw the potential for

technical failures on the computer equipment.

Improvements. Though nurses generally valued the iDoseCheck interface they did offer

suggestions that would improve the utility and efficiency of the tool. The ‘clear’ button was not

well used and could be deleted or moved to a more prominent location. Nurses indicated they

would like information about how to dilute the drug and how fast to push it. They did not want to

access the drug manual for this information and felt it could be a ‘pop-up’ message at the bottom

of the screen.

Accessibility was consistently cited by nurses as a factor that would improve the use of

iDoseCheck in their departments. This included improved access to computers, more computers,

better location of computers, and greater integration with other electronic devices and software.

The ideal solution for many nurses was to create iDoseCheck for mobile devices or integrate

iDoseCheck into the medication dispensing cabinet or into the electronic health record for

8
comprehensive functionality. Specifically in terms of awareness, one nurse suggested prompts in

their new medication cabinet to instruct users not to forget about iDoseCheck.

Nurses highly endorsed the usefulness of iDoseCheck and suggested many other drugs which

they felt should be included. High alert drugs such as opioids were first and foremost, however,

antibiotics, and resuscitation drugs were also suggested. “Antibiotics, we give so many that we

do not necessarily know the parameters for all of them and we do not have the time, to be honest,

to go look up everything.” Regarding potential use in a pediatric resuscitation, one nurse stated

“Yes, trying to do all of the math and everyone is basically yelling at you for the meds. Having

the iDoseCheck and knowing that you have that double check as you are drawing up five or six

meds at the same time would be so much easier.”

Conclusion

This proof of concept research indicated that iDoseCheck is easy to learn, simple to use

and provides nurses with sound technologic support for the independent double check. This

support has the potential to decrease wrong dose medication errors and save lives. Currently

iDoseCheck-Morphine is accessed via a web link (www.idosecheck.com), however, nurses

indicated that it is critical to have flexible access and the tool should be available at the point-of-

care. For example, during resuscitation drugs are typically prepared and double-checked at the

patient’s bedside. Depending on hospital context, there may not be easy access to a computer or

to the web. Having iDoseCheck on a mobile device would enable nurses refer to the double-

check information at the time of drug administration.

Next Steps. Consultation with nurses, pharmacists and physicians is currently underway

to determine the drugs, features and platforms that will best serve the needs of healthcare

9
professionals that provide pediatric care. Of particular concern and interest are the needs of

healthcare professionals in community hospitals that provide substantial pediatric care but do not

have the same comfort with and knowledge of pediatric medications.

10
 Reference List

1. Gonzales K. Medication administration errors and the pediatric population: A systematic

search of the literature. Journal of Pediatric Nursing 2010;25:555-65

2. Ferranti J, Horvath MM, Cozart H et al. Reevaluating the safety profile of pediatrics: a
comparison of computerized adverse drug event surveillance and voluntary reporting in
the pediatric environment. Pediatrics. May 2008;121;e1201-e1207. Available from:
American Academy of Pediatrics, Elk Grove Village, IL. Accessed May 22, 2013

3. Kaushal R, Bates DW, Landrigan C et al. Medication errors and adverse drug events in
pediatric inpatients. JAMA 2001;285:2114-20

4. American Academy of Pediatrics. Policy Statement: Prevention of medication errors in

the pediatric inpatient setting. Pediatrics 2003;112:431-6

5. Cousins DH, Sabatier B, Begue D et al. Medication errors in intravenous drug

preparation and administration: a multicentre audit in the UK, Germany and France. Qual
Saf Health Care 2005;14:190-5

6. Brady AM, Malone, AM, and Fleming, S. A literature review of the individual and
systems factors that contribute to medication errors in nursing practice. J Nurs Manag
2009;17:679-97

7. Eisenhauer LA, Hurley AC, and Dolan N. Nurses' reported thinking during medication
administration. J Nurs Scholarsh 2007;39:82-7

8. Wright K. The assessment and development of drug calculation skills in nurse education-
-a critical debate. Nurse Educ Today 2009;29:544-8

9. McMullan M, Jones R, and Lea S. Patient safety: numerical skills and drug calculation
abilities of nursing students and registered nurses. J Adv Nurs 2010;66:891-9

10. Gladstone J. Drug administration errors: a study into the factors underlying the
occurrence and reporting of drug errors in a district general hospital. J Adv Nurs
1995;22:628-37

11. Shanks LC and Enlow MZ. Medication calculation competency. Am J Nurs 2011;111:67-
9

12. Grandell-Niemi H, Hupli M, Leino-Kilpi H et al. Medication calculation skills of nurses

in Finland. J Clin Nurs 2003;12:519-28

11
13. Calliari D. The relationship between a calculation test given in nursing orientation and
medication errors. J Contin Educ Nurs 1995;26:11-4

14. Belela AS, Peterlini MA, and Pedreira ML. Medication errors reported in a pediatric
intensive care unit for oncologic patients. Cancer Nurs 2011;34:393-400

15. Colquhoun M, Orrbine E, Sheppard I et al. National collaborative: Top five drugs
reported as causing harm through medication error in pediatrics. Dynamics 2009;20:20-2

16. Wong IC, Ghaleb MA, Franklin BD et al. Incidence and nature of dosing errors in
paediatric medications: a systematic review. Drug Saf 2004;27:661-70

17. ISMP. ISMP's List of high-alert medications. 2008. Available at:

http://www.ismp.org/tools/highalertmedications.pdf. Accessed May 25, 2013

18. Grissinger M. The virtues of independent double checks: they really are worth your time.
Pharmacy and Therapeutics 2006;31:492

19. Gosbee LL. Exploration of two double check methods. Test report, Prepared for the
Institute for Safe Medication Practices Canada. Toronto, ON: ISMP, 2006

20. Grasha AA. A cognitive systems perspective on human performance in the pharmacy:
implications for accuracy, effectiveness and job satisfaction. Executive Summary Report,
Report No. 062100. Alexandria, VA: National Association of Chain Drug Stores, 2000

21. Gertman, D., Blackman, H., and Marble, J. The SPAR-H human reliability analysis
method. Prepared for the Division of Risk Analysis and Applications, Office of Nuclear
Regulatory Research, U.S. Washington, D.C.: Nuclear Regulatory Commission, 2005

22. System Reliability Centre. Technique for human error rate prediction (THERP).
Rome:Alion Science and Technology, 2005

23. Parush A, Campbell C, Ellis J, Vaillancourt R, Lockett J, Lebreux D, Wong E, and

Pascuet E. A Human Factors Approach to Evaluating Morphine Administration in a
Pediatric Surgical Unit. In: Duffy, V ed. Advances in Human Factors and Ergonomics in
Healthcare. CRC Press, 2010:468-477

24. Dickinson A, McCall E, Twomey B et al. Paediatric nurses' understanding of the process
and procedure of double-checking medications. J Clin Nurs 2010;19:728-35

25. Ellis J, Parush A, Taign M et al. Preventing Medication Errors with High Alert Drugs:
Testing the iDoseCheck. Canadian Journal of Nursing Informatics. August
2012;7;Available from: CNIA, Accessed May 25, 2013.

26. Brooke J. SUS: A "quick and dirty" usability scale. In: Jordan PW, Thomas B,
Weerdmeester A, and McClelland IL eds. Usability Evaluation in Industry. London:
Taylor & Francis, 1996:189-194

12
27. Bangor A, Kortum PT, and Miller JT. An empirical evaluation of the System Usability
Scale. International Journal of Human-Computer Interaction 2008;24:574-94

28. Bangor A, Kortum, PT, and Miller, JT. Determining what individual SUS scores mean:
adding an adjective rating scale. Journal of Usability Studies 2009;4:114-23

29. Sauro J. A Practical Guide to the System Usability Scale. Denver:CreateSpace, 2011

30. Elo S and Kyngäs H. The qualitative content analysis process. Journal of Advanced
Nursing 2007;62:107-15

13