International Journal for Quality in Health Care; Volume 19, Number 5: pp.

259 266 Advance Access Publication: 23 August 2007

10.1093/intqhc/mzm031

Improving quality through effective implementation of information technology in healthcare

JOHN VRETVEIT1,2, TIM SCOTT3, THOMAS G. RUNDALL4, STEPHEN M. SHORTELL4 AND MATS BROMMELS1,5
Medical Management Centre, The Karolinska Institute, Stockholm, Sweden, 2Faculty of Medicine, Bergen University, Norway, 3School of Management, University of St Andrews, UK, 4School of Public Health, University of California, Berkeley, CA, USA, and 5School of Public Health, Helsinki University, Finland
Downloaded from http://intqhc.oxfordjournals.org/ at 11200 Instituto Politcnico de Leiria on October 4, 2013
1

Abstract
Objectives. To describe an implementation of one information technology system (electronic medical record, EMR) in one hospital, the perceived impact, the factors thought to help and hinder implementation and the success of the system and compare this with theories of effective IT implementation. To draw on previous research, empirical data from this study is used to develop IT implementation theory. Design. Qualitative case study, replicating the methods and questions of a previously published USA EMR implementation study using semi-structured interviews and documentation. Setting. Large Swedish teaching hospital shortly after a merger of two hospital sites. Participants. Thirty senior clinicians, managers, project team members, doctors and nurses. Results. The Swedish implementation was achieved within a year and for under half the budget, with a generally popular EMR which was thought to save time and improve the quality of patient care. Evidence from this study and ndings from the more problematic USA implementation case suggests that key factors for cost effective implementation and operation were features of the system itself, the implementation process and the conditions under which the implementation was carried out. Conclusion. There is empirical support for the IT implementation theory developed in this study, which provides a sound basis for future research and successful implementation. Successful implementation of an EMR is likely with an intuitive system, requiring little training, already well developed for clinical work but allowing exibility for development, where clinicians are involved in selection and in modication for their department needs and where a realistic timetable is made using an assessment of the change-capability of the organization. Once a system decision is made, the implementation should be driven by top and departmental leaders assisted by competent project teams involving information technology specialists and users. Corrections for unforeseen eventualities will be needed, especially with less developed systems, requiring regular reviews of progress and modications to systems and timetables to respond to user needs. Keywords: evaluation, health care, information technology, quality

Information technology has great potential for improving quality and safety as well as for reducing costs and creating new service innovations [1]. Electronic medical record (EMR) is at the centre of a health information technology system and most western countries, and services have plans for, or have already introduced EMRs. Prescription order entry is often part of an EMR system and has been found to reduce medication errors and adverse drug events [1]. Many safety

problems are the result of poor communication between shifts and across professions and departments: EMRs can improve communications, but there is no strong evidence of the impact of EMRs on communication processes and safety outcomes, or efciency and savings gains [1]. Although there are many advantages with EMRs and information technology, healthcare has not been able to make use of the potential in the same way as many other

Address reprint requests to: John vretveit, Medical Management Centre, The Karolinska Institute, Floor 5, Berzelius va g 3, Stockholm SE-17177, Sweden. Tel: 46-31-69-39-28; Fax: 46-31-69-1777; E-mail: jovret@aol.com
International Journal for Quality in Health Care vol. 19 no. 5 # The Author 2007. Published by Oxford University Press on behalf of International Society for Quality in Health Care; all rights reserved

259

J. vretveit et al.

industries. In particular, implementation experience has been varied and sometimes negative, notably in public health care systems, and where the EMR is part of a larger health information system serving many purposes and connecting different organizations [2]. One example is the slow progress in the UK: by 2003 3% of NHS hospitals were meeting the target to have an electronic patent record by 2005 [3], but the target has now been put back to 2007 and possibly beyond.

Methods
The objectives were to describe and assess an implementation in one hospital and analyse this in relation to factors suggested by previous research to be important for successful implementation as well as in relation to a published USA case study, which used similar methods [22, 4, 23]. The Swedish study reported here and used prospective and concurrent semi-structured interviews of a selected sample of 30 informants half way through and 3 months after implementation. Selection criteria were that the informants were knowledgeable about how the system was chosen, designed, implemented and about a wide range of perceptions and results. The informants were: the project leader, four part-time project leaders, three persons from a supervisory group, four heads of division, seven heads of clinics, one instructor, ve nurses, four doctors and one doctor secretary. These data were supplemented by hospital documentation and observation visits. The interviews were transcribed, coded and collated to create themes. Unclear responses and contradictory reports were claried with informants. Themes were only retained when more than four respondents described the same items (Table 1).

Previous research
The review of research for this study found a little research, which could help more successful implementation and a lack of theory about implementation of this type of intervention to an organization [4]. These ndings were also described in earlier reviews which report studies of the implementation and impact of EMRs being relatively few, mostly retrospective, without controls, with most data from informants selfreports, and often from surveys. Many of the studies are in a small number of well-resourced US health systems, which have developed EMRs suited to their needs over a number of years, and this experience may not be generalizable. The research does not reect experience with some more userfriendly EMR systems developed in recent years in this fast-moving eld. There are large differences in the settings, implementation processes and the type of EMR system studied. Studies dene the start, nish and scope of implementation differently: guidance given by a nurse to a junior doctor about how to use the system is considered an implementation activity in one study and routine operation in another. Some studies include factors, such as physician champion actions, as part of the intervention, others separate these as conditions, which help or hinder implementation. Success is dened differently, with some studies not considering the perspectives and assessments of different stakeholders, or work- and clinical-process redesign as part of implementation or as a benet. However, some ndings are reported repeatedly, especially factors for successful implementation. These may be classied in terms of features of the EMR system [1, 5 13], the implementation process [6, 8, 11, 14 16], or of the leadership [11, 14 16], resources [1, 5, 6, 8, 14, 15, 17] and the recipient organizations culture [15, 18 20]. There is some weak evidence that implementation is more difcult where there is a culture of undeveloped or antagonistic relations between professions, departments or with management, and that implementation can either improve or reduce cooperation. One study of diffusion of innovation in many different settings found that new techniques spread rapidly when the technique fulled these conditions: relative advantage ( perceived as better), compatible with values and needs), low complexity (ease of understanding and use), trial-ability ( possibility of experimentation) and observability (visible examples elsewhere) [21].

Findings
Preparation In 2003, two Stockholm hospitals 40 km apart merged to form the one Karolinksa University Hospital. Interviewees reported that there was a need to form a common EMR to increase integration and allow communication between the two sites, and, ultimately, other services. Three independent appraisal studies were carried out. The reports were made to one session of heads of department: interviewees describe a dening moment when the chief medical ofcer asked for a show of hands from the group about which system should be adopted, and nearly all voted for one system. The meeting recommended that all departments at one site (Solna) would move from ve different types of medical records to the one system used by the other site, and this was accepted by top management. The new system would need to be installed in 40 clinics with 7000 users at the Solna site. Some new hardware had to be installed, as well as considerable changes to software. The new system contained patient administration, clinical medical records and referral- and replies to referral information, but it was not an entirely paperless record: there were still many documents such as EKG and pictures (e.g. radiology). Analysis of documents and interviews show four key choices, which were confronted and made before implementation, all related to making the key main change within 1 year using limited internal implementation capacity, and delaying other changes to later phases. The rst choice was about the extent to which top-level management and the IT department should prescribe the

260

Information technology in healthcare

Table 1 Interview questions How is the new system intended to be different from the old? What were the objectives, expected costs, results and timetable of the implementation of the new system? Exactly which activities were planned to implement the system? Which activities have been carried out to implement the system? What are the effects of these changes so far for personnel? For management? For patients? Are there any objective data to show the effects? What further activities to implement the system and results are expected? What helped and hindered the implementation activities and what could have been done differently? Practice culture. And how would you describe the culture of your unit/practice/department/hospital/region? What makes it unique in terms of the way things are done around here? E.g. Like a family; Entrepreneurial; Bureaucratic; Is efciency of operation highly valued; Other. Cultural mediation. Would you say that the culture of the unit/organization(s)/region has affected how the system was implemented? In what ways? (E.g. Pace of implementation; Choice of initial implementation sites; Clinical leadership for the system). Leadership. Do you think leadership styles inuenced system implementation in this clinic/hospital/region? Dening moments/critical events/turning points. Have there been any dening moments in the implementation of the system: any critical event marking a turning point in clinicians interpretation and use of the system? Previous IT implementations. How has your experience of system implementation been affected by previous IT implementations? Use of EHR. In your experience, what changes has system implementation of actually made to clinical practice or care processes? E.g. Has it changed how consultations are conducted? How a clinicians work is organized? How clinical work is coordinated between individuals/across sites? Automation of existing care processes? Redesign of care processes? For chronic disease management? Chronic disease management. Would you say that the system is an effective vehicle to develop more effective chronic disease management programs (at unit/org/regional levels)? E.g.: Diabetes? Asthma? Congestive heart failure? Depression? Other illnesses? System functionality. Which, in your view, are the most useful aspects or functions of System to improve chronic disease management? E.g. Electronic patient record? Access to records from other clinicians and sites? Tests and procedures ordering? Diagnosis and treatment outcomes reporting? Drug order entry? Automated alerts and reminders? Electronic communication with patients? Health history? Any other comments. Can you think of anything else that might help us to learn from implementing system at the hospital? Anyone else that we should talk to about this?

system and its details, and how much and what type of independence the departments should have. The preparation process involved the departments, and top leadership made the timetable and managed the project tightly, but departments could choose at what time during the year they would implement and some of the details of the screens which would be created by the hospital IT department. The second key issue was whether to use an outside implementation service or use the limited in-house resources for implementation. The solution was to manage the project internally, phase-in the change over time, and to use external consultants for specic changed which needed to be made. This was possible because of a third choice and decision, which was not to attempt to implement computer physician order entry (CPOE) to order medications at the same time but to phase this in after the EMR and in the following year. The disadvantage was that some departments would loose CPOE, which was functional in the current system. The fourth choice was not to implement functions which different departments needed for research during the EMR phase, but to modify the EMR in the second year to allow this.

Project process to introduce the EMR Once the decision was made in 2004 about which system to introduce, interviewees reported that senior leadership made it clear that departments only had choice about small screen modications and their date for implementation in 2005. Each department was required to nominate staff internally to form a department project group to work with the information technology department to ne-tune the system for the department and carry out implementation. During 2005, all clinics had the same interventions from the project members and the plan was almost exactly followed.

Effects of the system on personnel and work An analysis of the interviews carried out half way through implementation (June, 2005) through to 3 months after implementation (March, 2006) identied common themes that are listed below and illustrated with typical quotes from informants. Approximately 95% of the comments were positive about the implementation process and the new system.

261

J. vretveit et al.

More complete and better information on the system and likely increase in patient safety (e.g. clearer medications information)
A lot easier to nd patient information. I am certain it saves time because of this, and improves patient care because we dont have to wait to get the information from another system

taken to start using the new system. More details of the implementation and impact are provided in a full report [23]. Factors hindering and helping the change The factor most often reported to hinder implementation was not allowing for extra personnel time:
It was difcult to be able to prepare for TC and at the same time do the ordinary work Staff had to work over time to be able to do all that was expected of them

New and better ways to work were being discovered.

In the emergency room the new system allows a real time list of patients in ER with basic information which doctors easily and quickly see. This is very useful where two or more doctors are involved with a patient before one patients information could have been held on ve different systems, many of which could not be accessed

One interviewee thought that the earlier experience implementing an EMR in 2000 was a hindrance because,
The last change like this was complicated. It took a lot to learn the new system, and there were many problems. So our experience with large IT changes like this was not entirely positive

Likely increase in patient safety (e.g. clearer medications information)

A lot easier to nd patient information. I am certain it saves time because of this, and improves patient care because we dont have to wait to get the information from another system

Time savings (for example far fewer telephone calls as a result of the whole hospital using the same system)
Emergency room personnel are very positive as the new system allows them to follow patients minute to minute and see which part of the department the patient is in. This saves work

The merger had happened recently, so new heads of units were covering both sites and other personnel had been changed. There were also some initial disagreements about whether or how much departments should pay for the system.
We needed better information about how much time and money we should have set aside in the department for this project.

Improving integration of the two merged sites Potential for development (e.g. clinics could to use electronic prescriptions and electronic dictation in the future). There were a few disadvantages reported: The speed of implementation prevented developing new procedures
People did not get time and help to adjust their routines to the new system. It would have been much better to change routines while changing the system. There was no time for development. Mostly we just put what we did on the computer

The short time for implementing the system hindered the possibility to give all staff better preparation.
If you had a detailed doctor- or, clinic-specic question you did not ask it because there were too many different people there; secretary nurses and people from different clinics.

Interviewees at Karolinska reported that the following factors helped implementation: Personnel saw the benets of having the same system covering two sites (e.g. allowing easy staff and patient information transfer).
Once the merger was decided and we could see it was happening, we were all motivated to get a common record for all departments and sites: everyone could see the benets and necessity for this

Personnel time was diverted from clinical work for implementation (Difculties getting the time of physicians and personnel to attend training and help adapt the system to their department needs)
Some local project groups had difculties getting time for working with TC. Not all head of clinics and head of units understood that it had to take time to prepare for TC

The costs of the implementation at this 700 bed 7000 employee site of the teaching hospital was  $700,000 USD, under half of the $1.5 m budgeted for consultants, development of integration, education, equipment, information material and to upgrade the hardware server. However, this does not include the costs of working time used by clinic project personnel and other personnel in planning and making adaptations, implementation, training or other time

The other site was already using the EMR and had developed it to be user-friendly. The IT department did not have to make major changes apart from increasing the capacity of the system. The system had a good reputation and many people did not like the old systems. The new EMR was said by the users to be a very easy and usable.
The system itself is intuitive and can be tted to the medical work which is done now and also to the work if it is reorganised This new system saves time because it is quicker to see where to go for information and to access it

262

Information technology in healthcare

Table 2 Summary of EMR implementation Type of implementation change System selection Design and testing Implementation process Main factors helping implementation A two teaching hospital merger led to a change of an existing EMR used by Karolinska site to the EMR used at the other site. Implementation successful Selection of the system was made locally by the hospital with extensive consultation with clinical personnel Already tried and tested at the other site, but also successfully piloted in one department at the Karolinska site Selection, planning and full implementation made as planned and for half the budget Consultation before implementation Consensus about need for the system and which system was best Prioritzation and driving by management team Competent IT project leader and team Tried and tested system User friendly intuitive system needing little training Potential for development of system Order entry not difcult to integrate Recent merger not complete with new people in post Time spent by department personnel on implementation was taken from ordinary work time Some had difculties involving doctors in the preparation work Initial disagreements about much departments should pay for the system No extra time burdens and increased efciency Better coordination of long term patients reported $700,000 for 700 bed, 7000 employee hospital site

.............................................................................................................................................................................

Main factors hindering implementation

Main impact Cost of implementation

Personnel were already used to EMR systemsit was not a change-over from a paper system, but a new EMR system. Many were dissatised with their system, and with having ve different medical record systems in the hospital. The system demanded little extra work to operate it and needed little time-off for training or to adjust to it, Senior management said it was the highest priority project and made it so, as did heads of clinics. There was no problem getting resources. The hospital management group continually pointed out the importance of the project. The project leader was said by many to be very competent and that the project was well planned and organized, in part because of previous experience of introduction at the other site and familiarity with the system. A well-functioning local IT-department in the hospital helped in the implementation process (Table 2).
Comparison to ndings from other studies The review of research identied factors found by previous studies to be important to successful implementation. This could be viewed as the best evidence-based theory of EMR implementation given the current state of scientic knowledge in the eld. Following case study methods for analytic generalization [24], empirical ndings from the study are compared with these factors in Table 3.

Table 3 Factors important for implementation shown by previous research Factor important for implementation The EMR System Ease of navigation, efciency in use and accessibility Physician acceptance and implementers responsiveness to concerns Absence of system failures No conicting suitability (managerial/ clinical) Implementation process User involvement in selection and development Education provided at the right times, amount and quality Previous computer or EMR experience Leadership Strong management support Physician champion Resources Adequate people and nancial resources Organization culture and climate Academic medical centre more change ready Karolinska EMR implementation Yes Yes Yes Yes

....................................................................................

Yes Yes Yes Yes Yes Yes Yes

263

J. vretveit et al.

Table 4 Presence of features predicting successful innovation implementation (Rogers, 1995) Rogers 1995 theory Relative advantage (perceived as better) Compatibility (consistent with values and needs) Complexity (ease of understanding and use) Trialability (possibility of experimentation) Observability (visible examples elsewhere) Karolinska implementation Yes (in pre-implementation assessment, and with experience during implementation) Largely (but prescription order entry and use for research would need to be developed in year 2) Yes (intuitive) Yes (piloted in one department) Yes (at the other hospital site)

....................................................................................

Comparison to Rogers theory of diffusion of innovation Rogers theory for the successful adoption of an innovation is based on research of different interventions and different situations. Evidence from these studies provides some limited conrmation of Rogers theory [21] (Table 4). One possible starting point for future research to develop EMR implementation theory is to combine the above framework with Rogers theory. The two models above were derived from empirical research but the above only provides conrmatory evidence of both modelsresearch to disconrm each of the factors is needed.

some other studies because the research was carried out prospectively and concurrently, drew on detailed project documentation and involved a systematic comparison with a theory of information technology implementation derived from research evidence and a previous study before the empirical data gathering. Although there are limitations there are some tentative conclusions, which may be drawn about why the implementations took the course it did, and why the ndings differed from the less successful Kaiser implementation, which was studied using the same methods. The elements of the theory summarized above also support these conclusions. The relatively successful Karolinska implementation could be due to: Consultation before implementation Consensus about a need for the system and which system was best Prioritzation and driving by management team Competent IT project leader and team Tested, user-friendly and intuitive system needing little training Potential for development of the system Medication order entry not difcult to integrate after implementation The evidence from this and other research is that an EMR designed to meet many different needs often does not meet local clinical work needs, is more difcult to implement, and can reduce productivity and access to information critical for patient care and safety. A new system will need to have benets which signicantly outweigh these disadvantages and which are clearly communicated, if it is perceived to be less user-friendly and requires extra time for operation, which is often reported to be the case in previous research. Time and resources will be needed to develop the system. Practical implications In this study, the comparison with other research suggests general lessons for others implementing an EMR system in a hospital: Choose a system which allows a range of needs to be met and is a tried and tested in a similar setting, The overriding choice criteria should be a system that works for clinical personnel and saves time. Resistance is not always irrational. If personnel do not think it is easy to use and will save time then implementation will be signicantly more difcult and possibly impossible. The system should be intuitive, requiring little or no training, The system should be easy to modify and develop, within limits, for different departments and uses. The decision about the system should be participatory, but once made, implementation should be directed and driven. Balance local control of selection, implementation and clinical participation with meeting higher-level requirements. Involve each level in different ways, with clear and appropriate parameters about which decisions can be

Discussion
A recent review found serious shortcomings in most studies of information technology in health care, but also some evidence of improved quality of care for patients. It found that most health care providers needed more information about how to implement IT successfully [1]. The research reported in this paper used observation and documentation but relied largely on self reports by a limited sample of informants, which is one of the limitations found in many studies. The EMR system and the implementation process are both similar and different to other EMR systems and to implementations under other conditions. The study site did not change-over from a paper system, but replaced ve old systems with one system, and personnel were already used to working with EMRs. Many EMR implementations require more development work before or during implementation, and some are of paper to computer systems, which are a far more substantial change. Generalizations in this eld need to be make with caution: complex interventions in complex social organizations depend for their effectiveness on the conditions under which they are introduced (the conditionality of interventions [25]. However, the ndings from this study are stronger than

264

Information technology in healthcare

made locally and which require higher-level decisions about common standards. Assess and address the presence and absence of prior and concurrent factors, which have been repeatedly shown in research to help and hinder implementation. Findings from other research and these case studies suggest that EMR implementation is a conditional intervention and success depends on many prior and concurrent environmental factors [25]. The less change the EMR system demands and the fewer other changes, which are occurring at the same time, then the more likely implementation will be successful. Organizations, which have strong formal and informal change capacity may be able to manage more changes at the same time. Future research is needed of implementations of different EMRs in different situations, reported in a standardized way to allow comparisons. Theory and knowledge could be developed if studies built on previous research to test hypotheses, especially about which conditions are critical for successful operation for particular systems and how this is dened by different parties.

modied and which saves time and increases productivity. This theoretical model for successful implementation is based on a review of research and supported by the evidence of this study. It provides a theoretical basis for future research and a practical tool for implementing an EMR and realizing the potential of this method for improving safety and the quality of patient care.

Funding
Funding for the Sweden study was provided by Stockholm County Health care and other sponsors of the Medical Management Centre. Ethical approval. Granted under existing agreements for research with Stockholm County Health care, Sweden. The Kaiser Permanente Hawaii Institutional Review Board gave ethical approval for the USA study. Statement of independence of researchers from funders. All researchers were independently funded and have no non-research relationships to the organizations studied or to any information technology suppliers or consultants.

Conclusions
Health services do not have a good history of cost effective implementation IT and especially of EMRs. The potential for increasing safety and productivity of this quality intervention is largely unrealized. The study of EMR implementation presented in this paper was carried out using the same case study method of an implementation in the USA. After reviewing previous research, the research team expected the EMR planning and implementation in this study would follow the same problematic pattern of other implementation. The ndings of the research team, who were entirely independent from the hospital, were unexpected and contrary to most previously reported research: the Karolinska implementation was successful, on time and within budget. Empirical data of respondents perception of what helped and hindered implementation show a consensus about the main helping factors. These were the local hospital control of selection of the system, employee involvement in many different ways, leadership and support by a competent on site information technology department, and decisive and full leadership backing. The Karolinska experience suggests that a tried and tested EMR, which is accepted by physicians can be implemented successfully in 1 year into a teaching hospital with some experience of computerization. It did not show whether such a system allowed the clinical work and process redesign, which some thought necessary and as a missed opportunity during the implementation. Other ndings about what helped and hindered were derived by comparing interviewee data to a simple implementation theory developed from a review of previous research. The ndings from this revealed the importance of organizational, leadership and cultural factors, as well as a user-friendly EMR, which assists clinical work, is easily

References
1. Shekelle PG, Morton SC, Keeler EB. Costs and Benets of Health Information Technology. E006. Rockville, MD: Agency for Healthcare Research and Quality. April 2006. Evidence Report/ Technology Assessment No. 132 (Prepared by the Southern California Evidence-based Practice Center under Contract No. 290-02-0003). 2. Hendy J, Reeves BC, Fulop N, Hutchings A, Masseria C. Challenges to implementing the national programme of information technology (NPfIT): a qualitative study. BMJ 2005;331:331 336. 3. NHS Executive. Information for Health. An Information Strategy for the Modern NHS 1998 2005. NHS Executive: Leeds, 1998. 4. Ovretveit J. Summary review of research into IT implementation in healthcare, Stockholm: Karolinska Institutet Medical Management Centre. 2006. Click iDisk Public Folder (Graphite) at top of web site: http://homepage.mac.com/johnovr/FileSharing1.html (16 May 2007 last date accessed). 5. Brailer D, Terasawa E. Use and Adoption of Computer-Based Patient Records. Oakland, CA: California HealthCare Foundation, 2005. 6. Miller RH, Sim I. Physicians use of electronic medical records: barriers and solutions. Health Aff (Millwood) 2004;23(2):116 26. 7. Sitting D, Kuperman G, Fiskio J. Evaluating physician satisfaction regarding user interactions with an electronic medical record system. Proc AMIA Symp 1999:400 504. 8. Gamm LD, Barsukiewicz CK, Dansky KH et al. Investigating changes in end-user satisfaction with installation of an electronic medical record in ambulatory care settings. J Healthc Inf Manag 1998;12(4):53 65. 9. Ayadin C, Forsythe D. Implementing computers in ambulatory care. Proc AMIA Symp 1997:677 81.

265

J. vretveit et al.

10. Blair D, Schutte P. The electronic medical record in multi-site family practice Part 2: the implementation phase. J Med Pract Manage 2003;19(3):131 36. 11. Chang M, Starren J. Software engineering risk factors in implementation of a small medical record system: the problem of scalability. Proc AMIA Symp 2002:145 49. 12. Gadd C, Penrod L. Dichotomy between physicians and patients attitudes regarding EMR use during outpatient encounters. Proc AMIA Symp 2000:275 79. 13. Wager K, Lee F, White A. Life after a disastrous electronic medical record implementation: one clinics experience. 2005, http://www.idea-group.com/cases/details.asp?id182. 14. Wager K, Lee F, White A, Ward M, Ornstein S. Impact of an electronic medical records system on community based primary care practices. J Am Board Fam Pract 2000;13(5):338 48. 15. Townes P, Benson D, Johnston P, Vaughn C. Making EMRs really work: the southeast health centre experience. J Ambul Care Manage 2000;23(2):43 52. 16. Dansky K, Gamm L, Vasey J, Barsukiewicz C. Electronic medical records: are physicians ready? J Healthc Manag 1999; 44(6):440 55. 17. Audet A, Doty M, Peugh J, Shamasdin J, Zapert K, Schoenbaum S. Information Technologies: when will they make it into physicians black bags?. MedGenMed 2004;6(4), http:// www.medscape.com/viewarticle/493210.

18. Retchin S, Wenzel R. Electronic medical record systems at academic health centres: advantages and implementation issues. Acad Med 1999;74(5):493 98. 19. Berg M. Health Information Management. London: Routledge, 2004. 20. Gustafson D, Sainfort F, Eichler M, Adams L, Bisognano M, Steudel H. Developing and testing a model to predict outcomes of organizational change, Health Serv Res 2003;38(2):751 76. 21. Rogers EM. Diffusion, Spread and Sustainability of Innovations. 4th edn, New York: Free Press, 1995. 22. Scott T, Rundall T, Vogt T, Hsu J. Kaiser Permanentes experience of implementing an electronic medical record: a qualitative study. BMJ 2005;331:1313 1316. 23. vretveit J, Granberg C. Evaluation of the Implementation of an Electronic Medical Record at the Solna site of Karolinska University Hospital. Stockholm: Karolinska Institutet Medical Management Centre, 2006. 24. Yin R. Case Study Research: Design and Methods, Beverly Hills: Sage, 1994. 25. vretveit J. Framework for quality improvement translation: understanding the conditionality of interventions. Jt. Comm. J. Qual Patient Saf., 2004; Global supplement, August:15 24. Accepted for publication 28 June 2007

266