Scot M. Silverstein, MD Drexel University, College of Information Science and Technology, Philadelphia, PA sms88@drexel.
edu Remediating an Unintended Consequence of Healthcare IT: A Dearth of Data on Unintended Consequences of Healthcare IT Abstract: Background: Case reports, systematic statistical data and other information on unintended consequences (UC’s) of healthcare information technology (HIT) is relatively scarce despite ample literature on potential HIT benefits. This impedes optimal efforts at computerization of healthcare, and can and should be remediated. Objectives: To illustrate the relative scarcity of information on HIT UC’s, suggest contributing factors, and recommend tactical measures for improvement such as better user reporting of HIT UC’s and better diffusion of existing literature on the phenomenon. Methods: A number of recent indicators for scarcity of UC information were compiled and possible reasons described. Examples of suboptimal adverse results disclosures in related domains (e.g., the pharmaceutical industry) that may hold lessons for HIT were included. Results: UC information on HIT is relatively scarce likely due to a variety of influences and complex interactions among and between medicine, informatics, government and industry that, left unaddressed, may lead to delays or other harm to good faith efforts to computerize informational aspects of healthcare delivery and research. Conclusions: The relative scarcity of definitive information on the extent of HIT UC’s should be addressed in a responsible and ethical manner by clinicians, regulators and other stakeholders if this technology is to be successfully rolled out nationally.
Keywords: Medical Records Systems, Computerized Computer Systems Evaluation Medical Errors
“Most especially must I tread with care in matters of life and death.” Hippocratic Oath, Modern Version  “ICT (Information and communications technology) use leads to multiple, and often paradoxical, effects.” Kling, Sawyer and Rosenbaum, Understanding Social Informatics  1. Background Medical error reduction is an international issue, as is the implementation of healthcare information technology (HIT) as a potential means of achieving it. Biomedical informatics researchers have encountered numerous instances in which HIT applications such as EHR, CPOE and CDSS seem to facilitate errors rather than reduce their likelihood. With a heightened awareness of these issues, informaticians can facilitate better system design, implementation, and research in such a way that might help avoid or minimize avoid unintended consequences of these errors.  Unintended consequences (UC’s) are outcomes of actions that are not originally intended in a particular situation. In the clinical domain, UC’s, including those related to the use of healthcare IT (HIT) are usually equated with clinical side effects or adverse medical events and poor outcomes.  Information systems are not intended to disrupt clinicians’ workflow, cognitive processes or actions, so HIT-related “close calls” or “near misses” might also be considered as HIT UC’s. These are clinician errors of commission or omission related to unintended HIT malfunction, cognitive or work disruption, and other issues that do not result in patient harm due to redundancies and human checks and balances in the healthcare delivery process that correct for the error before harm occurs. UC case reports, rates, statistics and other information (henceforth “UC information”) are systematically reported by manufacturer and specific product, and are available in many domains such as drug adverse events and automobile defects. When, however, information is needed on the incidence of UC’s related to HIT applications such as electronic health record (EHR) systems or computerized order entry (CPOE) systems, buyers largely need to rely on the vendors themselves with their inherent conflict of interest, on word of mouth, anecdote, and perhaps scant peer reviewed literature. There is no systematic, comprehensive, trustworthy source of UC information, especially if the information is sought about a specific HIT product version or use of a product in specialized clinical settings such as subspecialty units. UC’s could be minor and limited, or could be widespread and clinically significant. It is difficult for applied informatics personnel and others involved in HIT selection and implementation to ascertain the overall UC incidence nationwide – or worldwide - without good data.
While there is an abundance of literature on the positive attributes of HIT towards improving healthcare quality, efficiency, cost effectiveness and other issues in specific clinical settings, there is an apparent scarcity of data on in situ UC’s related to HIT. Safety is unproven by impartial, methodological study and the true risk/benefit ratio is unknown. 2. Methods A number of recent indicators for scarcity of UC information were compiled and possible reasons described. Examples of suboptimal adverse results disclosures in related domains (e.g., the pharmaceutical industry) that may hold lessons for HIT were included. A number of tactics for remediation of HIT UC information scarcity are suggested that are practicable at the level of the individual researcher and clinician, as well as at the policy-making level. 3. Results 3.1 Recent indicators of scarcity of detailed, comprehensive UC information in HIT UC information diffusion (as well as other information that could place HIT in a negative light such as implementation conflicts and failures) seems muted. Published information appears spotty compared to other technological domains and detailed, comprehensive statistics and case detail appear uncommon. HIT is an unregulated industry. There are no federal databases to turn to regarding HIT UC’s, nor is there a commercial source of comprehensive information. The Joint Commission recently reported in a Sentinel Events Alert  that “as health information technology (HIT) and converging technologies—the interrelationship between medical devices and HIT—are increasingly adopted by health care organizations, users must be mindful of the safety risks and preventable adverse events that these implementations can create or perpetuate … There is a dearth of data on the incidence of adverse events directly caused by HIT overall (emphasis mine). The Joint Commission did state that there was some known data on HIT adverse events related specifically to pharmaceuticals. The JC reported that the United States Pharmacopeia MEDMARX database  included 176,409 medication error records for 2006, of which 1.25 percent resulted in harm. Of those medication error records, 43,372, or approximately 25 percent, involved some aspect of computer technology such as information management systems, confusing displays and software. Lack of other sources of HIT UC information, however, led to the overall statement about a dearth of data. A longitudinal citation analysis conducted across a significant portion of the Medical Informatics periodical literature from 1994-2005, focusing on the identification of strings (linear citation patterns) and networks (multi- branched citation patterns) of highly cited MI-related articles indexed in the Web of Science, demonstrated an interesting pattern. The temporal strings are
representative of persistent, highly visible research themes in MI. Themes found not wellrepresented included sociotechnical issues such as IT difficulties and UC’s, and ethics and legal issues in medical informatics. These were felt to represent a gap in research and authorship. The gaps suggested that medical informatics research and training, while facilitating the improvement of healthcare, may itself benefit from an evaluation of these themes.  These gaps are likely in part due to unpopularity of subjects that could place HIT in a negative light. Another illustration of an apparent lack of enthusiasm for publication on clinical IT difficulties including UC’s is a 2006 study that was conducted of access characteristics to a first-of-its-kind HIT difficulties web resource placed online in the late 1990’s . The web site explicitly addressed issues of health IT implementation difficulties, usage problems and failures via detailed case accounts in a de-identified format. It was found in the study that the website was one of few relevant sites retrieved via major search engine queries on “healthcare IT failure” or related concepts, usually at a very high ranking. With that observation in mind, it was hypothesized that accesses (“hits”) upon the website might reflect a significant portion of the demand for information on this issue, the site serving as a de facto gauge or “barometer” of interest in the topic. Interest in reading about healthcare IT difficulty and UC’s via the Web appeared ongoing by searchers of a variety of demographics in numerous countries.  Yet, the website remained nearly unique after nearly a decade online and is rarely cited or described in the HIT literature. A search of a major FDA data resource, the Manufacturer and User Facility Device Experience (MAUDE) database , showed little data on HIT. MAUDE is a national biomedical database containing reports of adverse events involving medical devices. The data consists of voluntary reports since June 1993, user facility reports since 1991, distributor reports since 1993, and manufacturer reports since August 1996. While MAUDE contains many thousands of adverse events reports on medical devices (reporting of which is largely mandatory), it contains a mere handful (approximately two dozen) voluntary case reports of HIT malfunctions and UC’ over t least the past decade. Most MAUDE HIT adverse events reports are, in fact, from a single vendor. Many of these malfunctions were potentially clinically significant and easily could have, or did, lead to clinical UC’s [Table 1]. Another aspect of the scarcity of HIT UC data in MAUDE is notable. Reports from many other vendors were absent altogether in the MAUDE database. For many HIT vendors the MAUDE database reported no records at all over the past decade. It is unlikely that UC’s do not occur with other HIT vendor products. It is more likely that they were simply not reported, although such reporting would have been of benefit to system designers, implementers and purchasers. A 2009 AMIA research report based on a workshop at the 2006 national meeting, attended by over fifty informatics specialists across ten AMIA working groups, observed that there have been few systematic and thoughtful publications describing lessons learned from IT interventions that
had null, negative, or disappointing outcomes.  Workshop attendees also observed that ITrelated failures in health care often are covered up, ignored, or rationalized, so mistakes are repeated. Noted were a small number of high-profile health IT failures that adversely affected patient care and well known institutions suspending their systems or halting their development due to physician protest, extreme overspending, errors, and delays. However, the true extent of these issues domestically and internationally is not robustly known due to a scarcity of literature and trustworthy data sources. The Washington Post reported that more than two dozen doctors, academics, patients and computer programmers when interviewed opined that HIT can increase errors, add hours to doctors' workloads and compromise patient care.  Since the literature does not support knowing whether these problems are widespread or not, such reports have led to a Senate Finance Committee investigation. The United States Senate Committee on Finance has jurisdiction over the Medicare and Medicaid programs. The Ranking Member of the Committee Senator Charles Grassley, over the past year, has received unsolicited HIT complaints similar to those received by the Washington Post from numerous patients, medical practitioners and technologies engineers, who felt strongly enough to report these issues to the Senator. These complaints include, for example, faulty software that miscalculated intracranial pressures and interchanged kilograms and pounds, resulting in incorrect medication dosages. Since the true extent of the UC’s is unknown, and since Sen. Grassley has a fiduciary and ethical responsibility to investigate, an investigation has commenced. A letter of inquiry was sent to ten major HIT vendor and consulting firms, requesting an extensive variety of documents related to industry practices, including handling of HIT defects and UC’s.  On occasion, explicit case reports of HIT UC’s do appear, usually in the news and only after events of high visibility. For example, a report from the Veterans Administration about problems with VistA appeared in early 2009. After an upgrade, 41 of the 153 VA medical centers reported information displayed under the wrong name, and failure to display “stop” orders at 9 VA center resulting in infusion of drugs such as heparin for up to 11 unordered hours. The authors concluded that although many advocates of electronic medical records systems promote them as a means to reduce mistakes affecting patient care, the problems in the VA system suggest that malfunctions of these systems or problems with the way they are implemented have the potential to lead to medical errors. No patients were injured on this occasion. Yet, this was only by serendipity, not by design.  The true extent of HIT implementation and usage difficulties, UC’s and “close calls” such as the VistA malfunction are unknown. Quality studies of UC’s that do appear in the literature are uncommon. A literature search shows they largely appear confined to the low circulation informatics journals, not general medicine
journals or journals likely to be read by policy makers, and appear to be authored by a relatively small set of informatics researchers. The studies, using various methods including empirical observation, user surveys and intra-expert discussions, indicate that UC’s are common, for example with CPOE, but detail is lacking that would be useful to HIT decision makers on product differentiation by vendor, version etc. [18, 19] A systemic literature review also found wide variance in study quality including validity measures and measurement issues.  In all, the available UC information seems underwhelming considering the size and budget of the healthcare and HIT industries and the age of the latter, even at the relatively low adoption rates of HIT in 2009. 4. Discussion 4.1 A reminder that HIT is an experimental technology The UC’s related to HIT are not entirely avoidable. It is possible, however, to avoid or minimize studying them, reporting them, and revealing their occurrence to the research community, purchasers, and to the public whose medical care increasingly depends on HIT. This results in a scarcity of UC information. The term “experiment” comes from 14th century Middle English, from the Anglo-French esperiment, from Latin experimentum, from experiri: an operation or procedure carried out under controlled conditions in order to discover an unknown effect or law, to test or establish a hypothesis, or to illustrate a known law.  It must be remembered that HIT is an experimental technology, not a technology proven to be safe and efficacious in the complex environments of healthcare delivery. Its use is an experiment, an operation or procedure carried out under controlled conditions in order to discover an effect. Profound alterations in the provision of medical care mediated by HIT may occur as a result of the experiment. It is perhaps not unreasonable to lightly study the UC’s of long-proven technologies. HIT, however, while having the potential for many benefits, is also capable of facilitating known and unknown, predictable and unpredictable UC’s that can result in undesired or adverse clinical outcomes. Many unintended and undesired consequences of Healthcare Information Technologies (HIT) flow from interactions between the HIT and the healthcare organization’s sociotechnical system—its workflows, culture, social interactions, and technologies.  A January 2009 National Research Council report notes that HIT at present does not adequately support clinician cognitive processes, will not be sufficient to achieve medical leaders' vision of health care in the 21st century and may even set back the cause . The report concedes that success of HIT will “depend upon accelerating interdisciplinary research in biomedical informatics, computer science, social science, and health care engineering.”  A technology
that does not support critical aspects of the work of its users, and whose success depends on further intensive multidisciplinary research across disparate disciplines, cannot reasonably be considered a proven technology. It therefore does not seem reasonable to commit low levels of attention to study of HIT UC’s. Experimental technologies need to be treated as such in terms of researcher and user obligations to report potentially adverse effects. We need to better understand the incidence of UC’s and how to prevent them before a national rollout of HIT gets underway, if such a rollout is to be conducted to the highest scientific and ethical standards of medicine. 4.2 UC information scarcity as an unintended consequence of HIT The intended consequences of HIT R&D and implementation should include an increase in the knowledge of both HIT benefits and possible drawbacks, for protection of patient safety. This seems a reasonable and ethical position. Evaluation methods in biomedical informatics must address not only a range of different types of information resources (Including HIT), but also a range of questions about them, from the technical characteristics of specific systems to their effects on people and organizations. The reasons for performing evaluation studies include promotional, scholarly, pragmatic (to help developers know what is most effective, and what does not work), ethical, and medicolegal (to assure users that an information resource is safe and effective). The methodologies and expertise exist to conduct both objectivist and subjectivist studies of health IT that could reveal the incidence of UC’s in greater detail.  Thus, the scarcity of studies and data on UC’s could itself be considered an unintended consequence of HIT research and efforts to diffuse this technology throughout the healthcare system. 4.3 Proposed reasons for scarcity of comprehensive UC data: 4.3.1 Loss of scientific skepticism In Dr. Octo Barnett’s Ten Commandments (1970), the tenth commandment was perhaps the most important. Dr. Barnett wrote “Be optimistic about the future, supportive of good work that is being done, passionate in your commitment, but always guided by a fundamental skepticism.”  Fundamental skepticism must promote consideration of UC’s and caution about claims of the absolute benevolence of HIT. Loss of fundamental skepticism may be one factor that has contributed to a relative scarcity of knowledge of HIT UC’s. Instead, a marketing-driven atmosphere of exuberance seem to prevail, to the extent that billions of dollars have been earmarked in the American Recovery and Reinvestment Act (ARRA) of 2009 for a national rollout of HIT in five years, with penalties then accruing to non-adopters.
4.3.2 Adverse consequences of challenging the HIT industry status quo, real and perceived Another likely reason for the scarcity is unpopularity (due to opposition or “pushback”) of writing about UC’s and other aspects of HIT that may put the technology in an unflattering light. Such material and opinions are contrary to the current popular culture about computers in general, and about HIT specifically. A marketing mindset-based environment openly promotes HIT as a highly benign and extremely beneficial technology. A very large industry stands to benefit from increasing sales of HIT, recently amplified further by the recent Electronic Health Records provisions in the ARRA. Researchers in the field stand to benefit from both government and industry largesse if they propose research or publish results on the positive side of the spectrum. Research and results that could potentially tarnish HIT’s reputation or injure the market for robust HIT sales would not surprisingly be unpopular among the HIT industry and government stakeholders and consultants who serve this industry. HIT evaluation studies, with some notable exceptions, tend to emphasize the benefits and downplay or ignore the problems and UC’s. They often fail to take into account workflow disruptions, distracting workarounds and improvisations, communication failures, decreased productivity, treatment delays, incorrect tests, close calls, and other adverse incidents. Several notable exceptions by authors such as Han et al.  , Koppel et al. [28, 29] and others did indeed receive “pushback”, sometimes in the form of notably harsh criticism. Examples include published JAMA letters calling Koppel’s research “disingenuous at best“ due to versioning issues  or “misleading and inaccurate” , an accusation that Han’s conclusion of an unintended increase in pediatric mortality due to a new CPOE is not at all supported by his team’s analysis , and the involuntary (but fortunately temporary) pulling of Dr. Jon Patrick’s essay on major EMR problems and physician dissatisfaction in the ED’s of the Australian state of New South Wales  apparently due to governmental intervention or pressure. (Dr. Patrick’s essay reappeared in a revised form under the rubric of an Opinion Editorial.)  Such pushback, especially when governmental officials may be involved, can potentially be career damaging. 4.3.3 Contractual nondisclosure clauses When a UC or “close call” (an HIT-related event caught early by clinicians before patient harm accrues) does not result in patient harm, and in the absence of reporting requirements, such incidents will most likely not appear in the literature. In fact, even if harm occurs, contractual clauses agreed to by hospital governance that hold the HIT vendor harmless for any HIT UC-related injuries and prohibit clinicians from divulging the problems outside of their organization (“gag clauses”) may prevent reporting, in a manner distinctly different than reporting of drug adverse events such as through MedWatch, the FDA Safety Information and Adverse Event Reporting Program. 
HIT contracts may permit vendors to retain company confidential knowledge about designs, faults, software operations, and glitches. Their counsel have crafted contractual terms that absolve them of liability and other punitive strictures, while compelling users’ nondisclosure of their systems’ problematic, even disastrous, software faults. This may create a culture of enforced nonsharing of HIT problems.  4.3.4 Lack of HIT regulation Unlike the pharmaceutical industry where FDA regulates research IT used in clinical trials and data management, and may inspect and validate the IT for robust and secure operation, there is no such regulation of HIT. This is despite the fact that pharmaceutical research IT does not have a potential for short term or immediate patient harm as does HIT, and that unregulated HIT may be of similar complexity to regulated pharmaceutical research IT. The paradoxical lack of regulation of HIT may lead to systems more prone to cause UC’s, and may decrease stakeholder motivations to report these problems. It should be noted that HIT other than the simplest of systems will likely soon be subjected to regulation as medical devices in a number of countries, including Sweden and other members of the EU , Canada , the United States  and other countries as well. Considering the advances in HIT in the past decade or two and the increasingly deep penetration of this technology into the complex ”metabolic-pathways-like” complexity of healthcare delivery and the increasing chances for clinically significant UC’s, this redefinition and regulatory oversight is probably well past due. 4.3.5 Bypassing oversight review Implementations in hospitals and physicians’ offices of new HIT systems, or revisions and customizations of existing HIT may bypass ordinary patient and clinician protections. There is no requirement that institutional research governance officials review HIT projects, although as pointed out earlier these systems are probably best thought of as still experimental in nature. The implementation is generally led by IT personnel, often lacking biomedical backgrounds, and/or multidisciplinary steering and operations oversight committees where power structures may impede optimal clinician and clinician/informaticist decision making. It is the committee that thus holds the responsibility to report adverse events, and if dominated by hospital executives with a career advancement stake in HIT implementation, UC’s might not be reported, or differing accounts of system problems that may minimize the downsides can arise between the IT steering/operations committees and senior executives.  4.3.6 Bypassing traditional patient protections Lack of informed consent provisions and procedures for patients may mute another possible source of UC reports (the patients themselves) by leaving patients in the dark about mediation of
their care by a new and/or modified HIT system. The lack of awareness might prevent a patient from even recognizing that an adverse outcome may be related to an UC of HIT, if this information is not explicitly revealed to them. There is strong motivation not to reveal such information, or to settle off the record if injuries occurred, in that such revelations could lead to litigation and publicity that could place HIT in a negative light. Interestingly, NIH requires that research on new biomedical tools and technologies be conducted will full patient informed consent, attention to patient protections, and protection of researchers as well. For example, federal regulations for protection of human research subjects mandate that NIH grant proposals (including those that might involve new, novel or altered HIT such as CDSS algorithms, IT-mediated real time home health care data transmission, IT-mediated interventions, and other uses of HIT) undergo a review process under guidelines that call for explicit consideration of informed consent and protections. Failure to provide for this consent and protections can lead to a poor score or rejection of the proposal [Table 2]. The lack of informed consent processes in HIT may not only may reduce UC information, but is also a significant incongruity with potential liability implications. 4.3.6 Market and regulatory capture The Washington Post reported that the influential HIT trade group, the Healthcare Information and Management Systems Society (HIMSS), had worked closely with technology vendors, researchers and other allies in a sophisticated, decade-long campaign to shape public opinion and win over Washington's political machinery. HIMSS represents 350 companies and about 20,000 members. Corporate members include massive government contractors such as Lockheed Martin and Northrop Grumman, health-care technology giants such as McKesson, Ingenix and GE Healthcare, and drug industry leaders, including the Pharmaceutical Research and Manufacturers of America (PhRMA).  This is a wealthy and powerful organization. Large scale lobbying can lead to a trade association’s domination of a product market over neutral, objective observers. It may also lead to “regulatory capture”, where powerful organizations with private interests may “capture” the government regulators through well placed influencers and functionaries in order to foster regulations (or lack thereof) supporting their private goals.  HIMSS and other large IT trade organizations have a fiduciary responsibility to represent their constituent vendor and consulting companies to improve their reputation, sales and financial positions. This is manifested by lobbying activities on government and press/public relations and marketing activities. The natural tendency of such a responsibility is to de-emphasize UC’s that could inhibit sales or tarnish a company’s products and market reputation. A similar inhibitory effect on third parties’ (e.g., researchers, clinicians) propensity to potentially tarnish the image of health IT may also occur, since speaking and research fees from the trade
organizations are dependent on the loyalty of the third party to presenting HIT as safe and effective. 4.3.7 Failure to report safety issues in a study In the conduct of research, there is a tendency to minimize reporting of UC’s such as adverse clinical events while reporting positives. Reports of clinical trials usually emphasize efficacy results, especially when results are statistically significant. Poor safety reporting can lead to misinterpretation and inadequate conclusions about the interventions assessed. For example, in a recent study investigators at Paris Diderot University in France assessed 133 articles reporting the results of randomized controlled trials published in 2006 in six prestigious English language journals with high impact factors, including the New England Journal of Medicine, Lancet, JAMA, British Medical Journal, and Annals of Internal Medicine. The review revealed important heterogeneity and variability in the reporting of harm-related results. Adverse events were mentioned in some manner 88.7% of the 133 trials reports. However, no information on severe adverse events was given in 27.1% of the articles. No information on withdrawal of patients owing to an adverse event was given in 47.4% of the articles. Restrictions in the reporting of harm-related data were noted in 43 articles (32.3%), with a description of the most common adverse events only (n=17), severe adverse events only (n=16), statistically significant events only (n=5), and a combination of restrictions (n=5). The authors concluded, "the reporting of harm remains inadequate."  An accompanying editorial raised concerns about why the reporting of adverse events is so heterogeneous, again with possible applicability to many areas in biomedicine including HIT: Perhaps conflicts of interest and marketing rather than science have shaped even the often accepted standard that randomized trials study primarily effectiveness, whereas information on harms from medical interventions can wait for case reports and nonrandomized studies. Nonrandomized data are very helpful, but they have limitations, and much harm will remain long undetected if we just wait for spontaneous reporting and other nonrandomized research to reveal them. In an environment where effectiveness benefits are small and shrinking, the randomized trials agenda may need to reprogram its whole mission, including its reporting, toward better understanding of harms.  Similar issues may prevail in evaluation studies of HIT. 4.3.8 Suppression of negative studies Related to the above, health care corporations may regard clinical trials more as marketing tools than as science. Clinical research studies that failed to provide results favorable to a sponsor's product may be simply suppressed due to the profit motive.  Recent examples include the suppression of negative studies on acute migraine headache medication sumatriptan in favor of studies with positive results.  Information on
defective non-IT-related medical devices may also be suppressed. In another recent example, a company will plead guilty to two criminal misdemeanors for failing to properly alert the U.S. Food and Drug Administration about problems with some of its implantable defibrillators.  Many other examples have been documented in the pharma and medical device sectors , and a similar tactic may occur in the highly profitable HIT sector as well. 5. Recommendations for remediation of scarcity of UC information 5.1 Scientific rigor and candor Broadly speaking, in a field with downsides, it has been said that there are: 1) those who know about the problems but fail to speak, 2) those who see the problems but fail to act, and 3) those who see, know and speak out. The field of HIT needs more of the third group. 5.2 Nondisclosure clause elimination Nondisclosure clauses in HIT contracts should be dropped or refused. This measure is needed to increase transparency and data sharing, along with regulatory requirements that UC data be recorded, reported without fear of adverse consequence, and shared and diffused transparently across institutions, medical societies, and geographic boundaries. 5.3 Reporting of HIT UC’s Clinicians should manage HIT UC’s as they manage drug adverse events, and be educated to do so. As an example, a clinician could report UC’s as they occur via the FDA MedWatch system. 5.4 Standardized service level agreements SLA’s from HIT vendors calling for rapid remediation of serious HIT defects, especially those that could result in patient harm in the short or medium term, should be a sine qua non of the HIT market, and at reasonable costs. 5.5 Sharing of liability as appropriate to the circumstance. Koppel and Kreda had suggested some reasonable compromises on this issue in their JAMA “Hold Harmless clause” article.  5.6 Increased cross disciplinary research focused on HIT UC’s The National Research Council recommendations on accelerating interdisciplinary research in biomedical informatics, computer science, social science, and health care engineering should be strongly supported by all HIT research funding agencies and stakeholders. 5.7 Rigorous evaluation and validation of HIT pre- and post-sales
Evaluation of health IT, both pre- and post-sales, that consider both the benefits and the downsides, and that accurately records and reports UC’s, should become a formally mandated process as in drug clinical trials. A post marketing surveillance system and registry analogous to FDA MedWatch, which allows individuals and organizations to report UC’s to an agency with authority to act upon the data, must also be developed if we are to successfully achieve national HIT. The methods and studies are best developed by neutral organizations and personnel without conflict of interest where possible, and with full disclosure of possible conflicts if they exist. 5.8 Informed Consent An informed consent process for patients, making them fully aware of the mediation of their care by HIT devices, should be developed and implemented. 5.9 Wider dissemination of recent reports on HIT 2009 is proving to be a pivotal year in healthcare IT. Authoritative articles and reports on health IT problems and UC’s have appeared at an accelerated pace. It is apparent that the HIT industry is coming under increased scrutiny, perhaps a necessary requirement for the IT reform needed to decrease UC’s. Dissemination of the most authoritative examples such as the Joint Commission Sentinel Events Alert, National Research Council report, JAMA and JAMIA articles, major newspaper reports, letters, and others [e.g., 5, 14, 15, 24, 50-55] might serve to return the HIT marketplace from a pseudoscientific, marketing-influenced exuberance to a condition of “sobriety” and scientific rigor. 5.10 Support of ongoing governmental investigations On Oct. 16, 2009 a letter was sent to major healthcare IT vendors and management consulting firms by Senator Charles E. Grassley initiating a Senate investigation of HIT and of HIT corporate practices.  This investigation should be supported and relevant information forwarded by involved stakeholders. 6. Conclusion UC information on HIT is relatively scarce likely due to a variety of influences and complex interactions among and between medicine, informatics, government and industry that, left unaddressed, may lead to delays or other harm to good faith efforts to computerize informational aspects of healthcare delivery and research. A computer-related, highly visible “Libby Zion” analog, for example, could cause a severe setback to the cause of healthcare automation. HIT should not challenge the resilience of the healthcare delivery system through engendering “near misses”, since eventually actual patient harm is likely to result. Such challenges must be prevented to the greatest extent practicable.
The scarcity of definitive information on the extent of HIT UC’s should be addressed in a responsible and ethical manner if this technology is to be successfully rolled out nationally over the next five to ten years, and internationally as well. This includes measures practicable at the level of the individual (e.g., improved reporting of UCs via existing resources such as FDA MedWatch, improved dissemination of UC literature to organizational HIT leadership) and at the level of policy-making (e.g., HIT regulation, requirements for patient informed consent). The appearance of this new journal Applied Clinical Informatics itself is a hopeful sign that these issues will receive the critical attention they merit. Finally, medical innovation that does not properly account for patient rights, provide robust and good faith research on UC’s, and patient protection from UC’s is not true innovation, but taking advantage of patients seeking care. Greater transparency and reporting about HIT UC’s, along with appropriate levels of validation and regulation of HIT, will benefit patients in the highest traditions of medicine.
Clinical Relevance Statement: The implications for practitioners of these results and recommendations are improvement of the quality of care they provide and the protection from HIT defects and liability for events that might be beyond their control. For consumers the implications are also improvement of care quality, reduction of error, and enhanced informed consent rights related to the conduct of care provision.
Table 1: Example cases of HIT issues in the MAUDE database that could have or did cause UC’s: Example 1: The issue involves powerchart local access medication administration task, used when certain cerner millennium solutions are not available. At powerchart local access sites that utilize coordinated universal time (utc) functionality, medication administration tasks might be displayed with incorrect times. When a pt download occurs from cerner millennium servers to powerchart local access, and there is no cerner millennium application session active, powerchart local access adds or subtracts the number of hours equal to the time zone difference from greenwich mean time. Scheduled medication administration tasks may show an incorrect administration time and the possibility exists for a pt to receive medications earlier or later than intended.  Example 2: Patient care delay. The issue involves functionality in cerner millennium powerchart office and powerchart core and affects users that utilize the powerchart inbox and message center inbox. In results to endorse or sign and review, if the user clicks ok and next multiple times in quick succession while attempting to sign a result or a document, the display could lag behind the system's processing of the action, and multiple results or documents could be signed without the user's review. In message center, when clicking ok and next or accept and next, or when deleting or completing messages and moving to the next task, a document could be signed or a message could be deleted without the user's review. Results could be endorsed or documents could be signed without physician review, which could impact patient care. Cerner received communication that a patient's follow-up care was delayed as a result of this issue.  Example 3: Microbiology set up a program within the cerner computer system to automate the reporting system for hsv testing. The system was tested with the assistance of cerner and found to be working appropriately. The new system was operational for approximately 3 weeks when it was determined that the first word of the sentence, "no" was inappropriately dropping off of the following sentence: "no herpes simplex virus type 1 or herpes simplex virus type 2 detected by dna amplification. " as such, two of five patients were incorrectly informed that they had hsv before the error was detected. One had started an antiviral creme treatment. The other three did not have follow-up visits until after the correct results were determined. Cerner has looked at the program and has not provided an answer for the system issue. In the interim, the previous manual review and entry process is being used. 
Table 2: NIH Human Subjects Protection Requirements for Grant Review Human Subjects Protection and Inclusion of Women, Minorities, and Children Guidelines for Review of NIH Applications Requirements for Review Federal regulations for the protection of human research subjects (45 CFR 46), require that the evaluation of research applications that involve human subjects take into consideration the risk to subjects, the adequacy of protections against risk, potential benefits of the research to subjects and others, and the importance of the knowledge to be gained. The NIH Peer Review regulations (42 C.F.R. 52h) specify that reviewers will take into account, in determining overall impact that the project in the application could have on the research field involved, the adequacy of the proposed protection for humans. Therefore, reviewers must evaluate the proposed plans to protect human subjects from research risks, as appropriate for the research proposed, as appropriate for the research proposed, as one of the review criteria that factor into the evaluation of scientific and technical merit. 
References: 1. Lasagna L. The Hippocratic Oath: Modern Version, 1964. http://www.pbs.org/wgbh/nova/doctors/oath_modern.html . Last accessed 20 Nov 2009. 2. Kling R, Rosenbaum H, Sawyer S. Understanding And Communicating Social Informatics: A Framework For Studying And Teaching The Human Contexts Of Information And Communication Technologies. 1st ed. Information Today; 2005. 3. Ash JS, Berg M, Coiera E. Some Unintended Consequences of Information Technology in Health Care: The Nature of Patient Care Information System-related Errors. J Am Med Inform Assoc. 2004;11:104-112. 4. American Medical Informatics Association. Anticipating and Addressing Unintended Consequences of Health Information Technology (HIT) and Policy. 9 Sep 2009. Available from: http://www.amia.org/files/shared/AMIA_Policy_Slides_09-09-09_for_posting.pdf . Last accessed 20 Nov 2009. 5. Joint Commission. Sentinel Events Alert: Safely implementing health information and converging technologies. 11 Dec 2008. Available from: http://www.jointcommission.org/SentinelEvents/SentinelEventAlert/sea_42.htm . Last accessed 20 Nov 2009. 6. Quantros, Inc. MedMarx Database. Available from: http://www.usp.org/products/medMarx/ 7. Silverstein SM. Contemporary Issues in Medical Informatics: Common Examples of Healthcare IT Difficulties [Internet]. Drexel University, College of Information Science and Technology, Philadelphia, PA. Available from: http://www.ischool.drexel.edu/faculty/ssilverstein/medinfo.htm . Last accessed 20 Nov 2009. 8. Silverstein SM, Wania C, Chen Y. Access Patterns to a Website on Healthcare IT Failure (poster). AMIA Annu Symp Proc. 2006:1095. Available from: http://www.ischool.drexel.edu/faculty/ssilverstein/failurecases/content/documents/FINAL_Healt hcare%20IT%20Failure_Oct_19_2006.ppt . Last accessed 20 Nov 2009. 9. McCain KW, Silverstein SM. Tracing Persistent Highly Visible Research Themes in Medical Informatics (poster). 2006 May. AMIA Spring Meeting, Phoenix, AZ 10. U.S. Food and Drug Administration. Manufacturer and User Facility Device Experience (MAUDE) database. Available from: http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfMAUDE/search.CFM . Last accessed 20 Nov 2009.
11. MAUDE Adverse Event Report. Available from: http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfMAUDE/Detail.CFM? MDRFOI__ID=722614 . 12. MAUDE Adverse Event Report. Available from: http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfMAUDE/Detail.CFM? MDRFOI__ID=946706 . 13. MAUDE Adverse Event Report. Available from: http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfMAUDE/Detail.CFM? MDRFOI__ID=753029 . 14. Kaplan B, Harris-Salamone K. Health IT Success and Failure: Recommendations from Literature and an AMIA Workshop. J Am Med Inform Assoc. 2009;16:291-299. 15. Mostrous A. Electronic medical records not seen as a cure-all. The Washington Post. 2009 Oct 25. Available from: http://www.washingtonpost.com/wpdyn/content/article/2009/10/24/AR2009102400967.html?hpid%3Dtopnews&sub=AR 16. United Senate Senate, Committee on Finance. Senator Charles E. Grassley, Letter to Cerner. 16 Oct 2009. Available from: http://media.washingtonpost.com/wpsrv/politics/documents/erecords102409.pdf. Last accessed 20 Nov 2009 . 17. Kuehn B. IT Vulnerabilities Highlighted by Errors, Malfunctions at Veterans' Medical Centers. J Am Med Inform Assoc. 2009;301(9):919-92 18. Ash JS, Sittig DF, Dykstra R, Campbell E, Guappone K. The unintended consequences of computerized provider order entry: findings from a mixed methods exploration. Int J Med Inform. 2009 Apr;78 Suppl 1:S69-76. 19. Ash JS, Sittig DF, Poon EG, Guappone K, Campbell E, Dykstra RH. The extent and importance of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc. 2007;14:415-423. 20. Weir CR, Staggers N, Phansalkar S. The state of the evidence for computerized provider order entry: a systematic review and analysis of the quality of the literature. Int J Med Inform. 2009 Jun;78(6):365-74. 21. Merriam Webster’s Online Dictionary. Available from: http://www.merriamwebster.com/dictionary/Experiment. 22. Harrison M, Koppel R, Bar-Lev S. Unintended Consequences of Information Technologies in Health Care—An Interactive Sociotechnical Analysis. J Am Med Inform Assoc. 2007 Sep–Oct; 14(5): 542–549.
23. National Academies, Office of News and Public Information. Current Approaches to U.S. Health Care Information Technology are Insufficient. 20 Jan 2009. http://www8.nationalacademies.org/onpinews/newsitem.aspx?RecordID=12572 . Last accessed 20 Nov 2009. 24. Stead W, Lin HS, editors. Computational Technology for Effective Health Care: Immediate Steps and Strategic Directions. 1st ed. National Academies Press; 2009. Available from: http://www.nap.edu/catalog.php?record_id=12572 . Last accessed 20 Nov 2009. 25. Friedman CP, Wyatt JC. Evaluation Methods in Biomedical Informatics. 2nd ed. Springer; 2005. 26. Collen M. A History of Medical Informatics in the United States 1950 to 1990. 1st Edition. American Medical Informatics Association; 1995. 27. Han YY, Carcillo JA, Venkataraman ST, Clark RS, Watson RS, Nguyen TS, Bayir H, Orr RA. Unexpected Increased Mortality After Implementation of a Commercially Sold Computerized Physician Order Entry System. Pediatrics. 6 Dec. 2005; 116(6):1506-1512. 28. Koppel R, Metlay JP, Cohen A, Abaluck B, Localio AR, Kimmel SE, Strom BL. Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors. J Am Med Inform Assoc. 2005;293:1197-1203. 29. Koppel R, Wetterneck T, Telles JL, Karsh B. Workarounds to Barcode Medication Administration Systems: Their Occurrences, Causes, and Threats to Patient Safety. J Am Med Inform Assoc. 2008;15:408-423. 30. 31. Keillor A, Morgenstern D [letter]. JAMA. 2005;294(2):178. Levick D, Lukens H [letter]. JAMA. 2005;294(2):179-180
32. Phibbs C, Milstein A, Delbanco SD, Bates D. No Proven Link Between CPOE and Mortality [letter]. Pediatrics, 19 Dec. 2005. 33. Patrick J. A Critical Essay on the Deployment of an ED Clinical Information System ‐ Systemic Failure or Bad Luck, version 5 [Internet]. Health Information Technology Research Laboratory, University of Sydney, Australia. Available from: http://www.it.usyd.edu.au/ %7Ehitru/essays/The%20Story%20of%20the%20Deployment%20of%20an%20ED%20Clinical %20Information%20System5.pdf . Last accessed 20 Nov 2009. 34. Health Information Technology Research Laboratory, School of Information Technologies, University of Sydney, Australia. Essays homepage [Internet]. Available from: http://www.it.usyd.edu.au/~hitru/index.php? option=com_content&task=view&id=91&Itemid=146 . Last accessed 20 Nov 2009.
35. U.S. Food and Drug Administration. MedWatch: The FDA Safety Information and Adverse Event Reporting Program. Available from: http://www.fda.gov/safety/MedWatch/ . 36. Koppel R, Kreda D. Health Care Information Technology Vendors' "Hold Harmless" Clause: Implications for Patients and Clinicians, JAMA. 2009;301(12):1276-1278. 37. Medical Products Agency, Sweden. Proposal for guidelines regarding classification of software based information systems used in health care. 2 Jun 2009. Available from: http://www.lakemedelsverket.se/upload/foretag/medicinteknik/en/Medical-Information-SystemsReport_2009-06-18.pdf Last accessed 20 Nov 2009. 38. Health Canada. Classification of Medical Devices Class I or Class II patient management software. 31 Aug 2009. Available from: http://www.hc-sc.gc.ca/dhp-mps/mdim/activit/announce-annonce/md_notice_software_im_avis_logicels-eng.php . Last accessed 20 Nov 2009. 39. Ray T. Electronic Health Record Systems Will Require 'Some Form of Oversight,' Government Advisor Says. Genome Web Pharmacogenomics Reporter. 27 Oct 2009. Available from: http://www.genomeweb.com/dxpgx/electronic-health-record-systems-will-require-someform-oversight-government-adv . Last accessed 20 Nov 2009. 40. Roche WF. Switch to electronic records alarms Jefferson Regional physicians. Pittsburgh Tribune-Review. 20 Oct 2009. Available from: http://www.pittsburghlive.com/x/pittsburghtrib/news/s_650606.html . Last accessed 20 Nov 2009. 41. National Institutes of Health. Human Subjects Protection and Inclusion of Women, Minorities, and Children, Guidelines for Review of NIH Applications. Available from: http://grants.nih.gov/grants/peer/guidelines_general/Human_Subjects_Protection_and_Inclusion. pdf . Last accessed 20 Nov 2009. 42. O’Harrow R. The Machinery Behind Health-Care Reform: How an Industry Lobby Scored a Swift, Unexpected Victory by Channeling Billions to Electronic Records. The Washington Post. 16 May 2009. Available from: http://www.washingtonpost.com/wpdyn/content/article/2009/05/15/AR2009051503667.html . Last accessed 20 Nov 2009. 43. Boehm F. Regulatory Capture Revisited – Lessons from Economics of Corruption. Internet Center for Corruption Research. University of Passau, Passau, Germany. 2007 Jul. Available from: http://www.icgg.org/downloads/Boehm%20-%20Regulatory%20Capture %20Revisited.pdf . Last accessed 20 Nov 2009. 44. Pitrou I, Boutron I, Ahmad N, Ravaud P. Reporting of Safety Results in Published Reports of Randomized Controlled Trials, Arch Intern Med. 2009; 169: 1756-1761
45. Ioannidis JP. Adverse events in randomized controlled trials: neglected, distorted, and silenced. Arch Intern Med 2009; 169: 1737-1739 46. Poses R. Healthcare Renewal Blog [Internet]. Roy Poses [cited 20 Nov 2009]. Available from: http://hcrenewal.blogspot.com/2009/11/suppression-of-clinical-trials-of.html . 47. Tfeft-Hansen PC. Unpublished clinical trials of sumatriptan. Lancet 2009; 374: 1501-2.
48. Lopatto E, Feeley J. Boston Scientific Pays $296 Million to Settle Probe. Bloomberg. 20 Nov 2009. Available from: http://www.bloomberg.com/apps/news? pid=20601103&sid=aW7jGYK573f4 . Last accessed 20 Nov 2009. 49. Healthcare Renewal Blog [Internet]. Query on index term “suppression of medical research” [cited 20 Nov 2009]. Available from: http://hcrenewal.blogspot.com/search/label/suppression%20of%20medical%20research . 50. The National Programme for IT in the NHS: Progress since 2006. Public Accounts Committee – Second Report. 14 Jan 2009. House of Commons. Available from: http://www.publications.parliament.uk/pa/cm200809/cmselect/cmpubacc/153/15302.htm . Last accessed 20 Nov 2009. 51. Hoffman S, Podgurski A. e-Health Hazards: Provider Liability and Electronic Health Record Systems. Aug 2009. Berkeley Technology Law Journal, Forthcoming Case Legal Studies Research Paper No. 09-25. Available from: http://papers.ssrn.com/sol3/papers.cfm? abstract_id=1463671## . Last accessed 20 Nov 2009. 52. Hoffman S, Podgurski A. Finding a Cure: The Case for Regulation And Oversight of Electronic Health Records Systems. Harvard Journal of Law & Technology 22; 1 (2008): 103165. 53. Silverstein SM. Health Care Information Technology, Hospital Responsibilities, and Joint Commission Standards [letter]. JAMA. 2009; 302:382. 54. Himmelstein DU, Wright A, Woolhandler S. Hospital Computing and the Costs and Quality of Care: A National Study. Am J Med, 20 Nov 2009, doi:10.1016/j.amjmed.2009.09.004 55. Leviss J, editor. H.I.T. or Miss: Lessons Learned from Health Information Technology Implementations. 1st ed. AHIMA Press; 2009.