carr_ch35.

qxd 11/21/00 9:41 AM Page 837

35
Risk Management
Program Evaluation
Christopher Cassirer

S
ince the medical professional liability insurance crises of the 1970s and 1980s, risk
management programs have been regarded by many as one of the most promising
responses to the problem of medical malpractice. In general, risk management pro-
grams are defined as the systems designed to prevent and control patient injury, enhance
quality, promote safety, and minimize the losses associated with medical malpractice
claims.

NEED FOR RISK MANAGEMENT PROGRAM EVALUATION

The objectives of these programs are to: (1) design and implement activities to
reduce the risk of injury associated with medical management of the patient, (2) reduce
and control the number and size of payments for medical malpractice claims and losses,
(3) identify the most economical approaches to financing risk whether it is through pur-
chased insurance and/or a variety of self-insurance alternatives, and (4) enhance quality
and improve patient safety.1
During the past thirty years, support for hospital risk management programs has
come from both federal and state legislatures, insurance companies, policy makers, man-
agers, and members of the patient care community. For example, in response to the med-
ical malpractice insurance crisis of the 1970s, the Health Care Financing Administration
(HCFA) introduced changes to the Medicare reimbursement policy which enabled hospi-
tals to consider alternative methods of financing risk.2 A condition of eligibility for reim-
bursement was that hospitals had to provide evidence that a risk management program
was in place to control losses.

837
carr_ch35.qxd 11/21/00 9:41 AM Page 838

838 Monitoring and Evaluating

In response to the crises of the 1970s and 1980s, many state legislatures passed
mandates requiring hospitals to implement risk management programs as a condition of
eligibility for licensure. Although mandates and regulations varied among the states, there
was a clear commitment to promote risk management as one of the more promising
responses to the medical malpractice problem.3
Health care provider associations and groups also demonstrated support for risk
management. The Department of Health and Human Services Task Force on Medical Lia-
bility and Malpractice and the American Hospital Association’s Medical Malpractice Task
Force issued statements advocating the strengthening and continued expansion of risk
management programs in hospitals. Further, the Joint Commission on the Accreditation
of Healthcare Organizations (JCAHO) introduced language requiring hospitals to imple-
ment various risk management program activities, such as linking risk management with
quality assurance activities in hospitals as a condition of continued accreditation.4
Despite the wealth of support that has been generated for risk management, there
is an absence of data or reliable information to suggest that risk management programs
are effective. In part this is due to the relative recency of risk management programs in
the health care industry. Second, there is little to no agreement among the professional
community about what is an effective risk management program. Third, there are real
difficulties associated with demonstrating the impact of risk management program activ-
ities on incidents and rates of adverse events and patient injuries and the frequency and
severity of medical malpractice claims. Fourth, measuring what is prevented continues
to be the most elusive goal in proving the value of risk management. Medical chart
reviews, occurrence screens, and incident reports made verbally or in writing can be
important sources of information about medical injury. Losses associated with malprac-
tice claims tend to be the focus of our evaluation activities. Risk managers, however, can
begin to enhance their skills and assume a leadership role by learning more about the
basic tools currently available to design, implement, and evaluate risk management pro-
gram effectiveness.
To help enable practicing risk managers with a better set of tools, this chapter will:
(1) present an empirically based conceptual framework to define risk management pro-
gram goals and objectives, (2) present information about the American Society for
Healthcare Risk Management’s (ASHRM) past efforts to develop a comprehensive risk
management program evaluation model that incorporates a systems perspective, a con-
tinuous quality improvement approach to program design, and evaluation, (3) review
recent research regarding hospitals’ efforts to implement the ASHRM model, (4) discuss
other studies and ongoing research efforts to evaluate risk management program effec-
tiveness, and (5) discuss future trends that will continue to affect efforts by professional
risk managers to develop tools and strategies, and recommend partnership strategies and
roles for the profession in building an evaluation model to demonstrate the effectiveness
of risk management programs in practice.

SETTING RISK MANAGEMENT PROGRAM OBJECTIVES

While risk management programs have a defined role in many types of health care
provider organizations, much of the history and current effort to evaluate programs is
based on our knowledge of hospital risk management practices. Since the 1970s, studies
have continued to show that approximately 80 percent of malpractice claims are the
carr_ch35.qxd 11/21/00 9:41 AM Page 839

Risk Management Program Evaluation 839

result of hospital-based adverse events and injuries to patients. Thus, efforts to develop
research-driven models, tools, and strategies to manage risk have been largely hospital-
based. Similarly, evaluation research and recommendations regarding the effective design
and development of risk management programs for health care organizations have been
largely hospital-focused.
The generally accepted frame guiding current efforts to conceptualize the goals and
objectives of risk management programs is based on the accumulating evidence drawn
from twenty years of research on rates of adverse events involving hospitalized patients
and the accumulating literature on medical malpractice closed claims studies. The most
important studies to date were conducted in the 1970s and 1980s. The results of these
investigations have impacted national policy formulation and state-enacted legislative
activities. Operationally, these studies have shaped the currently accepted framework for
defining the problem of medical malpractice and the goals and objectives of hospital and
health care risk management programs.

NATIONAL ASSOCIATION OF INSURANCE COMMISSIONERS

In response to the medical malpractice insurance crisis of the 1970s, the National Asso-
ciation of Insurance Commissioners (NAIC) began a comprehensive study based on the
malpractice claims files of all U.S. insurers that had written premiums of $1 million or
more in any year since 1970.5,6 The complete database included information on 71,788
claims closed by 128 insurers between 1975 and 1978.
Analysis of filed claims indicated that 78 percent of all incidents resulting in paid
claims occurred in hospitals and accounted for approximately 87 percent of all payments
to claimants. The majority of these claims were filed based on incidents and adverse
events occurring in operating rooms and emergency departments. Approximately one-
third of all paid claims alleged improperly performed procedures.
A similar analysis was conducted by the U.S. General Accounting Office (GAO) fol-
lowing the medical malpractice insurance crisis of the 1980s. In 1987, the GAO published
a study of 1,706 closed malpractice claims. Data were obtained from twenty-five profes-
sional liability insurance companies. This group was selected to represent a population
of 73,000 closed claims involving more than 100,000 providers and 102 insurance com-
panies as of 1984. The findings were similar to those of the NAIC researchers. Approxi-
mately 80 percent of closed malpractice claims were determined to be the result of
medical care related to injuries involving hospitalized patients.

CALIFORNIA HOSPITAL ASSOCIATION STUDY

The emerging evidence that most injuries occurred to hospitalized patients led to two
major studies of patient injury rates in hospitals in the 1970s and the 1980s. These stud-
ies measured rates of adverse events and adverse events due to probable negligence
occurring among hospitalized patients.
The first comprehensive investigation to estimate rates of adverse events and injuries
due to negligence was conducted by the California Medical Association and the California
Hospital Association in the 1970s. The purpose of the study was to determine the cost and
feasibility of a no-fault compensation system for medical malpractice. The analysis
carr_ch35.qxd 11/21/00 9:41 AM Page 840

840 Monitoring and Evaluating

involved reviews of 20,864 medical records drawn from a sample of twenty-three acute
care hospitals in California. Medical records represented patient differences in age, gen-
der, race, and payment source. Hospitals included in the sample represented differences
in size, location, region, ownership, and teaching status. Reviewed medical records were
assumed to represent all California hospital discharges during 1974.
Examination of the medical records involved the application of twenty screening cri-
teria. Trained medical chart reviewers applied the criteria and identified hospitalized
patient charts where there was evidence of an adverse event. Screened charts were then
subjected to a second review by teams of physicians and physician-attorneys to determine
if a potentially compensable malpractice event (PCE) had occurred. PCEs were defined
for the study as adverse events in which patients suffered temporary or permanent dis-
ability due to errors in health care management. Physician-attorneys then reviewed all
charts with evidence of a PCE to determine if a jury would be likely to decide in favor of
the injured patient in a legal review of the malpractice claim.
The researchers estimated that 970 or 4.65 percent of medical charts provided evi-
dence of patient injuries likely due to errors in health care management of the patient,
either prior to or during hospitalization. From those initial 970 records it was determined
that 17 percent would have been likely to result in a legal determination of negligence.
Based on this research, it was later estimated that one out of every 126 patients hospital-
ized in California in 1974 suffered a potentially compensable injury.7

HARVARD MEDICAL PRACTICE STUDY

The medical malpractice crisis of the mid-1980s prompted a second major study to
measure hospitalized patient injury rates. Similar to the study conducted in California, a
team of researchers at the Harvard University developed a population-based measure of
the incidence of hospitalized patient injuries and produced an estimate of the percentage
of medical care-related injuries due to probable negligence.
To estimate the incidence of patient injuries, the Harvard research team selected a
random sample of fifty-one acute care hospitals in New York in 1984. Hospitals were
stratified by size, geographic location, teaching status, and ownership. Second, a random
sample of approximately 31,000 medical records was selected from among the fifty-one
hospitals.
Medical records were reviewed by trained medical record analysts (MRAs) and
nurses to screen for adverse events. Adverse events were defined in the Harvard study as
evidence of substandard care, inappropriately performed medical procedures, or errors
in health care management. The MRAs relied on eighteen screens to identify adverse
events. In total, 1,133 adverse events were detected. Teams of physician-attorneys then
reviewed the adverse events to decide if there was any indication of probable negligence.
Of the 1,133 records initially selected, 280 were determined to be the result of probable
negligence in health care management of the patient.8,9
Based on their analysis, the Harvard research team estimated that 3.7 percent of
patients discharged from the fifty-one hospitals in New York in 1984 suffered an adverse
event. Of these adverse events, approximately 1 percent was determined to be due to
probable negligence. Initial findings indicated that adverse events ranged in hospitals
from .2 percent to 7.9 percent with a mean of 3.2 percent. Rates of adverse events due to
negligence ranged from 1 percent to 60 percent with a mean of 24.9 percent. Further
carr_ch35.qxd 11/21/00 9:41 AM Page 841

Risk Management Program Evaluation 841

analysis indicated that rates were not normally distributed among the hospitals. Charac-
teristics of hospitals examined for their possible association with rates of adverse events
(AEs) and negligent adverse events (NAEs) included hospital ownership, location, size,
proportion of minority discharges, and the teaching status of the hospitals.
The results of a multivariate regression analysis indicated that university teaching hos-
pitals had a higher rate of adverse events than affiliated and nonteaching hospitals. Hospi-
tals in upstate, nonmetropolitan statistical areas had significantly fewer adverse events than
hospitals closer to the major cities. Large hospitals had fewer adverse events than medium-
size hospitals. The only hospital characteristic, however, significantly associated with neg-
ligent adverse events among the hospitals was the proportion of minority discharges.
In the most recent studies conducted by another Harvard Medical Practice Study
team, approximately 15,000 medical charts were examined from hospitals in Colorado
and Utah in 1995. Methods of chart review to identify AEs and NAEs were similar to those
utilized in the studies examining rates of medical injury in New York hospitals. Patient
injury rates ranged from 3 to 4 percent.10 Similar to the findings of the New York medical
practice study, the rate of NAEs was determined to be less than 1 percent.
Based on this data, the Institute of Medicine (IOM) estimates that as many as 44,000
to 98,000 patients experience a preventable medical injury in the process of receiving
medical care in hospitals in the United States per year.11

RELATIONSHIPS BETWEEN PATIENT INJURIES

AND MALPRACTICE CLAIMS
Data on rates of patient injuries, however, provide only partial insight into the full scope
of the medical malpractice problem. Studies have shown that not all injuries result in the
filing of medical malpractice claims or legal determinations in favor of patients, even
when there has been evidence of probable negligence.
The first studies to examine relationships between patient injuries and malpractice
claims were conducted in the early 1980s. Danzon, for example, examined the early find-
ings of the California researchers with the goal of determining the number of patients who
were injured in the process of receiving medical treatment that later filed a malpractice
claim.12 A second goal of the study was to estimate the number of patients who recovered
damages when an adverse event was determined to be the result of probable negligence.
The results of these analyses indicated that less than one in ten patients suffered an
adverse event due to negligence in California hospitals in 1974 went on to file a medical
malpractice claim. Of those claims that were filed, slightly less than 40 percent resulted
in payments to claimants.13,14
The second major set of studies focusing on relationships between patient injuries
and malpractice claims was conducted by the research team at Harvard. They evaluated
in detail the 280 medical charts that suggested patients were injured due to probable neg-
ligence. Based on their research, it was determined that only forty-seven, or one out of
every 7.6 negligent adverse events resulted in filing claims for injuries due to probable
negligence, only one out of sixteen received compensation.
Similar to the findings of Danzon, the Harvard research team concluded that of all
those patients discharged from New York hospitals in 1984 who suffered an adverse event
due to probable negligence, few went on to file malpractice claims or recover damages
for their injuries. Although several medical malpractice insurance crises had occurred in
carr_ch35.qxd 11/21/00 9:41 AM Page 842

842 Monitoring and Evaluating

the past, the data suggest that the problem of medical malpractice may have been much
worse than indicated during the crises of the 1970s and 1980s.

CONCEPTUALIZING RELATIONSHIPS, PATIENT INJURIES,

AND MALPRACTICE CLAIMS
Relationships between the incidence of patient injuries, the rate of provider error, and
malpractice claims are represented by Figure 35.1.15–17 Area A represents all medical
injuries among hospitalized patients. Results of the California study, for example, indicate
that 4.65 percent of all patients discharged from hospitals in California in 1974 were
injured as a result of medical mismanagement of the patient. Similarly, the results of
the Harvard Medical Practice Study indicate that 3.7 percent of patients discharged from
hospitals in New York in 1984 suffered an adverse event.
Area B represents the extent of legal misconduct in medicine. Although the scope of
this dimension is unknown, the incidence of negligence in medical care that results in
patient injury can be identified. The Harvard studies, for example, indicate that approxi-
mately 1 percent of patients hospitalized in New York in 1984 suffered an adverse event
due to negligence. This percentage represents all of those patient injuries that should
result in the filing of medical malpractice claim.
Area D represents the estimates of negligent adverse events that have emerged as
medical malpractice claims. It suggests that the actual number of claims that emerge is
significantly less than the number expected if current systems for addressing injury and
claims were operating effectively. It also overlaps Area B to indicate that some claims are
filed when there is no evidence of negligence. Moreover, it suggests that claims are filed

FIGURE 35.1. Relationships Among Patient Injuries, Provider Errors, and

Malpractice Claims

C B

D E

A  Incidence of patient injuries  Unknown

B  Incidence of errors during medicine  Estimates of 3–4 percent (Harvard Studies)
C  Patient injuries due to errors in medical care  Less than 1 percent (Harvard Studies)
D  Filed malpractice claims (1 out of 8—Danzon; Harvard Studies)
E  Filed claims resulting in claimant compensation (1 out of 10—Danzon; Harvard)

Adapted from: National Association of Insurance Commissioners. NAIC Malpractice Claims: Medical Malpractice Closed
Claims, 1975–1978. Milwaukee, WIS. National Association of Insurance Commissioners, 1980; Orlikoff, J. E., Vanagunas, A. M.
Malpractice Prevention and Liability Control for Hospitals (2nd ed.). Chicago, Ill.: American Hospital Association, 1988; Morlock,
L. L., Cassirer, C., Malitz, F. E. “Hospital Risk Management and Professional Liability Claims Experience in Maryland.” Final
Report: Agency for Health Care Policy & Research, Grant Number 1 RO1 HS06735, 1997; Cassirer, C. Hospital Risk Manage-
ment Programs in Maryland (1995). Baltimore: Johns Hopkins University, 1997.
carr_ch35.qxd 11/21/00 9:41 AM Page 843

Risk Management Program Evaluation 843

due to legal misconduct by health care providers when no evidence of medical care
related injury appeared in the medical record.
Finally, Area E represents the number of claims that result in compensation to
claimants who are injured due to negligence in the process of receiving medical treatment.
Consistent with the findings from the major studies of medical injury and malpractice, it
suggests that fewer patients are compensated than expected. As noted, in the Harvard stud-
ies, only one out of every sixteen patients who suffered an injury due to negligence
received compensation under the current liability system.
This adapted framework (Figure 35.1) initially proposed by the National Association
of Insurance Commissioners (NAIC) in the late 1970s continues to provide the most com-
prehensive paradigm for characterizing the multidimensional nature of the medical mal-
practice problem in hospitals and highlights the major goals and objectives that hospital
risk management programs attempt to address. Referring to Figure 35.1, in theory, hos-
pital risk management programs have the potential to: (1) reduce the frequency of pre-
ventable adverse events (Area A), (2) reduce the number of patient injuries due to
provider errors (Area B), (3) decrease the number of malpractice claims (Area C),
(4) manage the number of claims that do emerge to control losses (Area D), and
(5) finance risk through the most economical methods to ensure an adequate source of
funds is available to pay for malpractice claims and expenses (Areas D and E).

ASHRM MODEL FOR RM EVALUATION

Prior to the 1980s, hospitals and other health care provider organizations had little guid-
ance or access to resources to help them respond to state mandates and other require-
ments to develop risk management programs. To facilitate hospitals’ efforts, the American
Hospital Association (AHA) chartered a subsidiary organization, named the American Soci-
ety for Healthcare Risk Management (ASHRM). For the past twenty years, ASHRM has
assumed a leadership role in helping to define the professional practice of health care risk
management.
Following the medical malpractice insurance crisis of the 1980s, ASHRM organized
a task force of experts to define a model and approach to developing and implementing
a hospital risk management program assessment tool. By 1991, the ASHRM task force
had produced a Hospital Risk Management Self-Assessment Manual for risk man-
agers. Included in the manual is a tool called the ASHRM Assessment Abstract. The
abstract includes a listing and description of the types of program elements and related
activities that ASHRM believes every hospital should have in place as part of developing
an effective program. These recommendations were designed to apply to all hospitals in
every state, regardless of whether or not states had a mandate.
The approach defined by ASHRM represents “better practices” in the profession as
defined by a leading society of risk management professionals. These better practices
reflect the accumulated wisdom of the experts serving on the task force and the experi-
ence of practicing risk managers. The history and support for the ideas synthesized in the
abstract can be found in the trade and professional literature, numerous published arti-
cles and books on the subject of health care risk management, as well as the limited sci-
entific research on the effectiveness of these approaches.18–25 The objectives of risk
management programs that are the focus of the ASHRM model have been described. The
recommendations of ASHRM are designed to represent the basics, a starting point for risk
management program development.
carr_ch35.qxd 11/21/00 9:41 AM Page 844

844 Monitoring and Evaluating

THE ASHRM MODEL PHILOSOPHY

The ASHRM Model for risk management is based on risk management theory, a systems
approach and a continuous quality improvement philosophy of program improve-
ment.26–30 Each activity recommended by ASHRM should have a structure in place to sup-
port it, an associated set of processes to accomplish the activity, and a set of defined
outcome measures for evaluating performance. Consistent with the theory of CQI, empha-
sis is placed on the development of process measures. Accumulated data and information
about performance is to be used to guide ongoing improvement to accomplish risk man-
agement program goals.

THE ASHRM MODEL ASSESSMENT PROGRAM

According to ASHRM, an effective hospital risk management program comprises five basic
program elements, each of which is delineated into specific dimensions of activity. The
five elements and related dimensions of activity recommended by ASHRM are as follows:

Organizational Structure
ASHRM recommends that a hospital risk management program should have an organiza-
tional structure to support the risk management function. Dimensions of the orga-
nizational structure that should be in place include activities to promote governing board
commitment, establishing the role and responsibilities for a designated hospital risk
manager, and activities to promote medical staff involvement in the hospital risk man-
agement program.

Governing Board Support Activities of the governing board assessed with the
ASHRM Assessment Abstract focused on whether the governing board supports the risk
management program, whether a written policy statement or plan describing the
risk management program has been developed, and if formal approval of the plan has
been secured from the governing board. Another activity assessed is whether the govern-
ing board receives reports from risk management at least twice per year.
Process measures for assessing governing board support focus on the content of the
written risk management plan. For example, whether goals and objectives have been
defined for risk management, whether the position of the risk manager in the organiza-
tional structure is identified in the plan, as well as other plan characteristics.
Other measures of process assessed whether there are established communication
channels between risk management and other organizational components and if there is
a description of the institutional policy of risk financing. Process measures also focused
on the written risk management reports to the governing body. The processes assessed
were whether the report to the governing body included information on insurance
issues, risk financing issues, and adverse events, as well as the hospital’s claims experi-
ence, among other measures.
Outcome measures focused on whether there is documentation of the governing
board approval of the risk management program and whether the governing board
updates the risk management plan at least annually. Another outcome measure assessed
is whether the governing body takes appropriate action on data from risk management
and if those actions are documented in the governing body meeting minutes.
carr_ch35.qxd 11/21/00 9:41 AM Page 845

Risk Management Program Evaluation 845

Designated Risk Manager One activity was assessed on this dimension of hos-
pital risk management programs. It focused on whether there is a designated person(s)
responsible for the hospital risk management program.
Process measures focused on the contents of the written job description for the risk
manager. Assessments pertained to the role of the risk manager in loss identification,
quality assurance, safety management, claims handling, risk financing, security, and
patient relations, among other roles. No outcome measures were assessed on this dimen-
sion of hospital risk management programs.
Medical Staff Involvement The ASHRM Assessment Abstract assessed whether
medical staff involvement in the hospital risk management program was structured.
Process measures further delineated the attributes of this structured involvement and
included items such as whether the medical staff had developed criteria in at least three
clinical areas to identify adverse patient events.
Other process measures assessed were whether there was a process for the medical
staff to review variations from these established criteria, whether the medical staff identi-
fied risk management problems in the delivery of patient care, whether the medical staff
recommended corrective action to resolve problems, whether the medical staff ensured
that problems were resolved, whether the medical staff participated in risk management
policy development, and whether the medical staff participated in the design of educa-
tional programs directed toward loss prevention.
Three outcome measures assessed on this dimension focused on whether the med-
ical staff had identified at least three risk management problems annually, whether the
medical staff had resolved at least three problems, and whether the number of claims
involving the medical staff had decreased over time.

Incident Identification and Analysis

The second element that ASHRM recommends should be in place is an incident identifi-
cation and analysis system. This element comprises two dimensions of activity including
activities designed to identify incidents of adverse events and activities directed toward the
analysis of incidents.
Risk Identification Systems Assessments of the risk identification system
focused on whether the hospital had developed systematic methods for reporting patient,
visitor, and employee-related adverse events. Assessments also focused on whether there
was a system to identify adverse events in high risk clinical areas and whether there was
a system developed to report adverse events to local and state entities.
Process measures for patient, visitor, employee and clinical high risk incident iden-
tification systems focused on whether reports produced by these systems identified the
number of incidents that occurred each month as compared to an activity base. Process
measures for the system for reporting adverse events to state and local entities were con-
cerned with whether the system identified an individual with reporting responsibility,
whether there was a process in place for determining which events were reportable, and
if a format had been established for reporting adverse events.
Outcome measures for incident reporting systems pertaining to patient, visitor,
employee, and high risk clinical areas focused on whether the reporting method in each
of the four areas increased the percentage of claims identified at the time of occurrence.
The outcome measure for the system of reporting adverse events focused on whether
local and state requirements were followed.
carr_ch35.qxd 11/21/00 9:41 AM Page 846

846 Monitoring and Evaluating

Risk Analysis One activity was assessed on this dimension of hospital risk man-
agement programs. The item focused on whether a process had been developed to ana-
lyze and trend risk identification data. Assessments of process measures related to this
activity focused on whether the analysis of identified risk was stratified by location, type
of occurrence, patient characteristics, and other characteristics. The outcome measure-
ment focused on whether loss prevention activities had been initiated as a result of the
identified problems.

Loss Prevention
The third basic element recommended by ASHRM is loss prevention, which comprises
several dimensions of activity: education programs to prevent incidents from occurring,
updating informed consent policies, monitoring hospital compliance with regulatory and
accreditation requirements, analyzing claims data, and engaging in activities around
patient and family relations.

Educational Programs Risk management educational activities assessed

included whether risk management information was presented to new employees, resi-
dents, and medical staff during orientation programs and whether continuing medical
education (CME) programs were conducted on risk management at least annually.
Process measures assessed focused on whether comparison data had been devel-
oped to examine the number of employees, residents, and medical staff exposed to
risk management out of the total number of these individuals who were hired annu-
ally. A process measure was also defined as whether the risk manager compares the
number of physicians in attendance at CME programs out of the total number of med-
ical staff.
The one outcome measure assessed was whether heightened awareness had been
observed regarding the institutional risk management program, risk manager functions,
and related policies and procedures.

Informed Consent One activity and one process measure were assessed pertain-
ing to informed consent. There were no outcome measures assessed. The activity
assessed focused on whether the hospital provided a current, updated policy and proce-
dure for obtaining and documenting informed consent. The process measure assessed
was defined as whether the informed consent policy and procedure had been updated
within the last two years.

Compliance with Regulations One activity pertaining to compliance with reg-

ulations was assessed. The item focused on whether the hospital monitored local, state,
federal, and JCAHO requirements for risk management. Process measures assessed
focused on whether the number and type of standards were monitored and if the hospi-
tal tracked the amount of time spent in reviewing requirements, educating staff, provid-
ing consultation, preparing for survey visits, and developing and implementing plans to
improve compliance with risk management program requirements. Outcome measures
assessed focused on whether monitoring activities and processes resulted in the absence
or reduction of the frequency and severity of regulatory citations, the frequency and sever-
ity of JCAHO contingencies, and the frequency and severity of claims related to noncom-
pliance with risk management regulations.
carr_ch35.qxd 11/21/00 9:41 AM Page 847

Risk Management Program Evaluation 847

Claims Data Analysis Activities The ASHRM Assessment Abstract assessed

whether hospitals had implemented a mechanism for analyzing claims and or incident
data and whether loss prevention issues had been identified. Process measures focused
on whether this information was reviewed periodically by specific categories including,
allegation, service, specialty, and location. The one outcome measure assessed focused
on whether claims reviews resulted in a lower frequency and severity of claims in areas
targeted by loss prevention efforts.

Patient and Family Relations Activities pertaining to patient and family rela-
tions focused on whether institutional mechanisms had been developed to assist and
respond to patients and families following an adverse event, the filing of a complaint, or
notification of lost or stolen property. The process measure assessed focused on whether
the hospital risk management program tracked the number and type of patient and fam-
ily interactions. The outcome measure assessed was whether claims had been averted due
to positive patient and family interventions.

RISK FINANCING POLICIES AND PROCEDURES

The fourth basic element that ASHRM recommends should be in place is a risk financing
policy and process for analyzing decisions about risk funding. Activities pertaining to the
risk financing dimension were defined as whether the hospital had implemented a mech-
anism for calculating the overall cost of risk, monitoring the cost of risk, comparing the
cost of risk with other institutions, and conveying findings to senior management and the
governing body.
Process measures focused on whether this data was compiled annually and included
information on risk financing costs, loss prevention costs, risk management program
administration costs, and information on underfunded losses. Another process measure
assessed focused on whether the cost of risk was tracked as a percentage of overall oper-
ating budget.
Assessments of outcome measures focused on whether the cost of risk was similar
to or lower than comparable institutions and whether the cost of risk was controlled or
reduced overtime.

Claims Management Policies and Procedures

The fifth basic element of a hospital risk management assessment program is developing
a claims management program. Claims management activities assessed with the ASHRM
Assessment Abstract focused on whether the hospital had developed a written claims
management policy and claims management process. Process measures assessed
included whether the policy was developed with senior management input, whether the
policy was reviewed at least annually, and whether the policy reflects a comprehensive
review of the process for conducting claims investigation, case analysis, settlement
processes, litigation management processes, and claims file maintenance.
Outcome measures assessed focus on whether the policy and policy review process
result in a lower frequency of claims involving wrongful claims management practices, a
lower frequency of claims involving “bad faith,” and favorable audit reports from exter-
nal sources such as excess insurance carriers.
Each program assessment element ASHRM Assessment Abstract is identified. The
number of recommended activities, processes, and outcome measures appears in sum-
mary form in Table 35.1.
carr_ch35.qxd 11/21/00 9:41 AM Page 848

848 Monitoring and Evaluating

TABLE 35.1. Hospital Risk Management Elements Measured on ASHRM

Assessment Abstract
Structures/ Process Outcome
Activities Measures Measures Total
Organizational Structure
Governing Board 3 16 3 22
Hospital Risk Manager 1 18 0 19
Medical Staff Involvement 1 10 3 14
TOTAL 5 44 6 55
Risk Identification and Analysis
Risk Identification 5 7 5 17
Risk Analysis 1 4 1 6
TOTAL 6 11 6 23
Loss Prevention
Education 4 4 1 9
Informed Consent 1 1 0 2
Compliance 1 6 3 10
Claims Data Analysis 2 4 1 7
Patient and Family Programs 3 1 1 5
TOTAL 11 16 6 33
Risk Financing 4 6 2 12
Claims Management 1 7 3 11
TOTAL ITEMS ASSESSED 27 84 23 134

COMMITMENT TO THE ASHRM ASSESSMENT MODEL AMONG

HOSPITALS IN THREE STATES
Despite the wealth of support that has been generated for hospital risk management pro-
grams there is a dearth of empirical evidence describing their effectiveness. One dimen-
sion of assessing risk management program effectiveness is to determine organizational
commitment to developing a risk management program in the first place.
Although the ASHRM Assessment Model and tool have been available since the mid-
1980s, it does not appear to have been widely utilized in practice. Several recent studies
undertaken by researchers from the Harvard and Johns Hopkins University have utilized
the ASHRM Assessment Abstract to evaluate the effectiveness of hospital risk management
programs. The results of the following investigations examined the extent to which hos-
pitals have developed effective risk management programs based on the model and
approach defined by ASHRM.31 Two key questions addressed in this research are: To what
extent do hospitals have risk management programs in place that incorporate the basic
elements and dimensions of an effective program as defined by ASHRM? And, what are the
barriers to adoption of the ASHRM model?

DEVELOPING AN EFFECTIVE ASHRM RISK MANAGEMENT

ASSESSMENT PROGRAM
In 1995, researchers from Johns Hopkins University and Harvard University conducted a
study to describe the level of hospital risk management program activity in three states:
Colorado, Utah, and Maryland. In this study, a modification of the ASHRM self-assessment
carr_ch35.qxd 11/21/00 9:41 AM Page 849

Risk Management Program Evaluation 849

tool was administered to risk managers in seventy-seven acute care hospitals along with
a second survey designed to collect information about perceived “better practices,”
including barriers and facilitators to enhancing program performance. In total, thirteen
hospitals in Utah, fifteen hospitals in Colorado, and forty-nine hospitals in Maryland com-
pleted a slightly modified version of the ASHRM Assessment Abstract and the second sur-
vey. The hospital risk management programs studied in Colorado and Utah are the same
hospitals studied in the recent findings on patient injury rates presented in the Institute of
Medicine report on medical error.32
Participation in the 1995 study of hospital risk management programs was motivated
by involving state chapters of ASHRM: the Utah Healthcare Associated Risk Managers
(USHRM), the Colorado Healthcare Associated Risk Managers (CHARM), and the
Maryland Society for Healthcare Risk Management (MSHRM). Each state chapter of
ASHRM supported the project by presenting their members with information about the
study including its design, specific aims, and benefits to the members. Each chapter pub-
lished an announcement of the study in their respective newsletters and/or sent a letter of
endorsement signed by the president of the local state chapter.
Informed consent to participate was a two-stage process. First, a letter describing the
study was sent to the hospital CEOs in each state. Follow-up phone calls were made to
each hospital to verify that the letter had been received and to determine if there were any
questions or concerns about participating in the study. Then, each hospital risk manager
received a telephone call to describe the project and request that the ASHRM Assessment
abstract be completed. Hospital risk managers had the opportunity to review the abstract
and to discuss the project internally with other hospital administrators prior to giving
their consent to participate.
Responding to the abstract required risk managers to indicate whether specific pro-
gram activities were “in-place” or “needed development.” To estimate the level of hospi-
tal risk management program activity, total raw scores were computed for each hospital
by counting the number of program activities “in place” and dividing that number by 134
(the total number of activities, processes, and outcome measures included in the
abstract). Sub-scores were also computed to estimate the level of activity on individual
program elements and dimensions of activity within each program element. Variation in
mean abstract activity scores are reported in the tables.

Total ASHRM Activity Scores

The results of the study of hospital risk management activity based on responses to the
ASHRM Assessment Abstract suggest that, on average, hospitals in these three states have
approximately 65 percent of the recommended risk management programs and activities
in place (Table 35.2).

Structure, Process, Outcome

Across the states, on average, hospitals had the greatest commitment to developing struc-
tures and activities to support the risk management function. The results indicate that, on
average, hospitals in the three states had 71 percent of the structures and activities in
place. There was, however, somewhat lesser commitment to developing the processes
associated with those activities. Hospitals in the three states had 64 percent of the
processes in place associated with structures and activities identified in the abstract.
The least amount of activity was in hospitals’ efforts to implement outcome measures to
assess risk management program performance. On average, hospitals in the three states
had 60 percent of the outcome measures in place recommended in the abstract.
carr_ch35.qxd 11/21/00 9:41 AM Page 850

850 Monitoring and Evaluating

TABLE 35.2. Mean and Range of Hospital Risk Management Program Activity
Scores (%)
Based on Risk Managers’ Responses to the ASHRM Assessment Abstract (1995):
Structure, Process, Outcome, and Total Scores*
Utah Colorado Maryland TOTAL
Mean Range Mean Range Mean Range Mean Range
Structure/ 69 (21–97) 62 (44–82) 74 (37–96) 71 (22–96)
Activities
Processes 61 (14–91) 54 (15–83) 66 (29–94) 64 (14–94)
Outcomes 70 (9–100) 68 (39–91) 56 (9–100) 60 (9–100)
TOTAL 64 (15–90) 58 (26–93) 66 (28–94) 65 (15–94)

*Activity scores (%) are defined as the total number of items “in-place” divided by the total number possible per dimension,
multiplied by 100.

ASHRM Program Elements

Across the states, 73 percent of the hospitals had the recommended organizational struc-
tures in place; 77 percent had implemented suggested risk identification and analysis sys-
tems; 65 percent had appropriate claims management functions in operation; 62 percent
had implemented suggested loss prevention strategies; but only 30 percent had adopted
recommended risk financing mechanisms (Table 35.3).

Barriers to Developing an Effective ASHRM Risk

Management Assessment Program
In addition to administering the ASHRM Assessment Abstract, a second survey was com-
pleted by the seventy-seven hospitals. Hospital risk managers responded to both open-
and closed-ended questions and participated in key-informant interviews in which they
discussed barriers to developing a program consistent with ASHRM’s recommendations.
Key issues that emerged regarding barriers to utilization and implementation of the
ASHRM Abstract focused on the following:
• Awareness: Although the ASHRM Assessment Abstract had been in publication
since 1991, many risk managers were not aware that this instrument and guidance was
available through the professional association. Others had copies of the Self-Assessment
Manual which included the abstract but had not put it into use.
• Turnover: Risk managers reported that turnover in responsibility for the risk
management function or set of activities within their organizations limited the opportunity
to implement and follow through on many of the activities recommended by ASHRM.
• Competing priorities: Risk managers reported that increasing market pressure
to control costs has resulted in budget and/or staff reductions to support risk manage-
ment activities. Designated risk managers found themselves with an increasing array of
responsibilities and insufficient time to focus on more than claims management and con-
ducting providing the basic in-services programs.
• Alternative models in place: Some risk managers reported that while they had
many of the dimensions and activities recommended by ASHRM, their programs were
based on a different philosophy or organizational arrangement that affected how risk
carr_ch35.qxd 11/21/00 9:41 AM Page 851

Risk Management Program Evaluation 851

TABLE 35.3. Major Program Elements and Dimensions

Mean and Range of Hospital Risk Management Program Activity Scores (Percent)
Based on Risk Managers’ Responses to the ASHRM Assessment Abstract (1995): Major
Program Elements and Related Dimensions*
Utah Colorado Maryland TOTAL
Mean Range Mean Range Mean Range Mean Range
Organizational Structure 72 (15–98) 64 (16–96) 75 (23–100) 73 (15–100)
Governing Board Support 66 (0–96) 68 (39–96) 80 (23–100) 76 (0–100)
Hospital Risk Manager 77 (5–100) 63 (5–100) 79 (16–100) 76 (5–100)
Medical Staff Involvement 74 (0–100) 57 (0–100) 62 (0–100) 63 (0–100)
Risk Identification 75 (39–100) 82 (48–100) 77 (26–100) 77 (26–100)
Risk Identification System 73 (41–100) 82 (65–100) 74 (35–100) 75 (35–100)
Risk Analysis System 83 (0–100) 80 (0–100) 84 (0–100) 83 (30–100)
Loss Prevention 54 (9–90) 57 (21–85) 65 (15–94) 62 (9–94)
Education 46 (0–67) 41 (0–100) 40 (11–100) 45 (0–100)
Informed Consent 82 (0–100) 94 (50–100) 88 (0–100) 88 (0–100)
Compliance 38 (0–100) 59 (10–100) 53 (0–100) 51 (0–100)
Patient/Family Relations 84 (20–100) 60 (20–100) 89 (40–100) 84 (20–100)
Claims Data Analysis 62 (0–100) 62 (0–100) 82 (0–100) 76 (0–100)
Risk Financing 31 (0–92) 19 (0–100) 33 (0–100) 30 (0–100)
Claims Management 70 (0–100) 30 (0–100) 47 (0–100) 65 (0–100)

*Activity scores (percent) are defined as the total number of items “in-place” divided by the total number possible per dimension, multiplied by 100.

management activities were implemented. For example, among those hospitals partici-
pating in integrated delivery systems, many reported that the corporate entity had cen-
tralized the risk management function. Specific activities may have been available through
the corporate office, but implemented specifically within their institutions. Other pro-
grams reported they did not have formal and separate risk management programs.
Instead, risk management activities were part of a patient advocate or patient ombudsman
function. Still another program reported that increased competition, consolidation, and
integration in the local market had led to a corporate decision to fold the risk manage-
ment function into the human resources department. Issues of physician credentialing,
review, and response to incidents of patient injury that are tied to risk management pro-
gram activities were considered by this institution to be “employee” performance issues
and best housed within the human resources function.
In general, there did not appear to be any consensus among the risk managers in the
three states regarding “one best way” to design or implement a risk management pro-
gram within their institutions. Instead, there were perceptions about “better practices”
and a suggestion that there may be a range of alternative models that have the potential
to improve quality, safety, and reduce risk. Other models include insurer-based initiatives
and medical specialty-focused risk management interventions, among others.
Risk managers did, however, identify opportunities to enhance the ASHRM model.
One suggestions was to review the relevance of the criteria defined by ASHRM. It has been
ten years since the abstract was published. Another recommendation was that while the
ASHRM model is conceptually sound, the model reflects errors of commission rather than
errors of omission. Much of the marketplace is highly penetrated by managed care orga-
nizations and payment arrangements, which create incentives to omit certain approaches
carr_ch35.qxd 11/21/00 9:41 AM Page 852

852 Monitoring and Evaluating

to diagnosis and related treatments. The consequences of these actions may not be appar-
ent for many years to come, creating concerns about how to best manage this emerging
exposure. The model designed by AHSRM is based on the assumption that injuries are the
result of committed acts. Thus, in the ASHRM model, an important focus of an effective
risk management program, is to design systems that utilize incident reporting to identify
and manage potentially compensable events.
Regardless of the approach or area of activity in medicine that is the focus of a hos-
pital- or organization-based risk management intervention, there remains an ongoing
substantive and empirical challenge to defining appropriate measures of risk manage-
ment program outcomes. The successful evaluation of risk management program effec-
tiveness to promote evidence-based management is dependent upon defining and
measuring outcomes. There are few if any who have been successful in reaching consen-
sus on what is an acceptable set of measures for assessing organizational risk and defin-
ing measures that can be utilized to inform decision making, prevent injury, and improve
the management of claims. Identification of incidents of patient injury continues to be a
challenge. Providers remain reluctant to report adverse events. Data and systems to
review and share information are under development. However, developing acceptable
measures of adverse event rates, relevant measurements of claims frequency, and sever-
ity that can be compared within and across organizations remains an important chal-
lenge. Fundamentally, risk management programs are designed to prevent patient injury
from occurring in the first place. Measuring the impact of injuries prevented remains a
largely undeveloped area of research and program evaluation.

RISK MANAGEMENT PROGRAM OUTCOME MEASURES?

Consistent with the objectives of hospital risk management programs, there are a limited
set of agreed upon, scientifically valid outcome measures that are both timely and mean-
ingful in practice. As noted, the ASHRM Assessment Abstract provides a starting point for
hospitals seeking to develop an effective program. However, these measures are largely
subjective, relying on self-assessment using a crude scale. Other measures that are of inter-
est but difficult to develop and compute include analysis of adverse events, measurements
of claims risk and claims management practices, as well as measures of safety and quality.

Adverse Events
A primary objective of hospital risk management programs is to prevent and control
patient injuries. Historically, the methods utilized to detect incidents of injury include
incident reports and occurrence screens. Incident reports are limited in that providers
are traditionally unwilling to report information. Occurrence screens have been broad
and poorly specified, often providing more data than information. The studies conducted
in California and by the Harvard researchers in which criteria were utilized to screen
medical charts formed the basis of much of the early work on occurrence screens. Med-
ical chart review to collect data on injuries continues to warrant substantial criticism.
Currently, the shift to managed care and related strategies to shape and direct
provider decision making is creating a climate in which more treatments and services are
omitted. Increasingly, incident identification requires an understanding of the clinical
decision making process and tracking and analysis of services omitted, as well as acts
committed to detect adverse events. Standardized measures of clinical decision making
carr_ch35.qxd 11/21/00 9:41 AM Page 853

Risk Management Program Evaluation 853

processes are not sufficiently developed to promote interhospital or interorganizational

comparisons.

Claims Risk Assessment

Efforts to assess hospital and organizational risk of experiencing a medical malpractice
claim are poor and challenging. A key issue for liability insurers is to estimate the likeli-
hood that a provider will be exposed to a malpractice claim. Lack of consensus exists
among the professional community about when an adverse event becomes a claim and
what the organizational and clinical indicators are that suggest differences in risk among
institutions. Traditionally, types of services offered has provided a starting point. Hospitals
providing obstetrical, surgery, and emergency services are at greater risk than hospi-
tals that do not offer these services. The Harvard studies have provided some recent evi-
dence to suggest that certain organizational characteristics are associated with greater
risk of exposure to claims and lawsuits. Understanding organizational risk and the devel-
opment of measures to assess risk remains an important area of development. This infor-
mation is critical to ensure that comparisons among organizations regarding the impact
of risk management programs are relevant.

Claims Management
Another important objective of hospital risk management programs is to prevent and con-
trol losses associated with medical malpractice claims. Malpractice claims management
is itself a complex phenomenon with an intricate array of processes and players. Deci-
sions about how to address a claim once it is filed are critical for ensuring that funds are
managed efficiently and effectively. Traditional measures have focused on the frequency
and severity of malpractice claims—that is, the number of claims and the amount spent
on claims. Opportunities exist in defining more specific and standardized measures of
claims management practices particularly ones that incorporate a time dimension. For
example, early intervention to address an adverse event before it becomes a claim is crit-
ical. Once a claim is made, managing financial losses and reducing time to resolution of
a claim can lower expenses.

Safety and Quality

Since the 1980s, there has been an important and ongoing discussion and belief among
policy makers and managers that risk management should overlap with quality manage-
ment initiatives within organizations. Currently, the discussion of both risk and quality has
evolved into a discussion of patient safety.33 The recent report from the Institute of Med-
icine (IOM) on error in medicine provides a detailed discussion of the emerging litera-
ture on patient injury rates, and risk and quality approaches to improving patient safety.
In its current form, the discussion of safety should be treated as a core value within
organizations. For the first time in history, health care providers are attempting to define
and measure how this value is transmitted and managed within organizations. Efforts to
produce assessments of safety and safe cultures are currently underway. For example,
models of how complex systems fail, adapted from other industries, are helping to frame
our approach to addressing medical injury and improving patient safety.34 Outcome
measures of safety and quality should be developed and assessed for their relevance and
use in managing risk.
carr_ch35.qxd 11/21/00 9:41 AM Page 854

854 Monitoring and Evaluating

Evaluation Studies of Risk Management Program Effectiveness

Despite the difficulties in measuring outcomes of risk management programs, to date
there have been a handful of research studies that have attempted to relate health care risk
management activities in hospitals to improvements in malpractice claims experience. As
noted, these investigations only provide limited insight into the true impact of risk man-
agement programs, given the complex relationship that exists between injuries and claims.

MARYLAND HOSPITAL STUDY

One of the most influential studies of hospital risk management program effectiveness was
conducted by Morlock and Malitz.35 In this study, researchers examined relationships
between hospital risk management activity among forty acute care hospitals in Maryland
and their medical malpractice claims experience. Using data from a 1980 survey of hos-
pital risk management activity, the researchers described risk management program
components including policies for handling medical incidents and characteristics of edu-
cational programs offered on quality, safety, and risk. The medical malpractice claims
experience of hospitals was assessed based on closed claims data for incidents occurring
between 1980 and 1982. All claims utilized in the analysis were closed by 1987.
Malpractice claims were aggregated for each hospital. Indicators were constructed
to measure the total number of filed malpractice claims per hospital bed, the number of
filed claims per 100 hospital beds, the number of filed claims settled privately per 100
hospital beds, total dollars awarded by the court system per bed, and total dollars in pri-
vate settlements per bed. These claims were examined for both claims arising from all
hospital incidents and for those claims in which hospitals were named as defendants.
Claims were adjusted for hospital bed size and the volume of services provided by devel-
oping an index measure. Although crude, this measure was assumed to adjust for differ-
ences in hospital exposure to risk.
The analysis indicated that after adjusting for differences in risk, the malpractice
claims experience of hospitals with the following risk management activities in place was
significantly better:
1. A policy of notifying clinical chiefs of adverse medical incidents.
2. A policy of specifying who had responsibility for informing patients and families of
errors.
3. Governing board receipt of risk management reports on a regular basis.
4. Governing board oversight of risk management or quality assurance activities.
5. Education efforts concerning the responsibilities of physicians and nurses in quality
assurance and risk management.
The results of this academic research investigation of Maryland hospitals remain the
only empirical support for hospital risk management programs.36,37

CLINICAL INDICATOR RESEARCH

The most recent evidence of risk management effectiveness has come from the profes-
sional liability insurance provider community. It reflects ongoing work in the area of risk
management services research including design, development, implementation, and
carr_ch35.qxd 11/21/00 9:41 AM Page 855

Risk Management Program Evaluation 855

creation of clinical indicators of program performance. MMI Companies, Inc., recently

published a report examining the impact of its risk management approach on the mal-
practice claims experience of its clients.
A key element of the MMI approach to risk management is the development of clin-
ical indicator programs in high risk medical specialties. According to a recent report
from MMI in which twelve years of data are summarized and analyzed, there is strong evi-
dence to suggest that their clinical risk modification programs are helping to lower the
cost of malpractice claims. For example, in one of several analyses over time, MMI indi-
cates that hospitals in full compliance with their clinical guidelines in emergency services,
perinatal, and perioperative services have a significantly lower average cost per malprac-
tice claim than hospitals with less than full compliance. Further analysis of this data sug-
gested that full adherence to MMI guidelines in all three areas combined reduced the
average cost per claim by almost $70,000. Moreover, hospitals in this category had aver-
age claims costs of $2,834 in comparison to average claims costs of $72,767 among hos-
pitals with no compliance.

FUTURE OF RISK MANAGEMENT EFFECTIVENESS

STUDIES—EVALUATION IN A CHANGING ENVIRONMENT
Evaluating the effectiveness of health care risk management programs continues to pre-
sent a number of challenges. The marketplace continues to change. Integration and man-
aging care remain quixotic goals. In the profession of health care risk management, there
no consensus among the profession regarding the design and development of risk man-
agement programs. As noted, recent studies indicate there is wide variation in commit-
ment in the level of risk management activity among hospitals. In addition, defining
outcome measures that can be tied to specific risk management activities in a meaning-
ful way is difficult and a relatively new area of research and risk management program
administration. Further, much of our knowledge about what works in risk management
continues to come from expert opinion, descriptive research, and a very small number of
scientifically based studies of the effectiveness of risk management programs.
Although “better practices” in the profession are evident, it is clear that insufficient
attention has been directed toward the continued development of new tools that help
demonstrate value-added risk management. The ASHRM Assessment Abstract, for exam-
ple, has been available for nearly a decade. Its use by professionals seems limited. Eval-
uation of the tool has only recently been initiated, an activity that should be led by the
profession.
As market pressure continues to increase and risk managers are called to demon-
strate the value-added of their programs, an important issue for the profession is to con-
sider taking on the challenge of defining the next steps in risk management program
evaluation. Similar to the past, perhaps a first step toward this goal is to utilize the pro-
fessional association of ASHRM and its state-affiliated chapters to create an interdiscipli-
nary group of professionals including physicians, nurses, risk managers, and health
service researchers, among others, to begin to address these issues.
As this group comes together, they should start by recognizing and defining the
areas of overlap with quality and patient safety inherent in risk management. Although
well appreciated conceptually, risk management is not independent of activities to
improve quality of patient care or to enhance efforts to improve patient safety. In fact,
from a historical perspective, safety and risk have been linked as organizational program
carr_ch35.qxd 11/21/00 9:41 AM Page 856

856 Monitoring and Evaluating

activities since the early 1970s.38 As part of its effort to define the next direction for risk
management program development and measures of effectiveness, attention needs to be
directed toward developing operational definitions of differences and core sets of activ-
ities that are independent of safety and quality and unique to managing risk. Then, risk
managers can begin to lead the design and development of an evaluation strategy that
can integrate and manage across these core functions in the related areas of quality,
safety, and risk.
As noted, the history of risk management is currently tied in large part to success
stories based on faith and anecdote rather than quantitative assessments of program
performance. The future of risk management program success, however, will require that
the profession develop a strategy that begins with evidenced-based management prac-
tices. Again, risk managers themselves must respond to the call for action to share the
learning about defining the models, tools, and approaches to measuring the impact of
their activities.
To continue the dialogue and enhance professional practice in evaluating risk man-
agement program effectiveness, risk management professionals must also continue to uti-
lize and work in partnership with ASHRM and other professional risk organizations to
create and experiment with new models and tools to help demonstrate value-added risk
management. As the nation continues to struggle with issues surrounding patient injury,
patient safety, and improving quality, the call goes out to the profession to reactivate its
historical role as a leader role in patient injury prevention.

CONCLUSION
The purpose of this chapter has been to highlight some of the key issues that pertain to
the evaluation of health care risk management programs. Hospitals played a major role
in this discussion. Despite many efforts to create change in factors that contribute to inci-
dents of patient injury in hospitals, these institutions continue to be a place where many
of the most severe and disabling medical injuries occur.
To help risk managers begin to reframe and rethink approaches to developing hos-
pital risk management program assessment tools in the future, this chapter also pre-
sented some of the history of medical injury in hospitals and a conceptual framework for
defining the objectives of hospital risk management program interventions. To date, this
framework remains one of the most comprehensive approaches to describing the
multidimensional character of the problem of medical injury. Although the focus here has
been to identify the role and objectives of hospital risk management programs, it also
provides a powerful framework for areas of opportunity to think through strategies for
relating to quality improvement and emerging patient safety initiatives.
In addition, this chapter presented information and reviewed recent research on
current efforts to adopt a hospital risk management program assessment model based
on the recommendations of ASHRM. A key finding from the research, noted previously, is
that there is wide variation in institutional commitment to development hospital risk
assessment programs. Further research is needed to identify and understand the factors
that may help explain this observed variation.
Finally, this chapter discussed the future of health care risk management and issued
a call to the profession to continue to pursue a partnership strategy and to reactivate its
role as a leader in the patient injury prevention and patient safety movement.
carr_ch35.qxd 11/21/00 9:41 AM Page 857

Risk Management Program Evaluation 857

Endnotes
1. Morlock, L. L., and others. “Medical Liability and Clinical Risk Management.” Managing
Quality of Care in a Cost Focused Environment. Tampa, Fla.: American College of
Physician Executives, 1999.
2. HCFA Manual, 1978.
3. U.S. General Accounting Office. Health Care Initiatives in Hospital Risk Management.
Washington, D.C.: GAO/HRD-89-79, 1989.
4. Morlock, L. L., and others. “Medical Liability and Clinical Risk Management.” Managing
Quality of Care in a Cost Focused Environment. Tampa, Fla.: American College of
Physician Executives, 1999.
5. Nat’l Assoc. of Ins. Comm., 1980.
6. Morlock, L. L., and others. “Medical Liability and Clinical Risk Management.” Managing
Quality of Care in a Cost Focused Environment. Tampa, Fla.: American College of
Physician Executives, 1999.
7. Danzon, P. M. Medical Malpractice: Theory, Evidence, and Public Policy. Cambridge,
Mass.: Harvard University Press, 1985.
8. Brennan, T. A., and others. “Incidence of Adverse Events and Negligence in Hospitalized
Patients. Results of the Harvard Medical Practice Study-I.” New England Journal of Medi-
cine, 324(6), Feb. 7, 1991, pp. 370–376.
9. Weiler, P. C. “Toward No-Fault Compensation/Organizational Liability.” Medical Malprac-
tice On Trial. Cambridge, Mass.: Harvard University Press, 1990.
10. Thomas, E. J., Studdert, D. M., Newhouse, J. P., and others. “Costs of Medical Injuries in
Utah and Colorado.” Inquiry, 36, 1999, pp. 255–264.
11. Kohn, L. T., Corrigan, J. M., Donaldson, M. S. “To Err is Human: Building a Safer Health
System.” Institute of Medicine, Washington, D.C.: National Academy Press, 1999.
12. Danzon, P. M. Medical Malpractice: Theory, Evidence, and Public Policy. Cambridge,
Mass.: Harvard University Press, 1985.
13. Ibid.
14. Morlock, L. L., and others. “Medical Liability and Clinical Risk Management.” Managing
Quality of Care in a Cost Focused Environment. Tampa, Fla.: American College of
Physician Executives, 1999.
15. Mills, D. H (ed.). California Medical Association and California Hospital Association’s
Report on Medical Insurance Feasibility Study. Sacramento, Calif.: Sutter Publications,
1980.
16. Orlikoff, J. E., and Vanagunas, A. M. Malpractice Prevention and Liability Control for
Hospitals (2nd ed.). Chicago: American Hospital Association, 1988.
17. Morlock, L. L., and others. “Medical Liability and Clinical Risk Management.” Managing
Quality of Care in a Cost Focused Environment. Tampa, Fla.: American College of
Physician Executives, 1999.
18. Wade, R. D. Risk Management HPL: Hospital Professional Liability Primer (1st ed.).
Columbus: Ohio Hospital Insurance Company, 1983.
19. Monagle, J. F. Risk Management: A Guide for Health Care Professionals. Rockville, Md.:
Aspen Publications, 1985.
20. Troyer, G. T., Salman, S. L. (eds.). Handbook of Healthcare Risk Management.
Rockville, Md.: Aspen Systems Corp., 1986.
21. Orlikoff and Vanagunas, 1977.
carr_ch35.qxd 11/21/00 9:41 AM Page 858

858 Monitoring and Evaluating

22. Orlikoff, J. E., and Vanagunas, A. M. Malpractice Prevention and Liability Control for
Hospitals (2nd ed.). Chicago: American Hospital Association, 1988.
23. Harpster, L. M., and Veach, M. S. (eds.). Risk Management Handbook for Health Care
Facilities. American Society for Health Care Risk Management: American Hospital Associ-
ation, 1989.
24. Morlock and Malitz, 1991.
25. Morlock, L. L., and others. “Medical Liability and Clinical Risk Management.” Managing
Quality of Care in a Cost Focused Environment. Tampa, Fla.: American College of
Physician Executives, 1999.
26. American Society for Healthcare Risk Management (ASHRM). Hospital Risk
Management Self-Assessment Manual. Chicago: American Hospital Association, 1991.
27. Monagle, J. F. Risk Management: A Guide for Health Care Professionals. Rockville, Md.:
Aspen Publications, 1985.
28. Orlikoff, J. E., and Vanagunas, A. M. Malpractice Prevention and Liability Control for
Hospitals (2nd ed.). Chicago: American Hospital Association, 1988.
29. Wade, R. D. Risk Management HPL: Hospital Professional Liability Primer (1st ed.).
Columbus: Ohio Hospital Insurance Company, 1983.
30. Ziegenfuss, J. T., and Perlman, H. “Decreasing Medical Malpractice.” Health Care
Management Review, 14(4), 1989, pp. 67–75.
31. Morlock, L. L., Cassirer, C., and Malitz, F. E., “Hospital Risk Management and Professional
Liability Claims Experience in Maryland.” Final Report: Agency for Health Care Policy &
Research, Grant Number 1 RO1 HS06735, 1997.
32. Kohn, L. T., Corrigan, J. M., Donaldson, M. S. “To Err is Human: Building a Safer Health
System.” Institute of Medicine, Washington, D.C.: National Academy Press, 1999.
34. Ibid.
35. Cook, R. I. A Brief Look at the New Look in Error, Safety and Failure of Complex
Systems. Cognitive Technologies Laboratory. Chicago: University of Chicago, 1999.
36. Morlock and Malitz, 1991.
37. U.S. General Accounting Office. Health Care Initiatives in Hospital Risk Management.
Washington, D.C.: GAO/HRD-89-79, 1989.
38. U.S. General Accounting Office. Testimony-Medical Malpractice: Experience With
Efforts To Address Problems. Washington, D.C.: GAO/T-HRD-93-24, 1993.
39. Wade, R. D. Risk Management HPL: Hospital Professional Liability Primer (1st ed.).
Columbus: Ohio Hospital Insurance Company, 1983.

Suggested Readings
American Hospital Association. Medical Malpractice Task Force Report on Tort Reform and
Compendium of Professional Liability Early Warning Systems for Health Care Providers.
Chicago: American Hospital Association, 1986.
American Society for Healthcare Risk Management (ASHRM). Hospital Risk Management Self-
Assessment Manual. Chicago: American Hospital Association, 1991.
Brennan, T. A., and others. “Incidence of Adverse Events and Negligence in Hospitalized Patients.
Results of the Harvard Medical Practice Study-I.” New England Journal of Medicine, 324(6),
Feb. 7, 1991, pp. 370–376.
Cassirer, C. Hospital Risk Management Programs in Maryland (1995). Baltimore: Johns
Hopkins University, 1997.
carr_ch35.qxd 11/21/00 9:41 AM Page 859

Risk Management Program Evaluation 859

Cook, R. I. A Brief Look at the New Look in Error, Safety and Failure of Complex Systems.
Cognitive Technologies Laboratory. Chicago: University of Chicago, 1999.
Danzon, P. M. Medical Malpractice: Theory, Evidence, and Public Policy. Cambridge, Mass.:
Harvard University Press, 1985.
Harpster, L. M., and Veach, M. S. (eds.). Risk Management Handbook for Health Care Facilities.
American Society for Health Care Risk Management: American Hospital Association, 1989.
Institute of Medicine. Joint Commission on Accreditation of Healthcare Organizations ( JCAHO).
Accreditation Manual for Hospitals. Chicago, Ill., 1989.
Kohn, L. T., Corrigan, J. M., Donaldson, M. S. “To Err is Human: Building a Safer Health System.”
Institute of Medicine, Washington, D.C.: National Academy Press, 1999.
Mills, D. H (ed.). California Medical Association and California Hospital Association’s Report
on Medical Insurance Feasibility Study. Sacramento, Calif.: Sutter Publications, 1980.
Monagle, J. F. Risk Management: A Guide for Health Care Professionals. Rockville, Md.: Aspen
Publications, 1985.
Morlock, L. L., and Malitz, F. E. “Do Hospital Risk Management Programs Make A Difference?:
Relationships Between Risk Management Program Activities and Hospital Malpractice Claims
Experience.” Law and Contemporary Problems, 54(2), Nov. 1991, pp. 1–22.
Morlock, L. L., and others. “Medical Liability and Clinical Risk Management.” Managing Quality
of Care in a Cost Focused Environment. Tampa, Fla.: American College of Physician Executives,
1999.
Morlock, L. L., Cassirer, C., and Malitz, F. E., “Hospital Risk Management and Professional Liabil-
ity Claims Experience in Maryland.” Final Report: Agency for Health Care Policy & Research,
Grant Number 1 RO1 HS06735, 1997.
National Association of Insurance Commissioners. Malpractice Claims: Medical Malpractice
Closed Claims, 1975–1980. Milwaukee, Wis.: National Association of Insurance Commissioners,
1978.
Orlikoff, J. E., and Vanagunas, A. M. Malpractice Prevention and Liability Control for Hospitals
(2nd ed.). Chicago: American Hospital Association, 1988.
Smith, D. G., and Wheeler, J. R. C. “Strategies and Structures for Hospital Risk Management
Programs.” Health Care Management Review, 17(3), Summer 1992, pp. 9–17.
Thomas, E. J., Studdert, D. M., Newhouse, J. P., and others. “Costs of Medical Injuries in Utah and
Colorado.” Inquiry 36, 1999, pp. 255–264.
Troyer, G. T., Salman, S. L. (eds.). Handbook of Healthcare Risk Management. Rockville, Md.:
Aspen Systems Corp., 1986.
U.S. General Accounting Office. Medical Malpractice: Characteristics of Claims Closed in 1984.
Washington, D.C.: GAO/HRD-87-55, 1987.
U.S. General Accounting Office. Insurance: Profitability of the Medical Malpractice and General
Liability Lines. Washington, D.C.: GAO/GGD-87-67, 1987a.
U.S. General Accounting Office. Health Care Initiatives in Hospital Risk Management.
Washington, D.C.: GAO/HRD-89-79, 1989.
U.S. General Accounting Office. Testimony-Medical Malpractice: Experience With Efforts To
Address Problems. Washington, D.C.: GAO/T-HRD-93-24, 1993.
Wade, R. D. Risk Management HPL: Hospital Professional Liability Primer (1st ed.). Columbus:
Ohio Hospital Insurance Company, 1983.
Weiler, P. C. “Toward No-Fault Compensation/Organizational Liability.” Medical Malpractice On
Trial. Cambridge, Mass.: Harvard University Press, 1990.
Ziegenfuss, J. T., and Perlman, H. “Decreasing Medical Malpractice.” Health Care Management
Review, 14(4), 1989, pp. 67–75.
carr_ch35.qxd 11/21/00 9:41 AM Page 860