Embed Doc
Copy Link
Readcast
Collections

CommentGo Back

Download

Medical Decision Making: Evidence Based or Expert Opinion?Avery Tung, M.D. Chicago, Illinois

318Page 1

1. Introduction

“I don’t believe in that study”

Recent studies have established that physician compliance with the results of clinical studies, externally promulgated guidelines, and recommended practices, is poor. In the non-anesthesia domain, examples of noncompliance include below-optimal rates of peri-MI aspirin and beta-blocker use, cancer screening, diabetes and prenatal care. Moreover, studies correlating these observations with age have found that older physicians are muchless likely to conform to literature-based guidelines and practice advisories than younger ones. Although few clear guidelines for anesthesia practice exist, clinical anesthesia practice also diverges frequently from that suggested byclinical trials. Examples include use of aprotinin for cardiac surgery, the BIS monitor, pulse oximetry, ultrasound-guided central line placement, PA catheterization, administration of albumin, and perioperative beta blockade.The startling finding that physicians often do not follow their own literature has led to an increased focuson how physicians make medical decisions. Historically, this process has involved a complex (and poorly defined)integration of multiple sources: clinical experience, peer experience, case reports, mechanism-based research,randomized trials, and intuitive judgment. Recently, however, rankings of decision tools have been proposed thatincrease emphasis on randomized controlled trials, and deemphasize intuition, clinical experience, and physiologicreasoning. Rankings such as these form the basis for the evidence-based medicine movement.While this approach seems rational (who would willingly disagree with the “evidence”?), it presents achallenging dilemma for clinicians when the results of clinical trials or meta-analyses conflict with experienceand/or clinical judgment. In these situations, the clinician is often forced to choose between two seemingly opposedversions of reality: that supported by clinical experience, and that defined by clinical trials.Although theoretically improbable, conflicts between experience and evidence are discouragingly common.In the ICU, where advanced medical technology is applied in an extremely complex setting, many therapies have noevidence-based support, and several are used routinely despite well-known evidence to the contrary. ICU strategiesdeemed ineffective by the literature include transfusion to hematocrits>30 mg/dl, nitric oxide, albumin for volumeexpansion, pulse oximetry, augmented oxygen delivery for sepsis, pressure control ventilation, SIMV for weaningfrom mechanical ventilation, and prone positioning.But how can clinical experience and the results of randomized trials diverge so dramatically? If a therapywere effective, wouldn’t it rapidly promulgate throughout medical practice regardless of whether studies proved itsefficacy? And, if therapies were ineffective, wouldn’t they be eliminated rapidly by generations of physiciansfrustrated by the futility of meaningless care? Moreover, if a therapy has been validated by generations of physicians, can a randomized trial finding no benefit have meaning?Discord between rational models of decision-making and actual human decision behavior is not new, andhas been extensively researched in economic and other social science contexts. This talk will briefly review two prominent examples of disagreement between clinical experience and clinical trials, draw from recent work inEconomics and Psychology to compare human and analytical decision making strategies, discuss uniquecharacteristics of human decision-making when risk and uncertainty are involved, and identify potential strategies toimprove decision-making in ambiguous or uncertain environments.

2. Albumin and the PA catheter

“We’ve been doing this for decades!”

Two irreconcilable conflicts between evidence and experience in anesthesia practice are the use of albumin(vs crystalloid) for volume replacement and the Pulmonary artery catheter (PAC) for hemodynamic monitoring. In both, plausible benefits were suggested by extensive clinical experience and research on basic mechanisms that predated the arrival of clinical trials testing their utility. In both, prospective randomized clinical trials demonstratedno meaningful benefit. And finally, both therapies remain in widespread clinical use today. Table 2 depicts theresults of different information sources with respect to these devices:

318Page 2Table 2: Levels of evidence and findings for PA catheter and albumin useClinicalExperienceMechanistic Rationale ObservationalTrialsProspectiveRCTsAlbumin Useful

•

Stabilizes oncotic pressure

•

Free radical scavenger

•

Anti-thrombotic agentMixed No benefit(harm?)Pulmonaryarterycatheter Useful

•

Directly measures PAP

•

Indirectly measures LAP

•

Measures cardiac output No benefit(harm?) No benefit(harm?) Neither therapy originated as the result of clinical trials. Albumin was developed in a protein chemistrylaboratory, and first used clinically to resuscitate wartime trauma victims. The PAC was introduced by cardiologiststo monitor patients with myocardial infarction and compromised heart function. Both, however, disseminatedrapidly and widely into clinical practice, and enjoyed prolonged periods of clinical use before the appearance of clinical trials invalidating their utility. The existence of clearly articulated mechanistic roles for each reinforcedtheir clinical usefulness, but also enhanced the sense of disbelief at trials reporting a lack of benefit. It seemed (andseems) counterintuitive that a therapy with supportive mechanistic evidence that is perceived as useful by physicianscould have no effect or even cause harm.Today, despite studies explicitly refuting clinical experience, widespread media attention chastising physicians for being stubborn, ignorant, or failing to test new technologies, and calls for a moratorium on both PAC and albuminuse, many anesthesiologists and critical care physicians (and other physicians) continue to use both albumin and thePAC, at times applying both therapies to the same patient.

3. Can so many physicians all be wrong?

“All those studies are flawed”

The most recent ASA practice guidelines on PAC use explicitly acknowledge the gap between clinical experienceand literature trials:

“…these benefits have not been demonstrated in currently available research because of deficiencies in study designand performance. It is suggested that a properly designed, randomized controlled trial with adequate sample size,well trained physicians and nurses, well defined interventions, and meaningful outcome measures would reveal thebenefits observed in practice”

Since these guidelines were published in October, 2003, four subsequent studies ranging in size from 150to 2,000 patients have emerged, all essentially demonstrating the same result: that benefits from PAC use are smallor nonexistent, and that in some cases the PAC may actually worsen outcome.Reasonable explanations exist for why the PAC does not improve outcomes. These include the risks of invasive vascular access, physician ignorance and conservatism, quality of PA catheter data, and a lack of consensusregarding the appropriate response to PAC information. But none satisfactorily explain why many physicianscontinue to find it useful. The argument that physicians do not understand how to use the monitor that they feel theyneed is relatively weak. Why would physicians choose to use a monitor that they a) don’t understand and b) areunable to use effectively?

4. Do Humans Make Decisions Rationally?

“I can’t sell that stock…I’ll lose money!”

The 2002 Nobel Prize in Economics suggests a possible answer to this dilemma. Awarded in part to Dr. DanielKahneman, a Princeton psychologist, this prize celebrated the development of a conceptual framework for observing, testing, and predicting human decision making behavior under conditions of risk and irreducibleuncertainty. In particular, Kahneman found systematic, pervasive, and replicable differences between humandecision making strategies (particularly when risk and uncertainty are involved) and those predicted by rationalmodels. He proposed that instead of calculating probabilistically, humans approach problems differently, frequently

318Page 3using mental “shortcuts” (called heuristics) that simplify the process of assessing risk and uncertainty. Becausethese shortcuts led to “severe, systematic biases”, he argued that many economic models based on the “rational”human might not adequately represent reality.Many of Kahneman’s findings are intuitively understood by physicians. Humans are risk averse withrespect to gains, but risk seeking with respect to losses. Thus, therapeutic choices framed in terms of survival elicitdifferent responses than choices framed in terms of dying. Humans diagnose intuitively based on ease of recallmore than statistical likelihood. Thus, medical students are more likely to think of obscure diseases with narrow but prominent symptom profiles than of more common diseases with more protean manifestations. And, humans aremore likely to identify by “matching” to mental models than by statistical likelihood. Thus, pretest probability isoften ignored in assessing differential diagnostic possibilities. In these respects, physicians are well calibrated to biases generated by their intuition.But some of his other observations (potentially less well known by physicians) may be relevant to the PACcontroversy. Humans not only judge based on ease of recalled examples (the “availability” heuristic) but also ondegree of plausibility. This tendency leads to what Kahneman termed the conjunction fallacy, wherein a morespecific (and plausible) but less likely scenario is favored over a more general one. Consider the following twostatements, for example:

Mr. F has had one or more heart attacks2.

Mr. F is over 55 years old, and has had one or more heart attacks

Even though the second statement is probabilistically less likely, it seems intuitively more likely because of itsgreater plausibility. Kahneman demonstrated this effect in 1983, posing the following question to 66 Bostoninternists:

A 55 year old woman had a pulmonary embolism documented angiographically 10 days after a cholecystectomy. Please rank the following conditions in terms of the probability that they will be experienced by the patient (1= most likely, 6= least):1.

Dyspnea and hemiparesis2.

Calf pain3.

Chest pain4.

Syncope and tachycardia5.

Hemiparesis6.

Hemoptysis

When this experiment was performed, the average rating for condition 1 was 2.7 and for condition 5 was 4.6, eventhough condition 5 is statistically more likely.It is easy to see how PA catheter data can reinforce such wayward intuition. Which might be more likely in theabove 55 year old woman?

Hemiparesis2.

A sudden rise in PA pressures to 60/30 mmHg, a patent foramen ovale, and hemiparesis

In this circumstance, one can posit that PAC placement simultaneously attributes causality to the hemiparesis,reinforces the diagnosis, and distorts the assessment of likely causes. Another heuristic, termed“representativeness”, may also alter assessments of PAC utility. Kahneman found that humans are much morelikely to use “goodness of fit” than probability when making a diagnosis. Thus, when confronted with thedescription:

Steve is very shy and withdrawn, invariably helpful but with little interest in people or in the world of reality. Ameek and tidy soul, he has a need for order and structure, and a passion for detail.

most would consider Steve a more likely librarian than salesman, even though there are many more salesmen thanlibrarians in the world. In this case, similarity to the mental model of a librarian intuitively induces a disregard for