You are on page 1of 7

Volume I • Number) • 1998

VALUE IN HEALTH

Economic Modeling and Sensitivity Analysis

Joel W. Hay, PhD


Department of Pharmaceutical Economics and Policy, University of Southern California, Los Angeles, CA

ABSTRACT _

The field of pharmacoeconomics (PE) faces serious con- model. Software distribution would allow other users to
cerns of research credibility and bias. The failure of re- validate the assumptions and calculations of a particular
searchers to reproduce similar results in similar settings, model and apply it to their own circumstances. Multivari-
the inappropriate use of clinical data in economic models, ate sensitivity analysis can also be used to present results
the lack of transparency, and the inability of readers to in a consistent and meaningful way that will facilitate
make meaningful comparisons across published studies comparisons across the PE literature. Using these meth-
have greatly contributed to skepticism about the validity, ods, broader acceptance and application of PE results by
reliability, and relevance of these studies to healthcare policy-makers would become possible. To reduce the un-
decision-makers. Using a case study in the field of lipid certainty about what is being accomplished with PE
PE, two suggestions are presented for generally applica- studies, it is recommended that these guidelines become
ble reporting standards that will improve the credibility requirements of both scientific journals and healthcare
of PE. Health economists and researchers should be ex- plan decision-makers. The standardization of economic
pected to provide either the software used to create their modeling in this manner will increase the acceptability of
PE model or a multivariate sensitivity analysis of their PE pharmacoeconomics as a practical, real-world science.

S everal challenges are threatenin.g the credibility


of the field of pharmacoeconomics (PE). Re-
sults are rarely reproducible from one study to an-
from one study to another, and valid comparisons
across published studies are almost impossible.
Variability in the different PE analyses performed
other, and comparisons between different studies may have several causes, including:
are often difficult. A lack of coherence between
• Invalid model assumptions or projections-for
the results of different economic analyses has re-
example, researchers can inadvertently misuse
sulted in general skepticism about what is being
economic models by substituting the price of
accomplished with PE studies. But there are some
one drug while assuming the effectiveness of an-
simple ways to narrow both the level of uncer-
other, or healthcare costs from one country to
tainty and the discrepancies between the results
an analysis in another country;
obtained with different economic models. A case
• Failure to provide sufficient information about
study in lipid PE is described, which provides a
modeling equations and parameters so that a
concrete example of the proposed enhancements
reader or user can meaningfully evaluate and re-
in PE model presentation.
produce the analyses;
• Inadequate sensitivity analysis-while most pub-
Current Challenges in the Field lished analyses include some type of sensitivity
of Pharmacoeconomics analysis, it is usually restricted to varying drug
price levels and a few other parameters.
Although economic modeling is essential to deter-
mine the cost-effectiveness of alternative therapies, When dealing with a complex nonlinear PE
such as those for lipid-lowering, methodological model, however, single parameter variation of re-
weaknesses are apparent in many published stud- sults is inadequate: it does not allow determina-
ies. Results of PE analyses are rarely reproducible tion of how results change when several different
parameters are altered simultaneously. Since pub-
lished PE studies will often have baseline results
Address correspondence to: Dr. Joel W. Hay, University of
Southern California, Department of Pharmaceutical Eco- with differences in parameter values, univariate
nomics and Policy, School of Pharmacy, CRP 140-J, 1540 sensitivity analysis will usually be insufficient for
East Alcazar St., Los Angeles, CA 90033. comparison of results across studies. This variabil-

© 15POR 1098-3015/98/$10.50/187 187-193 187


188 Hay

ity in results causes healthcare decision-makers to nomic analyses, there are a few problems with this
be skeptical of PE presentations. It seems to be a approach. Most PE models have some proprietary
widespread belief that any desired result can be value. Researchers and research sponsors would
obtained simply by altering some aspect of an eco- have to balance that against the scientific commu-
nomic model. Also, drug formulary decision-mak- nity's need-to-know. In addition, not all models are
ers often look at some of the key input parameters user-friendly. Frequently, these models are very
included in a particular PE analysis and assume complex and are not developed in ways that are
that, if they are different from their own experi- easy to fathom. Finally, users can take economic
ence, the analysis does not apply to their own cir- models out of context or use them inappropriately.
cumstances. Therefore, the study is often believed For instance, a model that was developed for one
to be irrelevant for their purposes. drug or population could be used to make inappro-
The credibility of pharmacoeconomics lies in de- priate projections to another drug or population.
veloping generally applicable standards of analysis
and presentation. If PE is to be a real-world science,
it must provide information that can be utilized by Multivariate Sensitivity Analysis
different users under different circumstances. There Another approach to model transparency that is
are some straightforward ways to narrow the level easier and more practical than distributing software
of uncertainty and lack of coherence between the models on the Internet is better use of PE model
results of different models. The key lies in the multivariate sensitivity analysis (MSA). MSA can
proper presentation of PE modeling results. Equa- be used to present results in a consistent and mean-
tions and parameters of an economic model need to ingful way to all users, including those who want to
be made more explicit and transparent. Presenta- compare different published studies. In addition,
tion of the model results also needs to adequately multivariate sensitivity analysis may allow a broader
show variations with changes in input parameters. applicability of results by policy-makers when mak-
These variations must be demonstrated in a much ing real-world decisions concerning which drug ther-
more comprehensive and systematic way than has apy is appropriate in each particular context, with
been usually done. different sets of patient and practice characteristics.
To accomplish this, the MSA should run a se-
ries of analyses of the PE model with different si-
Standardization of Economic Modeling:
multaneous perturbations along all the plausible
A Solution to the Problem
ranges of the model parameter space (drug prices,
Software Distribution ofPharmacoeconomic Models disease risks, survival, quality-of-life weights, dis-
One way to present an economic model more ex- count rate, healthcare utilization and costs, treat-
plicitly is to provide access to the software of the ment efficacy and safety parameters, etc.). The re-
PE models themselves. When researchers design a sults of these analyses would be (possibly) thousands
PE study, the underlying software model could be of sets of PE model results (e.g., cost-per-life-year-
provided via the Internet, so that others can di- saved estimates at different parameter values) and
rectly verify and validate the assumptions and specified parameter value choices. These model find-
equations of the model. Using one's own preferred ings can be easily summarized in a simple multivari-
parameter values (e.g., local costs, utilization rates, ate regression format. This would allow any user to
drug prices, etc.), it could be determined whether show how the changes in each of the important pa-
the model applies to one's own circumstances. rameters in a given model systematically affect the
These types of software models could be made reported results, such as cost per quality-adjusted
available to anyone who requests them after read- life-year (QALY) or cost per life-year saved. Cost
ing a published PE study. In addition, Internet soft- per QALY would be used as the regression equation
ware distribution might be one of the requirements dependent variable, and all variable model input pa-
of scientific journals prior to the publication of any rameters as the independent measures. Choices of
PE study. model parameter values could be either systematic
or random from the bounded parameter space. If
Limitations ofSoftware Distribution parameter values are chosen randomly and simulta-
Although software distribution of PE models would neously, a joint multivariate distribution of these pa-
greatly contribute to the diminution of skepticism rameters must be specified in advance. A case study
about the validity of these analyses and discrepan- in lipid PE, described below, demonstrates how this
cies between the results obtained with different eco- approach can be applied.
Economic Modeling 189

Case Study: A US Pharmacoeconomic large northeastern health plan on the 5-year cost of
Perspective on the Pravastatin West of treating a primary event of coronary heart disease.
Scotland Coronary Prevention Study These direct medical costs averaged about $50,000
over 5 years.
The West of Scotland Coronary Prevention Study
When a net cost analysis of pravastatin treatment
(WOSCOPS) [1] was a primary prevention trial
is performed, the medical care savings due to
designed to determine whether the administration
avoided CVD events per 1000 patients treated with
of the cholesterol-lowering drug, pravastatin, over
the drug works out to $1 million. Taking the cur-
a 5-year period would reduce the risk of nonfatal
rent wholesale list price of the drug into account,
myocardial infarction and death from coronary
minus a 13% discount, which is what is most rele-
heart disease. Scottish men between the ages of 45
vant to managed care, the net cost of the treatment
and 64, with hypercholesterolemia and no history
per day is about $1.49, or about $540 per year [3].
of myocardial infarction, were assigned to treat-
To determine the cost-effectiveness of prava-
ment with either pravastatin or placebo. There
statin treatment, we again used the WOSCOPS trial
was a similar distribution of other risk factors in
results, which, as described above, showed a signif-
both groups, but the primary inclusion criterion
icant risk reduction in coronary events with lipid-
was high levels of cholesteroL
lowering therapy. We evaluated the cost per life-
Results from the WOSCOPS trial demonstrated
year saved over the clinical trial period of 5 years.
that men treated with pravastatin showed a 20%
Using a managed care perspective, we also evalu-
reduction in blood cholesterol levels and a 26% re-
ated the changes in direct medical costs, survival,
duction in the levels of low-density lipoprotein
and employee productivity. Over the 5-year clinical
(LDL) cholesterol, while placebo produced no ef-
trial period of the WOSCOPS study, coronary
fect. Pravastarin lowered the risk of coronary events
events were observed in patients of both the prava-
(nonfatal myocardial infarction or death from cor-
statin and placebo groups. Using the WOSCOPS
onary heart disease) by 31 %; the occurrence of cor-
baseline average risk levels, both probabilities and
onary events was significantly lower in the treat-
costs were attached to each of the subsequent events
ment group relative to controls. Lipid-lowering
that occurred over the 5-year period. The economic
therapy also produced a 32% reduction in the risk
parameters used in our analysis were evaluated as:
of death from all cardiovascular causes and low-
1) an annual discount rate for costs and benefits at
ered the risk of death from any cause by 22%. Fi-
baseline of 3%; 2) the lag before any therapeutic
nally, the number of revascularization procedures
benefit was achieved; 3) therapy compliance of 78 %
was significantly reduced in men treated with
taken from the clinical WOSCOPS trial; 4) total
pravasrarin relative to the placebo group.
daily drug cost of $2.73 per day including drug initi-
ation and monitoring; and 5) risk reduction of
Economic Analysis: Application ofthe 21.8 % for initial CVD events.
Pharmacoeconomic Model to WOSCOPS Data The results of using these values in our economic
The economic analysis looked at direct treatment model are shown in Table 1. The number of life-
costs, including those associated with: 1) the drug years saved, net drug costs per day, and cost per
itself; 2) medication initiation and monitoring; 3) QALY saved were evaluated under different scenar-
treating therapeutic adverse events, which in the ios. Using the WOSCOPS baseline average risk
WOSCOPS trial were essentially zero; and 4) re- equations, the cost per QALY saved was estimated
ductions in treatment for coronary disease, or at $17,803. When men between the ages of 45 and
events avoided with pravastatin use. 64 meeting the current National Cholesterol Edu-
To project the analysis of the WOSCOPS clini- cation Program (NCEP) primary prevention guide-
cal trial into a United States (US) context, we fo- lines were considered in our economic model, the
cused on the initial events that were avoided in the cost per QALY saved was $17,585, which, interest-
Scottish trial, including strokes, angina, myocar- ingly, is very similar to the WOSCOPS result. If
dial infarctions, and death from cardiovascular employee productivity gains or screening and com-
disease (CVD), using US data from the Framingham pliance costs were included in the analysis, the pic-
estimates of survival [2]. A 20% additional reduc- ture changed. As can be seen in Table 1, if employee
tion in mortality beyond Framingham estimates was productivity gains were included in the analysis, the
used to account for some of the new therapies that cost per QALY saved became meaningless, because
have been added since their last estimates of post- the net daily cost of treatment was - $0.31 (imply-
event survival. In addition, data was taken from a ing a simultaneous savings in cost and QALYs).
190 Hay

Table I Cost-effectiveness of pravastatin in primary prevention of coronary heart disease: effect of varying input parameters
WOSCOPS baseline NCEP primary Employee productivity Screening and compliance
average risk* prevention population gains included costs included
Life-years saved O.IS 0.15 0.15 0.15
Pravastatin cost (net per day) $1.49 $1.50 -$0.31 $2.21
Cost per QALY saved ($US) $17,803 $17.585 nfa $26.279

*Data from [I].


n/a. not applicable due to negative net per day pravastatin cost; NCEP. National Cholesterol Education Program; QALY. quality-adjusted life-year: WOSCOPS.
West of Scotland Coronary Prevention Study.

On the other hand, with the inclusion of screening abetes, high density lipoprotein (HDL) cholesterol,
and compliance costs, the cost per QALY saved smoking, total serum cholesterol, annual discount
went up to $26,279. Therefore, different cost esti- rate, total daily medication cost, angina cost, CVD
mates can be obtained depending on which param- death cost, and myocardial infarction and stroke
eters are included in the analysis. The bottom-line costs.
numbers do not tell the full story; taking anyone of The results of this multivariate cost-effectiveness
the numbers by themselves is not very meaning- regression are shown in Table 2. The regression in-
ful. Results depend critically on where one puts the dicates that each year of age reduces the cost per
model parameters. Unfortunately, many people tend QALY by $553. Each point of diastolic blood pres-
to focus on these bottom-line numbers or best-case sure reduces the cost-per QALY by $422. If the
estimates that appear in published abstracts of par- patient has diabetes, that reduces the cost per life-
ticular PE studies. year saved by over $10,000. Each point of HDL
A number of univariate perturbations to the pa- cholesterol raises the cost per life-year saved by
rameters included in the model were performed. It about $400. Smoking reduces the cost per life-year
was found that regardless of how parameters were saved by $10,571. Each point of total serum cho-
altered in the analysis-whether they were screen- lesterol reduces it by about $60. For the annual
ing and compliance costs, very high-risk patients, discount rate, whether 3%, 4%, or 5% is chosen,
the average risk levels of the WOSCOPS or the cur- each additional percentage point adds $6031 to
rent NCEP guidelines-for ages between 45 and the cost per life-year saved. Each additional dollar
64, the cost per life-year saved with pravastatin of daily medication cost adds about $10,000 to
therapy was consistently less that $50,000, a value the cost per life-year saved. And, finally, each dol-
that is often considered to be the cutoff point for lar in increased medical cost offsets for events
cost-effective medical care. such as angina and myocardial infarction reduces
the cost per QALY by about 7 cents.
Multivariate Sensitivity Analysis Regression The baseline cost for a myocardial infarction
Cost-effectiveness analysis of pravastatin therapy treatment over 5 years is about $45,000. If a reader
can also be performed with several different simul- at a managed care plan thought that at his location
taneous perturbations of the model along the dif- the MI treatment cost was $80,000, then, using
ferent values of parameter space. To summarize this MSA cost-effectiveness regression, the cost
the results of the analysis in a meaningful way for per life-year saved would decline by about 30%.
readers and users, multivariate sensitivity analysis Thus, it is quite possible to input numbers from a
(MSA) regression was performed. The findings of different part of the country, or even from other
this MSA were presented in a simple multivariate countries, to project what the cost per life-year
regression format to allow demonstration of how saved would be with pravastatin in that particular
changes in each of the important parameters in the setting.
model would impact the results, taking into ac- This regression was validated by repeating the
count the simultaneous effect of all other model entire MSA regression in very different regions of
parameters. Cost per QALY saved was used as the the parameter space. One important feature of this
dependent variable in the context of this multivari- MSA regression is that it does not require that the
ate regression. A very good fit in terms of explain- values be very close to the means that were used
ing the cost per QALY as a function of several vari- from the WOSCOPS study. Even using widely dif-
ables was obtained (r 1 = 0.94). The independent ferent parameter values, robust estimates of the re-
variables included age, diastolic blood pressure, di- gression coefficients can be obtained. Hence, this
Economic Modeling 191

Table 2 Multivariate cost-effectiveness regression (correlation coefficient = 0.94)


Model parameter* Cost per QALY saved ($US) Pvalue
(Constant) 70,711.00 0.00
Age (per year increase) -953.00 0.00
Diastolic blood pressure (per point increase) -422.00 0.00
Diabetes -10,907.00 0.00
HDL cholesterol (per point increase) 383.00 0.00
Smoker -10,780.00 0.00
Total serum cholesterol (per point increase) -56.00 0.00
Annual discount rate (per percentage point increase) 6031.00 0.00
Total daily medication cost (per dollar increase) 10,571.00 0.00
Angina cost (per dollar cost offset) -0.Q7 0.00
CVD death cost (per dollar cost offset) -0.03 0.22
Myocardial infarction (per dollar cost offset) -0.07 0.00
Stroke cost (per dollar cost offset) -0.Q3 0.15
R' for regression = 0.94. While standard errors and statistical tests of significance are reported to allow confidence interval estimates, they are not fully validsince
the underlying modeling data in the regression are nonstochastic.
*Baselineparameter values: Age = 55.1 years; diastolic blood pressure (DBP) = B4.0mm Hg; diabetes = 1%; high density lipoprotein (HDL) 44.0 mm/dL; cur-
rent smoker = 44%;total serum cholesterol (TSC) 272.0 mm/dL; discount rate = 3%; dailydrug costs = $2.73.
CVD. cardiovascular disease; HDl, high density lipoprotein; QAlY, quality-adjustedlife-year.

MSA in a simple regression format tends to work PE model cost-effectiveness chart and determine
quite well. whether treatment would be cost effective (Fig. 1).
In presentation and use of these MSA regres- The Framingham CVD risk estimates were used
sion results, it is important to keep in mind that to project the baseline risk of coronary disease. The
the underlying regression coefficients are not esti- placebo risk group in the WOSCOPS trial fit very
mated from surveyor clinical trial data, but rather nicely with the Framingham risk equations [4,5].
from a PE model. In that sense the data used to esti- What the model shows, and what the clinical trial
mate the regression come from a model that was put data support, is that it does not matter hoio a pa-
together artificially; they are completely nonstochas- tient attains a particular level of CVD risk, at least
tic. Traditional statistical measures of quality of fit within the ranges of variation in the WOSCOPS
and confidence intervals for coefficient estimates are trial. Whether it is due to a combination of smok-
still informative, because they indicate the precision ing, diabetes, high LDL cholesterol levels, or other
of the MSA regression and allow the user to evalu- factors is not relevant. As long as a patient's 5-year
ate the accuracy of prediction. Nevertheless, these CVD risk can be projected, it can be plotted against
statistics do not meet classical requirements for sta- what the cost-effectiveness model demonstrates, For
tistical analysis. example, using this model, for people at the mean
5-year risk levels of the WOSCOPS trial (14%), the

Cost per Life-Year Saved with Lipid Therapy as a


Function of5-Year CVD Risk
Summarizing the model findings in a slightly dif- Cost per Quality-Adjusted Life Year Saved
ferent way, the model can be used to estimate the .
- ... .
S6O,000

cost per life-year saved with lipid therapy as a S55,000


S50 ,000
function of an individual's CVD risk over 5 years. --
.....
S45,OOO
S40,000
This may be quite useful for cost-effectiveness $35,000
•••-.'¢~-~
analyses, particularly for clinicians who want to SlO,OOO

consider economics as well as clinical responsibil-


$25, 000
-...:' ~
~'"".'"
, ."...
$20,000 II; . ..
ity. Any clinician or healthcare policy-maker can S\ 5,0 00
sio.oeo
easily obtain estimates of CVD risk from the S5,ooO
SO
Framingham study or from the American Heart 4% ' " 6% 7% 8% 9% 10% 11% 12% 13% I"" 1$% 16% 17% 18% 19% 20%
Association's Internet website (hrtp.z/www.arneri- 5·¥ear CVD Risk
canheart.org). Clinicians can enter patients' risk
factors and determine what the 5-year CVD risk is Figure I Cost-effectiveness and 5-year baseline cardiovas-
for different patients. They can then refer to the US pravastatin primary prevention projection.
cular risk:
192 Hay

cost per life-year saved is between $15,000 and these analyses, this model allows other users to
$20,000. use different values of the input parameters and
As the risk of CVD increases, the cost per life-year reliably apply it to their own circumstances.
saved is reduced, or, in other words, pravastatin
therapy becomes more cost-effective. The cost per
life-year saved for patients with a 20% risk of CVD Conclusions
is between $10,000 and $15,000; it increases to
The credibility of the field of pharmacoeconomics
between about $20,000 and $25,000 with a 13%
lies in developing generally applicable standards
risk and to approximately $56,000 with a 5% risk.
for analysis and presentation of results. Pharma-
Hence, using a $50,000 cost-effectiveness cutoff
coeconomic researchers should be expected to
point, treatment would be cost-effective when the
provide either the software or a comprehensive
patient faces a somewhat greater than 1% CVD risk
multivariate sensitivity analysis of their economic
per year. It all depends on what the cutoff point is or
model. This should be a requirement of both jour-
how much health plan-makers, consumer groups,
nals and healthcare plans. The standardization of
and employers are willing to pay for cost-effective
economic modeling will put our profession on a
medical treatment. If there is a willingness to pay for
more scientific basis, eliminate many uncertainties
proven cost-effective therapies that have a cost per
about PE studies, and allow comparisons across
QALY below $50,000, then pravastatin treatment
published studies. Multivariate sensitivity analy-
of patients with as-year CVD risk as low as 6%
sis should be included in all lipid PE studies and
should be an option.
other PE studies as well. This analysis will greatly
This model supports the view that it is cost-
assist users in translating model results to their lo-
effective to intervene at much lower levels of CVD
cal settings. It will allow meaningful comparisons
risk than what is typically thought of as in the ap-
of different models of the same disease or treat-
propriate range for treatment, based on current
ment. This kind of analysis will indicate to all us-
US NCEP guidelines, European lipid guidelines,
ers where problems with the data or modeling
and existing lipid PE published studies.
may lie. It will show where additional research is
Extensions o(the WOSCOPS Trial needed to obtain adequate precision for crucial PE
model parameters so that future modeling efforts
Extensions of the WOSCOPS study beyond the clin-
can usefully inform healthcare policy.
ical trial period were also done using this model.
Preliminary results for men demonstrate that the
lifetime cost per QALY is much better than either The assistance of the WOSCOPS Economic Study Group
the 5-, 10-, or 15-year treatment horizons. The and Yong Yuan, Pablo LaPuerta, and Daniel Pettitt in the
development of the underlying WOSCOPS Pravastatin
5-year treatment horizon is the most rigorous be-
PE Model is gratefully acknowledged. Development of
cause it uses what is known from the clinical trial the model was supported by a grant from Bristol-Myers
and goes no further. The lifetime model assumes Squibb. This article was prepared with the assistance of
that relative risk reduction persists as long as the BioMedCom Consultants inc., Montreal, Canada.
patients stay on pravastatin therapy. As age in-
creases to about 65-70 years, however, the cost per
QALY saved is not very different for the different
time horizons. References
Taking it a step further, results were projected Shepherd J, Cobbe SM, Ford I, et al, Prevention of
for women, assuming that the CVD risk reduction coronary heart disease with pravastatin in men with
for women on pravastatin who met the average hypercholesterolemia. West of Scotland Coronary
risk profile of the WOSCOPS clinical trial would Prevention Study Group. N Engl ] Med 1995;333:
be similar to that for men. Such a projection is 1301-7.
supported by the results from the CARE [6] and 2 Kannel W, Wolf P, Garrison R. The Framingham
Study, Section 35: Survival Following Initial Car-
LIPID [7] trials of pravastatin, where the reduc-
diovascular Events. Bethesda, MD: National Heart,
tion in CVD risk among treated women was at
Lung and Blood Institute, 1988.
least as great as that for men. Projections were 3 Hay J, Lapuerta P, Yuan Y, et al. Pravastatin cost-
made beyond the clinical trial period. The values effectiveness for primary prevention of cardiovascu-
derived from the analysis were favorable and quite lar disease in the United States. Abstract 1017.
similar to those obtained for men. But more im- American Heart Association 70th Scientific Session,
portantly than the actual values derived from Orlando, Florida. November 1997.
Economic Modeling 193

4 West of Scotland Coronary Prevention Study 6 Sacks FM, Pfeffer MA, Moye LA, et al. The effect
Group. Baseline risk factors and their association of pravastatin on coronary events after myocardial
with outcome in the West of Scotland Coronary infarction in patients with average cholesterol lev-
Prevention Study. Am J CardioI1997;79:756-62. els. N Engl J Med 1996;335:1001-9.
5 West of Scotland Coronary Prevention Study 7 Tonkin A (for the LIPID Study Group). Long-term
Group. Influence of pravastatin and plasma lipids intervention with pravastatin in ischaemic disease
on clinical events in the West of Scotland Coronary (LIPID). Late-breaking clinical trials plenary ses-
Prevention Study (WOSCOPS). Circulation 1998; sion. American Heart Association 70th Scientific
97:1440-5. Session, Orlando, Florida. November 1997.

You might also like