ORIGINAL RESEARCH ARTICLE PhormacoEconomics 1997 Jan: II (I): 71>-88

117[}-7tnoJQ7 J(DJHXJ75/S07.00JQ

© Adis Internationollimited. All rights resetved.

Sensitivity Analysis in Health Economic

and Pharmacoeconomic Studies
An Appraisal of the Literature
Karen E. Agro,l Carole A. Bradley,2 Nicole Mittmann,3 Michael Iskedjian,4 A. Lane
Ilersich 5•6 and Thomas R. Einarson 7
1 McMaster University, Hamilton, Ontario, Canada
2 Glaxo Wellcome Canada Inc., Mississauga, Ontario, Canada
3 Department of Pharmacology, University of Toronto, Toronto, Ontario, Canada
4 Faculty of Pharmacy, University of Toronto, Toronto, Ontario. Canada
5 Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
6 Canadian Coordinating Office for Health Technology Assessment, Ottawa, Ontario, Canada
7 Faculty of Pharmacy and Department of Clinical Pharmacology, Faculty of Medicine,
University of Toronto, Toronto, Ontario, Canada

Summary The objective of this study was to analyse the extent of reporting of sensitivity
analyses in the health economics. medical and pharmacy literature between jour-
nal types and over time.
90 articles were chosen from each of the bodies of literature on health eco-
nomics. medicine and pharmacy. MEDLINE. EMBASE and International Phar-
maceutical Abstracts were searched for English-language economic studies
published between 1989 and 1993.
The studies chosen for inclusion had to be original articles published in one
of the selected journals between January 1989 and December 1993, involving a
comparison between drugs, treatments or services. and evaluating both costs and
outcomes. 123 articles initially met these criteria; however. 16 were inappropri-
ate. 17 were randomised out. leaving 90 studies (73%) that were used (30 from
each literature group). Data were extracted independently by 5 raters using a
validated checklist. Inter-rater reliability was assessed by calculating K.
53 of the 90 articles (59%) conducted sensitivity analyses. 39 (74o/c) stated
explicitly that a sensitivity analysis was being performed; this was noted in the
Methods section of 35 papers (67%).80% of health economics journals. 70% of
medical journals and 20% of pharmacy journals conducted sensitivity analyses.
Despite the fact that all published pharmacoeconomic guidelines suggest the
use of sensitivity analysis. only 59% of studies between 1989 and 1993 did so.
Improvement is required. especially in the pharmacy literature. No time trends
in the conduct of sensitivity analyses were detected. However. the sample may
not have been sufficient to detect such trends. Pharmacoeconomic guidelines
should provide more details on preferred methods of sensitivity analysis and on
desired parameters.
76
A sensitivity analysis is an integral component
of any sound pharmacoeconomic study.[l) It is a
'process through which the robustness of an eco-
nomic model is assessed by examining the changes
in results of the analysis when key variables are
varied over a specified range'. (2)
Sensitivity analysis represents a method of test-
ing the validity of statements made from results
that were based on certain assumptions in the phar-
macoeconomic study. It is concerned with explor-
ing the consequences of the uncertainty in both the
parameter estimations and assumptions made in the
pharmacoeconomic model. Sensitivity analyses
are advocated in pharmacoeconomic research be-
cause a number of assumptions and uncertainties
are incorporated into the analysis, and thus the in-
tegrity of the results may be compromised if those
assumptions prove to be inaccurate.(3)
Recent pharmacoeconomic guidelines have
consistently recommended the use of sensitivity
analysis.l4- l2 ) However, those guidelines have not
specified which method or approach should be em-
ployed, nor have they indicated the parameters of
the sensitivity analysis.
Recently, Briggs et aI.(13) reviewed the areas of
uncertainty found in an economic study, the poten-
tial statistical analysis required to address these
areas of uncertainty, and the appropriateness of dif-
ferent types of sensitivity analysis methods.l 13 )
These areas of uncertainty relate to 4 general com-
ponents, namely:
• the sample data
• the generalis ability of the results
• the extrapolation of the results
• uncertainty in the analytical methods.
In addition, Briggs and colleagues(13) reviewed
4 common sensitivity analysis methods: simple,
threshold, analysis of extremes and probabilistic
(the Monte Carlo simulation). However, the field
of pharmacoeconomics is evolving. Since that re-
view, at least 2 new methods of conducting sensi-
tivity analysis have been published.[3.l4)
The objective of this study was to analyse the
extent of the reporting of sensitivity analyses in the
pharmacoeconomic and health economic literature
to Adls International Umlted. All rights reserved.
Agro e/ al.
between journal types and over time. The develop-
ment of a checklist was required to perform this
analysis. The purpose of the checklist was to iden-
tify whether a sensitivity analysis had been con-
ducted, to report on the type of analyses, amI io
document the parameters of the sensitivity analy-
ses. The checklist was then applied to a selected
sample of articles, and descriptive statistics were
compiled comparing results between journal types
and over time.
This paper extends the recent work of Mason
and Drummond[IS) and Briggs and Sculpher[16 1 by
examining the conduct of sensitivity analysis be-
tween 3 journal types (health economic, medicine,
pharmacy) over a pre-specified time period. This
paper also adds to the research of Briggs et al. I131
by reviewing 2 new methods for the conduct of
sensitivity analysis, pharmacoeconomic guideline
recommendations on the necessity of sensitivity
analysis, and the advantages and disadvantages of
each sensitivity analysis method.
Methods
We used a 2-stage approach to examine the phar-
macoeconomic and health economic literature.
First, we developed a checklist as our data collec-
tion instrument. To develop and validate the instru-
ment, issues pertaining to pharmacoeconomic
studies were identified from the literature. Phar-
macoeconomic guidelines were retrieved and re-
viewed to ascertain their requirements with respect
to sensitivity analysis. For the purposes of this re-
search, pharmacoeconomic guidelines were de-
fined as documents that outlined the essential ele-
ments of a pharmacoeconomic or health economic
evaluation. We accepted published guidelines that
originated from regulatory agencies, or from indi-
vidual authors or groups of authors.
Development of Checklist
Pharmacoeconomic guidelines from both regu-
latory agencies and individual authors are summa-
rised in table I. While all guidelines that we reviewed
recommended performing a sensitivity analysis,
there was a lack of consistency in identifying both
PharmocoEconomlcs 1997 Jon; 11 (1)
Sensitivity Analysis 77

Table I. Summary of the pharmacoeconomic guideline recommendations regarding the use of sensitivity analysis. All guidelines
recommended that a sensitivity analysis should be performed
Guideline Type recommended Variables tested
clinical economic
Regulatory guidelines
Ontariol4] Use of wide confidence limits on utilities; a Measurement techniques Not specified
method to test 'robustness of qualitative used to convert clinical
conclusions', in order to 'identify areas effects into values
where further research is needed to more (quality-adjusted life-years or
precisely estimate the values of those hea~hy year equivalents)
variables to which the resu~ is sensitive'
Canada l5] MonteCario Assumptions Sampling error; assumptions
Australial6] Confidence intervals Assumptions such as Marginal costs as different
outcome estimates; upper proportions of average costs;
and lower confidence lim~s upper and lower confidence limits
Plausible values, including values in the Key parameters; range of Key parameters; range of potential
confidence interval or other statistical potential assumptions assumptions
measure (simple one-way or multi-way
analysis)
Independent authors
Guyatt et a1. 18] Not specified Key parameters Key parameters; medical values
with no substantial evidence of
their true value
Jolicoeur et al. [9] Threshold analysis Key variables; discount rate Key variables
Drummond et 81.1'°] Confidence intervals on cost data from Estimates with imprecision; Estimates with imprecision; value
randomised, controlled clinical trials value judgements or judgements or methodological
methodo[ogical controversy controversy
McGhan & Lewis[l1] On a range of plausible variables (simple Key variables Key variables
analysis)
Garattini et 81.1'2) Simple; full confidence intervals; analysis Efficacy rates Costs as unit costs or volumes;
of extremes method differs if volume is a
deterministic or stochastic variable;
5% discount rate

the preferred sensitivity analysis methods and the
parameters on which the analyses should be per-
formed, Therefore. to generate a meaningful check-
list. we reviewed the strengths and weaknesses of
each sensitivity analysis method (table II).[[3]
With respect to our review of the phar-
macoeconomic guidelines (table I). it was evident
that. although all guidelines recommended using a
sensitivity analysis. inconsistencies existed re-
garding the type of analysis recommended and on
the types of variables that should be tested. Accord-
ing to the guidelines reviewed. the common types
of sensitivity analysis recommended included sim-
ple. threshold analysis and confidence intervals.
Confidence intervals are recommended for use as
a plausible range for estimation of parameters in a
sensitivity analysis. where such data are available.
These guidelines also provided some indication
of the variables that should be tested, In general.
the variables suggested were either clinical or eco-
nomic parameters that were critical to the analysis
or were associated with known imprecision. One
guideline. the Division of Drug Marketing and
Communications (DDMAC) economic guidelines
from the US,l2°] was excluded from our analysis
because the report was unpublished at the time of
writing this report,
The checklist was then constructed using infor-
mation from the guidelines. the strengths and
weakness of the various sensitivity analysis meth-
ods. and their application to the different types of
health economic study (predictive. prospective and
retrospective). A copy of the checklist appears in
fig. 1.

© Adis Inlemanonal Urnlled. All rights reserved. PharmocoEconomics 1997 Jan; 11 (1)
78 Agro e/ al.

Table II. Methods of conducting a sensitivity analysis: advantages and disadvantages

Method Definition Advantages Disadvantages
Simple
one-way Selecting plausible values for a single Easy to use; flexibility in parameter May be difficult if the variables
parameter choice used are dependent on each other
multi-way Selecting plausible values for a single Easy to use; flexibility in parameter Cumbersome if >2 inputs are
parameter; > 1 variable examined choice varied simultaneousIyl13.1 7J
simultaneouslyl13,
Threshold Varying specific input parameters until a Meaningful when the value of an input Difficult if the variables used are
break-even point is reached 18, variable is unknown, such as the price dependent on each other; used for
of a new drug;113, comparable with single perameters only;
95% CI of parameter as a statistical cumbersome if more than one
test variable is changedI13.17J
Probabilistic Uncertainties In all values and probabilities Easy to use; may be conducted via Assumes parametric distribution,
(i.e. Monte are simultaneously evaluated; a parametric computer program and assumes probability and utility
Carlo) model evaluates over entire range for all distributions are independen~17J
parameters
Analysis of extremes
simple Evaluating extreme estimates from the Useful when the base-case value has Extreme value choices may
base-case analysis;113, commonly referred been estimated by experts, but the approach, but never achieve, the
to as 'worstlbest case analysis' 'distribution' between the outer limits is best/worse case values;113, unable
unknownl131 to assess intermediate values
within the bounds of the besVworse
case scanarios
CI partial Generating Cis from RCTs for input A statistical approach to evaluating the No projection of the confidenca
variables; the bounds of the Cis are not limits on input variables intervals for the input variables
re-entered into the economic evaluation through to the cost-effectiveness
to generate bounds on the economic and cost-benefit ratios etc. Umited
end-points (i.e., no CI calculated for to data generated from RCTs with
cost-effectiveness ratio) single parameter changes only
CI full Generating Cis from RCTs for input A statistical approach to evaluating Umited to data generated from
variables; the bounds of the Cis are the limits on input variables and on RCTs; using Cis to determine
re-entered into the economic evaluation economic end-points; facilitates sample sizes may require the
to generate bounds on the economic sample size calculations that are inclusion of more patients in the
end-points (i.e., a cost-effectiveness ratio needed to show an 'economically RCT, thus exposing increasing
with a CI) important differenca';114. 191increases numbers of patients to a new drug,
objectivity in the decision to adopt simply to obtain economic data;114,
one technology over another!14, single parameter changes only
Rank-order A combination of a CI and threshold Comprehensive (examines all Does not test altemate hypotheses
stability analySis, ROSA provides a plausible range parameters); tests model stability;13' or indirect costs per se; examines
analysis in which inputs and values may be found avoids parameter selection biasl31 single parameters only
(ROSA)13, (CI component); calculates a break-even
point for all inputs. Elasticities test the
stability of the mode~3'
Abbreviations: CI = confidence interval (most often 95%); RCT = randomised controlled trial.

Selection of Literature health economics, medical and pharmacy. The

In the second phase of this study, the literature
was analysed using the validated checklist (fig. 1).
To facilitate this phase of the research, a literature
sample was identified using the methods described
by Bradley et aI.l21) These authors chose 3 specific
journals from the literature in each of 3 fields:
health economics journals were Pharmaco-
Economics, International Journal of Technology
Assessment in HealthCare and Journal of Health
Economics. The medical journals examined were
New England Journal of Medicine, Medical Care
and Journal of the American Medical Association.
From the pharmacy literature, the chosen journals

© Adls International Umlted. All rights reserved. PharmacoEconomics 1997 Jan: 11 (1)
Sensitivity Analysis 79

P/eae Reviewer's initials Article number

ch«:Ic Hi

~""r~
I) What was the type of pharmacoeconomlc atudy?

Cost analysis ~.ll.'

0
0
One group
Two groups ~
'N-
0 More than two groups
Economic analysis ':(i
0 Cost-effectiveness analysis .. ,
0 Cost-benefit analysis
0 Cost-minimisation analysis .,;.
0 Cost-utility analysis

II) What waa the type of study design? .'.

0 Predictive (i.e. modelling, decision tree)

0 Prospective: RCT
0 Prospective: Non-RCT
0 Retrospective '.
0 Mixed (pleaae check off above-mentioned components)
0 Cannot access

e
III) Waa a sensitivity analysis performed?

Yes
0 Explicitly - stated so by the authors
0
0 No
Implicitly - implied by the study
':1
'(,-,
IV) What type of sensitivity analySis waa perfonned? (Check all applicable)

0 Simple '.

0 Threshold
0 Probabilistic (i.e. Monte Carlo method)
Analysis of extremes
0 Confidence intervals (Cls)-Partial: Cis are given for various input variables but the inputs have not been
placed back into the economic or cost analysis to regenerete the economic end-point
D Confidence intervals (Cls)-Full: Cis around an economic end-point i.e. around a CEA ratio
D Cannot assess

V) On what parametenl waa the sensitivity analysis performed? Check all applicable

Inputs-rates
D Success probabilities for the drug in question
D Epidemiologic rates (i.e. prevalence)
D Other, please specify
Inputs-direct costs
D Drugs
D services
D Others, pleaae specify
D Inputs-indirect costs
Please list
Discount rate
D Inputs
D OUtcomes (i.e. OALYs)
0 Cannot assess

I
Fig. 1. Final sensitivity analysis evaluation form. Abbreviations: CEA ~ cost-effectiveness analysis; OALYs ~ quality-adjusted life-years;
RCT ~ randomised controlled trial.

~ Adis In!ema!1onal Umi!ed. All rights reserved. PharmacaEcanamics 1997 Jan; 11 (1)
80
were The Annals of Pharmacotherapy, American
Journal of Health System Pharmacy and Hospital
Pharmacy.
MEDLINE, EMBASE and International Phar-
macy Abstracts were searched for English-language
economic studies. Economic studies were defined
as those reports that evaluated the impact of drug
therapy, medical technologies, services or treat-
ment programmes with respect to both clinical and
economic outcomes.l ll A 5-year time frame was
specified a priori and used, because it represented
a reasonable time period in which to assess changes
in publication quality. However, the sample size
may not have been sufficient to detect time trends.
Inclusion criteria for studies required that arti-
cles were original research articles published in
one of the selected journals between January 1989
and December 1993, involving a comparison be-
tween drugs. treatments or services, and assessing
both costs and outcomes in cost-effectiveness,
cost-benefit. cost-utility or cost-minimisation stud-
ies. Simple cost analyses, financial feasibility anal-
yses. editorials. commentaries and review articles
were excluded. We aimed to randomly select, by
lottery. 30 articles per journal type from the articles
identified through the literature search.
Application of Checklist
An initial checklist was developed and tested by
5 raters on a random selection of 10 papers from
our sample of 90 articles. From the initial testing.
the checklist was refined and applied to the same
10 articles from the sample.
We then applied the checklist to the sample of
90 articles. The analysis was performed by 5 stu-
dents with postgraduate degrees in health fields in-
cluding I Doctor of Pharmacy candidate, I Doctor
of Philosophy candidate. 2 Master of Science can-
didates. and I Master of Science graduate (3 of
these individuals were registered pharmacists).
Raters underwent 30 minutes of formal (content)
instruction before validation of the checklist and I
hour of formal training before the evaluation of
reliability.
II:> Adls International Umlted. All rights reseIVed.
Agrot'tal.
Inter-rater reliability was assessed by calculat-
ing 1C between raters for 3 key questions on the
validated checklist for 10 randomly selected arti-
cles. The 3 key questions were as follows.
• What was the type of study design"!
• Was a sensitivity analysis performed?
• What type of sensitivity analysis was per-
formed?
The third question comprised a checklist of
parts (simple. threshold. probabilistic. analysis of
extremes. confidence intervals - partial, confidence
intervals - full. cannot assess). Agreement was de-
fined as different raters agreeing on all parts of the
type of analysis.
The rate of agreement (A) was calculated on a
per question basis by the following equation:
A = sum of observed agreementsll: total possi-
ble agreements.
The total possible agreements per question for
5 raters was n(n-1)/2 = 5(5-1)/2 = 10.
Classification of Studies
We classified the study designs into either pre-
dictive, prospective or retrospective.l22 ) Predictive
studies involved the use of modelling or decision-
tree analyses. Prospective trials involved the gen-
eration of costs and/or effects in either a random-
ised controlled trial or nonrandomised controlled
trial. Retrospective studies involved the gathering
of data on costs and/or effects that had already
been incurred.
Descriptive statistics were calculated for the 90
studies. Absolute values and/or percentages were
reported for the outcome variables: incidence of
conducting sensitivity analyses, type of economic
analysis performed and type of sensitivity analysis
performed. Once the studies that performed a sen-
sitivity analysis had been identified. this sample
was analysed in order to determine the section of
the paper in which the authors first mentioned their
plan to conduct a sensitivity analysis (Methods,
Results or Discussion).
PharmacoEconomlcs 1997 Jon; 11 (1)
Sensitivity Analysis 81

Table III. Sample selection of articles for literature analysis Literature Analysis
(reproduced from Bradley et al.. 1211 with permission)
Selection step _J_ou_r_na_l...!.ty-'-pe_ _ _ _ _ _ _ __
health medical pharmacy total
Results of the literature search and sample se-
economics lection are presented in a previous publication by
Met initial inclusion 45 38 40 123 Bradley and colleagues.l 21J Initially. 123 articles
criteria met the criteria, but 16 were inappropriate and 17
Inappropriate after 5 7 4 16
were randomised out (table III). Thus. 90 articles
review"
Randomised outb 10 6 17
were examined: 30 from health economics jour-
Final sample of 30 30 30 90 nals, 30 from medical journals and 30 from phar-
articles rated macy journals (appendices A, B and C).
a Appeared to meet criteria on initial examination. but on closer Data on study type were gathered through the
scrutiny. proved not to meet criteria because they were not
original articles, not cost-analysis studies, did not use health checklist (fig. 1). The majority of the 90 articles
outcomes. or involved discounting only. were either predictive [n = 39 (43%»). retrospec-
b A sample size of 30 per joumal type was decided a priori; tive [n = 24 (27%») or prospective nonrandomised
articles were randomly deleted to arrive at that total.
controlled trials [n = 20 (22%)]. The remainder in-
cluded prospective randomised controlled trial
studies [n = II (12%»), or those of mixed design [n
Results
=5 (6%)] (table IV).
When types of economic analyses were re-
viewed with respect to journal type. health eco-
Checklist Reliability
nomics and medical journals utilised the predictive
The inter-rater agreement of the 5 raters on eval- model (i.e. decision-tree analyses) more frequently
uating the level of reporting of sensitivity analyses than the other designs. having 18 (60%) and 16
in the 10 pharmacoeconomic articles (according to (53%) occurrences of the predictive design, re-
the question 'Was a sensitivity analysis perfonnedT) spectively (table IV). Pharmacy journals generally
was 82%, while 1( was 0.75 [standard error (SE) = approached their studies using the prospective
O.072J. The respective agreement and 1( for the nonrandomised controlled trial or retrospective de-
question on the type of study design were 84% and signs. with 13 (43%) and 11 (37%) instances. re-
0.76 (SE =0.055), and for the type of sensitivity spectively.
analysis performed were 72% and 0.69 (SE = From the sample of 90 papers examined, 53
0.055). Therefore, the calculated coefficients (59%) of the articles conducted a sensitivity anal-
showed acceptable agreement among raters. Raters ysis. Of the 53 articles that conducted sensitivity
then analysed the balance of the 90 articles inde- analyses, 39 (74%) of the authors stated explicitly
pendently. that a sensitivity analysis was being perfonned.

Table IV. Proportion of models used in economic analyses'

Joumaltype Predictiveb Prospective randomised Prospective nonrandomised RetrospectiveC Mixed"
controlled trial controlled trial
Health economics 18 6 3 5 2
Medicine 16 3 4 8
Pharmacy 5 2 13 11 2
Totale 39(43%) 11 (12%) 20 (22%) 24 (27%) 5(6%)
a More than 1 model was used in some papers; 1 paper was not assessable.
b Modelling or decision-tree analyses.
c For example. retrospective case analyses or retrospective chart reviews.
d A combination of the different model types.

(i;; Adis Intemotional Umited. All rlghts reserved. PharmacoEconomlcs 1997 Jan; 11 (1)
82
Table V. Incidence 01 sensitivity analysis conduct by joumaJ type and over lime (figures in parentheses are percentages)
Journal type 1989 1990
Health economics 010 (0) 414 (100)
Medicine 314 (75) 7/9 (78)
Pharmacy 0110 (0) 0i6 (0)
Total 3115 (20) 11119 (58)
a X2 = 26.33, dI = 2, P < 0.001; pharmacy journals versus health economic and medicine journals.
When the reporting of sensitivity analysis was
evaluated by journal type (table V), 80% of health
economics and 77% of medical journal articles had
conducted sensitivity analyses in their phar-
macoeconomic or health economic studies, while
only 20% of pharmacy journal articles had done so.
There were no significant trends with respect to
sensitivity analysis reporting over time. Although
there appeared to be an upward trend in sensitivity
analysis reporting for the entire sample, the sample
size may not have been sufficient to detect time
trends.
In the assessment of the type of sensitivity anal-
ysis performed, the majority of studies [n = 37
(41%)] used a simple sensitivity analysis tech-
nique. Threshold determination was used in 10
(11 %), analysis of extremes in 9 (10%) and partial
confidence intervals in 6 (7%) (table VI).
We noted the first mention in each paper of ei-
ther the plan to conduct a sensitivity analysis or the
presentation of the final results of a sensitivity
analysis, whichever appeared first. Of the 53 pa-
pers that conducted a sensitivity analysis, 29 (55%)
indicated their plan to conduct a sensitivity analy-
sis in the Methods section of the article, while in
23 (43%), this information appeared in the Results
section.
Sensitivity analyses were generally conducted
on inputs such as rates (e.g. epidemiological, suc-
cess probabilities) and direct costs (e.g. drugs, ser-
vices). Indirect costs and discount rates on inputs
and outcomes were also commonly analysed. Be-
cause raters were asked to specify the exact param-
eters on which the sensitivity analysis was con-
ducted, we were able to identify less prominent, but
nonetheless emerging, parameters on which sensi-
tivity analysis was being conducted in the sample.
© Adls InternaUonal Umlted. All rights reserved.
Agroet al.
1991 1992 1993 Total"
314 (75) 9112 (75) 8110 (80) 24130 (80)
617 (86) 314 (75) 416 (67) 23130 (n)
1/6 (17) 516 (83) 012 (0) 6/30 (20)
10117 (59) 17122 (n) 11117 (65) 53190 (59)
Two areas - compliance rates and test charac-
teristics (e.g. laboratory costs, test sensitivity and
specificity) - were noted by the raters as areas on
which analyses had been commonly conducted.
Discussion
This study evaluated the extent of sensitivity
analysis reporting in the pharmacoeconomic liter-
ature by journal type and over time. The inter-rater
reliability on 3 key questions met our pre specified
value, which indicated that the raters were judging
the articles in a similar fashion. The 1C values were
considered acceptable for our purposes.
Approximately 59% of the sample of90 articles
conducted a sensitivity analysis. Sensitivity analy-
ses were more often conducted in health economic
(80%) and medical journals (77%) compared with
pharmacy journals (20%). This finding may have
been a reflection of the editorial policies, knowl-
edge and experience of those individuals associ-
ated with health economic and medical journals.
The frequency of use of sensitivity analyses in
pharmacy journals was significantly lower than the
other 2 (table V). Based on our findings, there ap-
pears to be a need for increased use of sensitivity
analyses in future pharmacoeconomic and health
economic studies, particularly in the pharmacy lit-
erature.
The relatively low frequency of sensitivity anal-
ysis was identified as a deficiency in the literature.
Of those studies that did perform a sensitivity anal-
ysis, as many as 14 (26%) did not explicitly state
that such an analysis had been planned or was being
conducted by the authors. In light of the fact that
all pharmacoeconomic guidelines to date have sug-
gested the inclusion of sensitivity analysis, these
analyses should be included and explicitly stated
PharmacoEconomics 1997 Jan: 11 (1)
Sensitivity Analysis
as having been planned in the Methods section of
future studies. This addition would assist the
reader in assessing the quality of the evaluation as
well as in interpreting and extrapolating the results.
Of the studies that conducted a sensitivity anal-
ysis, 29 (55'k) papers indicated their intention to
do so in the Methods section of the paper. In some
cases. it was difficult to assess where the sensitivity
analysis was first mentioned in the body of the pa-
per. This barrier was encountered in some articles
found in the journals Medical Care and Interna-
tional Journal of Technology Assessment In
Healthcare, because those journals did not have a
Methods section per se. Perhaps journals need to
ensure that such important information is not omit-
ted from the Methods section of their manuscripts,
or at least be added as a requirement before accep-
tance of the manuscript by the journal.
The simple sensitivity analysis method was em-
ployed most frequently. The appropriateness of the
widespread use of the simple analysis methodol-
ogy is unknown, given the lack of direct compara-
tive data using different methodologies. However,
based on the known advantages and disadvantages
of the different sensitivity analysis methods, no
one method seems to be ideal for all purposes. To
date. rank-order stahility analysis (ROSA pi is the
most comprehensive method available. but re-
quires the addition of a Monte Carlo simulation[l7j
in order to cover alternative hypotheses that are not
addressed with ROSA alone. In the future. re-
searchers may wish to use more than one method
in a particular pharmacoeconomic study. in order
to compensate for the deficiencies in each method.
Further research is necessary to prospecti vely
compare the effects of different combinations of
sensitivity analysis methods.
Table VI. Type of sensitivity analysis performed (figures in parentheses are percentages)
Joumaltype Simple Threshold
Health economics 15/30 (SO) 6/30 (20)
Medicine 19/30 (63) 4/30 (13)
Pharmacy 3/30 (10) 0/30 (0)
Total 37/90 (41) 10/90 (11)
I&; Adis IntemaHonol Umited. All rights reserved.
83
There appears to be an appropriate upward trend
in the incidence of sensitivity analysis in recent
years. However, there were few data points, since
only 5 years were covered in our study, and no time
trends in the conduct of sensitivity analyses were
detected. There were too few time points after the
release of the pharmacoeconomic guidelines to
perform an analysis to measure their impact.
One hypothesis to explain these results is the
timing of the literature with respect to the release
of the guidelines. The literature reviewed in this
study was published between January 1989 and
December 1993. The regulatory guidelines were
published between 1992 and 1995. while the inde-
pendent author guidelines were published between
1986 and 1993. Therefore. based on the time frame
used. most of the studies in our sample were pub-
lished before the release of the pharmacoeconomic
guidelines. An evaluation of the impact of the
guidelines would require a time series comparing
the situation before with that after release of the
guidelines. Conducting a time series was beyond
the scope of the present research, but would be a
worthwhile endeavour. As a second hypothe-
sis. it is possible that sensitivity analyses
were performed. but not reported. Finally, a third
hypothesis could be related to the inadequacies
of the current methods and the present lack of one com-
prehensive sensitivity analysis method. In the fu-
ture, it may be useful to develop a sensitivity anal-
ysis method that could be used for all variables and
with any study design. Alternatively. a set of guide-
lines could be developed indicating the preferred
approach for various types of studies/analyses.
There was also a high degree of inconsistency
in the parameters on which the sensitivity analyses
were conducted. This variability between studies
Probabilistic Analysis of Confidence intervals Cannot Total
extremes partial full assess
3/30 (10) 7/30 (23) 3/30 (10) 3/30 (10) 2130 (7) 39/30
1/30 (3) 2130 (7) 1/30 (3) 0/30 (0) 1/30 (3) 28130
0/30 (0) 1130 (3) 2130 (7) 0/30 (0) 0/30 (0) 6/30
4190 (4) 9/90 (10) 6/90 (7) 3/90 (3) 3/90 (3) 72190 (80)
PharmacoEconomics 1997 Jan; 11 (1)
84
increases the heterogeneity of the research. Of
those parameters chosen. success rates were one of
the more common parameters examined. However.
it was not clear from the studies whether the suc-
cess rates were generated from intention-to-treat or
per-protocol analyses. The difference in success
rates between these 2 study designs may have had
an impact on the overall results of the economic
study. II) keeping with the goal of transparency in
research. such information should be stated explic-
itly by the authors of a pharmacoeconomic study.
In addition. regulatory agencies and individual au-
thors of pharmacoeconomic guidelines need to ad-
dress this issue and determine on what parameters
sensitivity analyses should be conducted and what
sensitivity analysis methods should be used. Also.
the quality of sensitivity analyses is extremely im-
portant. Further research should address the issue
of qUality.
With respect to the pharmacoeconomic and
health economic guidelines. limited information
has been provided regarding which sensitivity
analysis method should be used. This variability
observed between the guidelines illustrates the
need for further research in the area of sensitivity
analysis. Consensus may not be required on which
type of test should be universally employed, but
rather on which type of analysis should be used for
certain study types or variables/uncertainty being
tested. Briggs and colleagues (13 ) have summarised
the types of test that might be used to deal with
different types of uncertainty (data variability,
generalisability, extrapolation, analytical method).
Perhaps the authors of future pharmacoeconomic
guidelines or updates of current guidelines could
consider this information.
Similarly, as illustrated by the guidelines, re-
searchers in this field have not agreed on which
variables the sensitivity analysis should be con-
ducted. Jacobs and coworkers (23 ) commented that
researchers and authorities should reconsider the
use of sensitivity analyses and decide whether
these should be used to address uncertainty in fun-
damental principles broadly or whether they should
be applied only to examine the impact of varying
© Adls Intematlonal Umlted. All rights reserved.
Agro et al.
different inputs on the cost-effectiveness ratios. In
the future, pharmacoeconomic guidelines should
identify the parameters on which an analysis
should be conducted, given a certain study design.
Such guidance would assist thc uscrs of pharo
macoeconomic studies in achieving consistent
methodology across studies, on which they could
base policy decisions.
One aspect not examined in our checklist was
the use of sensitivity analysis versus subgroup
analyses. Subgroup analyses seek to identify
groups to whom the results apply. Subgroup anal-
yses provide insight into the generalisability of the
results and the extrapolation of the findings to
specified patient populations. On the other hand.
sensitivity analyses address the validity of the data.
They answer the question 'Are the results true or
artifactual?,. Both subgroup and sensitivity analy-
sis should be identified a priori and not applied a
posteriori. Future pharmacoeconomic guidelines
must address subgroup analysis and differentiate it
from sensitivity analysis.
A need exists to develop pharmacoeconomic
guidelines that incorporate all aspects of sensitivity
analyses, which can be accepted by all parties (gov-
ernment, industry and academia). A mechanism
must be in place to update the guidelines on a reg-
ular basis. given the emergence of new methodol-
ogies in the developing field of health economic
and pharmacoeconomic analysis.
Conclusions
This research has shown that a large number of
pharmacoeconomic guidelines recommend em-
ploying sensitivity analysis, but only 59% of the
studies in our sample actually conducted such an
analysis. Study authors need to pay close attention
to conducting sensitivity analyses in order to help
verify the validity of pharmacoeconomic research
results. Regulatory agencies, industry and acade-
mia need to update their guidelines according to the
results of new methods of conducting sensitivity
analyses.
PharmacoEconomics 1997 Jan: 11 (1)
Sensitivity Analysis
Appendix A
Papers from health economics journals that
were included in the study:
A I. Bergemann R. Brandt A. Siegrist W. Cost-effectiveness
study of lipid lowering therapy in hyperlipoproteinaemia type
lib and type IV (Fredrickson). PhannacoEconomics 1993; 3:
131-9
A2. Cantor SB. Carson RW. Spann S1. A cost-effectiveness
analysis of epoctin usage for patients with AIDS. Phar-
macoEconomics 1993; 3: 2+4-9
A3. Castiel C. Hervo C. Gaillard M. et a!. Cost-utility analysis
of early thrombolytic therapy. PharmacoEconomics 1992; I:
438-2
A4. de Lissovoy G, Elixhauser A. Luce BR, et a!. Cost-analysis
of imipenem-cilastatin versus clindamycin with tobramycin
in the treatment of acute intra-abdominal infection. Phar-
macoEconomics 1993; 4; 203-14
A5. Fendrick AM. Javitl Je. Chiang YP. Cost-effectiveness of
the screening and treatment of diabetic retinopathy: what are
the costs of underutilization') Int J Technol Assess Health
Care 1992; 8: 694-707
A6. Fendrick MA. de Pouvourville G. Bitker C. et a!. Treatment
of symptomatic cholelithiasis in France. Int J Technol Assess
Health Care 1992; 8: 166-84
A7. Garrelts Je. Smith DF. Ast D. et al. A comparison of
the safety. timing and cost-effe.tiveness of administering
antibiotics by intravenous bolus (push) versus intravenous
piggyback (slow infusion) in surgical prophylaxis. Pharmaco-
Economics 1992: I: 116-23
A8. Hatziandreu EJ. Hatzakis A. Hatziyannis S. et a!. Cost-ef-
fectiveness of hepatitis-B vaccine in Greece. Int J Technol
Assess Health Care 1991; 7: 256-62
A9. Henriksson P. Stege R. Cost comparison of parenteral es-
trogen and conventional hormonal treatment in patients with
prostatic cancer. Int J Technol Assess Health Care 1991: 7:
no-s
A 10. Hillson SD. Rich EC. Two strategies for prophylaxis of
fatal postoperative pulmonary embolism.lnt J Technol Assess
Health Care 1990: 6: 470-9
A II. Hjalte K. Lindgren B. Persson U. Cost-effectiveness of
simvastatin versus cholestyramine: results for Sweden. Phar-
macoEconomics 1992; I: 213-6
A 12. Johannesson M. Wikstrand J. Jonsson B. et al. Cost-effec-
tiveness of antihypertensive treatment: metoprolol versus thi-
azide diuretics. PharmacoEconomics 1993: 3: 30-44
A 13. Johannesson M. Economic evaluation of hypertension
treatment. Int J Technol Assess Health Care 1992: 8: j06-23
A14. Johnson NE. Nash DB. CarpenterCE. et al. Ondansetron:
costs and resource utilization in a US teaching hospital set-
ting. PharmacoEconomics 1993: 3: 471-81
A 15. Jonsson B. Horisberger B. Bruguera M. et a!. Cost-benefit
analysis of hepatitis-B vaccination: a computerized decision
model for Spain. Int J Technol Assess Health Care 1991: 7:
379-402
A 16. Jonsson B. Haglund U. Cost-effectiveness of misoprostol
in Sweden. Int J Technol Assess Health Care 1992: 8: 234-44
A17. Lynch A. Malek M. Davey PO. et a!. Costing wound in-
fection in a Scottish hospital. PharmacoEconomics 1992: 2:
163-70
'" Adis Intema~onal limited. All righls reserved.
85
A 18. Moller G. Goldie I. Jonsson E. Hospital care versus home
care for rehabilitation after hip replacement. Int J Technol
Assess Health Care 1992: 8: 93-101
A 19. Osterhaus n. Gutterman DL. Plachetka JR. Healthcare
resource and lost labour costs of migraine headache in the US.
PharmacoEconomics 1992: 2: 67-76
AlO. Pashko S. Johnson DH. Potential cost savings of oral ver-
sus intravenous etoposide in the treatment of small cell lung
cancer. PharmacoEconomics 1992: I: 293-7
All. Rutten-van Malken MPMH. Van Doorslaer EKA, Jansen
MCC. et al. Cost effectiveness of inhaled corticosteroid plus
bronchodilator therapy versus bronchodilator monotherapy
in children with asthma. PharmacoEconomics 1993: 4: 257-
70
A22. SCOll WG. SCOll HM. Annual costs of benign prostatic
hyperplasia in New Zealand. PharmacoEconomics 1993; 4:
455-68
A23. Scott WG, Tilyard MW, Dovey SM. et al. Roxithromycin
versus cefaclor in lower respiratory tract infection: a general
practice pharmacoeconomic study. PharmacoEconomics
1993; 4: 122-30
Al4. Scott TE. Jacoby I. Clinical decision analysis as a means
of technology assessment. Int J Technol Assess Health Care
1992; 8: 185-97
A25. Sesso R. Eisenberg JM. Stabile e. et a!. Cost-effectiveness
analysis of the treatment of end-stage renal disease in Brazil.
Int J Technol Assess Health Care 1990; 6: 107-4
A26. Sintonen H. Alander V. Comparing the cost-effectiveness
of drug regimens in the treatment of duodenal ulcers. J Health
Economics 1990; 9: 115-1O I
A27. Tulloch JFC, Antczak-Bouckoms AA, Ung N. Evaluation
of the costs and relative effectiveness of alternative strategies
for the removal of mandibular third molars. Int J Technol
Assess Health Care 1990; 6: 505-15
A28. van Hout BA. Wielink G. Bonsel GJ. et al. Effects of ACE
inhibitors in heart failure in the Netherlands: a phar-
macoeconomic model. PharmacoEconomi" 1993: 3: 387-97
A29. van Hout B. Bon,el G. Habbema D. et al. Heart transplan-
tation in the Netherlands: costs, effects and scenarios. J
Health Econ 1993; 12: 73-93
A30. Walker A. Whynes DK. Participation and screening pro-
grammes for colorectal cancer: more would be beller? J
Health Econ 1991: 10: 207-5
Appendix B
Papers from medical journals that were in-
cluded in the study:
B I. Appel U. Steinberg EP. Powe NR. et al. Risk reduction
from low osmolality contra't media: what do the patient'
think it is worth? Med Care 1990: 28: 324-7
B2. Barrett BJ. Parfrey PS. Vavasour HM. et a!. A comparison
of nonionic. low-osmolality radiocontrast agents with ionic.
high-osmolality agents during cardiac catheterization. N Engl
J Med 1992: 326: 431-6
B3. Bird JA. McPhee SJ. Jenkins C. et al. Three strategies to
promote cancer screening: how feasible is wide-scale imple-
mentation" Med Care 1990: 28: 1005-2
B4. Brignoli Gable e. Sedory Holzer S. Engelhart L. et a!.
Pneumococcal vaccine: efficacy and associated CO" savings.
JAMA 1990: 264: 2910-5
PharmacoEconomocs 1997 Jan: 11 (1)
86
B5. Buhaug H, Skjeldestad FE, Backe B, et aI. Cost effective-
ness of testing for chlamydial infections in asymptomatic
women. Med Care 1989; 27: 833-41
B6. Castellano AR, Nettleman MD. Cost and benefit of second-
ary prophylaxis for pneumocystis carinii pneumonia. JAMA
1991; 266: 820-4
B7, Chalfin DB, Holbein EM, Gein AM, et al. Cost-effective-
ness of monoclonal antibodies to gram-negative endotoxin in
the trealment of gram-negative sepsis in ICU patients. JAMA
1993; 269: 249-54
B8. Cohen IL, Lambrinos J, Fein IA. Mechanical ventilation for
the elderly patient in intensive care: incremental changes and
benefits. JAMA 1993; 269: 1025-9
B9. Cummings SR, Rubin SM, Oster G. The cost-effectiveness
of counseling smokers to quit. JAM A 1989; 261: 75-9
BID. Eckman MH, Beshansky JR, Durand-Zaleski I, et al. An-
ticoagulation for noncardiac procedures in patients with pros-
thetic heart valves: does low risk mean high cost? JAMA
1990; 263: 1513-21
B II. Eckman MH, Levine HJ, Pauker SG. Effect of laboratory
variation in the prothrombin-time ratio on the results of oral
anti-coagulant therapy. N Engl J Med 1993; 329: 696-702
B12. Edelson IT, Weinstein MC, Tosteson ANA, et al. Long-
term cost-effectiveness of various initial monotherapies for
mild to moderate hypertension. JAMA 1990; 263: 407-13
B 13. Edelson IT, Tosteson AN, Sax P. Cost-effectiveness of
misoprostol for prophylaxis against nonsteroidal anti-inflam-
matory drug-induced gastrointestinal tract bleeding. JAMA
1990; 264: 41-7
B14. Ehreth J, Chapko M, Hedrick Sc. Comparison of uti liz a-
tion and cost among contract adult day health care, VA adult
health care, and customary care. Med Care 1993; 31 (9
Suppl.): SS83-93
B 15. Finkler MD, Wirtschafter DO. Cost-effectiveness and ob-
stetric services. Med Care 1991; 29: 951-63
B 16. Garber AM, Fenerty JP. Costs and benefits of prenatal
screening for cystic fibrosis. Med Care 1991; 29: 473-89
B17. Goldman L, Weinstein MC, Goldman PA, et aI. Cost-ef-
fectiveness of HMG-CoA reductase inhibition for primary
and secondary prevention of coronary heart disease. JAMA
1991; 265: 1145-51
B 18. Hayashida M, Alterman AL, Mclellan AT, et al. Compar-
ative effectiveness and costs of inpatient and outpatient de-
toxification of patients with mild-to-moderate alcohol
withdrawal syndrome. N Engl J Med 1989; 320: 358-65
B19. Hillman BJ, Joseph CA, Mabry MR, et al. Frequency and
costs of diagnostic imaging in office practice, a comparison
of self-referring and radiologist-referring physicians. N Engl
J Med 1990; 323: 1604-8
B20. Hillner BE, Smith TJ, Desch CEo Efficacy and cost-effec-
tiveness of autologous bone marrow transplantation in meta-
static brea~t cancer. JAM A 1992; 267: 2055-61
B21. Hillner BE, Smith J1. Efficacy and cost effectiveness of
adjuvant chemotherapy in women with node-negative breast
cancer: a decision-analysis model. N Engl J Med 1991; 324:
160-8
B22. Kay BJ, Share DA, Jones K, et al. Process, costs, and
outcomes of community-based prenatal care for adolescents.
Med Care 1991; 29: 531-42
B23. Krunlholz HM, Pasternak RC, Weinstein MC, et al. Cost
effectiveness of thrombolytic therapy with streptokina~e in
© Adls Intemational Umlted. All rights reserved.
Agro e/ al.
elderly patients with suspected acute myocardial infarction.
N Engl J Med 1992; 327: 7-13
B24. Mast EE, Berg JL, Hanrahan LP, et .1. Risk factors for
measles in a previously vaccinated population and cost-effec-
tiveness of revaccination strategies. JAM A 1990; 264: 2529-
33
B25. Mohle-Boetani JC, Schuchat A, Plik'ytis BD, et .1. Com-
parison of prevention strategies for neonatal group B strepto-
coccal infection: a population-based economic analysis.
JAMA 1993; 270: 1442-8
B26. Schapira DV, Studnicki J, Brodham DO, et al. Intensive
care, survival, and expense of treating critically ill cancer pa-
tients. JAMA 1993; 269: 783-6
B27. Schulman KA, Glick HA, Rubin H, et al. Cost-effective-
ness of HA-IA monoclonal antibody for gram-negative sep-
sis: economic assessment of a new therapeutic agent. JAMA
1991; 266: 3466-71
B28. Schulman KA, Kinosian B, Jacobson TA, et al. Reducing
high blood cholesterol level with drugs; cost effectiveness of
pharmacologic management. JAMA 1990; 264: 3025-3
829. Soghikian K, Casper SM, Fireman BH, et al. Home blood
pressure monitoring: effect on use of medical services and
medical care costs. Med Care 1992; 30: 855-65
B30. WickizerTM, Wheeler JRC, Feldstein PJ. Does utilization
review reduce unnecessary care and contain costs? Med Care
1989; 27: 632-47
AppendixC
Papers from pharmacy journals that were in-
cluded in the study:
C I. Barriere SL. Formulary evaluation of second-generation
cephamycin derivatives using decision analysis. Am J Hosp
Pharm 1991; 48: 2146-50
C2. Calvo MV, Pilar del Val M, Mar Alvarez M, et al. Decision
analysis to assess cost-effectiveness of low-osmolality con-
trast medium for intravenous urography. Am J Hosp Pharm
1992; 49: 577-84
C3. Chin A, Gill MA, Ito MK, et al. Cost analysis of two
c1indamycin dosing regimens. DlCP Ann Pharmacother 1989;
23: 980-3
C4. Chin A, Gill MA,lto MK, et al. A cost comparison of intra-
muscular versus intravenous imipenem. Hosp Pharm 1989;
24: 905-9
C5. Colburn PA, Carver PA, Montgomery PA, et al. Appropriate
but not cost effective ceftazidime use in a university hospital.
Hosp Pharm 1989; 24: 911-4, 16, 28
C6. Cooke J, Cairns CJ, Tillotson GS, et al. Comparative clini-
cal, microbiologic, and economic audit of the use of oral
ciprofloxacin and parenteral antimicrobials. Ann Phar-
macother 1993; 27: 785-9
C7. Cramer R, Wright C. Changing physician prescribing habits
through a cost-effective first generation cephalosporin formu-
lary. Hosp Pharm 1989; 24: 33-8
C8. Destache CJ, Meyer SK, Padomek MT, et aI. Impact of a
clinical pharmacokinetic service on patients treated with
aminoglycosides for gram-negative infections. DICP Ann
Pharmacother 1989; 23: 33-8
C9. Goldman MP. Ciprofloxacin drug utilization review and
prospective drug use evaluation. DICP Ann Pharmacother
1990; 24: 82-6
PharmacoEconomics 1997 Jan: 11 (1)
Sensitivity Analysis
CIO. Grasela TH. Paladino JA. Schentag JJ. et al. Clinical and
economic impact of oral ciprofJoxacin as follow-up to paren-
teral antibiotics. DICP Ann Pharmacother 1991; 25: 857-62
CII. Moran LJ. Carey P, Johnson CA. Cost-effectiveness of
epoetin alfa therapy for anemia of end-stage renal disease.
Am J Hosp Pharm 1992: 49: 1451-4
C 12. Hatoum HT. Microcost analysis of inpatient dispensing
and administration of oral solids. Am J Hosp Pharm 1990: 47:
8()(1-S
CU. Iglar AM. Osland CS. Ploetz PA, et al. Time and cost
requirements for decentralized pharmacist activities. Am J
Hosp Pharm 1990; 47: 572-8
C14. Jones RA. Lopez LM, Beall 00. Cost-effective imple-
mentation of clinical pharmacy services in an ambulatory care
clinic. Hosp Pharm 1991; 26: 778-82
CIS. Johnson MS. Pesko LJ, Wood CF, et al. Cost and accept-
ability of three syringe-pump infusion systems. Am J Hosp
Pharm 1990; 47: 1794-8
C16. Lau NC, Caldwell RD. Arford PH. Cost comparison of
oral. nasogastric, and intramuscular cimetidine drug delivery
systems. Hosp Pharm 1992; 27: 946-7, 950-2. 954, 956, 962
C 17. McCombs JS, Nichol MB. Pharmacy-enforced outpatient
drug treatment protocols: a case study of medical restrictions
forcefaclor. Ann Pharmacother 1993; 27: 155-60
C 18. McDonough KP. Weaver RH. Viall GD. Enalapril to
lisinopril: economic impact of a voluntary angiotensin-con-
verting enzyme-inhibitor substitution program in a staff-
model health maintenance organization. Ann Pharmacother
1992; 26: 399-404
C 19. Mueller BA. Abel SR. Impact of college of pharmacy-
based educational services within the hospital. DICP Ann
Pharmacother 1990; 24: 422-4
C20. Nolly RJ, Skoutakis VA. Cost considerations of intrave-
nously administered histamine-2-receptor antagonists. DICP
Ann Pharmacother 1989; 23 Suppl. 10: S23-8
Cli. Plumridge RJ. Cost comparison of intravenous antibiotic
administration. Hosp Pharm 1991; 26: 600. 603-4, 615
cn. Roach AC, Kernodle DS, Kaiser AB. Selecting cost-effec-
tive antimicrobial prophylaxis in surgery: are we getting what
we pay for? DICP Ann Pharmacother 1990; 24: 183-5
C23. Rupp MT. Value of community pharmacists' interventions
to correct prescribing errors. Ann Pharmacother 1992; 26:
1580-4
C24. Seay RE, Edgren BE. Schilling CG. Cost avoidance using
syringe pumps to administer fat emulsion in a neonatal inten-
sive-care unit. Am J Hosp Pharm 1991; 48: 1972-4
C2S. Smith KS. Briceland LL. Nightingale CH. et al. Formulary
conversion of cefoxitin usage to cefotetan: experience at a
large teaching hospital. DICP Ann Pharmacother 1989; 23:
1024-9
C26. Souney PF. Stoukides CA. Pharmacoeconomic aspects
and formulary considerations related to histamine-2-receptor
antagonists. DICP Ann Pharrnacother 1989: 23 Suppl. 10: S29-
35
C27. Stroup JW. Iglar AM. Implementation and financial anal-
ysis of an operating room satellite pharmacy. Am J Hosp
Pharm 1992; 49: 2198-202
C28. Steele MA. Bess DT. Franse VL. et al. Cost effectiveness
of two interventions for reducing outpatient prescribing costs.
DICP Ann Pharmacother 1989; 23: 497-500
:r Adls International Umited. All rights reserved.
87
C29. Taylor JT, Kathman MS. Documentation of cost savings
from decentralized clinical pharmacy services at a commu-
nity hospital. AmJ Hosp Pharm 1991; 48: 1467-70
C30. Wang Jc, Conly JM, Shafran SD. Appropriateness of met-
ronidazole use in a teaching hospital. Am J Hosp Pharrn 1989;
46: 1385-9
Acknowledgements
This project is a research initiative through the Doctor of
Phannacy programme. Faculty of Pharmacy. University of
Toronto. Toronto. Ontario. Canada.
References
I. Drummond MF, Stoddart GL, Torrance GW. Methods for the
economic evaluation of health care programmes. Toronto:
Oxford Medical Publications, 1987
2. Glossary of terms used in health economics. and phar-
macoeconomic and quality-of-life analyses. Pharmaco-
Economics 1994; 6: 401-4
3. Einarson TR. Arikian SR. Doyle J. Rank-order stability analysis
(ROSA): testing pharmacoeconomic data. Med Decis Mak-
ing 1995; 15: 367-72
4. Ontario Ministry of Health. Ontario guidelines for economic
analysis of pharmaceutical products. Toronto: Ontario Min-
istry of Health, 1994
5. Canadian Coordinating Office for Health Technology Assess-
ment (CCOHTA). Guidelines for economic evaluation of
pharmaceuticals. 1st ed. Ottawa: CCOHTA. 1994
6. Australia Commonwealth Depanment of Health Housing and
Community Services. Guidelines for the pharmaceutical in-
dustry on preparation of submissions to the pharmaceutical
benefits advisory committee. Canberra: Australia Common-
wealth Depanment of Health Housing and Community Ser-
vices 1992
7. Task Force on Principles for Economic Analysis of Health Care
Technology. Economic analysis of health care technology: a
report on principles. Ann Intern Med 1995; 122: 61-70
8. Guyau G, Drummond M, Feeny D. et al. Guidelines for the
clinical and economic evaluation of health care technologies.
Soc Sci Med 1986; 22: 393-408
9. Jolicoeur LM. Jones-Grizzle AJ, Boyer JG. Guidelines for per-
forming a pharmacoeconomic analysis. Am J Hosp Pharm
1992;49: 1741-7
10. Drummond M. Brandt A. Luce B. et al. Standardizing method-
ologies for economic evaluation in health care. Int J Technol
Assess Health Care 1993; 9: 26-36
II. MeGhan WF. Lewis NJW. Issues in health policy: guidelines
for pharmaeoeconomic studies. Clin Ther 199~; 14: 486-94
12. Garattini L. Grilli R. Scopelliti D. et al. A proposal for Italian
guidelines in pharmacoeconomics. Pharmacoeconomics
1995: 7: 1-6
13. Briggs A. Sculpher M. Buxton M. Uncertainty in the economic
evaluation of health care technologies: the role of sensitivity
analysis. Health Econ 1994; 3: 95-104
14. Sacristan lA. Day SJ. Navarro O. et al. Use of confidence in-
tervals and sample size calculations in health economic stud-
ies. Ann Pharmacother 1995: 29: 719-25
15. Mason J. Drummond M. Reporting guidelines for economic
studies. Health Econ 1995; 4: 85-9
PharmocoEconomics 1997 Jan: 11 (1)
88
16. Briggs A, Sculpher M. Sensitivity analysis in economic eval-
uation: a review of published studies. Health Econ 1995; 4:
355-71
17. Doubilet P. Begg CB. Weinstein MC. et al. Probabilistic sen-
sitivity analysis using Monte Carlo simulation. Med Oecis
Making 1985; 5: 157-77
18. Pauker SG. Kassirer JP. The threshold approach to clinical
decision making. N Engl J Med 1980; 302: 1109-17
19. Sacristan JA. Hernandez J. Soto J. 'Economic relevance' in
pharmacoeconomic studies [letter). BMJ 1993; 306: 147
20. Division of Drug Marketing Advertising and Communications
(DDMAC). Principles for the review of pharmacoeconomic
promotion - draft. USA: 1995
© Adls international Urnlted. All righls reserved.
Agro ,'f al.
21. Bradley CA. Iskedjian M. Lanctot KL. et aI. Quality assessment
of economic evaluations in selected pharmacy. medical, and
health economics journals. Ann Pharmacothcr 1995; 29: 681-9
22. Einarson T. Shear N. Oh P. Models for pharmacoeconomic anal-
ysis. Can J Clin Pharmacol. In press
23. Jacobs P. Bachynsky J. Baladi J. A comparative review of phar-
macoeconomic guidelines. Phannacoeconomics 1995: 8: 182-9
Correspondence and reprints: Dr Thomas Einarson, Faculty
of Pharmacy, University of Toronto, 19 Russell St., Toronto,
ON M5S 252, Canada.
E-mail: 76766.72@Compuserve.com
PhonnocoEconomlcs 1997 Jon: II (I)