Economic evaluation: a

reader’s guide to studies of

cost-effectiveness
Background
• Economics is, essentially, the science of making choices.

• Health economics provides a framework for informing

decisions (choices) based on maximising outcomes from
available resources: what option(s) would provide the greatest
health gain for the resources (people, time, money...)
available.

• A framework for understanding economic

evaluations= compares the costs and the health outcomes of
two or more treatment approaches.

• Best quality cost-effectiveness evidence comes from

economic evaluations conducted within (parallel to)
• an RCT = trial-based evaluations
Opportunity cost Opportunity cost
The two basic study designs of economic evaluations

Trial-based evaluations Resource use and health-related quality of life data are recorded for all
participants over the duration of a clinical trial;

Model-based evaluations Data from multiple sources, such as randomised controlled trials,
observational studies, epidemiological data, and administrative records, are
combined;
Mathematical models are used to estimate costs, effects, and cost-
effectiveness of hypothetical (modelled) scenarios;
Useful when no single trial has collected all of the required data, when
results from one context are to be applied in a different setting or
population, or to evaluate more complex scenarios or long-term outcomes
beyond the feasible scope of a randomised trial
 PICOT framework
• Population
• Intervention
• Comparison
• Outcome
• Time frame
Outcome

• The Outcome is perhaps the biggest difference between an economic evaluation and
an RCT.
• In an RCT, there is usually one primary outcome – such as a patient-reported
outcome measure, clinical measurement, or physical performance test.
• In an economic evaluation, there are two outcomes:
• the costs (the net investment)
• the effects (the consequences resulting from that investment)
• These are typically reported as a ratio of one over the other, such as cost per
quality-adjusted life-year (QALY) gained.
• In practice, these two outcomes can be subsequently combined by valuing the
effects in the same units as the costs
 The Costs Outcome
Narrowest Broader Broadest

• Payer perspective • health system perspective

The societal perspective
• which also counts up other
• only costs considered are what a in which wider, non-health system
healthcare utilization
payer (typically an insurance financial consequences are tallied up,
company or government downstream from the decision
such as the out-of-pocket costs borne
reimbursement agency) pays to to invest (e.g. costs borne by
by patients, cost burdens to family and
the provider for delivering the other parts or payers in the
caregivers such as time off work to
treatment or program. health system
care for the ill patient or provide them
• additional costs from dealing
transport, government-paid social
with adverse events, and cost-
benefits such as disability benefits or
savings from reduced
unemployment benefits, and
healthcare utilization in other
productivity losses through sick leave
areas such as imaging
and other time off work, reduced
specialist consultations,
duties
surgeries, or medication
consumption)
so understanding which perspective is being used is crucial to interpreting the results and comparing results across studies.
There is no consensus regarding what perspective is most appropriate to report; often studies will report two or more
perspectives to aid comparability among studies.
methods of measuring
costs

• The two main methods of measuring costs are by

• Patient- reported instruments, such as a log-book or

a questionnaire

• Extracting data from administrative databases

Cost updating
• Discounting rate -3%

• CPI index

(5600/5000 ) *100 = 112

Types of economic evaluation
• Each has its useful place, but CUA has the
advantage of a common unit of effect
(QALYs, or less commonly DALYs,
disability-adjusted life years) that is
comparable across diseases and settings,
and thus are the most commonly seen in
the clinical literature; this article will
focus on CUAs.
 Amlodipine p=Pfiizer Amlodipine Siam Amlodipine Berlin

Reduce blood presser Reduce blood presser Reduce blood presser

 Telmesatan Ramipril amlodipine

Reduce blood presser Reduce blood presser Reduce blood presser

CUA
• The basic unit of health effect is known as Utility
• Utility is typically derived from health states captured using a quality
of life survey instruments
• such as the SF-12, SF-36, EQ-5D, HUI3, AQol-8D, 15D, or QWB.
• Utility can theoretically have a negative value, for health states are
considered worse than death

• Utility can theoretically have a negative value, for health states are
considered worse than death

• It reflects how the patient feels about his or her health at any
particular point in time.
• The utilities are then summed over the time spent in each given
health state to calculate quality-adjusted life years, or QALYs.
Quality-adjusted life years, or QALYs

• One QALY is equivalent to one year spent in perfect health (or at least self-perceived full health).

The person with chronic disease in which they experience utility of 0.5, will experience 1⁄2 a
QALY in one year, and 1 QALY over two years.
Quality-adjusted life years, or QALYs
• QALYs are thus a measure of the total amount of (quality-adjusted) health experienced by an individual over a
period of time; so even though utility is measured on a scale from 0 to 1, the QALYs reported in a given
study can range from 0 (or potentially negative) to the length of follow-up (in years)

• utility can also be defined in terms of disability (the basis of disability-adjusted life years, DALYs), and used in
economic evaluation in terms of, e.g. cost per DALY averted.
Timeframe

• longer the timeframe, the greater the time available for possible the
costs and effects to accumulate.
• Time horizon is therefore crucial to the interpretation of an economic
evaluation: if the intervention has very large up-front costs and a
very long period of effect (joint replacement surgery, for example)

• short (e.g. six-month) time horizon will not show very favour- able
cost-effectiveness, whereas a long time horizon (e.g. 15 years, or
lifetime) is much more likely
Interpreting the results of an economic evaluation study

• Incremental cost-effectiveness ratio, or ICER.

• This is typically the net input costs (in monetary units) to achieve each unit of effect; for
CUAs that unit is QALYs
• negative ICER would be a good thing, as it would imply a cost-saving paired with an effect
gain,
• but that assumption can be a trap
• As the ICER is a ratio, it can become negative if either the numerator (net input cost) or
the denominator (QALYs) is negative
 WTP is specific to each context – e.g. a national
health system may have a stated or widely-
accepted willingness- to-pay threshold
(US$100,000 in the USA
GBP£20,000– 30,000 in the UK).
THB 160,000 in Thailand

The WHO guidance indicates that a cost-per-QALY of less than 1

× GDP is considered highly cost-effective,
between 1×and 2×GDP cost-effective, and more than 3×GDP
not cost-effective, but caution these rough guides are not
intended to be used in country-level decision-making.

Recent evidence suggests lower thresholds

(often around 0.5×GDP may also be valuable to
identify more highly cost-effective interventions
– better reflecting the ‘opportunity cost’, or what
is given up by not using the resources on
something else.
Incremental net monetary benefit (INMB, or NMB).
• These are more straightforward to interpret than ICERs, and avoid the ambiguity of what a positive or
negative ICER means.
• The NMB is a unit estimate, calculated by the product of incremental effects (e.g. QALYs gained) and
willingness-to-pay, minus incremental costs.
• Exp : Willingness-to-pay of €30,000 per QALY
• if an intervention results in an average gain of 0.5 QALYs and has net costs of €10,000 per patient
• the NMB would be (0.5 × €30,000) = € 15,000 (the amount we would potentially be willing to pay
for this health gain
• the NMB would be € 15,000 - €10,000 = + € 5,000
• positive NMB means that the treatment is cost-effective at the given willingness-to-pay threshold
 Uncertainty

• Sensitivity analyses are often conducted and presented to show what would be the results
if the range or value of some key data inputs – such as the costs of the intervention or
other cost input

• the cost-effectiveness accept-ability curve (CEAC) is often reported

• The CEAC visualizes the probability that delivering the intervention or program will be cost-
effective across a range of willingness-to-pay thresholds

• Tornado diagram
One way sensitivity : TORNADO diagram
TWO way –sensitivity
 THE COST-UTILITY ANALYSIS OF
OF PROPHYLAXIS CYTOMEGALOVIRUS (CMV)
IN HIGH PREVALENCE OF CMV SEROPOSITIVE

RECIPIENTS IN KIDNEY TRANSPLANTS

NARISA RUENROENGBUN
DOCTOR OF PHILOSOPHY
(CLINICAL EPIDEMIOLOGY)
 Country/ Alternative
Authors/year Target populations Results of economic evaluation Direct costs
Currency strategies
For the 5-year time horizon:
VGC 200 vs 100-day regimen Drug/months/ Functioning graft/
Prophylaxis 100 days
Blumberg et al (2010) ($14,859 /QALYs gained) IPD admit/ Functioning graft/ Acute
USA / US$ High-risk (D+/R-) Pre-emptive 100 days
rejection/ Graft failure + dialysis/
Prophylaxis dominate over Maintenance dialysis
pre-emptive

Drug / DNA PCR monitoring /

Luan et al (2009) Prophylaxis 100 days
Prophylaxis not dominate over Inpatient care /outpatient
Luan et al (2010) USA / US$ Overall Pre-emptive 100 days
pre-emptive medication/Physician/nurse monitor
Luan et al (2011) fee

Drug prophylaxis / PCR quantitation

United States Renal
test / Inpatient care / Patient with a
Data System and Prophylaxis therapy
Organ Procurement USA / US$ High-risk (D+/R-) 6m dominate over 3m functioning transplant/ Patient on
6 m vs 3 m maintenance dialysis
and Transplantation (ICER=-1M/QALYsgained)

Limitations:
1. Lack of consensus in non-high risk (D+/R+) group and low to middle-income countries
2. No one accounted cost of long-term outcome of graft failure and acute rejection over ten years period.
3. Due to there was no study direct comparing cost-effectiveness study of10-year long-term clinical
effectiveness in the real-world practice of prophylaxis and the pre-emptive approach in high-risk (D+/R-) and
none-high-risk (D+/R+, D-/R+, D-/R-) groups about its benefit gain compared to expensive cost.

36
Part 2 : Cost Effectiveness analysis Statistic
Study analysis
Design and Patients Population

Study design
A cross-sectional study : Ramathibodi Hospital
This study was conducted between August- December 2021 after approval from
Hospital Ethics Committee.

Study patients
Renal transplant recipients received an operation from Ramathibodi Hospital from 2010 to 2020.

Inclusion criteria Exclusion criteria

• Kidney recipients who age 18-year-old
and elder
• Kidney recipients who still active follow
up at OPD visit at Ramathibodi hospital.
• Kidney recipients experienced graft loss
within 1 month after the transplant.
• Who was not participate in this study.
• Lost follow up more than 4 visits during
receiving CMV prevention strategies in 3-6
months

37
 Part 2 : Cost Effectiveness analysis Statistic
Outcomesanalysis
of Interest

Retrieved form hospital database 2010-2020

Utility score EQ-5D-5L Cost collected as cost per time (visit) per day.

Medication cost/ Cost of prophylaxis, cost of GC, cost

Direct medical of VGC
well status Laboratory data
cost Treatment and Nursing care
cost of transportation / cost of beverages/
Direct non-
Graft loss cost of nursing or caregiver , and uncover
medical cost medication/devices cost.

Opportunity lost from wage loss

CMV infection Indirect cost collected the duration of admission x number
of opportunity losses per day.

Acute graft
rejection

In overall patients, high-risk group (D+/R-) and non- high-risk group (D+/R+, D-/R+, D-/R-).

38
 Part 2 : cost Effectiveness analysis
ICER
Prophylaxis vs no prevention
Pre-emptive vs no prevention
Prophylaxis vs Pre-emptive
Scenario 1. Overall population with actual cost.
Scenario 2. Overall population with predicted full-
cost prophylaxis.
Scenario 3. A subgroup in high-risk group with
actual cost.
Scenario 4. A subgroup in the high-risk group with
predicted full-cost prophylaxis.
In overall patients, high-risk group (D+/R-) and non- high-risk group (D+/R+, D-/R+, D-/R-).
Statistic
Outcomesanalysis
of Interest
Probability of transition state
Well to graft loss
Well to death
Well to death via graft loss
Probability = 1- baseline survival function!(#!$% '(!))*'*!+$)
The counterfactual analysis individual predicted difference
from observation value between each comparison from the
condition that
“ if those patients get intervention 1 (prophylaxis) or 2
(pre-emptive) instead of did not receive any intervention ”
STATA command ‘margin, at((asobs) _all) subpop(if
strategy_prevention ==0) at(strategy_prevention =1)
at(strategy_prevention =2) generate(predicted)’.
39
 Part 2 : cost Effectiveness analysis Statistic
Statisticanalysis
method
outcome Descriptive analysis Conventional analysis Treatment effect model
N(%) chi-square Cox-regression analysis
Or Fisher;exact test Significant covariables at
Mean (S.D.) (p<0.1) in univariate and
Demographics (Anova, T test ) (p<0.05) in multivariate
Median (IQR) analysis
(quantile regression)
Double robust estimator IPWRA (T/O)
Linear regression with Treatment model: multinomial logistic
Significant covariables at regression
Utility Score Mean (S.D) (p<0.1) in univariate and Outcome model : Linear regression
(p<0.05) in multivariate Assumption Checking
analysis 1. Conditional treatment overlap
2. Sufficient treamtent overlab

-Cox-regression to estimate the baseline survival function

Estimate individual treatment effect by counter factual effect base on

X1 No prevention =. Y10 Estimate if Y1 1 receives prophylaxis or pre-emptive

Probability of
Mean(S.D)
transition state X2 Pre-emptive =. Y21 Estimate if Y2 0 receives no treatment or prophylaxis
In overall patients, high-risk group (D+/R-) and non- high-risk group (D+/R+, D-/R+, D-/R-).
Probability = 1- baseline survival function!(#!$% '( !))*'*!+$)

40
Econo
Part 2 : cost Effectiveness analysis Statistic method :Markov

• Markov models were developed based on the nature of post-transplantation

with CMV infection
Overall population High risk (D+/R-)
Prophylaxis vs no prevention
Pre-emptive vs no prevention
1 Prophylaxis vs Pre-emptive

1. ICERs (cost per QALYs gained) were calculated according to a

3 2
societal perspective
2. Model-based one CMV infection rate yearly of transplantation
3. A cycle length of 1 year
4. Future costs and QALYS were discounted at 3% per year.
5. All costs were adjusted to 2021 value using CPI index.

Sensitivity analysis: one-way sensitivity with cost of medication, cost of

prophylaxis or pre-emptive etc. and presented as a Tornado diagram.

41
 Part 2: Cost-Utility Analysis

Result: ICER/QALYs 42
ICER/
Cost Incr Cost
List Scenario Strategy Model QALY Incr QALY QALY gain
(THB) (THB)
(THB)
No
prevention
36,860,396.63 11.07

Pre- Pre-emptive vs
Actual emptive
42,863,518.23 6,003,121.60 8.06 3.01 1,991,334.36
No prevention
cost-
prophylaxis Partial- Prophylaxis vs
prophylaxis
43,231,506.13 6,371,109.50 9.93 1.14 5,572,736.05
1. Overall No prevention
Prophylaxis vs
367,987.90 1.87 196,642.13
Pre-emptive

Full cost- Full- Full-prophylaxis vs

prophylaxis prophylaxis
43,670,485.94 806,967.71 1.87 431,220.28
Pre-emptive

No
prevention
39,735,379.22 11.71

Pre- Pre-emptive vs
emptive 41,860,142.14 2124,762.916 7.23 4.49 473,671.92
Actual No Prevention
cost-
prophylaxis Prophylaxis vs
2. High Risk Prophylaxis 38,577,840.67 -1,157,538.55 8.31 3.40 - 340,452.51
(D+/R-) No Prevention

Prophylaxis vs
-3,282,301.47 1.08 -3,029,256.77
Pre-emptive

Full cost- Full- Full prophylaxis vs

prophylaxis prophylaxis
39,623,224.31 -2,236,917.83 1.08 -2,064,465.60
Pre-emptive
 Part 2: Cost-Utility Analysis

Result: The Tornado diagram of ICER in the overall population

43
Question