ANALYS CONTAC

MENU
IS T
Data ANALYSIS

ANALYSIS
MAURISH FATIMA (KEMU)

• President Society of Prevention

Advocacy and Research KEMU
(SPARK)
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

The Research World

You after learning meta-

analysis
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

OUTLINE OF SESSION
• Principles of a meta-analysis
• Steps in a meta-analysis
• Presenting your results
 ANALYS CONTAC
MENU
IS T

Study level
● Study A Outcome data Effect
measure
Review level

● Study B Outcome data Effect Effect

measure measure

● Effect
Study C Outcome data
measure
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

What is a meta-
anlaysis?
BRAINSTORMING!!!
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Combines the Estimate an

01 results from two 02 Average or
or more studies common effect
SYSTEMATIC
REVIEW

OPTIONAL
03 PART OF A
META-ANALYSIS

SYSTEMATIC
REVIEW
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

WHY META-
AnAlYSIS?
• Quantify treatment effects and their
uncertainty
• Increase power
• Increase precision
• Explore differences between the results
• Settle controversies from conflicting
studies
• Generate new hypotheses
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

SOFTWARES
YOU NEED!!
• R-package
• STATA
• Revman
• Statsdirect
• Meta-XL
• Open-MetAnalyst
 ANALYS CONTAC META
MENU
IS T ANALYSIS

NOT TO
?
PERFORM
A META-
ANALYSIS ?
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

WHEN NOT TO DO A META-

ANALYSIS
APPLES AND
ORANGES
1 •

•
Each included study must address
the same question
Consider outcomes and
comparisons
Requires your subjective •
Garbage in-
garbage out
A meta-anlaysis is only as good as
judgement

• Combining a broad mix of stuides
answers broad questions
• Answer maybe meaningless and
genuine effects maybe obscured if
studies are too diverse
2
the studies in it
• If studies included are biased then
meta-analysis will be incorrect
• If serious reporting bias is present
then unrepresentative set of
studies will give misleading
results
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Steps in meta-analysis
Step 1 Step 2 Step 3
Identify comparisons to be Identify outcomes to be Collect data from each
made reported and statistics to be relevant study
used

Step 4 Step 5 Step 6

Combine the results to Explore differences between Interpret the results
obtain the summary effect studies
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Selecting COMPARISONS
Hypothetical review: Caffeine for day-time
drowsiness Break your topic 70
into pairwise %
comparisons
Caffeinated
Decaffeinated Use your own
coffee judgement to 30
decide what to
%
group and what
should be a
separate
comparison
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

SELECTING OUTCOMES AND

EFFECT MEASURES MD/
OR SMD
ASLEEP AT THE irritability
END OF TRIAL
Outcome
s
Effect
Measure

rr
For each For each
comparison, outcome, select
select an effect
outcomes measure

Depends on Depends on
data available
in included
studies
data available
in included
studies
HEADACHES
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Common type of outcome data

BINARY
OR
CONTINUO
DICHOTO
US
MOUS

EXAMPLES??
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

TYPEs of effect measures

BINARY/DICHOTOMOUS DATA

RISK RATIO

RISK
ODD RATIO DIFFERENC
E
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

TYPEs of effect measures

CONTINUOUS DATA

MEAN
DIFFERENC
E
STANDARDI
ZED MEAN
DIFFERENC
E
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

WHAT IS Risk???
Number of EVENTS OF
RISK INTEREST
TOTAL NUMBER OF
OBSERVATIONS
EXAMPLES • 1 day of the week is
MONDAY
• What is the probability • There are 7 days in a
that today is Monday? week
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

WHAT ARE ODDS???

PROBABILITY OF
ODDS OCCURRENCE OF
OF THE
AN
PROBABILITY
EVENT
EVENT NOT OCCURRING
EXAMPLES • 1 day of the week is
Monday
• What are the odds • There are 6 days in a
that today is Monday? week when its not
Monday
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

EXAMPLES
In a study conducted at RMU, 20 students
were identified taking SSRIs and 5 of them
developed emotional blunting

Risk NUMBER OF STUDENTS REPORTING

EMOTIONAL
NUMBER BLUNTING
OF STUDENTS
TAKING RISK
SSRISOF
=5/20
=0.25 DEVELOPING
EMOTIONAL
BLUNTING IS 25% IN
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

EXAMPLES
In a study conducted at RMU, 20 students
were identified taking SSRIs and 5 of them
developed emotional blunting

NUMBER OF STUDENTS REPORTING

odd EMOTIONAL BLUNTING
NUMBER OF STUDENTS WHO
=5/15 DIDN’T REPORT EMOTIONAL
=1/3 THE CHANCES OF DEVELOPING
BLUNTING
EMOTIONAL BLUNTING ARE 1/3 to not
having emotional blunting among STUDENTS
TAKING SSRI
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

OLL!!
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Comparison between risk and odd

•

The difference between risk and odds is small

when the event is rare but can be large for
common events
RISK = ODDS/ 1+ODDS
Event Total Risk Odds
ODD = RISK/ 1-RISK 5 100 0.05 0.0526
50 100 0.5 1
95 100 0.95 19
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Relative measures (RISK

RATIO/ODD RATIO)
Risk and odds measure the likelihood of an
event (e.g. of having emotional blunting with
ssri)
In order to compare between groups (e.g.
SMOKER’S vs. NON-SMOKER’S, or in an RCT
patients receiving treatment vs. placebo) we need to
use relative effect measures
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Relative measures (RISK

RATIO/ODD
• RISK RATIO) Group A
(treatment)
RATIO
ALLO Risk Ratio =
WS US Risk in
treatment/Risk
TO in control
COMP
Group B
ARE (Control)

BETW
EEN
GROU
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Relative measures (RISK

RATIO/ODD RATIO)
Risk ratio=risk in Odds ratio = odds in
treatment group/ risk in treatment group/ odds in
control group control group
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Relative measures (RISK

RATIO/ODD RATIO)Event Non-Event Total

Treatment a b a+b

Control c d c+d

Total a+c b+d N

Risk in treatment = a/ a+b

Risk in control = c/c+d
Risk Ratio = Risk in treatment/ Risk in control
=
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Relative measures (RISK

RATIO/ODD RATIO)Event Non-Event Total

Treatment a b a+b

Control c d c+d

Total a+c b+d N

Odds in treatment = a/ b
Odd in control = c/d
Odd Ratio = Odd in treatment/ Odds in control
=
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Relative measures (RISK

RATIO/ODD RATIO)Dead Alive Total

Treatment 10 90 100

Control 14 86 100

Total 24 176 200

Risk in treatment = 10/ 100 = 0.1

Risk in control = 14/100 =0.14
Risk Ratio = 0.1/ 0.14
= 0.71
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Relative measures (RISK

RATIO/ODD RATIO)Dead Alive Total

Treatment 10 90 100

Control 14 86 100

Total 24 176 200

Odd in treatment = 10/ 90 = 0.1

Odd in control = 14/86 =0.16
Odd Ratio = 0.1/ 0.16
= 0.62
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

WHAT IS Risk difference???

RISK = Treatment group risk –control
difference group risk
• A measure easy to understand but clinical interpretation depends
on context
• A treatment reduces the risk of an adverse event by RD= Risk
2%: difference =
• From 70% risk to 68% or from 3% to 1%?
• May give impossible values if applied in different populations
• RD of -10% applied to a population with 7% Control Group Risk
gives –3% Treatment Group Risk
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

DIRECTION OF EFFECTS!!
• Is a risk ratio or odds ratio defined as
treatment over control or control over
treatment?
• Is rd defined as treatment – control or
control minus treatment?

• Make sure you are clear about how you are

defining your effect measure
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

CONTINUOUS DATA
MEAN = MEAN OUTCOME IN
difference INTERVENTION – MEAN
Haemoglobin OUTCOME INFortifie
CONTROLUnfortified group
levels are on d
average 0.5 g/L group
larger in the n 271 269
fortified group Mean Hb 121.0 120.5
compared with
SD 10.1 9.5
the unfortified
group Mean Difference (95% CI) 0.5 (-1.1, 2.2)
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

CONTINUOUS DATA
STANDARDIZE MEAN OUTCOME IN INTERVENTION –
D MEAN = MEAN OUTCOME
STANDARD IN CONTROL
DEVIATION OF
difference OUTCOME AMONG PARTICIPANTS
SMD expresses the size of the intervention effect
relative to the variability observed
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

CONTINUOUS DATA
WHEN TO
USE MD
AND SMD??
same scale??? MEAN
DIFFERENCE!!
different STANDARDIZ
scale??? ED MEAN
DIFFERENCE!!
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

CALCULATING
SUMMARY
• collect a summary statistic from each contributing study
• STATISTICS
how do we bring them together?
• treat as one big study – add intervention & control data?
• breaks randomiZation, will give the wrong answer
• simple average?
• weights all studies equally – some studies closer to the truth
• weighted average
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

WHAT IS THIS???

A FOREST
PLOT!!
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

weighti
ng
studies
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

• more weight to the

studies which give More
more information: participants

more participants,
more events,
narrower confidence More events
interval
calculated using the
effect estimate and
Narrow
its variance Confidence
interval
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

EXAMPLES

STUDY ID CAFFEINATED DECAFFEINATED WEIGHT

Amore-Coffea 2000 10/34

2/31 6.6%

Deliciozza 2004 10/40 9/40

21.9%

Morrocona 1998 3/15 1/17

2.9 %

Norscafe 1998 19/68 9/64

26.4 %
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

HETEROGENEIT
Y
IN META-
ANALYIS
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

?
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

HETEROGENEITY
Heterogeneity in meta-analysis refers
to the variation in study outcomes
between studies
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

HETEROGENITYCLINCAL
heterogeneit
y
TYPES
Statistical heterogeneity
METHODO
LGICAL
heterogenet
y
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Sources of Heterogeneity
Different
study Different
settings outcomes

By different -Length of times

people -On different people
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Dealing with statistical heterogeneity

Test for
Moderate heterogeneity existence of
heterogeneity
25% 75% Cochran’s Q:
based on chi-
square
50% I square is used
High to indicate
Low heterogeneity heterogeneity heterogeneity
 ANALYS CONTAC
MENU
IS T

What to do with heterogeneity?

Check that data 0 0 Don’t run a meta-
are correct 1 2 analysis
Results maybe misleading

EXPLORE
HETEROGENEI 0 0 IGNORE IT
-Fixed effect model ignore
TYanalysis
-subgroup
-Meta-regression
3 4 heterogeneity
-ignoring may mean that an
intervention effect actually doesn’t
exist
Random effects
meta-analysis
0 0 EXCLUDE
Incorporates heterogeneity but is not
a substitution for a thorough 5 6 STudies
Sensitivity analysis
investigation
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

SUBGROUP and meta-

regression
• Observational in nature
• Characteristics should be pre-specified: keep it
minimum
• Conclusion from such analysis should be
interpreted with caution
• Subgroups: splitting all the studies into groups
for comparisons
• Meta-regression: an extension of subgroup
analysis, allows investigation of continuous and
categorical variables
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Assessment of a heterogeneity
from a forest plot
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Significant
Results

DO EXPLAIN
YOUR
HETEROGENEITY!
Heterogeneity
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Fixed effect Random

model effect model

Assumes that ea
ch
has its own uniqu study
e
size and true eff effect
ect size
vary across studie
s

Assumes all studie

s have
the same true eff Conduct, if heter
ect size ogeneity is
present

Conduct if heterog Allows us to addr

eneity is
little or absent heterogeneity tha ess
t ca
readily be explaine nnot
d by
other factors
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Fixed effect model

HETEROG
ENEITY
More A p-value
PRESENT?? that is too
weight to A narrow
large studies small
confiden
ce
interval
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

RANDOM effect model

HETEROG
More
ENEITY
weight to PRESENT?? A larger p-
small A wider value
studies confiden
ce
interval
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Random vs fixed effect model

HETEROG
ENEITY
absent??
The random-effects method and the
fixed-effect method will give
identical results when there is no
heterogeneity among the studies.
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Random effect model

 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

FIXED effect model

 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Fixed effect Random

model effect
For dichotomous or
continuous data
DerSimonian and Laird
inverse-variance
inverse variance
straightforward, general
method

for dichotomous data only

Mantel-Haenszel
Restricted maximum
good with few events likelihood
weighting system depends
on effect measure

Peto
for odds ratios only
good with few events and
small effect sizes (OR close
to 1)
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

“USE RANDOM
EFFECT
MODEL!!”
—SOMEONE
FAMOUS
 ANALYS CONTAC
MENU
IS T

FOREST PLOT
effect of vitamin D supplementation on total cancer mortality
 ANALYS CONTAC
MENU
IS T

FOREST PLOT
effect of vitamin D supplementation on total cancer mortality
 ANALYS CONTAC
MENU
IS T

FOREST PLOT
effect of vitamin D supplementation on total cancer mortality
 ANALYS CONTAC
MENU
IS T

FOREST PLOT
effect of vitamin D supplementation on total cancer mortality
 ANALYS CONTAC
MENU
IS T

FOREST PLOT
effect of vitamin D supplementation on total cancer mortality
 ANALYS CONTAC
MENU
IS T

FOREST PLOT
effect of vitamin D supplementation on total cancer mortality
 ANALYS CONTAC
MENU
IS T

FOREST PLOT
effect of vitamin D supplementation on total cancer mortality
 ANALYS CONTAC
MENU
IS T

FOREST PLOT
effect of vitamin D supplementation on total cancer mortality
 ANALYS CONTAC
MENU
IS T

FOREST PLOT
effect of vitamin D supplementation on total cancer mortality
 ANALYS CONTAC
MENU
IS T

FOREST PLOT
effect of vitamin D supplementation on total cancer mortality
 ANALYS CONTAC
MENU
IS T

FOREST PLot
continuous outcomes
 ANALYS CONTAC
MENU
IS T

FOREST PLot

LOGO
FOR
COCHRA
NE
LIBRARY
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Interpreting confidence intervals

always present estimate with a confidence interval
precision
• point estimate is the best guess of the effect
• CI expresses uncertainty – range of values we can be reasonably sure includes the true effect

significance
• if the CI includes the null value
• rarely means evidence of no effect
• effect cannot be confirmed or refuted by the available evidence
consider what level of change is clinically important
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

Publication bias
Positive studies are more likely to be published
than negative studies
Reasons Positive
-Language barriers Studies
-unwillingness to publish negative findings
-hard to publish negative findings

Negative
Studies
OVER ESTIMATE OF EFFECT SIZE!!!
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

FUNNEL PLOT
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

HOW TO DEAL WITH IT?

FUNNEL PLOT

Funnel plots are scatter plots of the effect size estimate versus the
standard error (or sample size) of each study. They can be used to assess
publication bias and heterogeneity
-A symmetric funnel plot indicates low heterogeneity and no publication
bias.
-Asymmetry can indicate heterogeneity or publication bias.
-Statistical tests can then be used to confirm this.
Number of studies usually have to be >10 for funnel plots to be effective
in quantifying heterogeneity. Studies <10 may benefit from Doi plots
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

HOW TO DEAL WITH IT?

-Egger’s regression test The test of Begg assesses if

-Begg’s rank correlation test
there is a significant correlation
between the ranks of the effect
estimates and the ranks of their
variances. The test of Egger
uses linear regression to assess
the relation between the
standardized effect estimates
and the standard error (SE).
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

FUNNEL PLOT

For total
mortality
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

FUNNEL PLOT

For total
CANCER
INCIDENC
E
 ANALYS CONTAC
MENU
IS T
Data ANALYSIS

THANK YOU!!!! QUESTIONS??