PREVENTIVE MEDICINE
EBM 2: ASSESSING ARTICLES ON THERAPY
Dr. Sta. Maria
EBM Moodle Course Book (2020), Activity 2 Discussion PPT (2020), Supplementary videos (Sketchy EBM)
ARTICLES ON THERAPY
INTRODUCTION
• Treatment
o Any intervention that is intended to improve the course of disease
after it is established
o Special case of interventions in general that might be applied at any
point in the natural history of the disease
▪ From disease prevention to palliative care at the end of life
o Usually thought of as medications, surgery, or radiotherapy, but can
take on any form
▪ Ex: relaxation therapy, laser surgery, changes in the
organization and financing of healthcare
• Regardless of the nature of a well-intentioned intervention, the principles
by which it is judged superior to other alternatives are the same
• Comparative Effectiveness
o Head-to-head comparison of two or more interventions
▪ Ex: drugs, devices, tests, surgery, or monitoring (all effective
and current options for care)
• Also important to compare all clinically important results of the
interventions (beneficial and harmful)
o Results can help clinicians and patients understand all
consequences of choosing one course of action over another when
both have been considered reasonable alternatives
• Questions on the effectiveness of therapy are usually phrased like:
o P – the patient population with a certain disease or condition
o E – the exposures (or treatments) to be administered to these
patients
o O - the outcomes (or conditions) that the treatments are intended
to prevent
Experimental Treatment vs. Control Treatment
• Experimental treatment: usually the newer treatment
• Control treatments can either be:
o A passive control with no expected effect (ex: placebo)
o An active control
▪ The treatment considered to be the most effective at the time
of investigation (the standard treatment)
• Superiority Trial
o Trial authors attempt to prove that the experimental treatment is
superior to the control
o New vs. Placebo
• Equivalence Trial
o Trial authors are content to prove that the new and control
treatments are equivalent
o New vs. Standard
• Non-Inferiority Trial
o Some authors set out to prove that a new treatment is at least
equivalent, but mostly superior, to standard therapy
o New vs. Standard
MISLEADING RESULTS
• Research studies attempt to estimate the underlying truth
• Unfortunately, the true impact may never be known
o Studies may be flawed in their design or conduct (systematic errors)
o Even if studies are perfectly designed and executed, it is still
uncertain that the underlying truth was arrived at
• The utility of studies in making clinical decisions is largely determined by
internal and external validity
littlemarmaid
o Internal Validity: the extent to which the design and conduct of the
study minimizes bias
o External Validity: the extent to which the results of a study can be
generalized to the target population/patient
• Bias
o A systematic error that results in incorrect assumptions of the true
effect of an exposure on the outcome of interest (Shi, 2018)
▪ Bias in design, implementation, analysis, and interpretation
can affect the conclusions drawn from the research findings
and impact patient care
o An error in methodology that results in a systematic deviation of
the results away from the “truth” (EBM video)
o Bias can occur before, during, or after (publication stage) the
research period
• Random Error vs. Bias
Random Error Bias
Results are being pulled in all Results are being pulled
directions (random) in one direction
Creates unreliable results Creates misleading results
Widens the confidence interval
BIAS IN ARTICLES OF THERAPY
• For all potential sources of bias, it is important to consider the likely
magnitude and likely direction of the bias
o Ex: if all methodological limitations of studies were expected to bias
the results towards a lack of effect, and the evidence indicates that
the intervention is effective, then it may be concluded that the
intervention is effective even in the presence of these potential
biases.
Selection Bias
• Systematic differences between baseline characteristics of the groups
that are compared
• Randomization
o Prevents selection bias in allocating interventions to participants
• Success depends on fulfilling several interrelated processes
o Sequence Generation
▪ Allocating interventions to participants must be specified,
based on some chance (random) process
▪ Ex: Simple Random Sequence is a suitable method for
assigning interventions
o Allocation Concealment/Allocation Sequence Concealment
▪ Steps must be taken to secure the strict implementation of
that schedule of random assignments by preventing
foreknowledge of the forthcoming allocations
▪ Conceal upcoming allocations from those involved in
enrolment into the trial
Performance Bias
• Systematic differences between groups in the care that is provided, or in
the exposure to factors other than the interventions of interest
• Blinding/Masking of Study Participants
o Participants are prevented from knowing information that may
somehow influence them/alter the results
o Minimizes bias and maximizes the validity of the results
▪ Reduces the risk that the knowledge of which intervention
was received (instead of the intervention itself) will affect the
outcomes
o Ensures that the compared groups receive a similar amount of
attention, ancillary treatment, and diagnostic investigations
1
• Blinding is not always possible APPRAISING DIRECTNESS
o Ex: impossible to blind people as to whether or not major surgery (WILL I READ THE ARTICLE?)
has been undertaken • Done before even reading the article
• How well do the P, E, O of the study (research question) correspond with
Detection Bias the P, E, O of the clinical question?
• Systematic differences between groups in how outcomes are determined • Problem of directness is very common
• Blinding/Masking of Outcome Assessors o Due to limitations in sample size, methodology, and budget,
o Reduce the risk that knowledge of which intervention was received, researchers are constrained to answer questions that may be
rather than the intervention itself, affects outcome measurement significantly different from those asked by healthcare providers
o Especially important for the assessment of subjective outcomes • DECISION POINT: The PEO of the article and the PEO of the question
▪ Ex: degree of post-operative pain (those who know that they need to be the same. If they are, proceed with reading the article
have been given treatment may be more likely to feel pain
control)
APPRAISING VALIDITY
Attrition Bias (WILL I BELIEVE THE ARTICLE?)
VALIDITY
• Systematic differences between groups in withdrawals from a study
• The validity of trials comparing two therapies depends almost entirely on
• Withdrawals lead to incomplete outcome data
the fairness of the comparisons between patients in the treatment and
• Two Reasons for Withdrawals
control groups
o Exclusions: situations where some participants are omitted from
• There are 8 criteria in assessing validity
reports of analyses, despite outcome data being available
o A “yes” answer to a question means that the criterion is satisfied
o Attritions: situations in which outcome data are not available
o The more “yes” answers, the more certain that the comparisons are
fair and that the odds are not stacked in favor of 1 group (bias)
Reporting Bias
• Appraising validity looks into the internal validity of the study
• Systematic differences between reported and unreported findings
• Outcome Reporting Bias/Selective Reporting Bias
Question 1: Were patients randomly assigned to treatment groups?
o “within-study publication bias”
• Random Allocation/Randomization of the assignment of patients to the
o Analyses with statistically significant differences between
experimental or control group
intervention groups are more likely to be reported than non-
o Patients have an equal chance of being assigned to any one of the
significant differences
treatment groups
o May be one of the most substantial biases affecting results from
o Only randomization can create truly comparable groups
individual studies
• A powerful technique, but does not always succeed in creating groups
with similar prognoses
Other Biases
o Investigators may make mistakes that compromise randomization
• Relate mainly to particular trial designs
o Randomization may fail due to simple bad luck
o Carry-over in cross-over trials
▪ Usually happens when there is a small sample size
o Recruitment bias in cluster-randomized trials
• TIP: Look for “randomization”, “randomized”, or “randomly assigned” in the
• Some can be found across a broad spectrum of trials
title, abstract, or methodology
o Contamination: the experimental and control interventions get
“mixed”
Question 2: Was allocation concealed?
▪ Ex: participants pool their drugs
• Randomized Sequence Generation
• Some sources of bias that are only found in a particular clinical setting
o Necessary but not a sufficient safeguard against bias in intervention
allocation
SUMMARY:
o Ineffective if the sequences are not protected by adequate
concealment of the allocation sequence
• Knowledge of the next assignment can cause selective enrolment of
participants based on prognostic factors
o Participants who would have been assigned to an intervention
deemed to be “inappropriate” may be rejected
o Other participants may be deliberately directed to the “appropriate”
intervention
▪ Accomplished by delaying a participant’s entry into the trial
until the next appropriate allocation appears
• Deciphering allocation schedules may occur even if concealment was
attempted
o Many allocation schemes have been deciphered by investigators
because the methods of concealment were inadequate
o Ex: unsealed allocation envelopes may be opened; translucent
envelopes may be held against a bright light to reveal the contents
Sequence Generation; • TIP: To ascertain allocation concealment, look for a phrase or paragraph
Selection Bias
Allocation Sequence Concealment stating one of the following:
Performance Bias Blinding of Participants, Personnel o That a clinician recruiting a patient must contact a remote center
Detection Bias Blinding of Outcome Assessment (either a computer or a person) to obtain a patient treatment
Attrition Bias Incomplete Outcome Data assignment
Reporting Bias Selective Outcome Reporting o That envelopes containing the allocation sequence were opaque,
:) sealed, and numbered
o That packages containing the medications were indistinguishable

littlemarmaid 2
 • TIP: Look for use of identical preparations
Question 3: Were baseline characteristics similar at the start of the trial? o Blinding is more difficult (or even impossible) when the
• Baseline characteristics of treatment groups in an RCT tend to be very interventions involve diet, educational maneuvers, or surgical
similar because of randomization procedures.
o BUT sometimes inequality between treatment groups may arise
due to chance alone Question 5: Were the caregivers blinded to the treatment assignment?
• Reviewing the baseline characteristics allows us to check how successful • When caregivers are aware of the treatment group to which patients are
randomization and allocation concealment were assigned, they may treat the patients in the two groups differently.
o Compare the characteristics of patients within the treatment and o Ex: A clinician who knows his or her patient is on a placebo may
control groups in terms of prognostic factors (factors that could worry and decide to take better care of that patient. A clinician who
affect the outcome of the study) knows a patient is on treatment may decide to monitor more
• TIP: Look for the table of baseline characteristics, usually “Table 1” frequently in case there are any side effects.
o These changes in care may make a treatment appear better or
BLINDING worse than it is.
• An attempt to make the various participants in a study unaware of the • TIP: Look for use of identical preparations.
treatment group that patients have been randomized to o Keep in mind that blinding is not always possible (diet, educational
o Knowledge can cause the participants to act differently and maneuvers, surgical procedures)
diminish the internal validity of the study
• Masking is the more appropriate metaphor, but blinding is the more Question 6: Were outcome assessors blinded to the treatment assignment?
time-honored term • If people who determine outcome measurements are aware of
• Different types of people can be blinded in a clinical trial intervention assignments, bias could be introduced into these
o Participants assessments
▪ Ex: patients or healthy people • Outcome assessments may be made by the participants, healthcare
o Healthcare Providers/Caregivers providers, or by independent assessors
▪ Ex: doctors, nurses, people responsible for care • All outcome assessments can be influenced by lack of blinding, but more
o Outcome Assessors so in trials with subjective outcomes
▪ Primary Data Collectors (ex: interview staff responsible for o Ex: pain, number of days with a common cold
measurement/collection of outcome data) o It is important to consider how subjective or objective an outcome
▪ Secondary Assessors (ex: external outcome adjudication is when considering blinding
committees) • The importance and feasibility of blinding may differ across outcomes
▪ Can also be the participant or healthcare provider themselves within a study
o Data Analysis/Statisticians o Seemingly objective assessments (ex: doctors assessing the degree
o Manuscript Writers of psychological or physical impairment) can also be somewhat
• Tools subjective
Type Tool • TIP: First decide who is assessing a particular outcome: the patient, the
Participants Blinding of Participants caregiver, or an investigator.
Healthcare Providers/Caregivers and Personnel o If it is the patient or caregiver, the tips following Questions #4 and
Blinding of Outcome #5 will apply
Outcome Assessors
Assessment o if investigators are outcome assessors, look for statements
Data Analysts/Statisticians *not explicitly covered indicating that they were “blinded,” that clinical data were withheld,
Manuscript Writers by the tool or that their assessments were independent.

Question 4: Were patients blinded to the treatment assignment? INTENTION TO TREAT AND PER PROTOCOL ANALYSIS
• Lack of blinding of participants or healthcare providers could bias the • ISSUE: should the results of a randomized controlled trial (RCT) be
results by affecting the actual outcomes of the participants in the trial analyzed and presented according to the treatment assigned or according
o Due to lack of expectations in a control group, or to the one the patients received
o Due to differential behaviors across intervention groups
▪ Ex: differential drop-out, differential cross-over to an
alternative intervention, or differential administration of co-
interventions
• In empirical studies, lack of blinding in randomized trials is associated
with more exaggerated estimated intervention effects, by 9% on
average, measured as odds ratio (Pildal 2007)

• Blinding vs. Allocation Concealment

o Often confused with each other, but the two are very different
Allocation Concealment Blinding
Performed BEFORE a patient is
randomized into a treatment Performed AFTER randomization
group
Entails measures to ensure that
Entails measures to ensure that
the allocation sequence is not
the treatments being compared
known by those involved in
are indistinguishable from each
assigning trial participants to
other
treatment

littlemarmaid 3
• Reasons why participants may not get the treatment they were **Some textbooks only have 2 categories of analysis (ITT and per-
assigned/randomized to: protocol). In such cases, per-protocol analysis refers to BOTH:
o Getting the wrong intervention • Analysis where non-compliant participants are excluded
o Participants with adverse effects may decide to stop being • Analysis according to the treatment each patient received, regardless
compliant with the protocol of the group they were assigned to
• Non-compliant participants may create a bias in the data o “As Treated Analysis” is just a more specific term for this
• Intention-to-Treat (ITT) Analysis
o Answers the question: which treatment choice is best at the time the Question 7: Were all patients analyzed in their originally randomized groups?
decision must be made?
• As in the real world, patients in trials may not comply with or adhere to
o Analysis according to which group the patients were
an assigned treatment protocol
assigned/randomized, regardless of whether the patients received
o May happen because they might:
the treatment they were supposed to receive
▪ Forget how to take a drug properly
o Advantages
▪ Suffer intolerable side effects
▪ The question corresponds to the one faced by clinicians (to
▪ Refuse to go on for no apparent reason
offer the treatment or not?)
o A dilemma arises: should they be analyzed in their original
▪ The groups compared are highly randomized, so the
treatment group (ITT analysis) or should they be excluded so that
comparison has the full strength of an RCT
only compliant patients are included (censored/per-protocol
o Disadvantages
analysis)
▪ Because some patients do not receive the treatment they
• Two reasons why non-compliant patients should be analyzed in their
were assigned to, differences in effectiveness will tend to be original groups (ITT Analysis):
obscured
o A patient’s ability to comply is considered part of the performance
▪ Chances of observing a misleadingly small effect/no statistical
(success or failure) of a therapy
effect are increased
▪ Patients stop taking treatment when they don’t feel well
▪ Uncertainty whether the problem is the treatment itself or
▪ Excluding non-compliant patients from the analysis (per-
that the assigned treatment was not received
protocol) can make a treatment appear better than it is
• Per-Protocol/Explanatory Analysis**
▪ An ITT analysis will provide a more conservative measure of
o Answers the question: is the experimental treatment itself better?
the effect of treatment
o Analysis according to the treatment each patient received,
o Censoring non-compliant patients from an analysis may disturb the
regardless of the treatment they were assigned/randomized
balance of baseline characteristics achieved by randomization
o Assess whether taking the treatments, rather than being offered
▪ An ITT analysis preserves this balance and provides the most
them, makes a difference
valid results
o Disadvantages
• Three reasons for using a Per-Protocol Analysis:
▪ The study is no longer an RCT, unless most patients receive
o Unlike policy-makers, patients and their healthcare providers may
the treatments they were assigned
be less interested in whether a treatment will work even if some
▪ It is simply a cohort study
people don’t take it (an ITT analysis) and more interested in
▪ Concern about dissimilarities among groups, other than the
whether it can work if someone does decide to take it (a per-
experimental treatment (confounders)
protocol analysis)
o Must use methods to achieve comparability, like with any non-
o When studying adverse effects, an ITT analysis may underestimate
experimental study
harm
▪ Restriction, matching, stratification, or adjustment
▪ Includes people who stopped taking the treatments and
• In general…
therefore have a lower chance of experiencing adverse events
Intention-to-Treat Explanatory/Per-Protocol o When studying non-inferiority (i.e., whether a new treatment is at
More relevant to effectiveness Consistent with the purposes of least equivalent, but possibly superior, to standard therapy), an ITT
questions efficacy trials analysis favors equivalence (no difference)
Usually used for primary analysis ▪ Includes people who stopped taking the treatment
Both are reported ▪ Makes the treatments look equal and thus favor a non-inferior
Both are legitimate, with the right one result
depending on the question being asked
Will give similar results when patients in a trial follow the treatment REMEMBER:
to which they were assigned/randomized The ITT analysis should always be sought
• Most valid
ADDITIONAL NOTES (from the video): • Usually yields the most conservative results
There are 3 ways to deal with the data of non-compliant participants:
• Per-Protocol Analysis** BUT per-protocol analysis should also be used when appropriate
o Only the data from compliant participants will be examined • Appropriate when
o Remove all non-compliant data o Patients want to know whether a drug works
• As-Treated Analysis** o Looking for adverse events
o Examines participants’ data in the group they actually o Studies attempt to prove equivalence or non-inferiority
conformed to, regardless of the group they were randomized to
• Intention-to-Treat Analysis Usually the results should be the same under either analysis. When they
o All the participants’ data are kept in the group that they were differ, the results should be regarded with some doubt.
initially assigned to by the randomization process

Please watch for visuals:

https://www.youtube.com/watch?v=Kps3VzbykFQ&feature=emb_title

littlemarmaid 4
• TIP: To make sure that investigators included non-compliant patients in Attrition Bias
an analysis, look for the term “intention-to-treat” under the analysis • The overall effect of people being lost-to-follow-up
section of the article. • Occurs when there is non-random loss-to-follow-up in one group or
o If not stated explicitly, look for some indication that patients another
retained their original assignment even if they were non-compliant
o If this also is not stated, as a last resort, check that the number of BEWARE IF:
patients randomized is equal to the number of patients analyzed at • Lost-to-follow-up rates > effect size
the end of the study. This suggests (but does not guarantee) that all • Lost-to-follow-up rate is much higher in one group than the other
patients were included in the analysis
Please watch for visuals:
Question 8: Was the follow-up rate adequate? https://www.youtube.com/watch?v=dMfC-SSBZi0&feature=emb_title
• Adequacy of Follow-Up
o Refers to the minimization of the number of patients who drop-out • TIP: Drop-out rates should be shown in diagrams (Consort diagram) that
of the study depict the flow of patients from recruitment to the end of the study
o Drop-outs are NOT THE SAME non-compliant patients o OR at the very least, they should be mentioned in the manuscript
▪ Non-Compliant Patients: stop adhering to treatment, but itself
researchers can still determine what happened to them (they o If they cannot be found in either place, check whether the number
can still be included in the analysis) of patients randomized is equal to the number of patients analyzed
▪ Drop-Outs: leave the study and are lost to follow-up (no data at the end of the study
on their outcomes) ▪ If fewer patients were analyzed, you will need to decide
o Drop-outs usually leave the study because of adverse events or whether this is due to per-protocol exclusions (the lesser evil)
dissatisfaction or drop-outs (the greater evil)
o The greater number of patients lost to follow-up, the more the
study validity is threatened • DECISION POINT: Will we believe the article?
• Sensitivity Analysis o If all criteria have been fulfilled, it is reasonable to say that there is a
o Determines if there is a significant drop-out low risk of bias in the study
o Assumes the worst- and best-case scenario o Can move on to appraising the results
▪ Tries to find out whether the conclusions are sensitive to the
extreme scenarios
▪ If conclusions differ in these scenarios, the conclusion is “soft” APPRAISING RESULTS
▪ If the conclusions remain the same, it is “robust” QUESTION 1: HOW LARGE WAS THE TREATMENT EFFECT?
• The magnitude of the treatment effect may be expressed by comparing
ADDITIONAL NOTES (from the video): outcomes in the treatment and control groups
• Types of Outcomes
Missing participants really interfere with making any conclusions about
o Dichotomous
the study’s results:
▪ There are only 2 possible results
• The missing participant may have had a good outcome
▪ Usually summarized as proportions or rates
o Could make the experimental group better than the control
o Continuous
group
▪ There is a wide range of possible results
• The missing participant may not have had a good outcome
▪ Summarized as means
o Could make the experimental group equivalent to the control
group (not better, not worse)
Risk Ratios
o Other factors (cost, compliance, adverse effects) will have to
come into play in order to make a decision • Variable: Dichotomous
• The missing participant could have had a horrible adverse effect • Summary of Results Within Each Group: Proportion
o Could make the control group the better choice
Diseased Not Diseased Total
How to find out if participants went missing: A+B
Treatment A B
• Look for a consort diagram Total Exposed
o Shows the number of people approached for the study, those C+D
Control C D
who were included/excluded, the number of people randomized Total Unexposed
into each group, and what happened to them throughout the B+D
A+C N(A+B+C+D)
study Total Total Non-
Total Diseased Total Participants
Diseased
If participants were lost to follow-up, think about what the results would
look like if the missing participants had a good outcome, not a good • Risk in the Treatment (Rt) or EER = A/(A+B)
outcome, a bad outcome • Risk in the Control (Rc) or CER = C/(C+D)
• Relative Risk (RR) = Rt/Rc
Is there an acceptable lost-to-follow-up level? • Absolute Risk Reduction (ARR) = Rc – Rt (expressed in %)
• Some people say <5% is good, and >20% is bad • Relative Risk Reduction (RRR) = (Rc – Rt)/Rc
• BUT the problem is that the small percentage of participants who = ARR/Rc
went missing may have had a horrible outcome/death, and that = 1-RR
would change the results dramatically • Interpretation
• No level of lost-to-follow-up is reassuring o Assumes that the outcomes reported are bad (ex: death) rather than
• Each study has to be looked at specifically for this question good (ex: survival)
o If the reported outcome is good, interpretations are reversed

littlemarmaid 5
 o Interpretation for hazard ratio is the same as the interpretation of QUESTION2: HOW PRECISE WAS THE ESTIMATE OF THE TREATMENT EFFECT?
relative risk • Results for both individual studies and meta-analyses are reported with a
Comparison of Results Interpretation point estimate and an associated confidence interval
Between 2 Groups (Outcome = Bad) o Point Estimate
<1: Treatment is beneficial ▪ The best estimate of the magnitude and direction of the
Relative Risk, Hazard Ratio 1: No effect experimental intervention’s effect compared with the
>1: Treatment is harmful comparator intervention
>0%: Treatment is beneficial o Confidence Interval
Relative Risk Reduction, ▪ Describes the uncertainty inherent in any estimate
0%: No effect
Absolute Risk Reduction ▪ A range of values within which we are reasonably sure the
<0%: Treatment is harmful
true effect lies
ADDITIONAL NOTES (from the video): o Ex: ‘The odds ratio was 0.75 with a 95% confidence interval of 0.70
to 0.80’
• In EBM, risk means probability, not harm
▪ The point estimate is 0.75
• Absolute Risk Reduction
▪ The confidence interval is 0.70-0.80
o Answers the question: what proportion of participants benefited
from the experimental intervention?
Width of the Confidence Interval
o Usually the best way to look at risk
• If the confidence interval is relatively narrow (ex: 0.70-0.80), the effect
• Relative Risk
size is known precisely
o Answers the question: what proportion of participants who were
affected in the control group would still be affected in the • If the interval is wider (0.60-0.93), the uncertainty is greater
experimental group? o BUT can still have enough precision to make decisions about the
• Relative Risk Reduction utility of the intervention
o Answers the question: what proportion of affected individuals in • Very wide intervals (0.50-1.10) indicate little knowledge about the effect
the control group would benefit from being in the experimental o This imprecision affects the certainty in the evidence
group? o Further information is needed to draw a more certain conclusion
o Be skeptical if only the RRR is reported
Interpreting a 95% Confidence Interval
Please watch for visuals: • Loose interpretation: Indicates a range within which we can be 95%
https://www.youtube.com/watch?v=377DQHYka60&feature=emb_title certain that the true effect lies
• Strict interpretation is based on the hypothetical notion of considering
Survival Curve/Kaplan Meier-Curve the results that would be obtained if the study were repeated many
times
• Variable: Dichotomous
o If a study were repeated infinitely often, and on each occasion, a
• Summary of Results Within Each Group: Rate
95% confidence interval was calculated, then 95% of these intervals
would contain the true effect

Factors that affect the width of the confidence interval

• Sample size
o The width of the confidence interval is largely dependent on the
sample size of the study
o Larger studies tend to give more precise estimates of effects, and
have narrower confidence intervals
• Variability of outcome measurements
o For continuous outcomes
o How widely individual results vary between people in the study
o Measured as standard deviation
• Risk of the event
o For dichotomous outcomes
o More frequent events allow more precision, and narrower
• Survival Curve
confidence intervals
o Graphic illustration of the difference in rates in the control group
• The number of events observed
and the treatment group
o For time-to-event outcomes
• Hazard Ratio
• All these quantities are used in the computation of standard errors of
o A comparison between 2 rates
effect estimates
o Hazard Ratio (HR) = rate in the treatment/rate in the control
o Where the confidence interval is derived

Mean Difference
Interpreting Confidence Intervals
• Variable: Continuous
• Superior
• Mean Difference = mean in control – mean in treatment
o Both ends of the CI are on the side of benefit
o The treatment is beneficial
• Inferior
o Both ends of the CI are on the side of harm
o The treatment is harmful
• Inconclusive
o One end reflects important benefit, and the other end reflects
important harm

littlemarmaid 6
• Equivalent
o When one end reflects a small/unimportant benefit, and the other
end reflects a small/unimportant harm
o The two treatments are the same or equivalent
• Non-Inferior
o When one end reflects small/unimportant harm, and the other end
reflect benefit
o The treatment is at least as good or even better than the control
treatment
o Make sure that the boundaries for unimportant harm were set
BEFORE the study started
▪ These boundaries are called non-inferiority margins
• DECISION POINT: Does it work?
ASSESSING APPLICABILITY
SCRAPS
• Decisions about applicability are provided by clinical trials
o Check if patients’ characteristics satisfy the inclusion and exclusion
criteria
o Look at treatment effects in subgroups of participants that more
closely approximate the individual patient (subgroup analysis)
• SCRAPS: Sex, Comorbidity, Race, Age, Pathology, Socioeconomic factors
Sex
• Consider physiological, hormonal, or biochemical differences between
sexes that might affect the RRR of an intervention
• Ex: women have a greater reduction in stroke incidence compared to
men when treated with aspirin.
Comorbidities
• Consider coexistent conditions that could affect applicability
• Ex: Studies show that response to measles vaccination is reduced in
malnourished children
Race
• Consider racial differences that might affect applicability
• Examples:
o Black hypertensives are more responsive to diuretics than whites
o East Asians are more likely than whites to develop the adverse
effect of cough from angiotensin-converting enzyme (ACE)
inhibitors
Pathology
• Consider differences in the actual diseases under study
• At times, diseases with the same name are conditions with slightly
different pathology
o Can lead to significant variations in response to treatment
o Ex: Malaria in Zimbabwe is different from malaria in the Philippines
▪ Manifested as a difference in treatment response
• Also includes the difference in the severity of the disease
Socioeconomic Factors
• Most trials are carried out in ideal conditions (efficiency trial)
o Difficult to apply to everyday life
• Problem of provider compliance and of patient compliance
INDIVIDUALIZING THE RESULT
• The next step after ensuring that the biologic and socioeconomic factors
will not compromise the effectiveness
• Estimate the benefits, risks, and costs to the individual patient
littlemarmaid
• While studies report the effectiveness of a treatment in a population as a
whole, the benefits, risks, and costs vary greatly from patient to patient
o This variability affects the ARR to a greater extent than the RRR
(which, as we previously stated, is a very stable statistic)
o The main source of variation is the patient’s baseline risk
▪ Aka the risk for an event when the patient does not receive
treatment
o Variation in baseline risk is very common in medicine
▪ Patients may have mild, moderate, or severe forms of the
same disease
▪ They may have variable combinations of prognostic factors for
an adverse outcome
▪ Some individuals may come in early and others late in the
course of an illness
ADDITIONAL NOTES (from the video):
Number Needed to Treat (NNT)
• Number of patients needed to be exposed to an intervention to
affect positively 1 patient
• NNT = 1/ARR
• Interpretation:
o Ex: NNT = 3
o For every 3 patients treated, 1 benefits from the treatment
o Only tells you who will benefit, not what the outcomes are for
the other patients
• Application: What is a good NNT?
o In general, lower NNTs are better than higher NNTs
o BUT keep in mind the benefits of the study, the harms the
patient is exposed to, and the cost of the treatment
Please watch:
https://www.youtube.com/watch?v=SdtNJeB2i60&feature=emb_title
SHARING DECISION
• The information about the impact of the treatment on the specific
individual can be used to help patients reach a decision to treat or not to
treat
• Healthcare providers tend to make decisions for patients most of the
time
o Acceptable for treatments where the evidence is clear, the benefits
far outweigh the harm, and cost is not a problem
o Involve patients in the decision-making process when evidence is
unclear, the difference between benefit and harm is a toss-up, or
cost is potentially prohibitive
▪ Provide information in a clear and concise, but balanced
manner
• Look at the adverse effects, benefits, risks, cost
• Discuss with the patient and let them decide if they will agree to the
treatment
7
 CLINICAL CASE SCENARIO • Q2: Was allocation concealed? Yes
Matilda is a 50-year-old Female, known Diabetic (Type 2 Diabetes) for 5 years. She
is on Basal Insulin at 15 units once a day and Metformin 500mg/tab, 1 tab twice a
day. She is also hypertensive with previous history of myocardial infarction a
month ago. No other comorbid conditions. Other current medications are:
Atorvastatin 40 mg/tab 1 tab once a day in the evening, Aspirin 80 mg/tab 1 tab
once a day after lunch, Verapamil (Isoptin) 180 mg SR tab once a day, ISDN
(Isordil) 5 mg/tab 1 tab SL as needed for angina. She came in for her regular follow • Q3: Were baseline characteristics similar at the start of the trial? Yes
up. She had no subjective complaint except for nocturia and polydipsia. Her blood
pressure was 130/80, HR 88 bpm, RR 18 cpm, afebrile, BMI of 23 kg/m2. Other
physical examination findings were unremarkable. Laboratory result showed a
HbA1C of 8%, FBS 200 mg/dl, Creatine of 120 umol/L (eGFR 45 mL/min/1.73m2),
TC 150 mg/dl, LDL 80 mg/dl, HDL 50 mg/dl, SGPT 40 U/L. Matilda’s family o Plus, baseline characteristics table in supplementary file (section 1)
physician, Dr Sims, was thinking of giving her another hypoglycemic drug. Dr. Sims
heard from a round table discussion he attended that SGLT 2 inhibitors particularly • Q4: Were patients blinded to treatment assignment? Yes
Empagliflozin can prevent cardiovascular events (myocardial infarction and stroke) • Q5: Were caregivers blinded to treatment assignment? Yes
and cardiovascular death. Wanting to know the risk and benefit of this drug, he
searched an article which he can use to discuss to Matilda.

• Focused Clinical Question

o Among patients with Type 2 Diabetes Mellitus with previous myocardial o Double-blind pertains to the blinding of participants and personnel
infarction (P), how effective is Empagliflozin (E) compared to standard (caregivers)
treatment/placebo (C), in preventing cardiovascular events and death
(O)? • Q6: Were outcome assessors blinded to treatment assignment? Yes
• PECO of the Clinical Question
P (Population) Patients w/ Type 2 DM w/ previous MI
E (Exposure) Empagliflozin
C (Comparator) Standard treatment/placebo
O (Outcome) Preventing cardiovascular events and death

File Links: Article, Supplementary File, Study Protocol

• Q7: Were all patients analyzed in the groups to which they were
APPRAISING DIRECTNESS originally randomized? Yes
Question Article
• Eligible patients with type 2 diabetes
were adults (≥18 years of age)
• Body-mass index of 45 or less
Adult patient with • Estimated glomerular filtration rate
Type 2 Diabetes (eGFR) of at least 30 ml per minute
P
Mellitus with previous per 1.73 m2
Myocardial infarction • With established cardiovascular
disease • Q8: Was the follow-up rate adequate? Yes
• Glycated hemoglobin level of at least o Consort Diagram from the supplementary file:
7.0% and no more than 10.0%
E Empagliflozin Empagliflozin
Standard
C Placebo
Therapy/Placebo
• Primary Outcome: composite of death
from cardiovascular causes, nonfatal
myocardial infarction, or nonfatal
Cardiovascular Events
O stroke
and Death
• Secondary Outcome: composite of
the primary outcome plus
hospitalization for unstable angina
o Using Dr. Reandelar’s Calculator:
APPRASING VALIDITY
• Q1: Were patients randomly assigned to treatment groups? Yes

o Therefore: assuming worst- and best-case scenario, both the

control and treatment groups have the same conclusion
▪ Drop-outs were not many
▪ Follow-up is adequate

littlemarmaid 8
APPRAISING RESULTS HbA1C: 8% Inclusion: received stable glucose-
lowering therapy for at least 12
weeks before randomization

Patient is on
Pathology and had a glycated hemoglobin level
basal insulin
of at least 7.0% and no more than
and metformin
10.0%.

Subgroup: beneficial
Socioeconomic
N/A N/A
Factors

• Subgroup Analysis:

• Point-Estimates of the Study

o Hazard Ratio = 0.86
▪ Treatment is beneficial
▪ Treatment effect is on the side of empagliflozin
o ARR = 6.5%
o NNT = 16
• 95% Confidence Interval: 0.74-0.99
o Lower limit = 0.74
o Upper limit = 0.99
o Both are on the side of benefit
o Safe to say that empagliflozin is superior to the placebo or standard
therapy in terms of the primary outcome (preventing death from
cardiovascular causes, non-fatal stroke, non-fatal MI)

ASSESSING APPLICABILITY
Individualizing the Result
Patient Article
• Step 1: suppose based on my clinical intuition, a patient like Matilda has a
Both male and female were
baseline risk of 17% per year
included
Sex Female • Step 2: RR = 0.86
Subgroup: both males and females • Step 3: Rt = 14.6%
inconclusive result o Rt = Rc x RR = 17 x 0.86 = 14.6%
Inclusion: BMI: 45 or less • Step 4: ARR = 2.4%
o ARR = Rc – Rt = 17% - 14.6% = 2.4%
BMI: 23 kg/m2 Subgroup: BMI of less than 30 was • Step 5: NNT = 42
found to have beneficial efffect o NNT = 100/2.4 = 42
Inclusion: patients with established o Need to give empagliflozin to 42 patients with a baseline risk of
Co-Morbidity Previous MI cardiovascular disease 17% per year to prevent 1 patient from having the primary
Subgroup: inconclusive results outcome
Inclusion: at least 30 ml per minute
per 1.73 m2 Sharing Decision
eGFR: 45/4
• Adverse Effects of the Treatment (Genital Infection)
Subgroup: inconclusive
o Rc = 1.8%; Rt = 6.4%
Included Asians o RR = 3.5
Race Asian
Subgroup: beneficial o ARR = -4.8
Inclusion: >18 years of age ▪ Risk increase instead of risk reduction (the result is negative)
Age 50 years old o ARI = 4.8%
Subgroup: inconclusive o NNH = 21

littlemarmaid 9

• Benefit:
o The risk of cardiovascular death, nonfatal stroke or nonfatal MI falls
from 12.1 to 2.4% per year
• Risk:
o The risk for genital infection increases from 1.8 to 4.8%
• Cost:
o Matilda will spend 18,000 pesos per year, you need to give it for
3.1 years to have the benefit. hence you will spend approximately
54,000 pesos to gain the benefit of taking empagliflozin.
• Let Matilda decide if she will agree to receive empagliflozin as an
additional drug to her current medicines for T2 DM. Then if Matilda do
not agree offer her another alternative.
ADDITIONAL NOTES FROM ACTIVITY 2 DISCUSSION:
APPRAISING VALIDITY
Were patients randomly assigned to treatment groups?
• Answer ‘Yes’ if a random component was used in the sequence
generation process.
o Examples:
▪ Computer-generated random numbers
▪ Reference to a random number table
▪ Coin tossing
▪ Shuffling cards or envelopes
▪ Throwing dice
▪ Drawing lots.
o Minimization is generally implemented with a random element (at
least when the scores are equal), so an allocation sequence that is
generated using minimization should generally be considered to be
random
• Answer ‘No’ if no random element was used in generating the allocation
sequence or the sequence is predictable.
o Examples:
▪ Alternation
▪ Methods based on dates (of birth or admission)
▪ Patient record numbers
▪ Allocation decisions made by clinicians or participants
▪ Allocation based on the availability of the intervention
▪ Any other systematic or haphazard method
• Answer ‘No information’ if the only information about randomization
methods is a statement that the study is randomized.
• In some situations, a judgement may be made to answer ‘Probably no’ or
‘Probably yes’
o Example:
The context of a large trial run by an experienced clinical trials unit,
absence of specific information about generation of the
randomization sequence, in a paper published in a journal with
rigorously enforced word count limits, is likely to result in a
littlemarmaid
response of ‘Probably yes’ rather than ‘No information’.
Alternatively, if other (contemporary) trials by the same investigator
team have clearly used non-random sequences, it might be
reasonable to assume that the current study was done using similar
methods.
Was allocation concealed?
• Answer ‘Yes’ if:
o The trial used any form of remote or centrally administered method
to allocate interventions to participants,
o The process of allocation is controlled by an external unit or
organization,
o The process of allocation is independent of the enrolment
personnel (e.g. independent central pharmacy, telephone or
internet-based randomization service providers).
o Envelopes or drug containers were used appropriately.
▪ Envelopes should be opaque, sequentially numbered, sealed
with a tamper-proof seal and opened only after the envelope
has been irreversibly assigned to the participant
▪ Drug containers should be sequentially numbered and of
identical appearance, and dispensed or administered only after
they have been irreversibly assigned to the participant.
• This level of detail is rarely provided in reports, and a judgement may be
required to justify an answer of ‘Probably yes’ or ‘Probably no’
• Answer ‘No’ if
o There is reason to suspect that the enrolling investigator or the
participant had knowledge of the forthcoming allocation
Were baseline characteristics similar at the start of the trial?
• Note that differences that are compatible with chance do not lead to a
risk of bias. A small number of differences identified as ‘statistically
significant’ at the conventional 0.05 threshold should usually be
considered to be compatible with chance.
• Answer ‘Yes’ if:
o No imbalances are apparent or if any observed imbalances are
compatible with chance.
• Answer ‘No’ if there are imbalances that indicate problems with the
randomization process, including:
o Substantial differences between intervention group sizes, compared
with the intended allocation ratio
o A substantial excess in statistically significant differences in baseline
characteristics between intervention groups, beyond that expected
by chance
o Imbalance in one or more key prognostic factors, or baseline
measures of outcome variables, that is very unlikely to be due to
chance and for which the between-group difference is big enough
to result in bias in the intervention effect estimate.
• Also answer ‘No’ if there are other reasons to suspect that the
randomization process was problematic:
o Excessive similarity in baseline characteristics that is not compatible
with chance.
• Answer ‘No information’ when there is no useful baseline information
available
o Ex: abstracts, or studies that reported only baseline characteristics
of participants in the final analysis).
• The answer to this question should not influence answers to questions 1
or 2
o Example: if the trial has large baseline imbalances, but authors
report adequate randomization methods, questions 1 and 2 should
still be answered on the basis of the reported adequate methods,
and any concerns about the imbalance should be raised in the
answer to the question 3.
Congratulations for making it to the end of this trans!
Don’t forget to take a break!
10