HOW TO APPRAISE THE EFFECTIVENESS OF TREATMENT?
Richard Ronald B. Cacho, MD, FPOGP, FSGOP | Nov, 28, 2020
CASE:
A 50 year old female returns to your clinic for follow up 3 mos after
a negative ovarian biopsy. Her CA-125 was 4.3 (normal 35) and her
ultrasound showed a 30g ovary gland without nodularity or
induration.She has minimal symptoms. (the usual symptoms for
ovarian cancer is non specific GI symptoms). Despite the negative
biopsy the patient is particularly concerned about ovarian cancer
and asked you how to reduce her risk in the future. The patient
mentioned that she read about the trial medication for enlarged
ovaries that could reduce ovarian cancer. You recall that the results
of randomized trial to evaluate the effect of grapefruit on ovarian
risk. You tell the patient that you too have heard the media report
but you would like to review the trial publication before making a
treatment recommendation to her. The patient s very interested in
the appraisal evidence and sets an appointment to see you in 2
weeks
• Objective: to discuss on how to evaluate articles on therapy
that we come across and to provide care that more effective
for the individual patient.
Variables
• Questions on effectiveness of therapy should be phrased in term
of the variables:
P the patient population with a certain disease or condition
❖ Ex. Patients with ovarian cancer, asthma, HTN
E the exposures of treatments to be administered to the
patient
❖ Ex. Patients with HTN (receiving placebo and
receiving experimental ACEi)
❖ Experimental treatment
❖ Control therapy- neutral or active control
O the outcomes or conditions that the treatment are intended
to prevent or promote
❖ Dichotomous- only 2 possible results; proportions
answering YES or NO
❖ Continuous - wide range of possible results; means
and averages. It should be real numbers
APPRAISING DIRECTNESS
❖ Before even reading an article, the first thing we must do is
evaluate directness
❖ Problem of directness is very common
▪ Differences in the types of question asked by
researchers and clinicians according to the population
of interest(P), the exposures or treatments
evaluated(E), and the outcomes monitored(O)
POPULATION OF INTEREST
❖ Research question
▪ Because sample size is limited, researchers need to
restrict their studies to a few, broadly defined
subgroups of patients
▪ It should define the composition of his/her population
▪ Should be specific and a targeted population
▪ Ex.: the effect of treatment in the young and old
❖ Clinical question
▪ Clinicians are often interested in treatment effects in
many different subgroup
▪ Ex.: the effect of treatment in young and old, males and
females, sick and healthy, smokers, and non-smokers,
rich and poor
EXPOSURES EVALUATED
• Research question
➢ Researchers usually evaluated a specific exposure
▪ Ex: drug preparation, surgical technique or educational
strategy
➢ Researchers make inappropriate comparisons
▪ Ex.: high dose of drug VS. low dose of another
• Clinical question
➢ Clinicians are usually interested in variations of the
exposure
▪ Ex.: similar drug belonging to the same class, a similar
surgery strategy using a modified technique, or a
similar educational strategy using a different
educational tool
➢ Clinicians are only interested in fair comparisons
OUTCOME
• Research question
➢ Researchers usually monitor a selected few, easily
measurable outcomes
▪ Ex.: surrogate outcomes such as serum cholesterol
and blood pressure
▪ In OB: surrogate outcomes are birth of the baby,
APGAR score, pediatric aging
➢ Researchers monitor composite outcomes
▪ Ex.: instead of measuring the incidence of death alone,
that monitor the combined incidence of death, stroke
or heart attack
• Clinical question
➢ Clinicians are more interested in important effect that are
sometimes difficult and expensive to measure
▪ Ex.: clinical outcomes such as pain relief, disability,
overall quality of life or mortality
➢ Clinicians are interested in the effect of treatment on
separate outcome because in composite outcomes:
▪ The individual components may not be equal
importance to clinicians
▪ The effect of treatment may not be of the same
magnitude for each component
DEPARTMENT OF OBSTETRICS & GYNECOLOGY
APPRAISING VALIDITY
8 CRITERIA
1. Were patients randomly assigned to treatment groups?
- Have we allocated a population by chance?
- Randomization has a pivotal steps in determining the validity
of the study. It ensures that each subject has the same
probability of being selected for active treatment, protocols
rather than for a controlled treatment or a placebo
- It also allows study results to be generalized to a larger
population interest
- The strengths and weaknesses of each article must be
evaluated independently. If clinical practice
recommendations based on RCT are not available,
physicians may choose to look to non randomized studies
- Causation cannot be established using observation studies
o Ex. Control and case reports
2. Was allocation concealed?
3. Were baseline characteristics similar at the start of the
trial?
- Do we have baseline characteristics of all population?
- Look for the results: Demographics and description of study
participants
- Smaller studies have greater probability of having an
unequal distribution that larger studies
4. Were patients blinded to treatment assignment?
- subjects and researchers should not know which subjects
are assigned to the interventional control group. To reduce
chances of BIAS
- DOUBLE BLINDING → processes of keeping group
assignments concealed from study subjects and
investigators
- Double Blind is NOT possible in ex. Surgical and non
surgical
5. Were caregivers blinded to treatment assignment?
6. Were outcome assessors blinded to treatment
assignment? Our researchers, investigators
7. Were all patients analyzed in the groups to which they
were originally randomized?
All participants should be accounted for, all enrolled
subjects must be accounted for, they have to be part of
the population to be analyze, a large loss to follow up
may lead researchers to present to have bias, so do not
conceal any participants of the study, this principle is
called ‘intention to treat’ allows group randomization to
be preserve
8. Was follow-up rate adequate? 80% and above is a good
follow up rate
CORDIAL, Jonathan Michael | KUNADIA, Pratixa | LUTRANIA, Karen Grace
“You will never always be motivated. You have to learn to be disciplined.”
APPRAISING THE RESULTS
• Que. 1: how large was the effect of treatment?
➢ May be expressed by comparing outcomes in the treatment
and control groups
➢ Continuous variables: a range of possible results
▪ Expressed as the “mean difference”
▪ Mean difference=mean in control group – mean in
treatment group
➢ Dichotomous variables: only 1 or 2 possible results
➢ Dichotomous-population that can be group and be
separated in two distinct groups (male ,female) (Hot, Cold)
• Expressed as proportions or rates
▪ Hazards ratio=common expression comparing 2 rates
• Rate of outcomes on treatment/rate of outcomes
in control
• Similar to relative risk (RR)
Ways of expressing effectiveness
• Que 2: how precise was the estimate of
the treatment effect?
➢ Absolute risk reduction (ARR) Exact values or point
➢ Relative risk reduction (RRR) estimates
➢ Relative risk (RR)
▪ Ex.: warfarin reduces the risk
of stroke in patient with atrial fibrillation by 79% (RRR)
➢ Interval estimate: provides wide range of possible values of
the treatment effect
▪ Estimated at a 95% level of confidence (95% CI)= “we
are 95% sure that the true effect of the treatment lies
within this range.” Confidence interval compare to
point estimate is that you will give a lowerlimit and a
higher limit
▪ Ex.: warfarin reduces the risk of stroke in patients with
AF by 79% [95% CI: 52%, 90%]
DEFINITION OF TERMS
• Number needed to treat (NNT) – the number of patients who
need to be treated with the specified intervention to prevent one
bad outcome or produce one good outcome over the period of
time specified in the study.
2
DEPARTMENT OF OBSTETRICS & GYNECOLOGY
INDIVIDUALIZING THE RESULT

• Next step is to individualize the benefit, risks and costs to

your patient
• Will vary from patient to patient what we called in
clinics individualized type of treatment due to
patient’s baseline risk foe the event you are trying
to prevent

• Patient decision aids are used improve knowledge as well

as simulate participation in decision making
• Audio, video tapes, illustrations, reading
materials

• Confidence interval (CI) – a measure of the

precision of the results of a study
• P value – refers of the probability that any
particular outcome would have arisen by chance.
The smaller the P value the less likely due was by
chance and more likely due to the intervention.
Standard scientific practice, usually deems a P
value of less than 1 in 20 (expressed as P=0.05)
as “statistically significant.” The smaller the P
value the higher the significance.

• Different conclusion that can arise from looking at
confidence intervals:
1. When both ends of the CI are on the side of
benefit, the treatment is definitely beneficial.
2. When both ends of the CI are on the harm, the
treatment is definitely harmful’
3. When one end reflects important benefit and the
other end reflects important harm, then the study
is inconclusive.
4. When one end reflects a small unimportant
benefit and the other end reflects a small
unimportant harm, than for all intents and
purposes the two treatments being compared are
equal.
5 quick steps in using the baseline risk to estimate the effect of therapy
on an individual
1. Estimate you individual patients risk for an event without
treatment (Rc)
2. Estimate the RR using the study results
3. Estimate your individual patients risk for an event with
treatment (Rt)
4. Estimate the individualized absolute risk reduction (ARR)
5. Estimate the individualized number needed to treat (NNT)
or number needed to harm (NNH)
SUMMARY
❖ 1st you need to screen Screening an article

❖ Evaluation of articles on therapy

➢ When confronted with a therapy article:
1. Appraise directness: dose it directly a question
that is important to you?
2. If it dose, appraise validity: to decide the results
are credible
APPRAISING APPLICABILITY 3. If you are satisfied that the study is valid, appraise
the results.
• Next step is to decide if the results can be applied to our 4. Assess applicability
own patients 5. Individualize the results
• Check if patient’s characteristics satisfy the inclusion and
exclusion criteria
• Subgroup analysis
• Because data from subgroups are limited, healthcare
providers must decide on the applicability of trial results to
individual patients- based on the general information
available, consider biologic and socioeconomic issues.

3
CORDIAL, Jonathan Michael | KUNADIA, Pratixa | LUTRANIA, Karen Grace
“You will never always be motivated. You have to learn to be disciplined.”