MEDICAL RESEARCH 1

Module 1. 3
INTRODUCTION TO EVIDENCE-BASED MEDICINE
Dr. Mark Vincent Besa, MD, DPAFP| January 22, 2018
OUTLINE
B. We need information on the following:
I. Introduction to Evidence-Based Medicine  THERAPEUTICS
A. Medicine is not an exact science
 DIAGNOSTICS
B. We need information on
 HARM
C. Our information needs are not met
D. What is EBM  PROGNOSIS
E. The EBM triad  GUIDELINES
F. Advantages of EBM
G. Type of study
H. Rationale for appraisal of journals
I. The basics of critical appraisal
J. Relevance
K. Critical appraisal on therapy


OBJECTIVES:
No objectives were given.

REFERENCES:
1. Lecture Notes
2. PPT

Legend:
Remember Previous
Lecturer Book Trans Com
(Exams) Trans

    
C. Our information needs are not met.
I. Introduction to Evidence-Based Medicine
 TEXTBOOKS are out of date by the time they are
A. Medicine is not an exact science published.
 SPECIALISTS are not always available and may not
 Medicine is a dynamic field of knowledge.
have the answers we need.
 What may be considered as true now may not be as it
 Our COLLEAGUES too may not have the answers.
is tomorrow and vice versa.
 Information is disorganized in the internet and are
loaded with tons of UNVALIDATED information.

D. What is EBM?
 The conscientious, explicit, judicious use of current
best evidence in making decisions about the care of
individual patient.
 It means integrating individual clinical expertise with
the best available external clinical evidence from
systematic research.

 It changes from time to time depending on the economy

 Specific treatment needs evident based medicine.

Dr. David L. Sackett
The Father of Evidence-Based Medicine
1934-2015

TRANSCRIBERS || ESTOLANO, JUMAWAN CHECKER || BANTA R.O. 1

 MEDICAL RESEARCH 1 2021A

E. The EBM triad I. The Basics of Critical Appraisal

F. Advantages of EBM J. Relevance

 The very first step in the critical appraisal of a journal
article.
 The content of the journal has to be related to the
clinical case scenario or clinical dilemma.

1. Case from last week:

A 40 year old male patient came in your clinic came
in due to 2 weeks history of bloatedness which
progressed to burning epigastric pain that radiates
to his chest. You diagnosed him of GERD and wrote a
prescription of omeprazole. He now asks you
whether you can give him a tablet called Gaviscon as
his coworker is being treated with that medication and
G. Type of study has claimed to have improved with the medication.

PICOM:
• P - Adult males with GERD
• I – Gaviscon
• C- Omeprazole 40mg
• O - increase cure rates
• M - randomised controlled trial

 In therapy, the best study is “Randomized Control Trials”

then “Cohort” and so on.
 Critical Appraisal for:
1. Therapy
2. Diagnosis
3. Prognosis
4. Harm
5. Systematic review

H. Rationale for appraisal of journals

TRANSCRIBERS ||ESTOLANO, JUMAWAN CHECKER || BANTA R.O. 2


 VERY IMPORTANT NOTE:
If the article is not relevant, stop appraising
and move to the next journal or article.
II.Critical Appraisal on Therapy
Criteria for Appraisal:
1. Relevance
2. Validity questions
3. What are the results?
4. Can the results help my patient?
5. Conclusion
 Validity questions:
1. Was the assignment of treatment randomized?
YES or NO?
2. Were the groups similar at the start of the study?
YES or NO?
3. Was the follow-up complete? YES or NO?
4. Were the patients analyzed to the groups to which
they were originally randomized? YES or NO?
5. Were the subjects blinded to treatment? YES or
NO?
 If you answered 3 YES answers, consider your article
valid.
 If you are strict with the appraisal, you can consider 4
YES answers in order for your article to be valid.
 If your article is VALID you can now proceed with the
results.
 WHAT IS THE TREATMENT EFFECT?
• RISK IN CONTROL (Rc) = death, tx failure,
Adverse effect in control / N patients in
control
Probability of adverse effect or death on those
not given with therapy or given a placebo (water,
fake pill, etc.)
• RISK IN TREATMENT (Rt) = death, tx
failure, adverse effect in treatment / N
patients in treatment
Probability of adverse effect or death on those
given with the treatment or formula, assesses
how safe or unsafe a developing therapy can be
• ABSOLUTE RISK REDUCTION (ARR) =
(Rc-Rt)
The change in the risk of an outcome of a given
activity/therapy in relation to a comparison
treatment (usually control or placebo)
TRANSCRIBERS ||ESTOLANO, JUMAWAN
MEDICAL RESEARCH 1 2021A
• RELATIVE RISK: Rt/Rc
The ratio of the probability of developing adverse
effects/death or other risk in the treatment group
compared to those part if the control group.
Eg. What is the RR of dying or developing
symptoms from Dengvaxia compared to those
kids not given with the vaccine?
• RELATIVE RISK REDUCTION (RRR)=
1-RR
A result which tells by how much the treatment
reduced the risk of bad outcomes relative to the
control group who did not have the treatment.
Eg. What is the RRR of Dengvaxia given that all
recipients were saved from the risk of dengue
while some of participants of the control group
still happened to contract the disease.
 NUMBER NEEDED TO TREAT
• Average number of subjects needed
to be treated to prevent one
additional bad outcome
• NNT= 1/ARR
Computed as the inverse of absolute risk
reduction, wherein it is the number of patients
needed to be treated in order for one of them to
benefit from the therapy compared with the
control group. The closer the value to 1 means
that all or most of those under the treatment
benefited from the study.
 HOW PRECISE WAS THE ESTIMATE OF
TREATMENT EFFECT?
• To decide on precision, look at the 95%
confidence interval
• The closer the value, the more
precise.
95% confidence interval only suggests that results or
guarantee of a result falling into the range of
suggested values or result is 95% accurate and
statistically probable.
• Also, try looking at the p-value.
 P-VALUE: Helps determine the significance of a test.
• if the P-value is <0.05, there is significant
difference between the groups.
• if the P-value is >0.05, there is no significant
difference between the groups.
Significance of result in any form of research is
important since it will suggest that a treatment,
protocol, hypothesis etc. actually worked compared to
that of a positive or negative control.
III. Critical Appraisal on Diagnosis
A. Criteria for Appraisal on Diagnosis
1. Relevance
2. Validity
3. Results
4. Can the result be applied to the clinical scenario?
5. Conclusion
CHECKER || BANTA R.O. 3

B. Decision Analysis line
The critical appraisal of an article on diagnostics start with
the Decision Analysis Line which is a graphical/imaginary
representation of estimate (in percentage) that helps a
physician decide and improve the accuracy of a probable
diagnosis. Values are arbitrarily assigned and may vary due to
different factors like the skill level of the doctor, manifestation
of the patient, rarity of the case, etc.
1. Diagnostic Threshold (DT): arbitrary point at which you
rule out the disease below this point.
○ After this point, you will request for a
diagnostic test.
It is the minimal percentage needed to be satisfied for a
physician to consider whether an ongoing diagnosis/diagnoses
is worthy of consideration that may need for further
reevaluation and testing. An estimate below the DT would
suggest that a particular diagnosis is highly unlikely to occur
and it would save time, effort, and resources.
2. Therapeutic Threshold: arbitrary point at which you decide
to treat beyond this point
○ Below this point, you will request for a
diagnostic test.
Beyond this point, you may not need to request for
additional tests or requests since you as a physician is sold out
in the current diagnosis based on pre-existing lab results and
your assessment on the patient. It would save time and
needless effort and money if further tests will be requested
rather than going on with the mode of treatment since you are
of high confidence with your diagnosis.
3. Pre-Test Probability: probability that the patient has the
disease.
This is your initial assessment whether your patient has a
particular disease or manifestation of a particular pathology
given all the initial evidences and information that you can
gather. This would also guide a physician on which type of
tests or treatment he/she should consider and how probable
the patient has that initial diagnosis.
TRANSCRIBERS ||ESTOLANO, JUMAWAN
MEDICAL RESEARCH 1 2021A
4. Post-Test probability: probability that the patient has the
disease after doing the diagnostic exam.
After all the diagnostic exams, given that the pre-test
probability was within the scope of treatment, a post-test
probability can be assessed. The result can be an increase in
the level of confidence if the result of the diagnostic exam
supports or provide lead to your diagnosis. The result can also
be the same with the pre-test if the results were inconclusive or
may totally alter the course of treatment if the results suggests
that the symptoms were caused by different pathological
source.
C. Validity Questions
1. Was there an independent and blind comparison with a
reference standard? YES or NO?
2. Was an appropriate SPECTRUM of patients/subjects
included in the sample for the study? YES or NO?
3. Was a REFERENCE STANDARD done? YES or NO?
4. Were the methods described sufficient to allow for
replication? YES or NO?
 if you answered 3 YES answers, consider your article
valid
 if you are strict with the appraisal, you can consider 4
YES answers in order for your article to be valid
D. Results
 Look for the LIKELIHOOD RATIO(s).
 Likelihood ratio (LR) indicates by how much does a
given test result increases the pre-test probability of
the disease.
 LR of 1 means that the pre-test probability is the
same after doing the test.
 LR of >1 increases the probability that the disease is
present.
 LR of <1 decreases the probability of a disease
In order to compute for the Likelihood Ratio (LR), you have
to understand 2 very important values:
1. Sensitivity - the proportion of people with the disease in the
screened population who are identified as ill by the screening
test. (When the disease is present, how often does the test
detect it?)
2. Specificity - the proportion of disease-free people who are
so identified by the screening test. (When the disease is
absent, how often does the test provide a negative result?)
CHECKER || BANTA R.O. 4
 MEDICAL RESEARCH 1 2021A

Given 100 participants, 50 of them have cough. A “cough

analyzer” was used and tested and it detected a total of 70 E. Use of Likelihood Ratio and Fagan Nomogram
positives. Of which, 40 really have cough (A) and 30 tested
positive even if they don’t (B). The 10 legit positive with cough
not tested by the analyzer represents group (C) while the
remaining 30 who are cough-free and showed negative for
cough using the analyzer was group (D).

Sensitivity increases when the number of detected

positive (that really is positive for a certain criteria) is high with
minimal false positives. The highest sensitivity rating is 1 since
detecting all real positives with no false positives would result
to an equation of A/A or 1/1 which is 1. Higher sensitivity
means that a test/protocol is able to detect those who are really
positive for the criteria.

Specificity increases when the number of those without

the criteria was labeled as negative and the number of false
positives are very low. Like sensitivity, the highest score is 1.
The higher the sensitivity, the more confident the researcher
can be since it would mean that if the test screened a sample
as negative, it really doesn’t have any of the criteria being
tested. It is specific for the criteria and not otherwise.

Likelihood ratio of a positive test:

○ Probability that the test is (+) among those
with the disease
○ Probability that the test is (+) among those
without the disease
○ LR (+) = Sensitivity/(1-Specificity)
Mark your initial pre-test probability with a dot. Compute for the
LR and then put a dot or marker on the obtained value.
Connect these two dots and form a line extending until the
scale for post-test probability. The value that the line will
intersect will be your Post-test probability.
Some links on the use of Fonogram:
● http://araw.mede.uic.edu/cgi-bin/testcalc.pl

Likelihood ratio of a negative test: ● https://jamaevidence.mhmedical.com/Calculator.aspx

F. Can the Results Help Me in Caring for My Patients

○ Probability that the test is (-) among those
with the disease
IV.Critical Appraisal on Prognosis
○ Probability that the test is (- )among those
A. Criteria for Appraisal on Prognosis
without the disease
1. Relevance
2. Validity
○ LR (-) = (1-Sensitivity)/Specificity
3. Results
4. Can the results help me in patient care?
● It can change the PROBABILITY of the disease after
a diagnostic test is done.
TRANSCRIBERS ||ESTOLANO, JUMAWAN CHECKER || BANTA R.O. 5
 MEDICAL RESEARCH 1 2021A

B. Validity Questions CAN THE RESULTS HELP ME WITH PATIENT CARE?

1. Was there a representative sample of patients without the 1. Are the study patients similar to my own?
outcome at the start of observation?
2. Should I attempt to stop the exposure?
2. Was follow‐up sufficiently long and complete?
VI. Critical Appraisal on Systematic Review or
3. Were the criteria for determining the prognostic factor and Meta-analysis
A. Criteria for Appraisal on Diagnosis
outcome explicit and credible?
1. Relevance
4. Was there adjustment for other prognostic factors? 2. Validity
3. Results
4. Can the results help my patients?
 If you answered 3 YES answers, consider your article
valid.
B. Validity Questions
 If you are strict with the appraisal, you can consider 4
YES answers in order for your article to be valid. 1. Is the review address a focused clinical problem?

2. Is the criteria for searching and selecting articles for

C. Results inclusion and exclusion explicit and credible?
 How large is the chance of the outcome to occur
in a specified period of time? How precise were 3. Was the validity of included studies appraised and the
they? appraisal reproducible?

CAN THE RESULTS HELP ME IN CARING FOR MY
PATIENTS?
1. Are the study patients similar to my own?
 if you answered 3 YES answers, consider your article
valid
 if you are strict with the appraisal, you can consider 4
YES answers in order for your article to be valid

2. Can I use the results to decide on the intervention or C. Results

reassure my patient?
What are the overall results of the systematic review?
V.Critical Appraisal on Harm
In Meta-analysis, the result includes a Forest Plot.
A. Criteria for Appraisal on Harm
1. Relevance
2. Validity
3. Results
4. Can the results help me in patient care?

B. Validity Questions
1. Were there clearly identified comparison groups?

2. Were the exposures and outcomes measured in the same

way in the groups compared?

3. Was follow‐up sufficiently long and complete?

 if you answered 3 YES answers, consider your article

valid
 if you are strict with the appraisal, you can consider 4
YES answers in order for your article to be valid
C. Results

 What is the magnitude of the association between

exposure and outcome? Was the estimate of the
risk precise?

TRANSCRIBERS ||ESTOLANO, JUMAWAN CHECKER || BANTA R.O. 6

 MEDICAL RESEARCH 1 2021A

TRANSCRIBERS ||ESTOLANO, JUMAWAN CHECKER || BANTA R.O. 2