This action might not be possible to undo. Are you sure you want to continue?
1.Asking the Question 2.Searching the Literature 3.Appraising the Evidence 4.Integrating the Idea 5.Making a Decision
Components Of Critical Appraisal
Is the study valid?
VALIDITY / LACK OF BIAS
What are the results?
SIZE & PRECISION EFFECT
Will the results help me in caring for my patients?
APPLICABILITY & IMPACT
Validity vs Precision
Valid & Precise
Valid but NOT Precise
NOT Valid & NOT Precise
NOT Valid but Precise
User’s Guides .Example VALIDITY Was there an independent blind comparison with a reference standard? Did the patient sample include an appropriate spectrum of patients to whom the diagnostic test will be applied in clinical practice? Did the results of the test being evaluated inﬂuence the decision to perform the reference standard? Were the methods for performing the test described in sufﬁcient detail to permit replication? RESULTS Are likelihood ratios for the test results presented or data necessary for their calculation provided? APPLICABILITY Will the reproducibility of the test result and its interpretation be satisfactory in my setting? Are the results applicable to my patient? Will the results change my management? Will patients be better off as a result of the test? .
Other Available Users’ Guides Article on PROGNOSIS Article on HARM Overview / Meta-Analysis Clinical Practice Guidelines Economic Analysis Clinical Decision Rule ➡ Can be downloaded from websites: EBM Homepage JAMA .
Mariñas. MD. DPBA Clinical Epidemiology .Critical Appraisal: Articles on Therapy / Prevention Glenn D.
Was the assignment of patients to treatments randomized? 2. health workers. Were patients analyzed in the groups to which they were randomized? 4.Are The Results Of The Study VALID? 1. Were the groups similar at the start of the trial? 6. were the groups treated equally? . and study personnel “blind” to treatment? 5. Were all patients who entered the trial properly accounted for and attributed at its conclusion? 3. Were patients. Aside from the experimental intervention.
What Were The RESULTS? 7.How precise was the estimate of the treatment effect? .How large was the treatment effect? 8.
Were all clinically important outcomes considered? 11.Will The Results Help Me In Caring For My Patient? 9.Are the likely treatment beneﬁts worth the potential harms and costs? .Can the results be applied to my patient care? 10.
.. Is the research study an article on THERAPY / PREVENTION? .First Step.
Article On Therapy/Prevention Outcome (+) Disease Treatment (-) Disease Intervention THERAPY: Evaluation of intervention/s used for curing disease / reducing S/Sx PREVENTION: Evaluation of intervention/s used for reducing risk of disease or its complications .
Therapy Experimental group Subjects Control group Follow-up Outcome Randomized .Validity Appraisal .
Randomization “Random allocation” Each subject is given an equal chance of being assigned to either treatment or control group It removes investigator bias in the allocation of subjects It produces study groups comparable with respect to known as well as unknown variables .
Article on Therapy / Prevention Experimental group (T+) Cure rate (%) Similar in all aspects except for Tx status Control group (T-) Cure rate (%) .
Exceptions To Randomization Randomization is NOT possible when: No other known options for treatment Treatment of few subjects reverses uniform adverse outcomes Ethical issues .
Was the assignment of patients to treatments randomized? Answer should be YES. .Critical Appraisal .Therapy 1.
Therapy Experimental group Subjects Control group Follow-up Outcome Complete follow-up .Validity Appraisal .
Incomplete Follow-up Reasons for losses to follow-up / dropouts: Too Ill Already cured Migrated Solution: Assume WORST CASE SCENARIO Worst outcome happened to drop-outs .
8% Worst Case Scenario Assume the 16 lost to follow-up died death rate = 4 + 16 = 20 = 20% 84 + 16 100 .Example .WCS 100 subjects 4 deaths 16 lost to follow-up death rate = 4/84 = 4.
When Is Drop-out Rate Signiﬁcant? Drop-out rate > Differences in outcome rates between the 2 groups “5 and 20” rule of thumb Fewer than 5% loss probably leads to little bias while greater than 20% loss threatens validity! .
100 subjects 4 deaths Death rate= 4/100= 4% Group B.Effect Of Drop-outs Group A.100 subjects 7 deaths Death rate= 7/100= 7% Death rate A < Death rate B Group A.100 subjects 4 deaths 50 lost to ffup (50% DO) Death rate= 4/50= 8% Group B.100 subjects 7 deaths 2 lost to ffup (2% DO) Death rate= 7/98= 7% Death rate A > Death rate B .Example.
Therapy 2.Critical Appraisal . Was follow-up complete? Answer should be YES. .
Therapy Experimental group Subjects Control group Follow-up Outcome ITT .Validity Appraisal .
Even those who did not receive assigned treatment Reasons for “noncompliance” is often related to prognosis REQUIREMENT FOR ITT: Outcomes of “noncompliers” should be known Purpose: To preserve randomization .ITT “Intent-to-treat” analysis “Always analyze once randomized!” DO NOT EXCLUDE FROM DATA ANALYSIS ANY SUBJECTS ONCE RANDOMIZED.
5) Tx n X 1000 0/0 Y 1000 90/100 (90%) .Example of ITT Disease rate (compliers) 100/1000 (10%) 45/900 (5%) Disease rate (noncompliers) Disease rate Total 100/1000 (10%) 135/1000 (13.
Example of ITT Disease X Randomization Surgical Tx ASA Risk 3 excluded Medical Tx ASA Risk 3 included Falsely better outcome for Surgical Tx .
Therapy 3.Critical Appraisal . Were patients analyzed in the groups to which they were randomized? Answer should be YES. .
Therapy Experimental group Subjects Control group Follow-up Outcome Blinding .Validity Appraisal .
Blinding A process wherein the researcher makes sure that the subject. investigator. &/or data collector does not know of the subject’s assigned group Effect of Non-blinding: Favor treatment group over control Overestimate the true effect .
taste and texture but lacking the active ingredient Automatically results in double blinding Considered BEST way of blinding When blinding is NOT possible Use OBJECTIVE outcome measures Survival rate. Death rates .Blinding Placebo Indistinguishable from active treatment in appearance.
.Therapy 4. and study personnel “blind” to treatment? Answer should be YES. Were patients. their clinicians.Critical Appraisal .
Therapy Experimental group Subjects Control group Follow-up Outcome Groups similar .Validity Appraisal .
Were the groups similar at the start of the trial? Answer should be YES. Countercheck randomization Indirectly determines if sample size was adequate If answer is NO: Statistical analysis to adjust for differences Stratiﬁed analysis. regression .Critical Appraisal .Therapy 5.
Effect of Unequal Groups at Baseline GRP A w/ more of “good” prognostic factor GRP B w/ less of “good” prognostic factor Better outcome Worse outcome Overestimate of true effect (in Grp A) .
Validity Appraisal .Therapy Experimental group Subjects Control group Follow-up Outcome Groups treated equally .
were the groups treated equally? Answer should be YES.Critical Appraisal . Countercheck to BLINDING Ensures that outcomes are actually due to the experimental intervention . Aside from the experimental intervention.Therapy 6.
Cointerventions Treatment modalities applied to the subjects other than the treatment being studied Example Group A: MRM Group B: CBS + Chemotherapy COINTERVENTION .
Cointerventions GRP A: MRM Less subjects w/ chemotx GRP B: CBS More subjects w/ chemotx Worse outcome Better outcome Overestimate of true effect (in Grp B) .
Criteria for Validity Blinding Experimental group Subjects Control group Follow-up Outcome Groups treated equally Randomized (Groups similar) Complete follow-up ITT .
Is The Study Valid? Absolutely YES “Yes” to all questions Somewhat YES “Yes” to ﬁrst 2 questions NO “No” to any of the ﬁrst 2 questions Article may not be worth using at all .
.” .Remember.Isaac David Ampil II. MD . “An ounce of critical appraisal is worth pounds of futile reading..
How Large Was The Treatment Effect? Treatment T+ Compare with Non-treatment TOutcome O+ Outcome O+ .
Determining Risks O+ T+ Ta c Ob d Risk in T+ (Exptal grp)= a/a+b Risk of developing the outcome given the exptal tx Risk in T.(Control grp)= c/c+d Risk of developing the outcome if exptal tx is not given .
outcome if tx is not given) . outcome given the tx) ➡ a c / c + d (risk of dev.Comparing The Risks Between Groups O+ T+ Ta c Ob d Relative Risk (RR) / a + b (risk of dev.
RR) x 100% • Complement of RR expressed in % • Easier to interpret clinically than RR .Comparing The Risks Between Groups O+ T+ Ta c Ob d Relative Risk Reduction (RRR) Relative Risk Increase (RRI) ➡ (1 .
2= 0.5 RRR= (1 .Comparing The Risks Between Groups SSI+ Ab+ AbTotal 10 20 30 SSI90 80 170 Total 100 100 200 Risk of SSI w/ Ab= 10/100= 0.5) x 100%= 50% .1 / 0.0.1 Risk of SSI w/o Ab= 20/100= 0.2 RR= 0.
(a / a + b) • (risk of dev outcome w/o tx) .Comparing The Risks Between Groups O+ T+ Ta c Ob d Risk Difference Absolute Risk Reduction (ARR) Absolute Risk Increase (ARI) ➡ (c / c + d) .(risk of dev outcome w/ tx) .
Comparing The Risks Between Groups O+ T+ Ta c Ob d Number Needed to Treat (NNT) ➡ 1 / ARR • Reciprocal of ARR • Easier to interpret clinically than ARR .
Comparing The Risks Between Groups
SSI+ Ab+ AbTotal 10 20 30 SSI90 80 170 Total 100 100 200
Risk of SSI w/ Ab= 10/100= 0.1 Risk of SSI w/o Ab= 20/100= 0.2 ARR= 0.2 - 0.1= 0.1 NNT= 1 / 0.1= 10
Interpreting The Comparisons
If outcome is CURE: RR= Outcome in exptal grp / Outcome in control grp If numerator > denominator, RR>1, tx is beneﬁcial If numerator < denominator, RR<1, tx is harmful If numerator = denominator, RR=1, tx is useless ARR= Outcome in control grp - Outcome in exptal grp If ARR > 0, tx is harmful If ARR < 0, tx is beneﬁcial If ARR = 0, tx has no effect
Interpreting The Comparisons
If outcome is DISEASE: RR= Outcome in exptal grp / Outcome in control grp If numerator > denominator, RR>1, tx is harmful If numerator < denominator, RR<1, tx is beneﬁcial If numerator = denominator, RR=1, tx is useless ARR= Outcome in control grp - Outcome in exptal grp If ARR > 0, tx is beneﬁcial If ARR < 0, tx is harmful If ARR = 0, tx has no effect
1= 0.5) x 100%= 50% ARR= 0.1 Risk of SSI w/o Ab= 20/100= 0.0.5 RRR= (1 .2 .2 RR= 0.1= 10 .2= 0.1 / 0.Summary Of “Results” SSI+ Ab+ AbTotal 10 20 30 SSI90 80 170 Total 100 100 200 Risk of SSI w/ Ab= 10/100= 0.1 NNT= 1 / 0.0.
65 years old Interval estimate is more PRECISE than point estimate.60 years old 70% CI: 45 . .55 years old 99% CI: 15 .How Precise Was The Estimate Of Treatment Effect? Concept of Conﬁdence Interval Point estimate vs Interval estimate. eg: Estimating your age by point estimate Mean age= 48 years old Estimating your age based on interval estimate 95% CI: 32 .
55 y/o (more precise. the less precise but the more conﬁdent you are of the estimate 95% CI: 32 . more conﬁdent) . less conﬁdent) 99% CI: 15 .How Precise Was The Estimate Of Treatment Effect? Conﬁdence Interval Probability that the difference lies between a range of values at a given level of conﬁdence Gives an idea of the precision of the estimate The wider the interval.65 y/o (less precise.60 y/o 70% CI: 45 .
tx is useless RR= 2. tx is harmful If numerator = denominator. RR=1.4.86) 2.87 .86 means the RR could be: <1: harmful >1: beneﬁcial The result is thus NOT STATISTICALLY SIGNIFICANT.4.87 . tx is beneﬁcial If numerator < denominator.12 means tx is 2x more beneﬁcial than control 95% CI of 0. . RR>1.Using CI To Determine Statistical Signiﬁcance RR= Outcome in exptal grp / Outcome in control grp If numerator > denominator. RR<1.12 (95% CI: 0.
RR=1.86) 2. tx is useless RR= 2.98 .12 (95% CI: 1.98 . RR<1. tx is beneﬁcial If numerator < denominator.4.86 means the RR is always beneﬁcial The result is thus STATISTICALLY SIGNIFICANT. tx is harmful If numerator = denominator.12 means tx is 2x more beneﬁcial than control 95% CI of 1.4.Using CI To Determine Statistical Signiﬁcance RR= Outcome in exptal grp / Outcome in control grp If numerator > denominator. . RR>1.
.4.78 .12 (95% CI: -0.Using CI To Determine Statistical Signiﬁcance ARR= Outcome in control grp . tx has no effect ARR= 3.Outcome in exptal grp If ARR > 0.98) The ARR could be: <0: harmful >0: beneﬁcial The result is thus NOT STATISTICALLY SIGNIFICANT. tx is beneﬁcial If ARR < 0. tx is harmful If ARR = 0.
1 .4) .2 (1.2 .8 .34.0.92 .Using CI To Determine Statistical Signiﬁcance Treatment Surgery alone Surgery + postop chemo Surgery + postop chemo/RT Preop chemo + surgery +postop chemo 5 yr survival RR (95% CI) 0.5 (0.1) 3.3 (0.4.4 (0.2 .1) 14.1 (28.9) 22.214.171.124) Preop chemo/RT + surgery +postop chemo 32.
OR): Difference is not sifniﬁcant if the interval crosses 1 For Difference measures (ARR): Difference is not signiﬁcant if the interval crosses 0 .Using CI To Determine Statistical Signiﬁcance For Ratio measures (RR.
Applicability Criteria 9. Can the results be applied to my patient care? Answer should be YES. Patient must meet inclusion criteria and none of the exclusion criteria .
anatomic. QOL Quantity: survival. mortality Mechanistic outcomes vs. Biochemical.Applicability Criteria 10. Clinical outcomes Quality: morbidity. physiologic Serum PSA ECG ST elevation .Were all clinically important outcomes considered? Answer should be YES.
Are The Likely Treatment Beneﬁts Worth The Potential Harm And Costs? Harm & Cost Tx benefit NNT NNH Cost/life saved .
Are The Likely Treatment Beneﬁts Worth The Potential Harm And Costs? NNT= 1 / ARR You need to treat # pxs to produce 1 beneﬁt Low NNT: more beneﬁcial NNH= NNT x %side effects (in decimal) You need to harm # pxs to produce 1 beneﬁt Low NNH: more harmful Cost/life saved= (unit cost x #doses x duration) x NNT You need to spend # cost to produce 1 beneﬁt .
Are The Likely Treatment Beneﬁts Worth The Potential Harm And Costs? Cost-risk beneﬁt Need to treat __ persons to save 1 life at a cost of __ and produce __ adverse effects. Save __ lives at a cost of __ and produce __ adverse effects Example: NNT= 14 NNH= 7 Cost/beneﬁt= P100 You need to treat 14 persons to save 1 life at a cost of P100 but you produce 2 adverse effects .
A. 2008 .Sample C. For Treatment Title: Search Method: Source: P: Sample here I: C: O: M: Sample here Sample here Sample here Sample here Rt Rc RR RRR Randomized? Adequate follow-up? Intention-to-treat? Blinding? Baseline traits similar? Treatment the same? ARR NNT Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Endpoint Outcome1 Outcome2 Author’s conclusion: Reviewer’s conclusion: Reviewer: Juan dela Cruz Date: Feb 21. T.
middle esophagus. mediastinal lymphadenectomy increase survival? Citation: A prospective randomized trial of extended cervical & superior mediastinal lymphadenectomy for carcinoma of the thoracic esophagus.192 <0.ptxs < 70 yrs. Reviewer's conclusion: (soft).18 -0. for curative resection.Treatment Scenario: 60 y/o M w/ T2N0M0 SCC. w/ resectable SCC of thoracic esophagus I.51. Clin: more upper 1/3 in exptal (underest) & more lower 1/3 in exptal (overest).agree w/ authors but it definitely increases postop complcations Resolution of scenario: Ptx. sl. good risk. Hirayama K.43 NNT 6 2 p 0. underwent subtotal esophagectomy w/ 3-field lymphadenectomy but surgeon had to perform tracheostomy on the 7th postop day due to phrenic nerve palsy.RCT Outcome 5 yr death Tracheost. will the addition of cervical & sup. FPCS Date: 07/06/00 . net eff: overest Equal tx? N. Rc 52% 10% Rt 34% 53% RRR 35% -4.001 M. Reviewer's name: Isaac David Ampil II. DO ctrl =23%ctrl(vs 12%) WCS prod. Mori S.65 5. complication rate RCT? Y (1:1) Ff-up adeq? N. MD.3% RR 0.3 ARR 0.Subtotal esophagectomy w/ 2-field lymphadenectomy O. Author's conclusion: Three-field lymphadenectomy appears to prolong survival but the diff was not statistically significant. P.Subtotal esophagectomy w/ 3-field lymphadenectomy C. Am J Surg Vol 175..CAT . Jan 1998: 47. reversal of conclusion (over) ITT? Y Blinding? Y. Nishihira T. recurrence rate. ptx & observer Baseline same? N. p = NS.2 yr & 5 yr survival rates. Same ff-up but more postop RT/chemotx in controlunderestimn. Research Question: Among ptxs w/ resectable SCC of middle esophagus.
” .Remember..Isaac David Ampil II. “An ounce of critical appraisal is worth pounds of futile reading. MD ..
THANK YOU VERY MUCH! .
Group Session .
Histopath reveals adenomatous polyps. .Group Session A 61/M undergoes a colonoscopy as part of an executive check-up. He asks you if there is any way to prevent the recurrence of the polyps. knowing their malignant potential. Two polyps were found in the sigmoid and were removed. You recall having read an article in the New England Journal of Medicine on the use of Celecoxib for the prevention of adenomatous polyps.
This action might not be possible to undo. Are you sure you want to continue?