Critical Appraisal

On Article of Diagnostic Test (EBM(EBM-Diagnostic)

Critical appraisal is one step in the process of evidencebased clinical practice. To determine what is the best evidence, we need critical appraisal skills that will help us to understand the methods and results of research and to assess the quality of the research. Most research is not perfect, and critical appraisal is not an exact science - it will not give us the right answer. But it can help us to decide whether we think a reported piece of research is good enough to be used in decision making.

Critical Appraisal


- Worksheet for critical appraisal - Software : CAT Maker Main area of clinical objectives:  1. Diagnosis  2. Prognosis  3. Therapy/Treatment  4. Risk/Harm Others:  Systematic Review and Meta-analysis Meta Clinical Guidelines  Clinical Decision Making etc.

THREE MAIN ASPECTS TO BE APPRAISED: 1.

VIA

VALIDITY : VALID (CLOSENESS TO THE TRUTH) p IN THE METHODOLOGY SECTION IMPORTANCE : IMPORTANT (USEFULNESS) p IN THE RESULTS SECTION APPLICABILITY : APPLICABLE (CAN BE APPLIED IN CLINICAL PRACTICE) p IN THE DISCUSSION SECTION

2.

3.

DIAGNOSIS WORKSHEET
Citation: Are the results of this diagnostic study valid? Was there an independent, blind comparison with a reference (gold) standard of diagnosis? Was the diagnostic test evaluated in an appropriate spectrum of patients (like those in whom it would be used in practice)? Was the reference standard applied regardless of the diagnostic test result? Was the test (or cluster of tests) validated in a second, independent group of patients?

Are the valid results of this diagnostic study important?

SAMPLE CALCULATIONS
Target disorder (iron deficiency anemia) Present Diagnostic Positive test result (< 65 mmol/L) (serum Negative ferritin) (u 65 mmol/L) Totals 731 a 78 c 809 a+c Absent 270 b 1500 d 1770 b+d 1001 a+b 1578 c+d 2579 a+b+c+d Totals

Sensitivity = a/(a+c) = 731/809 = 90% Specificity = d/(b+d) = 1500/1770 = 85% Likelihood ratio for a positive test result = LR+ = sens/(1-spec) = 90%/15% = 6 Likelihood ratio for a negative test result = LR - = (1-sens)/spec = 10%/85% = 0.12 Positive Predictive Value = a/(a+b) = 731/1001 = 73% Negative Predictive Value = d/(c+d) = 1500/1578 = 95% Pre-test probability (prevalence) = (a+c)/(a+b+c+d) = 809/2579 = 32% Pre-test odds = prevalence/(1-prevalence) = 31%/69% = 0.45 Post-test odds = pre-test odds v LR Post-test probability = post-test odds/(post-test odds +1)

YOUR CALCULATIONS

Totals Target disorder Present Diagnostic test result Positive Negative Totals a c a+c Absent b d b+d a+b c+d a+b+c+d

Can you apply this valid, important evidence about a diagnostic test in caring for your patient?
Is the diagnostic test available, affordable, accurate, and precise in your setting? Can you generate a clinically sensible estimate of your patients pre-test probability (from personal experience, prevalence statistics, practice databases, or primary studies)? yAre the study patients similar to your own? yIs it unlikely that the disease possibilities or probabilities have changed since the evidence was gathered? Will the resulting post-test probabilities affect your management and help your patient? yCould it move you across a test-treatment threshold? yWould your patient be a willing partner in carrying it out? Would the consequences of the test help your patient?

Additional notes:

Example : an article of diagnostic test, entitle :

Bedside Diagnosis of Influenzavirus Infections in Hospitalized Children

Katherine A. Poehling, et al American Academy of Pediatrics

Background:  Influenzavirus has a significant impact on the pediatric population, with school-aged children having the highest infection rates.  For preventing nosocomial influenza infections and to facilitate prompt antiviral therapy, an accessible, rapid diagnostic method for influenzavirus is needed. Objective:  To compare the performance of a rapid diagnostic test (QuickVue Influenza Test; Quidel Corp, San Diego, CA) completed at the bedside of hospitalized children to viral culture and/or polymerase chain reaction (PCR) for influenzavirus.

Method:  Study population: 1) younger than 19 years and hospitalized with respiratory symptoms or 2) younger than 3 years and hospitalized with fever.  Sampel: 1) a primary admission diagnosis of an acute respiratory illness characterized by rhinorrhea, sore throat, cough, shortness of breath, or apnea or 2) a primary admission diagnosis consistent with a febrile illness and a temperature of 100.4F.  Broad inclusion criteria were chosen such that all children who were hospitalized with symptoms potentially related to influenza infections were eligible.

Sample in this study

Study design: prospective, cross sectional study from each child, 2 nasal swabs of the turbinates were obtained1 for influenzavirus culture and PCR and the other for the rapid diagnostic test. The rapid test results were compared with that of culture and PCR for influenzavirus. Influenza infection was defined as any sample with 1) a positive culture for influenzavirus or 2) 2 consecutive positive PCRs for influenza A or B. Each researcher was trained to perform and interpret the rapid diagnostic test at the bedside according to the manufacturers instructions. The laboratory technician who performed the culture and PCR was masked to the rapid diagnostic test results.

Results:

Sens: 74% Spec: 98% PPV: 74% NPV: 98%

DIAGNOSIS WORKSHEET
Citation: Are the results of this diagnostic study valid? Was there an independent, blind comparison with a reference (gold) standard of diagnosis? Was the diagnostic test evaluated in an appropriate spectrum of patients (like those in whom it would be used in practice)? Was the reference standard applied regardless of the diagnostic test result? Was the test (or cluster of tests) validated in a second, independent group of patients? Yes Yes

Yes Yes

Are the valid results of this diagnostic study important?

SAMPLE CALCULATIONS Totals Target disorder Present Diagnostic test result Positive Negative Totals a c a+c Absent b d b+d a+b c+d a+b+c+d

+ -

+ 14 5

5 209

Culture or PCR Present Quick Vue Influenza test Positive Negative 14 5 19 Absent 5 209 214

Totals 19 214 233

Totals
Sensitivity = a/(a+c) = 14/19 = 74% Specificity = d/(b+d) = 209/214 =98%

Likelihood ratio for a positive test result = LR+ = sens/(1-spec) = 74%/2% = 37 Likelihood ratio for a negative test result = LR - = (1-sens)/spec = 26%/98% = 0.27 Positive Predictive Value = a/(a+b) = 14/19 = 74% Negative Predictive Value = d/(c+d) = 209/214 = 98% Pre-test probability (prevalence) = (a+c)/(a+b+c+d) = 19/233 = 8% Pre-test odds = prevalence/(1-prevalence) = 8%/92% = 0.087 Post-test odds = pre-test odds v LR = 0.087 x 37 = 3.22 Post-test probability = post-test odds/(post-test odds +1)= 3.22/4.22=76%

Just to remind you:

Sensitivity: the percentage of persons with the disease of interest who have positive test results. = a/(a+c) x 100 Specificity: the percentage of persons without the disease of interest who have negative results. = d/(d+b) x 100

PPV: the percentage of persons with positive test results who actually have the disease of interest. = a/(a+b) x 100 NPV: the percentage of persons with negative test results who do not have the disease of interest. = d/(d+c) x 100

Likelihood Ratio


Likelihood is the probability of a particular test result for a person with the disease of interest divided by the probability of that test result for a person without the disease of interest. Likelihood Ratio for a positive test result (LR+) Likelihood Ratio for a negative test result (LR-)

 

(LR+) is the probability of a positive test result for a person with the disease of interest divided by the probability of a positive test result for a person without the disease. LR+ = Sensitivity / (1-Specificity) (1LR+ > 1: persons affected with disease of interest are more likely to have a positive test result than unaffected persons. The larger the value of the LR, the stronger the association between having a positive test result and having the disease of interest. LR+ value of 10 or greater is perceived as indication of a test of high diagnostic value.

 

Pretest probability of disease: the probability that a disease: person has the disease of interest before the test is performed. = prevalence Pretest odds of disease: the estimate before diagnostic disease: testing of the probability that a patient has the disease of interest divided by the probability that the patient does not have the disease of interest. Pretest odds= pretest probability/(1-pretest probability) probability/(1-

Posttest odds of disease: as the estimate after diagnostic testing of the probability that a patient has the disease of interest divided by the probability that the patient does not have the disease of interest. Posttest odds= pretest odds x LR +

Posttest probability = posttest odds/ (1+posttest odds)

a result of obtaining a positive test result, the estimated probability of the presence of disease has risen from 0.08 (pretest probability) to 0.76 (posttest probability). Diagnostic tests that produce big changes from pretest to post-test probabilities are important and likely to be useful to us in our practice

Software:

CAT Maker

Can you apply this valid, important evidence about a diagnostic test in caring for your patient?
Is the diagnostic test available, affordable, accurate, and precise in your setting? Can you generate a clinically sensible estimate of your patients pre-test probability (from personal experience, prevalence statistics, practice databases, or primary studies)? yAre the study patients similar to your own? yIs it unlikely that the disease possibilities or probabilities have changed since the evidence was gathered? Will the resulting post-test probabilities affect your management and help your patient? yCould it move you across a test-treatment threshold? yWould your patient be a willing partner in carrying it out? Would the consequences of the test help your patient?

No Yes Yes Yes Yes

Additional notes:

Thank you..