Critical Appraisal

On Article of Diagnostic Test

(EBM-Diagnostic)
 Critical appraisal is one step in the process of evidence-
based clinical practice.
 To determine what is the best evidence, we need critical
appraisal skills that will help us to understand the
methods and results of research and to assess the
quality of the research.
 Most research is not perfect, and critical appraisal is not
an exact science - it will not give us the “right” answer.
But it can help us to decide whether we think a
reported piece of research is good enough to be used
in decision making.
Critical Appraisal - Worksheet for critical appraisal
- Software : CAT Maker
  Main area of clinical objectives:
 1. Diagnosis

 2. Prognosis

 3. Therapy/Treatment

 4. Risk/Harm

  Others:
 Systematic Review and Meta-analysis

 Clinical Guidelines

 Clinical Decision Making etc.

THREE MAIN ASPECTS TO BE APPRAISED: VIA

1. VALIDITY :
VALID (CLOSENESS TO THE TRUTH)  IN THE
METHODOLOGY SECTION

2. IMPORTANCE :
IMPORTANT (USEFULNESS)  IN THE RESULTS SECTION

3. APPLICABILITY :
APPLICABLE (CAN BE APPLIED IN CLINICAL PRACTICE) 
IN THE DISCUSSION SECTION
DIAGNOSIS WORKSHEET
Citation:

Are the results of this diagnostic study valid?

Was there an independent, blind comparison with a

reference (“gold”) standard of diagnosis?
Was the diagnostic test evaluated in an appropriate
spectrum of patients (like those in whom it would be
used in practice)?
Was the reference standard applied regardless of the
diagnostic test result?
Was the test (or cluster of tests) validated in a second,
independent group of patients?
Are the valid results of this diagnostic study important?
SAMPLE CALCULATIONS
Target disorder Totals
(iron deficiency anemia)
Present Absent
Diagnostic Positive 731 270 1001
test result (< 65 mmol/L) a b a+b
(serum Negative 78 1500 1578
ferritin) ( 65 mmol/L) c d c+d
Totals 809 1770 2579
a+c b+d a+b+c+d
Sensitivity = a/(a+c) = 731/809 = 90%
Specificity = d/(b+d) = 1500/1770 = 85%
Likelihood ratio for a positive test result = LR+ = sens/(1-spec) = 90%/15% = 6
Likelihood ratio for a negative test result = LR - = (1-sens)/spec = 10%/85% = 0.12
Positive Predictive Value = a/(a+b) = 731/1001 = 73%
Negative Predictive Value = d/(c+d) = 1500/1578 = 95%
Pre-test probability (prevalence) = (a+c)/(a+b+c+d) = 809/2579 = 32%
Pre-test odds = prevalence/(1-prevalence) = 31%/69% = 0.45
Post-test odds = pre-test odds  LR
Post-test probability = post-test odds/(post-test odds +1)
YOUR CALCULATIONS

Totals
Target disorder

Present Absent
Diagnostic Positive a b a+b
test result
Negative c d c+d
Totals a+c b+d a+b+c+d
Can you apply this valid, important evidence about
a diagnostic test in caring for your patient?

Is the diagnostic test available, affordable, accurate, and precise in your

setting?
Can you generate a clinically sensible estimate of your patient’s pre-test
probability (from personal experience, prevalence statistics, practice
databases, or primary studies)?
Are the study patients similar to your own?
Is it unlikely that the disease possibilities or probabilities have changed
since the evidence was gathered?
Will the resulting post-test probabilities affect your management and help
your patient?
Could it move you across a test-treatment threshold?
Would your patient be a willing partner in carrying it out?
Would the consequences of the test help your patient?

Additional notes:
Example : an article of diagnostic test, entitle :

Bedside Diagnosis of
Influenzavirus Infections
in Hospitalized Children

Katherine A. Poehling, et al
American Academy of Pediatrics
Background:
 Influenzavirus has a significant impact on the pediatric
population, with school-aged children having the highest
infection rates.
 For preventing nosocomial influenza infections and to
facilitate prompt antiviral therapy, an accessible, rapid
diagnostic method for influenzavirus is needed.

Objective:
 To compare the performance of a rapid diagnostic test
(QuickVue Influenza Test; Quidel Corp, San Diego, CA)
completed at the bedside of hospitalized children to viral
culture and/or polymerase chain reaction (PCR) for
influenzavirus.
Method:
 Study population:
1) younger than 19 years and hospitalized with respiratory
symptoms or 2) younger than 3 years and hospitalized with
fever.
 Sampel: 1) a primary admission diagnosis of an acute
respiratory illness characterized by rhinorrhea, sore throat,
cough, shortness of breath, or apnea or 2) a primary
admission diagnosis consistent with a febrile illness and a
temperature of 100.4°F.
 Broad inclusion criteria were chosen such that all children
who were hospitalized with symptoms potentially related to
influenza infections were eligible.
Sample in
this study
 Study design:
prospective, cross sectional study  from each child, 2 nasal
swabs of the turbinates were obtained—1 for influenzavirus
culture and PCR and the other for the rapid diagnostic test.
The rapid test results were compared with that of culture and
PCR for influenzavirus.

 Influenza infection was defined as any sample with 1) a

positive culture for influenzavirus or 2) 2 consecutive positive
PCRs for influenza A or B.
 Each researcher was trained to perform and interpret the
rapid diagnostic test at the bedside according to the
manufacturer’s instructions.
 The laboratory technician who performed the culture and
PCR was masked to the rapid diagnostic test results.
 Results:
Sens: 74%
Spec: 98%
PPV: 74%
NPV: 98%
DIAGNOSIS WORKSHEET
Citation:

Are the results of this diagnostic study valid?

Was there an independent, blind comparison with a Yes

reference (“gold”) standard of diagnosis?
Was the diagnostic test evaluated in an appropriate Yes
spectrum of patients (like those in whom it would be
used in practice)?
Was the reference standard applied regardless of the Yes
diagnostic test result?
Was the test (or cluster of tests) validated in a second, Yes
independent group of patients?
 Are the valid results of this diagnostic study important?

SAMPLE CALCULATIONS
Totals
Target disorder

Present Absent
Diagnostic Positive a b a+b
test result
Negative c d c+d
Totals a+c b+d a+b+c+d
 + -
+ 14 5
- 5 209
 Culture or PCR Totals
Present Absent
Quick Vue Positive 14 5 19
Influenza
Negative 5 209 214
test
Totals 19 214 233

Sensitivity = a/(a+c) = 14/19 = 74%

Specificity = d/(b+d) = 209/214 =98%

Likelihood ratio for a positive test result = LR+ = sens/(1-spec) = 74%/2% = 37

Likelihood ratio for a negative test result = LR - = (1-sens)/spec = 26%/98% = 0.27

Positive Predictive Value = a/(a+b) = 14/19 = 74%

Negative Predictive Value = d/(c+d) = 209/214 = 98%

Pre-test probability (prevalence) = (a+c)/(a+b+c+d) = 19/233 = 8%

Pre-test odds = prevalence/(1-prevalence) = 8%/92% = 0.087

Post-test odds = pre-test odds  LR = 0.087 x 37 = 3.22
Post-test probability = post-test odds/(post-test odds +1)= 3.22/4.22=76%
Just to remind you:

 Sensitivity: the percentage of persons with the

disease of interest who have positive test results.
= a/(a+c) x 100

 Specificity: the percentage of persons without

the disease of interest who have negative results.
= d/(d+b) x 100
 PPV: the percentage of persons with positive
test results who actually have the disease of
interest.
= a/(a+b) x 100

 NPV: the percentage of persons with negative

test results who do not have the disease of
interest.
= d/(d+c) x 100
Likelihood Ratio
 Likelihood is the probability of a particular test
result for a person with the disease of interest
divided by the probability of that test result for a
person without the disease of interest.

 Likelihood Ratio for a positive test result (LR+)

 Likelihood Ratio for a negative test result (LR-)
 (LR+) is the probability of a positive test result for a
person with the disease of interest divided by the
probability of a positive test result for a person without
the disease.

 LR+ = Sensitivity / (1-Specificity)

 LR+ > 1: persons affected with disease of interest are
more likely to have a positive test result than unaffected
persons.
 The larger the value of the LR, the stronger the
association between having a positive test result and
having the disease of interest.
 LR+ value of 10 or greater is perceived as indication of
a test of high diagnostic value.
 Pretest probability of disease: the probability that a
person has the disease of interest before the test is
performed.
= prevalence

 Pretest odds of disease: the estimate before diagnostic

testing of the probability that a patient has the disease
of interest divided by the probability that the patient
does not have the disease of interest.
 Pretest odds= pretest probability/(1-pretest probability)
 Posttest odds of disease: as the estimate after diagnostic
testing of the probability that a patient has the disease of
interest divided by the probability that the patient does not
have the disease of interest.
 Posttest odds= pretest odds x LR +

 Posttest probability = posttest odds/ (1+posttest odds)

  a result of obtaining a positive test result, the estimated

probability of the presence of disease has risen from 0.08
(pretest probability) to 0.76 (posttest probability).

 Diagnostic tests that produce big changes from pretest

to post-test probabilities are important and likely to be
useful to us in our practice
 Software: CAT Maker……
Can you apply this valid, important evidence about
a diagnostic test in caring for your patient?

Is the diagnostic test available, affordable, accurate, and precise in your

setting? No
Can you generate a clinically sensible estimate of your patient’s pre-test
probability (from personal experience, prevalence statistics, practice Yes
databases, or primary studies)?
Are the study patients similar to your own?
Is it unlikely that the disease possibilities or probabilities have changed
Yes
since the evidence was gathered?
Will the resulting post-test probabilities affect your management and help
your patient? Yes
Could it move you across a test-treatment threshold?
Would your patient be a willing partner in carrying it out?
Would the consequences of the test help your patient?
Yes
Additional notes:
Thank you…..