IDC 213

Reporting on Clinical Trials and Diagnosis

Trans 4 Exam #1
Dr. Jacinto Mantaring
08/06/2019

OUTLINE V. Continuous Outcomes

I. Measures of Accuracy
II. Likelihood Ratios
III. Selecting a Test
IV. Dichotomous Outcomes
have AIDS?” It is the physician’s responsibility to determine his
Note: Dr. Mantaring merged the lectures on clinical trials and chances that he really has the disease.
diagnosis.  They can only be done in a 2x2 table analysis (i.e. 2 test results)
 They don’t demonstrate the change from pre-test to post-test
DIAGNOSTIC TESTS probability.
I. MEASURES OF ACCURACY  You can, however, derive the likelihood ratio from Sn and Sp.
GOLD STANDARD II. LIKELIHOOD RATIOS
+ - A. Definitions
A B 𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑜𝑓 𝑓𝑖𝑛𝑑𝑖𝑛𝑔 𝑖𝑛 𝑝𝑥 𝑤𝑖𝑡ℎ 𝑑𝑖𝑠𝑒𝑎𝑠𝑒
+
(True Positive) (False Positive) 𝐿𝑅 =
TEST 𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑜𝑓 𝑠𝑎𝑚𝑒 𝑓𝑖𝑛𝑑𝑖𝑛𝑔 𝑖𝑛 𝑝𝑥 𝑤𝑖𝑡ℎ𝑜𝑢𝑡 𝑑𝑖𝑠𝑒𝑎𝑠𝑒
C D
-
(False Negative) (True Negative) 𝑝𝑜𝑠𝑡𝑡𝑒𝑠𝑡 𝑜𝑑𝑑𝑠 𝑜𝑓 𝑡ℎ𝑒 𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 𝑜𝑓 𝑖𝑛𝑡𝑒𝑟𝑒𝑠𝑡
𝐿𝑅 =
Sn = A / (A + C) PPV = A / (A + B) 𝑝𝑟𝑒𝑡𝑒𝑠𝑡 𝑜𝑑𝑑𝑠 𝑜𝑓 𝑡ℎ𝑒 𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 𝑜𝑓 𝑖𝑛𝑡𝑒𝑟𝑒𝑠𝑡
Sp = D / (B + D) NPV = D / (C + D) Probability Odds
Sensitivity (Sn) Value Over 1 always Over (1 – numerator)
 The probability of getting a positive test result among those who Range of
have the condition of interest (i.e. disease) [0,1] [0,infinity]
values
 SnOut 𝐴 / (𝐴 + 𝐶) 𝐴/𝐵
o Getting a negative result from a highly sensitive test rules out LR+
𝐵 / (𝐵 + 𝐷) (𝐴 + 𝐶)/ (𝐵 + 𝐷)
a disorder 𝐶 / (𝐴 + 𝐶) 𝐶/𝐷
 Use a screening test that is sensitive LR-
𝐷 / (𝐵 + 𝐷) (𝐴 + 𝐶)/ (𝐵 + 𝐷)
o Objective is to test a whole mass of people and pick up those
who might need further work up (ex. screen tourists from Brazil  The likelihood ratio of a test is the ratio of the probability of the
to screen out Zika Virus) test result among those who have the condition of interest
Specificity (Sp) (numerator) compared to those who do not have the condition of
interest (denominator)
 The probability of getting a negative test result among those
o By convention, the numerator has those with the target
without the condition of interest
disorder and the denominator has those without the target
 SpIn
disorder
o Getting a positive result from a highly specific test rules in a
o “You are [LR] times more likely to get the [result] if you have
disorder
the disease than if you don’t have the disease
 Use a definitive test that is specific  Also defined as the post-test odds divided by the pre-test odds
Positive Predictive Value (PPV) of the condition of interest
 The probability of having the condition of interest among those o Hard to appreciate clinical application with this definition
who have a positive test o With this, you will notice that multiplying the pre-test odds with
Negative Predictive Value (NPV) the LR will get you the post-test odds
 The probability of not having the condition of interest among those  You can compute as many likelihood ratios as there are test
who have a negative test results
o In a 2x2 table, you have an LR+ and an LR-
 It is not correct to say “negative LR” and “positive LR.” It is the
Problems with Traditional Measures of Test Accuracy
 Sensitivity and specificity are NOT clinically useful. However, likelihood ratio of the test result. Thus, it is “the likelihood ratio of a
they are useful to describe the accuracy of the test. positive result” and “the likelihood ratio of a negative result.”
o Note: You will never find a 90% sensitive test and 90% specific  LR+
test. This is because of the receiver operating characteristic o The likelihood of getting a positive result among those who
(ROC). The more sensitive the test, the less specific, and vice have the disease compared to getting a positive result among
versa. In the ROC, you can graph the sensitivity against 1 those who do not have the disease
minus the specificity and look at the test graphically which is o True positive rate / false positive rate
closest to one. o Sensitivity / (1 – specificity)
o Sensitivity is sacrificed for specificity, and specificity is  LR-
sacrificed for sensitivity. o The likelihood of getting a negative result among those who
 Positive and negative predictive values change with prevalence. have the disease compared to getting a negative result among
Thus, they are clinically useful. those who do not have the disease
o Ex: In the clinical setting, a patient goes “I tested positive for o (1 – sensitivity) / specificity
HIV. What are my changes that I really do

Trans Group 23: Chua, de Chavez, De Guzman S., Dizon

 Reporting Results of Clinical Trials and Diganostic Tests Page 2 of 6
HS 202
 Pre-test odds  10/1 is the same as 1/10 in terms of magnitude (they are
o The odds of having the target disorder before doing any test reciprocals of each other)
(A+C/B+D)  LR of 10 means that you are 10 times more likely to get that result
o Ratio of those with the target disorder divided by those without if you have the target disorder
the target disorder  LR of 0.1 means that you are 10 times more likely to get that
o Pre-test probability / 1- pre-test probability result if you do not have the target disorder
 Post-test odds  How large is a likelihood ratio and how small is a small one?
o The odds of having a positive result or negative result given a (Sacket, 2000)
test o LR > 10 and < 0.1 – make large changes (from pre-test of
o Pre-test odds x LR post-test probability)
 Pre-test probability o LR 5-10 and 0.1-0.2 – make moderate changes
o Probability of the target disorder before a diagnostic test is o LR 2-5 and 0.2-0.5 – make small changes
ordered o LR < 2 and > 0.5 – make insignificant changes
o Prevalence of the disease before ordering a test
 Post-test probability
o Probability of the target disorder after a diagnostic test result is
known
o Post-test odds / (post-test odds + 1)
 Conversion
o Probability = odds / (1 + odds)
o Odds = probability / (1 – probability)
Calculating posttest probabilities from pretest
probabilities and likelihood ratios
 Calculate pretest probabilities (based on history/PE and initial
Figure 2. Likelihood ratios.
laboratories)
 Convert the pretest probabilities into pretest odds C. Advantages
 Perform the diagnostic test  Clinically useful
 Depending on the test results, multiply the likelihood ratio by the  Does not change with prevalence
pretest odds to compute the post test odds  Can be applied even in a larger than 2x2 table
 Convert the post test odds into post test probability  Measures and express diagnostic accuracy
 Make a decision to treat or not to treat  Gives a measure of the change from pre-test to post-test
probability or pre-test to post-test odds

III. SELECTING A TEST

In selecting a test, it depends on the researcher’s purpose.
Using Sn and Sp Using LRs
Rule In Test with high Sp Test with high LR+
Rule Out Test with high Sn Test with LR- near 0

Which test will you use?

Test A: 98% sensitivity and 60% specificity
Test B: 60% sensitivity and 98% specificity
Answer: it depends on the case.
A. Scenario 1
 A patient was referred for fever of 3-day duration.
 PE is unremarkable
 There is history of dengue in the community.
B. Scenario 2
 A patient was referred for fever of 5-day duration.
Figure 1. Bayes nomogram for estimating post-test probability
 PE is unremarkable
 There is history of dengue in the community.
B. Interpretation
 If your LR > 1 (the numerator with target disorder is greater than  Platelet count is 90,000
denominator without target disorder), then having the target C. Available Tests
disorder will mean you are more likely to get that test result Test A Test B
compared to not having the target disorder Sensitivity 98% 60%
 LR < 1 decreases the probability of having the target disorder Specificity 60% 98%
LR+ 2.45 30
 The larger the ratio, the larger the likelihood
LR- 0.03 0.41
 LR = 1
o The numerator and denominator are the same
o The test neither increases nor decreases the likelihood of
having the target disorder
 Values range from 0 to infinity

Trans Group 23: Chua, de Chavez, de Guzman S., Dizon

 Reporting Results of Clinical Trials and Diganostic Tests Page 3 of 6
HS 202
D. Rationalization
Scenario 1 Scenario 2 from pre-test to LR+, but in the
 Low index of  High index of post-test probability other direction
suspicion suspicion given that our pre-
Merits of the  No clinical indicator  Low platelet count test probability is
case other than fever & is bothersome (NV: low
history of dengue in 150k – 450k)  The patient looks  The patient looks
the community well like he/she has
Pre-test  Low  High  We want to use a dengue
probability (see Figure 3 for a graphical representation) test that is able to  We want to use a
 Specific  Sensitive Interpretation rule in dengue to test that is able to
 The test is able to  The test is able to be absolutely sure rule out dengue in
rule in the disease rule out the disease that the patient case our primary
in the setting of a in the setting of a does not have it working impression
Preferred test low probability high probability is flawed
 LR+ is preferably  LR- is preferably Test to use  Test B  Test A
big small
 A big LR+ can  A small LR- has the
make a big jump same effect as a big

Figure 3. Top: Scenario 2 – high pre-test probability; closer to the therapeutic threshold. Bottom: Scenario 1 – low pre-test probability; closer to the diagnostic threshold

CLINICAL TRIALS: o Point estimate – a single approximate value computed from a

IV. DICHOTOMOUS OUTCOMES sample
 Dichotomous outcomes - either it happened or didn’t o Interval estimate – consists of lower and upper limits which
 More clinically relevant outcomes contains the approximate
 Convert these outcomes to RR, RRR, ARR, NNT  Each result can be presented as a point estimate (RR, RRR, ARR,
o Riskt = Risk for treatment group NNT) or an interval estimate (CI)
o Riskc = Risk for control group o Example: RR = 0.65 [95%, CI 0.60, 0.70]
 Point estimate is 0.65
Formulae  This point estimate falls within the Interval
𝑅𝑖𝑠𝑘 𝑡  Interval estimates serve as a measure of precision
 Relative Risk (RR) =  The interval estimate is less than one and does not cross 1,
𝑅𝑖𝑠𝑘 𝑐
𝑅𝑖𝑠𝑘 𝑡−𝑅𝑖𝑠𝑘 𝑐
 Relative Risk Reduction (RRR) = meaning its beneficial (decreased risk) and is not useless
𝑅𝑖𝑠𝑘 𝑐
 Absolute Risk Reduction (ARR) = 𝑅𝑖𝑠𝑘𝑡 − 𝑅𝑖𝑠𝑘𝑐  95 % CI means you are 95 percent confident that the value
1 lies within the interval you have calculated
 Number Needed to Treat (NNT) =  However, we have to “interrogate” the confidence interval;
𝐴𝑅𝑅
We should do this as clinicians because CI can influence
Example: Weight Analogy (Wt as “risk”) treatment decisions
 When it comes to losing weight, how can you report your progress? o Ex: If CI [0.6, 0.95], will you apply the treatment?
 Given that your baseline weight is 100 kg, and that after weight  If near lower limit, it would be beneficial
program, your new weight is 70 kg  If near upper limit (close to 1 which is useless), treatment may
𝑊𝑡 𝑏 not be applied, especially if patient cannot afford it (may apply
o Relative weight =
𝑊𝑡 𝑐 treatment, though, if patient is rich)
𝑊𝑡 𝑏−𝑊𝑡 𝑐
o Relative weight reduction =
𝑊𝑡 𝑐
 Thus, individualize results!
o Absolute weight reduction = 𝑊𝑡𝑏 − 𝑊𝑡𝑐  Incidence vs. incidence density
 What are the results? o Incidence – number of people with the case (44 people with UTI)
o Relative weight = 70/100 = .70 or 70% o Incidence density – an event referring to the number of cases
(Interpretation: I am 70 percent of my baseline weight) per person per time – 44 incidents of UTI in 2 people in 5 years
o Relative weight reduction = 100-70/100 = .30
(Interpretation: I lost 30 percent of my weight) A. Meta-analysis and adverse events
o Absolute weight reduction = 100-70 = 30 kg  Meta-analysis is done for studies with the same Population,
(Interpretation: I lost 30 kg) Exposure, Outcome

When it comes to risk, how can you report your results?

 Interpretation of RR
o RR < 1 is beneficial
o RR = 1 is useless
o RR > 1 is harmful
 Definition of terms

Trans Group 23: Chua, de Chavez, de Guzman S., Dizon

 Reporting Results of Clinical Trials and Diganostic Tests Page 4 of 6
HS 202

 Based on Figure 5, we can obtain the outcome = number of

patients with at least 1 episode of common infection
o Rc = 289/324 = 0.892
o Rt = 286/314 = 0.911
o RR = 1.021 (0.97, 1.075)
o RRI = 0.021
o ARI = 0.019
o NNH (# need to harm) = 54

C. WHICH IS THE BETTER MEASUREMENT?

 Which is the better drug?
Table 1. Sample scenario comparing RR, ARR, and NNT

Figure 4. Diagram featuring Meta-analysis of Adverse Drug Events

 Interpretation of data for each study:

o STUDY 1 – Beneficial (p<0.05)
o STUDY 2 – Harmful (p<0.05)
o STUDY 3 – Useless (p>0.05)
o STUDY 4 – Tendency to Harm (p=0.05)
o STUDY 5 – Tendency to Benefit (p=0.05)
 Increase in sample size may confirm significance
o Several point estimates are combined to come up with “pooled”
estimate  From the critical appraiser’s (Dr. Mantaring’s) point of view, the
o Usually sample size of Study 3, 4 & 5 is small, so once you NNT is the most important
increase the number of samples, the confidence interval shrinks  For A to D, Rt is lower by half than Rc
and the ambiguity is resolved  RR is the same for all four drugs
o A meta-analysis can combine these points to get a true mean,  ARR and NNT are different. NNT makes it easier to appreciate the
but P, E and O between studies must be equivalent magniture and the benefit of the study
 Based on the ARR and NNT, DRUG D is the best among the four
B. Reporting the Results drugs
Sample study: D. THE P-VALUE
 Use of a fermented dairy probiotic drink containing Lactobacillus  Compare the following drugs below with the different p-values.
casei (DN-114 001) to decrease the rate of illness in kids: the Which is the better drug?
DRINK study
 A patient-oriented, double-blind, cluster-randomized, placebo-
controlled, clinical trial
 In this research, the results are presented in patient-days of illness
(this makes it possible for one patient to contribute several times)
o Results were presented in patient-days of illness
 The value “3” can be interpreted as 3 patients with the illness
or 3 days of illness in a single patient
o Rc = 0.098 pt-days of illness
o Rt = 0.078 pt-days of illness Figure 6. Sample scenario comparing durg outcomes with different p-
o RR = 0.79 values
o RRR = 0.21  P-value is only checked to show you the probability of committing
o ARR = 0.02 a Type 1 error but NOT for clinical relevance
o NNT = 50 to prevent 1 patient day of illness  It is more relevant to compute for RR, RRR, ARR, NNT, and get
the confidence interval
o Easier to compare across drugs

Table 2. Rt, Rc, RRR, ARR, and NNT based on the example in Figure
3 (confidence intervals in parentheses)

Figure 5. Distribution of Subjects for cumulated number of CID

during study product consumption

Trans Group 23: Chua, de Chavez, de Guzman S., Dizon

 Reporting Results of Clinical Trials and Diganostic Tests
HS 202
 Even if drug B has a good p-value, it is not the better drug. The
magnitude of the effect of Drug A (compared to more significant
than the effect of drug B). Drug B’s good p-value can be attributed
to its large sample size.
II. CONTINUOUS OUTCOMES
 For continuous outcomes, mean difference is used
 Mean difference = Mc - Mt
o Where Mc = mean in control group, and Mt = mean in treatment
group
o On the graph: favors intervention or favors control
o Numbers are relative: you want low values for cholesterol but
want high IQ
Trans Group 23: Chua, de Chavez, de Guzman S., Dizon
Page 5 of 6
Figure 4. Mean differences displayed as horizontal bars. Mean differences
less than 0 favor the treatment (treatment has a greater effect than the
control), whereas mean differences greater than 0 favor the control (control
has a greater effect than the treatment).
III. KEY POINTS
 How to compute statistics is not important
 How to interpret is more important
 No matter how the results are presented, they can often be
translated into useful clinical information
 Dichotomous outcomes are more clinically relevant outcome
REFERENCES
2021 trans, Doc Mantaring’s lecture slides
END OF TRANS
 Reporting Results of Clinical Trials and Diganostic Tests Page 6 of 6
HS 202
APPENDIX
EFFICACY ANALYSIS VS. INTENT-TO-TREAT ANALYSIS
 In efficacy or cencored analysis, the patients who dropout of the study are censored and only the compliers are analyzed
 In intention-to-treat (ITT) analysis, the dropouts are included in the analysis of intention to treat groups

EXAMPLE: HYPOTHETICAL TB TRIAL

2,000 patients were randomized into the following treatment allocation, producing the following outcomes:

Table 1. Treatment outcomes using efficacy analysis.

Treatment groups N Compliers outcome Dropout outcome Interpretation of
outcomes
6 mos 1000 100/1000 0/0 Out of 1000 patients, 100
had poor outcomes but
there were no dropouts
12 mos 1000 45/900 90/100 Out of 1000 patients, only
900 finished the study.
Out of the 900, 45 had
poor outcomes.
Out of the 100 who did not
finish the study, 90 had
poor outcomes.

Table 2. Treatment outcomes using efficacy analysis.

Treatment groups N Compliers outcome Dropout outcome Interpretation of
outcomes
6 mos 1000 100/1000 0/0 Out of 1000 patients, 100
had poor outcomes but
there were no dropouts
12 mos 1000 135/1000 90/100 Out of all 1000 patients,
135 had poor outcomems.
There were 100 dropouts
among which 90 had poor
outcomes

Table 3. Comparison between efficacy analysis and ITT analysis

Treatment groups N Compliers outcome Dropout outcome Overall outcome
6 mos 1000 100/1000 0/0 100/1000
12 mos 1000 45/900 90/100 135/1000
EFFICACY ANALYSIS ITT ANALYSIS
Does the drug work? Will the drug work?
IDEAL world REAL world

EXAMPLE: EFFECT OF DRUG X ON MORTALITY

 There are times when including the dropiouts in the analysis can change the results of the study
 Scenario: 200 patients were randomized into the following treatment allocations, but some were unanble to complete the study

Table 4. Treatment outcomes using efficacy analysis.

Treatment groups N Lost to follow-up Mortality
Drug X 100 6 3/94 (3%)
Placebo 100 6 5/94 (5%)
What if all those lost to follow-up in treatment groups had poor outcome, while those lost to follow-up in placebo had good outcomes?

Table 5. Treatment outcomes using efficacy analysis.

Treatment groups N Lost to follow-up Mortality
Drug X 100 6 3/94 (3%)
Placebo 100 6 5/94 (5%)
Including the dropouts in the outcomes, changed the mortality of Drug X from 3% to 9%, as compared to the 5% mortality of placebo. Hence, Drug X
turns out to have a higher association with mortality compared to placebo.

EXAMPLE: COMPARISON OF DROPOUT RATE

Table 6. Treatment outcomes using efficacy analysis.
Study ID Treatment dropout Control dropout Treatment outcome Control outcome
A 1% 1% 0.1% 0.4%
B 20% 20% 40% 80%
C 10% 1% 8% 20%
Which study has more robust results?
If the dropout rate is high, it is probably because of the treatment, and not just a results of chance.

Trans Group 23: Chua, de Chavez, de Guzman S., Dizon