Comparing variables

Statistical Comparison Tests

What tests should be used?

 Numerical data

Comparing independent groups

 Comparing groups
Numerical data

Independent / unrelated
groups

Normally distributed Not Normally Distributed

2 groups 2 groups
Independent t-test Mann-Whitney U-test

≥ 3 groups ≥ 3 groups
One-factor ANOVA Kruskal-Wallis test
 Example
• Is the average age of patients who receive
best supportive care significantly different
from chemotherapy patients?

• Or, could any difference seen be explained by

chance?
 Comparing 2 independent groups
Chemotherapy patients
Mean 66.7
80
Median 67.0
60
Standard 9.2
Frequency

40 Deviation
Interquartile 12.0
20
Range
0
0 20 40 60 80 100
Age at Diagnosis

Best supportive care patients

Mean 75.2
35
30
Median 76.0
25 Standard 9.4
Frequency

20 Deviation
15
Interquartile 12.0
10
5
Range
0
0 20 40 60 80 100
Age at Diagnosis
Null hypothesis (H0):
Average age is the same in the two groups

Alternative hypothesis (H1):

Average age is different

If p > 0.05 we don’t reject the null hypothesis

If p < 0.05 we reject the null hypothesis: there is a

statistically significant difference between the
groups
 Example
Is the average age of patients who receive best
supportive care significantly different from
chemotherapy patients?

Chemotherapy patients: mean age = 66.7

Best Supportive care patients: mean age = 75.2

T-test; p=0.02
 Comparing groups
Numerical data

Independent / unrelated
groups

Normally distributed Not Normally Distributed

2 groups 2 groups
Independent t-test Mann-Whitney U-test

≥ 3 groups ≥ 3 groups
One-factor ANOVA Kruskal-Wallis test
 Numerical data

Comparing dependent groups

 Dependent/related groups?
= paired data

• Repeat data on same group of patients

Eg. before and after some intervention

• Patient groups which have been one-to-one

matched for characteristics such as age, sex,
stage of cancer.
 Comparing groups
Numerical Data

Dependent/Related
groups

Not Normally
Normally distributed
Distributed

2 groups 2 groups
Paired t-test Wilcoxon signed rank test

≥ 3 groups
≥ 3 groups
Repeated Measures
ANOVA Friedman test
 Comparing 2 groups: Paired data

Time to produce organ at risk contours

25

20
time (mins)

15

10

0
Manual Contour Modified Contour
 Null hypothesis (H0): The average change
between manual and modified is 0

Alternative hypothesis (H1): The average

change between manual and modified is not 0

If p > 0.05 we don’t reject the null hypothesis

If p < 0.05 we reject the null hypothesis: there is a

statistically significant difference between the
groups
HADS anxiety scores were obtained for 30 cancer patients
before and after a session with a clinical psychologist to assess
whether there had been a significant change. The HADS scores
had a significantly skewed distribution.

Which is the most appropriate test?:

(a) Two-sample t-test

(b) Wilcoxon signed rank test
(c) One factor ANOVA
(d) Mann-Whitney test
(e) Paired t-test
Non numerical / categorical data

Comparing independent groups

 Comparing groups
Non-numeric/
categorical data

Independent / unrelated
Related groups
groups

Chi-square test
Fisher’s exact test McNemar’s test
Linear Trend test
 Comparing 2 groups with
a categorical outcome

Group 1 = female
Group 2 = male

Outcome = tumour side (left vs right)

 Comparing 2 groups with
a categorical outcome

Group 1 = female
Group 2 = male

Outcome = tumour side (left vs right)

Use a chi-square test

 Is there a difference in tumour side between
males and females?

Side/ Sex Left Right Total

Female 386 500 886

Male 457 644 1,101

Total 843 1,144 1,987

 Is there a difference in tumour side between
males and females?

Side/ Sex Left Right Total

Female 386 500 886

44% 56%
Male 457 644 1,101
42% 58%
Total 843 1,144 1,987
 Null hypothesis (H0): Tumour side is similar
between sexes (eg. 44% ≈ 42% for left side)

Alternative hypothesis (H1): Tumour side

is different between sexes (or tumour side is
associated with sex)

If p > 0.05 we don’t reject the null hypothesis

If p < 0.05 we reject the null hypothesis: there is a

statistically significant association between sex
and tumor side
Is there a difference in tumour side between
males and females?

Chi-square test; p=0.36

No significant difference in tumour side

No association between tumour side and sex

of patient
What is the chi-square test?

Compares observed and expected values

(under the null hypothesis) for each cell in
the table

Expected values are calculated by

multiplying the relevant column and row
totals and dividing by the grand total
What is the chi-square test?

The chi-square statistic =

sum of the terms

2
𝑂 −𝐸
𝐸
[one term for each cell in the table]

This numerical value is then compared

with the chi-square distribution
 Is there a difference in tumour side between
males and females?

Side/ Sex Left Right Total

Female 386 500 886

375.9 510.1
Male 457 644 1,101
467.1 633.9
Total 843 1,144 1,987

Expected values in red

 Is there a difference in tumour side between
males and females?
Side/ Sex Left Right Total

Female 386 500 886

375.9 510.1
Male 457 644 1,101
467.1 633.9
Total 843 1,144 1,987

Chi-square statistic =
2 2 2 2
386−375.9 500−510.1 457−467.1 644−633.9
+ + +
375.9 510.1 467.1 633.9
= 0.85 ; p=0.36
 Chi-square test

Can also be used for comparing more than 2

groups, and for outcomes that have more
than 2 categories.

Eg. Comparing patients grouped into 4

distinct categories of treatment and an
outcome measured at 12 months as ‘died’,
‘relapse’, ‘remission’.
 Fisher’s Exact test

Use instead of chi-square test if group sizes

are small

Eg. in previous example if only 4 females

and 5 males
 Linear trend test

If the grouping factor is an ordered category, eg

stage of tumour, and the outcome is a 2-category
variable, eg death within 12 months, a more
powerful analysis than a simple chi-square test is a
chi-square test for linear trend.

This method effectively fits a straight line to the

death rates corresponding to each stage.
 Comparing groups
Non-numeric/
categorical data

Independent / unrelated
Related groups
groups

Chi-square test
Fisher’s exact test McNemar’s test
Linear trend test
Non numerical / categorical data

Comparing dependent/related
groups
 Comparing paired groups with a
categorical outcome

One group with data before and after treatment

Outcome = binary (eg. yes vs no)

 Example

McNemar’s test is based on comparing the off-diagonal numbers:

Before ‘yes’ / After ‘no’ vs. Before ‘no’/After ‘yes’

ie. 785 vs 75
1000 cancer patients diagnosed via screening or presenting
with symptoms were classified according to their ER status
(positive vs negative).
What is the most appropriate test to use to see whether those
diagnosed via symptoms are more likely to be ER negative
than those diagnosed via screening?

(a) Fisher’s Exact test

(b) McNemar’s test
(c) Chi-square test for linear trend
(d) Wilcoxon signed rank test
(e) Chi-square test
 Scatterplot
Scattergram of creatinine vs. digoxin
140

120

100
Creatinine

80

60

40

20

0
0 20 40 60 80 100 120

Digoxin
 Correlation
Pearson correlation:

• Used for Normally distributed data

• Measures linear relation between variables
 Correlation

• r = 0 no relationship
• r = 1 perfect +ve relationship
• r = -1 perfect –ve relationship
 Pearson's correlation coefficient

1 0.93 0.75 -0.2

8
7
1.0
6 6 6
0.5
5
4 0.0
4
4
y

y
-0.5
3 2
2
-1.0
2
0 -1.5
0 1

-1 0 1 2 -1 0 1 2 -1 0 1 2 -1 0 1 2

x x x x

-1 -0.89 -0.8 -0.08

4 3
0 2
2
0 2

-2 0
-2 1
-2
y

y
-4 -4 -4 0

-6 -6
-6 -1
-8 -8
-2

-1 0 1 2 -1 0 1 2 -1 0 1 2 -1 0 1 2

x x x x
 Pearson's correlation coefficient

Sensitive to outliers

0.85 0.32
30
8
25

6 20
y

y
15
4

10

2
5

0 0

-1 0 1 2 -1 0 1 2

x x
 Correlation
Spearman correlation:

• Used for non-Normally distributed data

• Measures monotonic relation between
variables
Correlation does not imply
causation
 Correlation does not imply
causation

Eg.
Correlation between pork consumption and cirrhosis
mortality r = 0.40 (over 16 countries)
 Relationships
Continuous Data

Normally
Not Normally
distributed
Distributed

Pearson’s Spearman’s
correlation correlation
(2 variables) (2 variables)
 In a study looking at the relationship between pre-
treatment HADs anxiety score and post treatment
satisfaction score (SAT) were obtained on 130 women.
The Pearson correlation was -0.65; p<0.001.
What is the single best statement?

(a) HADs score explains 65% of the variability in satisfaction score

(b) We can conclude that an increasing HADs score is a cause of poor
SAT score
(c) A correlation of 1 is interpreted as showing no difference
between the two factors.
(d) There is a significant negative linear relationship between HADs
and SAT
(e) The correlation between HADs and SAT can be different from the
correlation between SAT and HADs
 Prediction

• Describe the relationship between two

variables
• Be able to predict the value of one
variable for a subject when only have data
on the other variable
 Regression
“How can we predict creatinine from digoxin?”
Scattergram of creatinine vs. digoxin
140

120

100
Creatinine

80

60

40

20

0
0 20 40 60 80 100 120

Digoxin
 Simple linear regression equation
Used to determine the exact linear relationship
between two variables x and y

y=b+ mx
b = intercept of line on y-axis (value of y when x is zero)
m = slope or gradient of line

x= independent variable
y = dependent variable
Linear Regression
 Creatinine vs. digoxin

Creatinine = b + m digoxin

Fitted regression equation:

Creatinine = 5 + 1.2 digoxin

 Creatinine vs. digoxin

Creatinine = 5 + 1.2 digoxin

For every unit increase in digoxin,

there is a 1.2 unit increase in creatinine
 Goodness of fit

• How well does the regression line

fit the data?
• How accurate are the predictions?
 R2

• The proportion of the total

variation explained by the model

• For simple linear regression = the

square of the correlation between
x and y.
 Multiple linear regression

More than one ‘x’ variable (independent

variable) in the regression equation

Eg.
Creatinine = b + m1 digoxin + m2 age
 Multiple linear regression

Can be used to adjust for confounding variables:

Eg.
“Is there a relationship between cholesterol and
age after adjusting for BMI?”

chol = b + m1 age + m2 BMI

 Confounders

• Complicate relationships between two

variables of interest .

• Related to both the dependent variable and

the independent variable
The predictive model for the link between tumour size and
age was derived for a cohort of 250 women with breast
cancer:
Tumour size = 2.47 + 0.10 x age ; p=0.13

(a) Age is treated as a confounder in the regression

(b) The model shows that tumour size significantly
increases by 0.10 for every 1 year increase in age
(c) The association between tumour size and age is not
statistically significant
(d) For every unit change in tumour size, there is a 0.10
change in average age.
(e) The correlation between tumour size and age is
0.10
 Prediction models

Outcome = continuous Outcome=binary

Predictors = continuous Predictors = continuous
and/or categorical and/or categorical

Linear Regression
Logistic Regression
Multiple linear regression
Multiple logistic regression
 Logistic regression

Dependent variable = binary (two categories)

Eg.

Predict ‘complications (yes/no) following op’

from ‘current systolic BP’
 Logistic regression

Predict ‘complications (yes/no) following op’

from ‘pre-op systolic BP’

Loge [p/(1-p)] = b + m (pre-op BP)

Where p = probability of having a complication

 Screening Tests – Mammograms
Test

Abnormal Normal Total

Cancer
5,747 963 6,710

Non-Cancer
174,310 1,653,760 1,828,070

Total 180,057
1,654,723 1,834,780

NCI-funded Breast Cancer Surveillance Consortium (HHSN261201100031C). Downloaded 09/01/2015 from the Breast Cancer Surveillance
Consortium Web site - http://breastscreening.cancer.gov/statistics/benchmarks/screening/2009/table7.html .
 Abnormal Normal Total

Cancer 5,747 963 6,710

Non-Cancer 174,310 1,653,760 1,828,070

Total 180,057 1,654,723 1,834,780

Sensitivity:
The percentage of the test being positive among the
people who really have the disease.
= 5,747 / 6,710 = 86%
 Abnormal Normal Total

Cancer 5,747 963 6,710

Non-Cancer 174,310 1,653,760 1,828,070

Total 180,057 1,654,723 1,834,780

Specificity:
The percentage of the test being negative among the
people who really don’t have the disease.
= 1,653,760/1,828,070 = 90%
 Abnormal Normal Total

Cancer 5,747 963 6,710

Non-Cancer 174,310 1,653,760 1,828,070

Total 180,057 1,654,723 1,834,780

Positive predictive value (PPV):

The percentage of test positive results that are true
positives
= 5,747 / 180,057 = 3%
 Abnormal Normal Total

Cancer 5,747 963 6,710

Non-Cancer 174,310 1,653,760 1,828,070

Total 180,057 1,654,723 1,834,780

Negative predictive value (NPV):

The percentage of test negative results that are true
negatives
= 1,653,760 / 1,654,723 = 99.9%
 Abnormal Normal Total

Cancer True positive False negative

Non-Cancer False positive True negative

Total
 Prevalence of disease

• PPV and NPV vary with prevalence

• Sensitivity and specificity do not vary with

prevalence
 Varying prevalence

Prevalence Sensitivity Specificity PPV NPV

1.0 per 1,000 0.86 0.90 0.85% 99.98%

3.6 per 1,000 0.86 0.90 3% 99.9%

20 per 1,000 0.86 0.90 15% 99.7%

 Relationship between PPV, sensitivity,
specificity and prevalence

𝑃𝑃𝑉 =

𝑠𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 × 𝑝𝑟𝑒𝑣𝑎𝑙𝑒𝑛𝑐𝑒
𝑠𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 × 𝑝𝑟𝑒𝑣𝑎𝑙𝑒𝑛𝑐𝑒+(1−𝑠𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦)×(1−𝑝𝑟𝑒𝑣𝑎𝑙𝑒𝑛𝑐𝑒)
 Relationship between NPV, sensitivity,
specificity and prevalence

𝑁𝑃𝑉 =

𝑠𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦 ×(1−𝑝𝑟𝑒𝑣𝑎𝑙𝑒𝑛𝑐𝑒)
1−𝑠𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 × 𝑝𝑟𝑒𝑣𝑎𝑙𝑒𝑛𝑐𝑒+𝑠𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦×(1−𝑝𝑟𝑒𝑣𝑎𝑙𝑒𝑛𝑐𝑒)
To assess the effectiveness of a new ultrasound screening test, 100
women with abnormal mammogram results were screened. 25
women were later found, by pathology, to have cancer. 20 of these
25 women had a positive ultrasound result. 65 of those without
cancer had a negative ultrasound.

(a) The PPV is 20/35

(b) The sensitivity of a test depends on the prevalence of cancer
(c) The sensitivity is 20/100
(d) The specificity is 65/75
(e) Specificity = 1 - sensitivity
Question 1

Multiple linear regression analysis (multivariable

analysis) can be used to:
Select the best statement

(a) adjust for confounding by adding the confounding

variable as an independent variable.
(b) determine if a study result is clinically important.
(c) adjust for loss to follow-up bias.
(d) calculate a study's power.
(e) correct the p-value for multiple comparisons.
Question 2

Select the best statement:

(a) The correlation coefficient, r, is dependent on the units of

measurement.
(b) In correlation, it matters which variable is put on the x-axis
and which variable is put on the y-axis.
(c) A correlation coefficient is always positive
(d) The Pearson correlation coefficient, r, is sensitive to
extreme values (outliers).
(e) Correlation is a measure of the relationship between the
population mean and the sample mean.
Question 3

Select the best statement concerning a scatterplot:

(a) It plots the distribution of potential confidence intervals in

a data set.
(b) It plots the distribution of a single variable.
(c) It provides a visual description of the distribution of
potential sample means drawn from a given population.
(d) It plots the standard errors of randomly selected data from
a given population.
(e) It plots the values of two numerical variables in a data set.
Question 4

The following is a linear regression model:

Energy Expenditure = 0.56 x (Caloric Intake) + 502.

Select the correct answer:

(a) The model adjusts for potential confounding by caloric intake.

(b) For every unit change in energy expenditure there is a 0.56 unit
change in mean caloric intake.
(c) For every unit change in caloric intake there is a 0.56 unit change
in mean energy expenditure.
(d) For every unit change in energy expenditure there is a change in
caloric intake of 0.56 +502.
(e) 502 is the value of caloric intake when energy expenditure is zero.
Question 5

Follow-up results for up to 84 months were reported for a series of 323 men with prostate
cancer
treated by brachytherapy (BXT). At entry, a Prognostic Index was calculated for each man,
and each classed as low, intermediate or high risk.
The table below gives summary statistics comparing the three risk groups for age and the
outcome measure which was the proportion with PSA £0.2 ng/ml at five years.

What is the most appropriate statistical test to compare age by the level of risk ?

(a) a one way analysis of variance

(b) a log-rank test
(c) a chi-squared test
(d) an unpaired t-test
(e) none of the above
Question 6

A cohort of cancer patients experiencing nausea and

vomiting during chemotherapy were given a new anti-
nausea treatment. A symptom score (1-10) was recorded
before and after treatment. The clinical effect of the
medication could be evaluated using:

(a) Wilcoxon signed rank test

(b) Spearman correlation coefficient
(c) Mann-Whitney U-test.
(d) McNemar’s test
(e) One-factor ANOVA.
Question 7

Select the best statement concerning Pearson's correlation coefficient:

(a) The associated hypothesis test has a null hypothesis that the correlation is
equal to one.
(b) It reflects the magnitude of the association for linear and non-linear
relationships between two numerical variables.
(c) A Pearson's correlation coefficient of zero indicates there is no linear
association between the two variables.
(d) A Pearson's correlation coefficient of positive one indicates there is no linear
association between the two variables.
(e) A Pearson's correlation coefficient of negative one indicates there is a non-
linear relationship between the two variables.
Question 8

Results of a liver scan (positive vs negative) and subsequent

pathology result (abnormal vs normal) were obtained for 344
patients. Abnormal pathology results were found for 258, and
positive liver scans were found for 263 patients. Of these 263
patients, 231 had abnormal pathology

(a)The sensitivity of the liver scan is 231/263

(b)The prevalence of abnormal pathology in the cohort is 258/344
(c)The specificity of the liver scan cannot be determined
(d)The PPV of the liver scan is 263/344
(e)The NPV of the liver scan cannot be determined