SAMPLE SIZE ESTIMATION

in Clinical Studies
• Istilah yang benar: besar sampel
• Jumlah subyek
• Jumlah pasien
• Jumlah sampel

Sampel darah, urin, etc → spesimen

 Which studies?
All studies require sample size estimation
except case report and case series
May use
– Formula
– Tables
– Computer generated
– Rule of thumbs
– Educated guess
 Usu. based on practical
purposes

Target population
(Domain)
Accessible
(Demographic & clinical)
population
(+ time, place)
Appropriate
sampling
technique

Actual
study Intended
subjects
Sample
Subjects [Subjects selected
completed for study]
[Non-response, drop outs,
the study
withdrawals, loss to follow-up]
 Sampling technique
Investigation
P
S S

Results

Inference

P value
Confidence intervals
 Why sample size?
To enable the investigator estimate the results of the
study with desired error in inferring into the
population
To ‘match’ the clinical importance and the statistical
significance: Too many subjects proves everything,
too few subjects proves nothing.
To keep the whole study in line with ethical
consideration: studies with too many or too few
subjects are unethical
Clinical importance vs. statistical significance

Cholesterol level,
mg/dl
300 220
Standard, n= 5000
R Clinical
Experimental, n=5000
218
300

t= df = 9998 p = 0.0023
Statistical
Statistically significant but clinically not important
Clinical importance vs. statistical significance

Success Failure

Old
0 10 10

New
3 7 10

Fischer exact test: p = 0.210

Clinically important but statistically insignificant

 Important concepts
Type 1 error (a): usu. 0.05, almost always 2-sided
Type 2 error (b): usu. 0.20, may 0.10 always 1-sided.
Power = (1-b) – ability to detect true difference
Prior knowledge of statistical characteristics of control
groups (proportion, mean, SD, etc)
Effect size = difference of outcome variable between
experimental and control groups. This should be based
on clinical judgment
 Commonly calculated sample size

Single proportion
Single mean
Difference between 2 proportions (independent) - X2 test
Difference between 2 means (independent) – t-test (ind)
Difference between 2 proportions (paired) – McNemar
Difference between 2 means (paired) – t-test (paired)
Equivalence (‘negative’) trials
Multivariate analysis (multiple / logistic regression)
How do experts develop
sample size formula?
Simple example: sample size for single proportion

z α 2 pq
n=
d2

n = number of participants
za = standard normal deviate for a, usu 1.96
p = estimated proportion, e.g. 0.6
q = 1-p, i.e. 0.4
d = acceptable deviation from estimate, e.g 0.1
d d
0 0.6 1

Then n = (1.962 x 0.6 x 0.4)/0.12 = 94

How to develop sample size formula
 Sample size formula for
single mean

n = number of subjects
s = SD (literature, pilot study, educated guess)
d = acceptable deviation from the predicted mean
 Sample size formula for
single proportion

n = number of subjects
Za = standard normal deviate for a
P = predicted proportion (literature, pilot study)
Q = (1-P)
d = acceptable deviation from predicted proportion
 Samplebetween
Difference size formula
2 means:
for mean independent
difference: independent
groups group

2 (z α + z β )s 2
n=
(x 1 − x 2 )2

n = number of participants per group

za = standard normal deviate for a
zb = standard normal deviate for b
x1 = mean of control (literature)
x2 = mean of experimental (judgment)
s = standard deviation of mean (literature)
Difference between 2 proportions:
independent groups
(z α 2PQ + z β P1Q 1 + P2 Q 2 ) 2
n=
(P1 − P2 ) 2

n = no of participants per group

Za = standard normal deviate for a
Zb = standard normal deviate for b
P2 = success rate in control group
P1 = success rate in intervention group
(P1-P2) = d = effect size =minimal clinically important
difference in success rate
P = (P1+P2)/2; Q = (1-P)
 Difference between 2 means:
paired groups

2
 (z α + z β )  s d 
n= 
 d 

Sd = SD of mean difference (not SD!)

d = minimal clinically important difference
Difference between 2 proportions:
paired data

{z a f + z β f − d2 } 2
np =
d2
alternativ e :
[z α + z b ] 2 f
np =
d2

np = number of pairs
f = proportion of participants pair with discordant
response (literature, pilot study)
d = (P1-P2)
 Drug B
Success Failure

Success a b
Drug A
Failure
c d

f= Proportion of participants with discordant

response = (b+c)/n
(previous evidence, assumption [educated guess],
pilot study)
 Equivalence trials
Non-inferiority trials
Negative trials
In contrast to usual trials which aim to prove that
the experimental drug gives better result than
standard drug or placebo, in equivalence trial
the investigator wish to show that the experimental
drug is not inferior to standard drug. The
experimental treatment may be cheaper, less
invasive, etc.
 Equivalence trial
2PQ (z α + z β ) 2
n1 = n 2 =
d2
n1, n2= no of participants per group
P = (P1+P2)/2; Q = (1-P)
d = maximal diff that clinically not important

1. Since we want to prove that P1 = P2, then (P1-P2) should be

0; since this is not possible, d should be as small as possible;
the usual used figure is d = 0.05
2. Since we want to detect the smallest difference, the power
should be as large as possible; b is usually set at 0.10 or
smaller
3. a of 0.05 is usually acceptable
 Sample size estimation for
multivariate analysis
Risk factors
A
B
Outcome
C
D

If risk factors AND outcome are numerical, → multiple regression

If risk factors are nominal, ordinal and numerical, while outcome is
nominal → logistic regression
 Sample size estimation for
multivariate analysis

Many attempts have been made to develop

sample size formula for multivariate analysis
However all formulas require estimation of
the overall correlation between independent
variables; this job is usually highly subjective
Many authorities recommend to using “rule
of thumbs”
Rule of thumbs for sample size estimation
for multivariate analysis (1)
The minimal number of subjects required should
be 5-50 (most frequently recommended: 10)
times as large as the number of independent
variables.
For example if we need to determine the
individual role of obesity, hypertension,
diabetes, LDL cholesterol level, sedentary life
style, and cigarette smoking for the
development of myocardial infarction, we would
need between 30 to 300 subjects (It is by no
means a smart estimation, is it?).
Rule of thumbs for sample size estimation
for multivariate analysis (2)

Assumption: One outcome usually

influenced by teens of “risk factors”
▪ > 400 - large
▪ 200-400 - moderate
▪ <200 - small
 Correction for predicted drop outs
Investigators may predict that a substantial
proportion of participants will not comply for one
or other reasons.
In this situation correction for predicted drop outs
is advisable. This does not exclude the maximal
attempts that should be made to keep them to
participate until the end of the study
Ideally no correction is needed for pragmatic
trials
Formula: n
n' =
1− f
Correction for predicted drop outs:
example
Suppose you have calculated that for your trial
160 participants will be needed per group.
When you predict that 15% of the participants
will not be comply for various reasons, the
number of participants in each group should be:
n
n' = n' =
160
= 188
1− f 1 − 0.15

Note that simply adding 15% from the calculated

number of participants (160+24=184) is not an
appropriate way for correction for drop outs
 Unequal allocation
So far it is assumed that the numbers of subjects all
treatment arm are the same
At times it is difficult (less patients, more expensive,
etc) to have participants for one treatment arm
and relatively easy to find participants for the
other treatment arm.
It is possible to reduce the number of participants
in one arm while maintaining the power of the
study, i.e. by increasing the number of participants
in the other treatment arm
Formula: n ' = ( c + 1 ) : 2 c  x n
c = number of case per control
 Unequal allocation
Suppose you have calculated that with equal
group size you would need 90 participants per
group. If you wish the experimental group half as
much as the control group, then:
n ' = ( c + 1 ) : 2 c  x n
c=2
n’ = [(2+1):4] x 90 = 68

It means that you will need 68 participants in

experimental arm and 136 in the other treatment
arm, or a total of 204 participants (vs. 180 total
in equal group sizes)
 Power calculation
It is not uncommon that at the end we only are
able to recruit less than intended number of
participants.
Similarly the results of the study (effect size,
proportions of success rate, etc) may be
substantially different from what have been
planned in sample size calculation.
Post-hoc power calculation may be performed by
entering the figures in the original formula used
 Power calculation: example
You have calculated the number of participants by using
formula for difference in proportion (equal sizes) by using
the following figures:
a= 0.05; b = 0.20; P1 (success rate in experimental
group) = 0.6; P2 (success rate in controls) = 0.4; and
found that 104 participants per group were needed.
At the end of the study you noted that P1 = 0.75; P2 =
0.45 while the numbers of participants in control and in
experimental groups were 90 and 92, respectively.
How do you estimate the actual power of the study?

(z α 2PQ + z β P1Q 1 + P2 Q 2 ) 2
n1 = n 2 =
(P1 − P2 ) 2
 Sample size for animal studies
Basically the same as for human subjects
Need for 3 Rs – to minimal number of subjects
➢ Replacement
➢ Reduction

➢ Refinement

Large effect size will reduce the no of subjects

Other “rule of thumbs” (Federer?)
 In general:

Smaller type I error (larger za) – more subjects

Smaller type II error (larger power) – more
subjects
More SD / variation – more subjects
Smaller effect size – more subjects
 Points to remember
Sample size calculation is just only an estimation. We
are supposed to use assumed figures rather than exact
ones. Do not use “strange figures” such as 13.4% or
0.279 for estimation. Figures quoted from literature
should be rounded to the nearest logical figures.

In many occasions pilot study may be needed if you

do not have the necessary assumed figures. This seems
to be logical, but results from such pilot study
frequently give only very rough estimate due to small
sample size.
 Points to remember (cont).
Effect size should NOT be based on literature, but on
clinical judgment, what is the minimum difference that
may change your current practice. Large effect size
will substantially reduce the sample size, but is not
acceptable because of 2 reasons:
➢ Very large effect size excludes the need to do the
study
➢ If your data ‘only’ give 30% effect size but your
sample size calculation used 40%, hypothesis
testing will result in ‘statistically not significant’ for
this remarkable clinical difference.
 Points to remember (cont).
Acceptable way to decrease sample size:
Use numerical instead of nominal variable
Use paired data instead of independent data
Use more precise variable
Use a more common outcome (if necessary surrogate
outcome; e.g. control of cholesterol level instead of
reduction of myocardial infarction)
Use unequal group sizes
 Epilogue

When calculating sample size you

should:
Be logic
Be parsimonious
Be realistic
Be creative
(Occasionally): Be stupid
 Take home message
a (za) - set by you – usu 0.05, 2-sided
b (zb)/power - set by you – usu 0.20, 1-sided
Proportion, mean, SD and other statistical
characteristics - literature, own data, pilot
Effect size – clinical judgment (less preferably
literature, pilot study)