You are on page 1of 100

# Unit 14: Nonparametric

Statistical Methods

Ramón V. León

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 1

Introductory Remarks
• Most methods studied so far have been based on
the assumption of normally distributed data
– Frequently this assumption is not valid
– Sample size may be too small to verify it
• Sometimes the data is measured in an ordinal scale
• Nonparametric or distribution-free statistical
methods
– Make very few assumptions about the form of the
population distribution from which the data are sampled
– Based on ranks so they can be used on ordinal data
• Will concentrate on hypothesis tests but will also
mention confidence interval procedures.
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 2
Inference for a Single Sample
Consider a random sample x1 , x2 ,..., xn from a population
with unknown median µ .
(Recall that for nonnormal (especially skewed)
distributions the median is a better measure of the center
than the mean.)

H 0 : µ = µ0 vs. H1 : µ > µ0
Example: Test whether the median household income of a
population exceeds \$50,000 based on a random sample of
household incomes from that population
For simplicity we sometimes present methods for one-sided tests.
Modifications for two-sided tests are straightforward and are given in the
textbook Some examples in these notes are two-sided tests.
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 3
Sign Test for a Single Sample
H 0 : µ = µ0 vs. H1 : µ > µ0
Sign test:
1. Count the number of xi 's that exceed µ0 . Denote this
number by s+ , called the number of plus signs. Let
s− = n − s+ , which is the number of minus signs.
2. Reject H 0 if s+ is large or equivalently if s− is small.
Test idea:
Under the null hypothesis s+ has a binomial distribution,
Bin (n, ½). So this test is simply the test for binomial proportions
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 4
Sign Test Example
A thermostat used in an electric device is to be checked for the
accuracy of its design setting of 200ºF. Ten thermostats were
tested to determine their actual settings, resulting in the
following data:

202.2, 203.4, 200.5, 202.5, 206.3, 198.0, 203.7, 200.8, 201.3, 199.0

H 0 : µ = 200 vs H1 : µ ≠ 200
s+ = 8 = number of data values > 200, so
10 10
  1 
1010 2
  1 
10
P-value = 2∑     = 2∑     = 0.110
i =8  i   2  i =0  i   2 

(The t test based on the mean has P-value = 0.0453. However recall that
the t test assumes a normal population)
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 5
Normal Approximation to Test Statistic
If the sample size is large ( ≥ 20) the common of S+ and S− is
approximated by a normal distribution with
1 n
E ( S + ) = E ( S− ) = np = n   = ,
2 2
 1  1  n
Var ( S+ ) = Var ( S − ) = np(1 − p) = n     =
 2  2  4
Therefore can perform a one-sided z - test with
s − n 2 −1 2
z= +
n4

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 6

P-values for Sign
Test Using JMP

## Based on normal approximation

to the binomial ( = z2 )
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 7
Treatment of Ties
• Theory of the test assumes that the distribution of
the data is continuous so in theory ties are
impossible
• In practice they do occur because of rounding
• A simple solution is to ignore the ties and work
only with the untied observation. This does reduce
the effective sample size of the test and hence its
power, but the loss is not significant if there are
only a few ties

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 8

Comfidence Interval for µ
Let x(1) ≤ x(2) ≤ ⋅⋅⋅ ≤ x( n ) be the ordered data values.
Then a (1-α )-level CI for µ is given by
x(b +1) ≤ µ ≤ x( n −b )
where b = bn ,1−α 2 is the lower α 2 critical point
of the Bin ( n,1 2 ) distribution.

## Note: Not all confidence levels are possible because

of the discreteness of the Binomial distribution

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 9

Thermostat Setting: Sign Confidence
Interval for the Median
From Table A.1 we see that for n = 10 and p=0.5,
the lower 0.011 critical point of the binomial
distribution is 1 and by symmetry the upper 0.011
critical point is 9.
Setting α 2 = 0.011 which gives 1-α = 1 − 0.022 = 0.978,
we find that
199.0 = x(2) ≤ µ ≤ x(9) = 203.7
is a 97.8% CI for µ .
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 10
Sign Test for Matched Pairs

## Drop 3 tied pairs. Then s+ = 20; s- = 3

7/26/2004 Unit 14 - Stat 571 - Ramón V. León 11
Sign Test for Matched Pairs

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 12

Sign Test for Matched Pairs in JMP
Pearson’s p-value is not the same
as the book’s two-sided P-value
because the book uses the
continuity correction in the
normal approximation to the
binomial distribution,
i.e, book uses z = 3.336 (Page
567) rather than z = 3.544745
used by JMP. Note that
(3.544745)2 = 12.5652

book
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 13
Wilcoxon Signed Rank Test
H 0 : µ = µ0 vs. H1 : µ ≠ µ0
More powerful than the sign test, however, it requires the
assumption that the population distribution is symmetric
1. Rank and order the differences in terms of their absolute value
Example 14.1 and 14.4: Thermostat Setting is 200° F

## 2. Calculate w+ = sum of the ranks of the positive differences

w+ = 6 + 8 + 1 + 7 + 10 + 9 + 2 + 4
3. Reject H0 if w+ is large or small
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 14
Wilcoxon Signed Rank Test in JMP

## This test finds a

significant
difference at
α=0.05 while the
sign test did not
at even α=0.1
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 15
Normal Approximation in the Wilcoxon
Signed Rank Test
For large n, the null distribution of W ∼ W+ ∼ W-
can be well-approximated by a normal distribution
with mean and variance given by
n(n + 1) n(n + 1)(2n + 1)
E (W ) = and Var (W ) = .
4 24
For large samples a one-sided (greater than median)
z-test uses the statistic
w+ − n(n + 1) / 4 − 1/ 2
z=
n(n + 1)(2n + 1) 24

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 16

Importance of Symmetric Population
Assumption

Here even though H0 is true the long right hand tail makes the positive
differences tend to be larger in magnitude than the negative differences,
resulting in higher ranks. This inflates w+ and hence the test’s type I error
probability.
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 17
Null
Distribution of
the Wilcoxon
Signed Rank
Statistics

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 18

Null Distribution of the
Wilcoxon Signed Rank Statistics

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 19

Wilcoxon Signed Rank Statistic:
Treatment of Ties
• There are two types of ties
– Some of the data is equal to the median
• Drop these observations
– Some of the differences from the median
may be tied
• Use midrank, that is, the average rank
For example, suppose
d1 = −1, d 2 = +3, d3 = −3, d 4 = +5 With ties Table A.10
Then is only approximate
(2 + 3)
r1 = 1, r2 = r3 = = 2.5, r4 = 4
2
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 20
Wilcoxon Sign Rank Test:
Matched Pair Design Notice that we drop the
three zero differences
Example 14.5: Comparing Two
Methods of Cardiac Output Notice that we average the
tied ranks

Two-Side P-values
Signed test: 0.0008
Signed Rank test: 0.0002
t-test: 0.0000671 (Page 284)

## (Notice that these tests require

progressively more stringent
population of differences)

JMP Calculation

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 22

Signed Rank Confidence Interval for the Median

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 23

Thermostat Setting: Wilcoxon Signed
Rank Confidence Interval for Median
From Table A.10 we see that for n = 10, the upper 2.4%
critical point is 47 and by symmetry the lower 2.4%
10(10 + 1)
critical point is - 47 = 55 - 47 = 8.
2
Setting α 2 = 0.024 and hence 1-α=1-0.048=0.952
we find that
+200.10 = x9 = x8+1 ≤ µ ≤ x47 = 203.55
is a 95.2% CI for µ

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 24

Inferences for Two Independent Samples
One wants to show that the observations from one population
tend to be larger than those from another population based on
independent random samples

## x1 , x2 ,..., xn1 and y1 , y2 ,..., yn2

Examples:
•Treated patients tend to live longer than untreated patients
•An equity fund tends to have a higher yield than a bond fund

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 25

Wilcoxon-Mann-Whitney Test Example:
Time to Failure of Two Capacitor Groups

## Reject for extreme values of w1.

7/26/2004 Unit 14 - Stat 571 - Ramón V. León 26
Stochastic Ordering of Populations
X is stochastically larger than Y ( X Y )
if for all real numbers u ,
P ( X > u ) ≥ P(Y > u )
equivalently,
P ( X ≤ u ) = F1 (u ) ≤ F2 (u ) = P(Y ≤ u )
with strict inequality for at least some u.
Denoted by X Y or equivalently by F1 < F2 )

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 27

Stochastic Ordering Especial Case:
Location Difference

## θ is called a location parameter

Notice that X 2 ≺ X 1 iff θ 2 < θ1
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 28
Wilcoxon-Mann-Whitney Test
H 0 : F1 = F2 ( X ∼ Y )
Alternatives :
One sided: H1 : F1 < F2 ( X Y)
Two sided: H1 : F1 < F2 or F2 < F1 ( X Y or Y X)

## Notice that the alternative is not H1 : F1 ≠ F2

(Kolmogorov-Smirnov Test can handle this alternative)

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 29

Wilcoxon Version of the Test
H 0 : F1 = F2 ( X ∼ Y ) vs. H1 : F1 < F2 ( X Y)
1. Rank all N = n1 + n2 observations,
x1 , x2 ,..., xn1 and y1 , y2 ,..., yn2
in ascending order
2. Sum the ranks of the x's and y's separately.
Denote these sums by w1 and w2
3. Reject H 0 if w1 is large or equivalently w2 is small

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 30

Mann-Whitney Test Version

## The advantage of using the Mann-Whitney form of the test is that

the same distribution applies whether we use u1 or u2

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 31

Null Distribution of the Wilcoxon-
Mann-Whitney Test Statistic

## Under the null hypothesis each of these 10 ordering has an

equal chance of occurring, namely, 1/10
5
  = 10
 2
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 32
Null Distribution of the Wilcoxon-
Mann-Whitney Test Statistic

## P ( w1 ≥ 8) = 0.1 + 0.1 = 0.2 (one-sided p-value for w1 = 8)

( H1 : X Y)
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 33
Normal Approximation of Mann-
Whitney Statistic
For large n1 and n2 , the null distribution of U can be
well approximated by a normal distribution with
mean and variance given by
n1n2 n1n2 ( N + 1)
E (U ) = and Var (U ) =
2 12
A large sample one-sided z - test can be based on the statistic
u1 − n1n2 2 − 1 2
z=
n1n2 ( N + 1) ( H1 : X Y)
12

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 34

Treatment of Ties
A tie occurs when some x equal a y.
– A contribution of ½ is counted towards both u1
and u2 for each tied pair
– Equivalent to using the midrank method in
computing the Wilcoxon rank sum statistic

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 35

Wilcoxon-Mann-Whitney Confidence Interval

## Example14.8 shows that [d(18) , d(63) ] = [-1.1, 14.7] is a 95.6% CI

for the difference of the two medians of the failure times of capacitors.
This example is in the book errata since Table A.11 is not detailed enough.
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 36
Wilcoxon-Mann-Whitney Test in JMP

With
continuity
correction.
Used in
the book
which gets
a one-
sided p-
value of
0.0502

Without
continuity
correction
7/26/2004 z2=1.6882 Unit 14 - Stat 571 - Ramón V. León 37
Inference for Several Independent Samples:
Kruskal-Wallis Test

is a completely
randomized
design

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 38

Kruskal-Wallis Test
H 0 : F1 = F2 = ⋅⋅⋅ = Fa vs. H1 : Fi < Fj for some i ≠ j

## Distance from the

Reject H 0 if kw > χ 2
a −1,α average rank
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 39
Chi-Square Approximation
• For large samples the distribution of KW under the
null hypothesis can be approximated by the chi-
square distribution with a-1 degrees of freedom
• So reject H0 if

kw > χ a −1,α

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 40

Kruskal-Wallis Test Example

Reject if
kw is large.
χ 3,.005
2
= 12.837
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 41
Kruskal-Wallis Test in JMP

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 42

7/26/2004 Unit 14 - Stat 571 - Ramón V. León 43
•Case method is different from Unitary method
•Formula method is different from Unitary method

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 44

Pairwise Comparisons:
Is Any Pair of Treatments Different?

• One can use the Tukey Method on the average ranks to make
approximate pairwise comparisons.

## • This is one of many approximate techniques where ranks are

substituted for the observations in the normal theory methods.

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 45

7/26/2004 Unit 14 - Stat 571 - Ramón V. León 46
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 47
Tukey’s Test Applied to the Ranks Averaged

Lack of
agreement
with the more
precise method
of Example
14.10. Here
Equation
method also
seems to be
different from
Formula and
Case method

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 48

Example of Friedman’s Test

## Ranking is done within blocks

χ 7,.025
2
= 16.012 - P-value =.0040 vs. .0003 for ANOVA table
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 49
Inference for Several Matched Samples
Randomized Block Design:
i a ≥ 2 treatment groups
i b ≥ 2 blocks
i yij = observation on the i-th treatment in the j-th block
i Fij = c.d.f of r.v. Yij corresponding to the observed value yij
For simplicity assume Fij ( y ) = F ( y − θ i − β j )
iθ i is the "treatment effect"
i β j is the "block effect"
i.e., we assume that there is no
treatment by block interaction

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 50

Friedman Test
H 0 : θ1 = θ 2 = ⋅⋅⋅ = θ a vs. H1 : θ i > θ j for some i ≠ j

## Distance from the total

Reject if fr > χ 2
a −1,α of the ranks from their
expected value when
there is no agreement
between the blocks
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 51
Pairwise Comparisons

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 52

Rank Correlation Methods
• The Pearson correlation coefficient
– measures only the degree of linear association between
two variables
– Inferences use the assumption of bivariate normality of
the two variables
• We present two correlation coefficients that
– Take into account only the ranks of the observations
– Measure the degree of monotonic (increasing or
decreasing) association between two variables

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 53

Motivating Example
( x, y ) = (1, e1 ), (2, e 2 ), (3, e3 ), (4, e 4 ), (5, e5 )

## Note that there is a perfect positive association between between

x and y with y = ex.

## •The Pearson correlation correlation coefficient is only 0.886

because the relationship is not linear

for these data

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 54

Spearman’s Rank Correlation Coefficient

## Ranges between –1 and +1 with rs = -1 when there is a perfect negative

association and rs = +1 when there is a perfect positive association

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 55

Example 14.12 (Wine Consumption and
Heart Disease Deaths per 100,000

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 56

7/26/2004 Unit 14 - Stat 571 - Ramón V. León 57
Calculation of Spearman’s Rho

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 58

Test for Association Based on Spearman’s Rank
Correlation Coefficient

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 59

Hypothesis Testing Example
H 0 : X = Wine Consumption and
Y = Heart Disease Deaths are independent.
vs.
H1 : X and Y are (negatively or positively) associated

z = rS n − 1 = −0.826 19 − 1 = −3.504
Two-Sided P − value = 0.0004

## Evidence of negative association

7/26/2004 Unit 14 - Stat 571 - Ramón V. León 60
JMP Calculations:
Pearson Correlation
350

300

## Heart Disease Deaths

250

200

150

100

50
0 2 4 6 8 10
Alcohol from Wine

## Plot is fairly linear

Pearson correlation

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 61

JMP Calculations: Spearman Rank Correlation

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 62

Kendall’s Rank Correlation Coefficient:
Key Concept Examples
Concordant pairs:
(1,2), (4,9) (1 - 4)(2 - 9)>0 Kendall’s
(4,2), (3,1) (4 - 3)(2 - 1)>0 idea is to
compare the
Discordant pairs: number of
(1,2), (9,1) (1 - 9)(2 - 1)<0 concordant
(2,4), (3,1) (2 - 3)(4 - 1)<0 pairs to the
number of
Tied pairs: discordant
(1,3), (1,5) (1 – 1)(3 – 5)=0 pairs in
(1,4), (2,4) (1 – 2)(4 – 4)=0 bivariate data
(1,2), (1,2) (1 – 1)(2 – 2)=0
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 63
(X, Y)
(1, 2) Kendall’s Tau
(3, 4) Example
(2, 1)
n  3
Number of pairwise comparisons =   =   = 3 = N
 2   2
Concordant pairs: Discordant pairs: Nc − Nd
τˆ =
(1,2) (3,4) (1,2) (2,1) N
(3,4) (2,1) 2 −1
=
3
Nc = 2 Nd=1 1
=
3
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 64
Kendall’s Rank Correlation Coefficient:
Population Version

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 65

Kendall’s Rank Correlation Coefficient:
Sample Estimate
Let N c = Number of concordant pairs in the data
Let N d = Number of disconcordant pairs in the data
n
Let N =   be the number of pairwise comparisons among
2
the observations ( xi , yi ), i = 1, 2,..., n. Then
Nc − Nd
τˆ = and N c + N d = N if no ties
N
Nc − Nd
τ=
ˆ if ties
( N − Tx )( N − Ty )
where Tx and Ty are corrections for the number of tied pairs.

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 66

Hypothesis of Independence Versus Positive Association

Wine data:
-.696 -4.164

JMP
Calculations:
Kendall’s
Rank
Correlation
Coefficient

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 68

Kendall’s Coefficient of Concordance
• Measure of association between several matched samples
• Closely related to Friedman’s test statistic
– Consider a candidates (treatments) and b judges (blocks) with each
judge ranking the a candidates
• If there is perfect agreement between the judges, then each
candidate gets the same rank. Assuming the candidates are
labeled in the order of their ranking, the rank sum for the ith
candidate would be ri= ib
• If the judges rank the candidates completely at random
(“perfect disagreement”) then the expected rank of each
candidate would be [1+2+…+a]/a =[a(a+1)/2]/a=(a+1)/2,
and the expected value of all the rank sums would equal to
b(a+1)/2

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 69

Kendall’s Coefficient of Concordance

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 70

Kendall’s Coefficient of Concordance
and Friedman’s Test

24.667
w= = 0.881
4(8 − 1)

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 72

Do You Need to Know More
“Nonparametric Statistical Methods, Second Edition”
by Myles Hollander and Douglas A. Wolfe. (1999)
Wiley-Interscience

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 73

Resampling Methods
• Conventional methods are based on the sampling
distribution of a statistic computed for the
observed sample. The sampling distribution is
derived by considering all possible samples of size
n from the underlying population.
• Resampling methods generate the sampling
distribution of the statistic by drawing repeated
samples from the observed sample itself. This
eliminates the need to assume a specific functional
form for the population distribution (e.g. normal).

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 74

Challenger Shuttle O-Ring Data

## Do we have statistical evidence that cold temperature leads to

more O-ring incidents?
•Notice that assumptions of two sample t test do not hold.
•What do we do?
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 75
Wrong t-test Analysis
Difference of Low
mean to High mean

## Notice that the assumptions of the

independent sample t-test do not hold,
i.e., data is not normal for each group.
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 76
Permutation Distribution of t Statistic

## Equivalent to selecting all simple random samples without replacement of

size 20 from the 24 data points, labeling these High and the rest Low
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 77
• A randomization test is a permutation test applied
to data from a randomized experiment.
Randomization tests are the gold standard for
establishing causality.
• A permutation test considers all possible simple
random samples without replacement from the set
of observed data values
• The bootstrap method considers a large number of
simple random samples with replacement from
the set of observed data values.

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 78

Calculation of t Statistics from 10,000
Bootstrap Samples
Think that we are placing the 24
Challenger data values in a hat.
And that we are randomly selecting
24 values with replacement from
the hat, labeling the first 20 values
High and the remaining 4 values
Low. We repeat these process
10,000 times. For each of these
10,000 bootstrap samples we
calculate the t-statistic. 35 t-
statistics values were greater than
-2 -1 0 1 2 3 4 5 6
or equal to 3.888 out of 10000 (if sp
= 0, t is defined to be 0). This gives
a bootstrap P-value of 35/10000 =
0.0035
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 79
Bootstrap Distribution of Difference
Between the Means
67 of the 10,000 differences
of the Low mean and the
High mean were greater than
or equal to 1.3. This gives a
bootstrap P-value of
67/10000 = .0067

Conclusion: Cold
weather increases the
-1 0 1 2 chance of O-ring
problems

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 80

Bootstrap Final Remarks
• The JMP files - that we used to generate the
bootstrap samples and to calculate the statistics -
are available at the course web site.
• There are bootstrap procedures for most types of
statistical problems. All are based on resampling
from the data.
• These methods do not assume specific functional
forms for the distribution of the data, e.g. normal
• The accuracy of bootstrap procedures depend on
the sample size and the number of bootstrap
samples generated

How Were
the
Bootstrap
Samples
Generated?

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 82

7/26/2004 Unit 14 - Stat 571 - Ramón V. León 83
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 84
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 85
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 86
Calculated Columns in JMP Samples File

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 87

7/26/2004 Unit 14 - Stat 571 - Ramón V. León 88
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 89
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 90
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 91
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 92
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 93
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 94
Bootstrap Estimate of the Standard Error of the Mean

## Summary: We calculate the standard deviation of the N

bootstrap estimates of the mean
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 95
BSE for Arbitrary Statistic

## Example: The bootstrap standard error of the median is calculated by

drawing a large number N, e.g. 10000, of bootstrap samples from the
data. For each bootstrap sample we calculated the sample median.
Then we calculate the standard deviation of the N bootstrap medians.

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 96

Estimated Bootstrap Standard Error for t-
statistics Using JMP

Note N =10,000
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 97
Bootstrap Standard Error Interpretation

## • Many bootstrap statistics have an

approximate normal distribution
• Confidence interval interpretation
– 68% of the time the bootstrap estimate (the
average of the bootstrap estimates) will be within
one standard error of true parameter value
– 95% of the time the bootstrap estimate (the
average of the bootstrap estimates) will be within
two standard error of true parameter value

## 7/26/2004 Unit 14 - Stat 571 - Ramón V. León 98

Bootstrap Confidence Intervals –
Percentile Method: Median Example
1. Draw N (= 10000) bootstrap samples from the data and for
each calculate the (bootstrap) sample median.
2. The 2.5 percentile of the N bootstrap sample medians will
be the LCL for a 95% confidence interval
3. The 97.5 percentile of the N bootstrap sample medians will
be the UCL for a 95% confidence interval

0.025 0.025

LCL UCL
7/26/2004 Unit 14 - Stat 571 - Ramón V. León 99
Do You Need to Know More?

## “A Introduction to the Bootstrap”

by Bradley Efrom and Robert J. Tibshirani.
(1993) Chapman & Hall/CRC