Chapter 6: Order Statistics

Random variable are denoted by uppercase (mayúscula) letters (like X, Y, Z , etc)

mathematical variables are denoted by lowercase letters (minúscula) (like x, y, z , etc )
observations are denoted by lowercase (minúscula) letters (like x, y, z , etc )
parameters of the distribution are usually denoted by greek letters (such as α, β, θ, η, etc)
Statistical Inference, August 26, 2020

cdf: cumulative distribution function (función de distribución acumulativa)

pdf: probability density function or pdf or density function (función de densidad de
probabilidad)
f X (·): the pdf of X. The suffix (subscript) indicates that f X (·) is the density of X
d
X  Y means that X and Y have identical distribution.
The set {x : f X (x) > 0} is called support set.

The set of random variables X1 , X2 , . . . , X n is said to be a random sample of size n from

a population with density function f X (x) if

1. X1 , X2 , . . . , X n are independent,

2. X1 , X2 , . . . , X n have identical detrition which is same as that of X.

Consequently, the joint pdf has the form

f X1 ,X2 ,...,X n (x1 , x2 , . . . , x n ),  f X (x1 ) f X (x2 ) · · · f X (x n ).

Let X1 , X2 , . . . , X n denote a random sample from a population with continuous cdf

FX . First let FX be continuous, so that the probability is zero that any two or more of
these random variables have equal magnitudes. In this situation there exists a unique
ordered arrangement within the sample. Suppose that X(1) denotes the smallest of the set
X1 , X2 , . . . , X n ; X(2) denotes the second smallest; ... and X(n) denotes the largest. That is

X(1)  min{X1 , X2 , . . . , X n },

X(2)  min {X1 , X2 , . . . , X n } − {X(1) } ,



X(3)  min {X1 , X2 , . . . , X n } − {X(1) , X(2) } , . . . ,



1
 X(n−1)  min {X1 , X2 , . . . , X n } − {X(1) , X(2) , . . . , X(n−2) } ,


X(n)  min {X1 , X2 , . . . , X n } − {X(1) , X(2) , . . . , X(n−2) , X(n−1) }



 max{X1 , X2 , . . . , X n }.

Then
X(1) < X(2) < · · · < X(n)
Statistical Inference, August 26, 2020

denotes the original random sample after arrangement in increasing order of magnitude,
and these are collectively termed the order statistics of the random sample X1 , X2 , . . . , X n .
The rth smallest, 1 ≤ r ≤ n, of the ordered X ’s, X(r) , is called the rth-order statistic. Some
authors use alternative notation

X1:n < X2:n < · · · < X n:n

to denote the order statistics. Further, by taking Yi  X(i) , i  1, . . . , n one can also define
order statistics as
Y1 < Y2 < · · · < Yn .

Some familiar applications of order statistics, which are obvious on reflection, are as
follows:

1. X(n) , the maximum (largest) value in the sample, is of interest in the study of foods
and other extreme meteorological phenomena.

2. X(1) , the minimum (smallest) value, is useful for phenomena where, for example,
the strength of a chain depends on the weakest link.

3. The sample median, defined as X([n+1]/2) for n odd and any number between X(n/2)
and X(n/2+1) , for n even, is a measure of location and an estimate of the population
central tendency.

4. The sample midrange, defined as (X(1) +X(n) )/2, is also a measure of central tendency.

5. The sample range X(n) − X(1) is a measure of dispersion.

6. In some experiments, the sampling process ceases after collecting r of the observa-
tions. For example, in life-testing electric light bulbs, one may start with a group of
n bulbs but stop taking observations after the rth bulb burns out. Then information
is available only on the first r ordered ”lifetimes” X(1) < X(2) < · · · X(r) , where r ≤ n.
This type of data is often referred to as censored data.

2
 7. Order statistics are used to study outliers or extreme observations, e.g., when so-
called dirty data are suspected.

8. Order statistics are used extensively in statistical inference.

For example, if a random sample of five light bulbs is tested, the observed failure times
might be (in months) (x1 , . . . , x5 )  (5, 11, 4, 100, 17). Now, the actual observations would
Statistical Inference, August 26, 2020

have taken place in the order x 3  4, x1  5, x2  11, x5  17, and x4  100. But the
"ordered" random sample in this case is (y1 , . . . , y5 )  (4, 5, 11, 17, 100).
The joint distribution of the ordered variables (that is Y1 , . . . , Yn ) is not the same as the
joint distribution of the unordered variables (that is X1 , X2 , . . . , X n ). Note that because of
ordering

1. the random variables Y1 , . . . , Yn are not independent,

2. the except is same cases they are not identically distribution,

3. the random variable Yj and Yj , 1 ≤ i , j ≤ n may have different distribution and

different from that of X.

The joint density of ordered statistics Y1 , . . . , Yn are obtained by applying transforma-

tion on X1 , X2 , . . . , X n . Let us consider the case of three variables, that is n  3. In this
case
f X1 ,X2 ,X3 (x 1 , x2 , x3 ),  f X (x1 ) f X (x2 ) f X (x 3 ),

where a < x 1 , x2 , x3 < b, and the transformation

Y1  X(1)  min{X1 , X2 , X3 }, Y2  min{{X1 , X2 , X3 } − {X(1) }}, Y3  max{X1 , X2 , X3 }.

Let us do some mathematics to compute the joint density of Y1 , Y2 , Y3 . Note that whenever
I do mathematics I use lowercase letters. Observe that

y1  x(1)  min{x 1 , x2 , x3 }, y2  min{{x1 , x2 , x3 } − {x (1) }}, y3  max{x1 , x2 , x3 }

which gives (3!  6) six different (distinct) possibilities for substitution, namely,

y1  x 1 , y2  x 2 , y3  x 3 ,

y1  x 2 , y2  x 1 , y3  x 3 ,

3
 y1  x 1 , y2  x 3 , y3  x 2 ,

y1  x 2 , y2  x 3 , y3  x 1 ,

y1  x 3 , y2  x 1 , y3  x 2 ,

y1  x 3 , y2  x 2 , y3  x 1 ,

It is clear that the above defined transformation is no one-to-one, and it may be carried
Statistical Inference, August 26, 2020

out by partitioning the domain into subsets A1 , A2 , . . . , A6 such that the transformation is
one-to-one on each subset. Let

S  {(x1 , x2 , x2 ) : a < x1 < b, a < x2 < b, a < x3 < b}

and corresponding to the above six equations define six sets A1 , A2 , . . . , A6 as

A1  {(x 1 , x2 , x3 ) : x1 < x 2 < x3 },

A2  {(x 1 , x2 , x3 ) : x2 < x 1 < x3 },
A3  {(x 1 , x2 , x3 ) : x1 < x 3 < x2 },
A4  {(x 1 , x2 , x3 ) : x2 < x 3 < x1 },
A5  {(x 1 , x2 , x3 ) : x3 < x 1 < x2 },
A6  {(x 1 , x2 , x3 ) : x3 < x 2 < x1 }.

One can show that A1 , A2 , . . . , A6 are mutually disjoint. That is A i ∩ A j  φ for 1 ≤ i ,

j ≤ 6. Here we will show that A1 ∩ A2  φ. Let x3  c is fixed number such that
x2 < c < b, then
(x1 , x 2 , c) ∈ A1 if and only if x1 < x2

and
(x1 , x 2 , c) ∈ A2 if and only if x1 > x2

Since the conditions x1 < x2 and x 1 > x2 are contradictory, the triplet (x1 , x2 , c) can not
belong to A1 and A2 simultaneously. Hence A1 and A2 are disjoint. Further, observe that
S  A1 ∪ A2 ∪ · · · ∪ A6 and the range of the transformation is B  {(y1 , y2 , y3 ) : a < y l <
y2 < y3 < b}.
Now, writing x1 , x2 , x 3 in terms of y1 , y2 , y3 (see definitions of A1 , A2 , . . . , A6 ), we have
six possible inverse transformations as

x 1  y1 , x 2  y2 , x 3  y3 ,

4
 x 1  y2 , x 2  y1 , x 3  y3 ,

x 1  y1 , x 2  y3 , x 3  y2 ,

x 1  y3 , x 2  y1 , x 3  y2 ,

x 1  y2 , x 2  y3 , x 3  y1 ,

x 1  y3 , x 2  y2 , x 3  y1 .
Statistical Inference, August 26, 2020

For a better understands of the transformation see Figure

One can check that for the transformation of n variables there are n! inverse transfor-
mations. Further, by using the definition of the Jacobian of transformation, namely,

∂x ∂x ∂x
 
 
 
 ∂u ∂v ∂w 
∂y ∂y ∂y
 
J(x, y, z → u, v, w)  
 
∂u ∂v ∂w

 

 ∂z ∂z ∂z 

∂u ∂v ∂w
 

one can check that

| J(x1 , x2 , x3 → y1 , y2 , y3 )|  1,

| J(x 1 , x2 , x3 → y2 , y1 , y3 )|  1,

| J(x 1 , x2 , x3 → y1 , y3 , y2 )|  1,

| J(x1 , x2 , x 3 → y3 , y1 , y2 )|  1.

| J(x 1 , x2 , x3 → y2 , y3 , y1 )|  1,

| J(x 1 , x2 , x3 → y3 , y2 , y1 )|  1,

Now, the joint density of Y1 , Y2 , Y3 is derived as

fY1 ,Y2 ,Y3 (y1 , y2 , y3 )  f X (y1 ) f X (y2 ) f X (y3 )(1) + f X (y2 ) f X (y1 ) f X (y3 )(1)
+ f X (y1 ) f X (y3 ) f X (y2 )(1) + f X (y3 ) f X (y1 ) f X (y2 )(1)
+ f X (y2 ) f X (y3 ) f X (y1 )(1) + f X (y3 ) f X (y2 ) f X (y1 )(1)
 6 f X (y1 ) f X (y2 ) f X (y3 )
 3! f X (y1 ) f X (y2 ) f X (y3 ), a < y1 < y2 < y3 < b

where the last line has been obtained because the product is commutative.

5
 Note that we can write the joint density of Y1 , Y2 , Y3 as

(function of y1 only) × (function of y2 only) × (function of y2 only).

But only the factorization of the joint density is not sufficient for independence. You have
to look at the support set of fY1 ,Y2 ,Y3 (y1 , y2 , y3 ). If the support set of the joint density is not
a cartesian product, the random variables are not independent. In this case the support
set of fY1 ,Y2 ,Y3 (y1 , y2 , y3 ) can not be written as cartesian product and therefore Y1 , Y2 , Y3
Statistical Inference, August 26, 2020

are not independent.

Now you can easily state the joint density of Y1 , . . . , Yn . This is Theorem 6.5.1 of the
book.

If X1 , X2 , . . . , X n is a random sample from a population with continuous pdf f X (x), then

the joint pdf of the order statistics Y1 , . . . , Yn is

fY1 ,Y2 ,...,Yn (y1 , y2 , . . . , y n )  n! f X (y1 ) f X (y2 ) · · · f X (y n ),

where a < y1 < y2 , · · · < y n < b.

Suppose that X1 , X2 , and X3 represent a random sample of size 3 from a population

with pdf
f (x)  2x, 0 < x < 1.

It follows that the joint pdf of the order statistics Y1 , Y2 , and Y3 is

fY1 ,Y2 ,Y3 (y1 , y2 , y3 )  3!(2y1 )(2y2 )(2y3 )

 48y1 y2 y3 , 0 < y1 < y2 < y3 < 1,

and zero otherwise.

Quite often one may be interested in the marginal density of a single order statistic,
say k, and this density can be obtained in the usual fashion by integrating over the other
variables. In this example, let us find the marginal pdf of the smallest order statistic,
Y1 . WE proceed as follows: first, by integrating y3 in the joint density fY1 ,Y2 ,Y3 (y1 , y2 , y3 ),
we get the marginal density fY1 ,Y2 (y1 , y2 ) and then, integrating y2 in the joint density
fY1 ,Y2 (y1 , y2 ), we get the marginal density fY1 (y1 ). Let us integrate y3 to get
∫ 1
fY1 ,Y2 (y1 , y2 )  fY1 ,Y2 ,Y3 (y1 , y2 , y3 ) dy3
y2

6
 ∫ 1
 48y1 y2 y3 dy3
y2
" #1
y32
 48y1 y2
2
y2

 24y1 y2 (1 − y22 ),

where 0 < y1 < y2 < 1. Now, integrate y2 in the joint density fY1 ,Y2 (y1 , y2 ) to get
Statistical Inference, August 26, 2020

∫ 1
fY1 (y1 )  fY1 ,Y2 (y1 , y2 ) dy2
y1
∫ 1
 24y1 y2 (1 − y22 ) dy2 .
y1

Substitute 1 − y22  z with −2y2 dy2  dz in the above integral to get

∫ 1 ∫ 0
1 1
y2 (1 − y22 ) dy2  − z dz  (1 − y12 )2 .
y1 2 1−y 2 4
1

Now, substituting appropriately, we get

fY1 (y1 )  6y1 (1 − y12 )2 0 < y1 < 1.

If we want to know the probability that the smallest observation is below some value, say
0.1, it follows that ∫ 0.1
p[Y1 ≤ 0.1]  6y1 (1 − y12 )2 dy1  0.03.
0

To derive the marginal pdf of the largest order statistic, Y3 , we proceed as follows:
first, by integrating y1 in the joint density fY1 ,Y2 ,Y3 (y1 , y2 , y3 ) we get the marginal density
fY2 ,Y3 (y2 , y3 ) and then integrating y2 in the joint density fY2 ,Y3 (y3 , y3 ) we get the marginal
density fY3 (y3 ). Let us integrate y1 to get
∫ y2
fY2 ,Y3 (y2 , y3 )  fY1 ,Y2 ,Y3 (y1 , y2 , y3 ) dy1
0 ∫ y2
 48y2 y3 y1 dy1
0
" # y2
y12
 48y2 y3
2
0
 24y23 y3 ,

7
 where 0 < y2 < y3 < 1. Now, integrate y2 in the joint density fY2 ,Y3 (y2 , y3 ) to get
∫ y3
fY3 (y3 )  fY2 ,Y3 (y2 , y3 ) dy2
0 ∫ y3
 24y3 y23 dy2
0
 6y , 5
0 < y3 < 1.
Statistical Inference, August 26, 2020

To compute the marginal density of Y2 we have two choices: we integrate y3 in the density
of fY2 ,Y3 (y2 , y3 ) or we may choose to integrate y2 in the density of fY1 ,Y2 (y1 , y2 ). The
marginal density of Y2 is derived as
∫ 1
fY2 (y2 )  fY2 ,Y3 (y2 , y3 ) dy3
y2
∫ 1
 24y23 y3 dy3
y2

 12y23 (1 − y22 ), 0 < y2 < 1.

It is possible to derive an explicit general formula for the distribution of the kth order
statistic in terms of the pdf, f X (x), and CDF, FX (x), of the population random variable X.
If X is a continuous random variable with f X (x) > 0 on a < x < b (a may be −∞ and b
may be +∞), then, for example, for n  3,
"∫ #
∫ b b
f y1 (y1 )  6 f X (y1 ) f X (y2 ) f X (y3 ) dy3 dy2
y1 y2
"∫ #
∫ b b
 6 f X (y1 ) f X (y2 ) f X (y3 ) dy3 dy2 .
y1 y2

Now, by using the definition of cdf, the integral in the square bracket is evaluated as
∫ b b
f X (y3 ) dy3  FX (y3 )  FX (b) − FX (y2 )  1 − FX (y2 ),

y2
y2

where we have used the result FX (b)  1. Now,

∫ b
f y1 (y1 )  6 f X (y1 )
 
f X (y2 ) 1 − FX (y2 ) dy2
y1
2
 3 f X (y1 ) 1 − FX (y1 ) , a < b < 1,


where the last line has been obtained by subtitling 1 − FX (y2 )  z and evaluating the
resulting integral.

8
 Similarly,
"∫ #
∫ y2 b
f y2 (y2 )  6 f X (y1 ) f X (y2 ) f X (y3 ) dy3 dy1
a y2
∫ y2  "∫ b
#
 6 f X (y2 ) f X (y1 )dy1 f X (y3 ) dy3
a y2

 6 f X (y2 )FX (y2 ) 1 − FX (y2 ) , a < y2 < b,

 
Statistical Inference, August 26, 2020

where we have used the result FX (b)  1 and FX (b)  0 and

∫ y3 ∫ y2 
f y3 (y3 )  6 f X (y1 ) f X (y2 ) f X (y3 ) dy1 dy2
a a
∫ y3 ∫ y2 
 6 f X (y3 ) f X (y2 ) f X (y1 ) dy1 dy2
a a
∫ y3
 6 f X (y3 ) f X (y2 )FX (y2 )dy2
a
2
 3 f X (y3 ) FX (y3 ) , 0 < y3 < b.


Next, consider the joint density of Y1 , . . . , Yn and integrate it with respect to y1 to get
the joint density of Y2 , . . . , Yn as
∫ y2
fY2 ,...,Yn (y2 , . . . , y n )  n! f X (y2 ) · · · f X (y n ) f X (y1 ) dy1
a
FX (y2 )
 n! f X (y2 ) · · · f X (y n ) , a < y2 < y3 · · · < y n < b.
1
Next, integrating y2 in the above density, the joint density of Y3 , . . . , Yn is derived as
y3
FX (y2 )
∫
fY3 ,...,Yn (y3 , . . . , y n )  n! f X (y3 ) · · · f X (y n ) dy2
a 1
[FX (y3 )]2
 n! f X (y3 ) · · · f X (y n ) , a < y3 < y4 · · · < y n < b.
(1)(2)
Further, integrating y3 in the above density, the joint density of Y4 , . . . , Yn is derived as
y4
[FX (y3 )]2
∫
fY4 ,...,Yn (y4 , . . . , y n )  n! f X (y4 ) · · · f X (y n ) dy3
a (1)(2)
[FX (y4 )]3
 n! f X (y4 ) · · · f X (y n ) , a < y3 < y4 · · · < y n < b.
(1)(2)(3)
Similarly, one-by-one, we can integrate y4 , y5 , . . . , y j−1 (in this order) to get

[FX (y j )] j−1
fYj ,...,Yn (y j , . . . , y n )  n! f X (y j ) · · · f X (y n ) , a < y j < · · · < y n < b.
(1)(2)(3) · · · (j − 1)

9
 Next, let us start integrating variables from the right. Integrating y n in the above density
using the result
b
1 − FX (y n−1 )
∫
f X (y n ) dy n 
y n−1 1
we get

[FX (y j )] j−1 1 − FX (y n−1 )

fYj ,...,Yn−1 (y j , . . . , y n−1 )  n! f X (y j ) · · · f X (y n−1 ) ,
Statistical Inference, August 26, 2020

(j − 1)! 1
a < y j < · · · < y n−1 < b.

Next, integrating y n−1 in the above density using the result

b
1 − FX (y n−1 ) [1 − FX (y n−2 )]2
∫
f X (y n−1 ) dy n−1 
y n−2 1 (1)(2)

we get

[FX (y j )] j−1 [1 − FX (y n−2 )]2

fYj ,...,Yn−2 (y j , . . . , y n−2 )  n! f X (y j ) · · · f X (y n−2 ) ,
(j − 1)! (1)(2)
a < y j < · · · < y n−2 < b.

Similarly, one-by-one, we can integrate y n−2 , y n−3 , . . . , y k+1 , k ≥ j, (in this order) to get

[FX (y j )] j−1 [1 − FX (y k )]n−k

fYj ,...,Yk (y j , . . . , y k )  n! f X (y j ) · · · f X (y k ) ,
(j − 1)! (n − k)!
a < y j < · · · < y k < b.

Now, substituting j  k in the above density we get Theorem 6.5.2 of the book.

If X1 , X2 , . . . , X n is a random sample from a population with continuous pdf f X (x). Then

the pdf of the kth order statistic Yk , 1 ≤ k ≤ n, is given by

n!
fYk (y k )  [FX (y k )]k−1 [1 − FX (y k )]n−k f X (y k )
(k − 1)! (n − k)!

if a < y k < b and zero otherwise.

For continuous random variables, the pdf’s of the minimum and maximum, Y1 and
Yn , which are special cases of the above result are

fY1 (y1 )  n[1 − FX (y1 )]n−1 f X (y1 ), a < y1 < b

10
 and

fYn (y n )  n[FX (y n )]n−1 n f X (y n ), a < y b < b,

respectively.
The densities of Y1 and Yn can also be obtained by differentiating corresponding CDF.
For discrete and continuous random variables, the CDF of the minimum or maximum of
Statistical Inference, August 26, 2020

the sample can be derived directly by following the CDF technique. For the minimum,

FY1 (y1 )  p[Y1 ≤ y1 ]  1 − P[Y1 > y1 ].

Next step needs some explanation. Note the Y1 is the minimum (smallest among X1 , . . . , X n )
and therefore if Y1 > y1 then X i > y1 for all i  1, . . . , n. Conversely, if X i > y1 for all
i  1, . . . , n, then Y1 > y1 . Now,

FY1 (y1 )  1 − P[Y1 > y1 ]

 1 − P[X1 > y1 , X2 > y1 , . . . , Yn > y1 ]
 1 − P[X1 > y1 ]P[X2 > y1 ] · · · P[X n > y1 ] (X1 , . . . , X n are independent)
 1 − {P[X > y1 ]} n (X1 , . . . , X n have identical distribution)
 1 − {1 − P[X ≤ y1 ]} n
 1 − [1 − FX (y1 )]n .

If FX is differentiable, then
d
fY1 (y1 )  FY (y1 )
dy1 1
d
 {1 − [1 − FX (y1 )]n }
dy1
 n[1 − FX (y1 )]n−1 f X (y1 ), a < y1 < b.

For the maximum,

FYn (y n )  P[Yn ≤ y n ].

Note the Yn is the maximum (largest among X1 , . . . , X n ) and therefore if Yn ≤ y n then

X i ≤ y n for all i  1, . . . , n. Conversely, if X i < y n for all i  1, . . . , n, then Yn ≤ y n . Now,

FYn (y n )  P[Yn ≤ y n ]
 P[X1 ≤ y n ]P[X2 ≤ y1 ] · · · P[X n ≤ y n ] (X1 , . . . , X n are independent)

11
  {P[X ≤ y n ]} n (X1 , . . . , X n have identical distribution)
 [FX (y n )]n .

If FX is differentiable, then
d
fYn (y n )  FY (y n )
dy n n
 n[FX (y n )]n f X (y1 ), a < y n < b.
Statistical Inference, August 26, 2020

Next, we will derive the joint density of Yj and Yk , 1 ≤ j < k ≤ n. To obtain the joint
density of Yj and Yk , 1 ≤ j < k ≤ n, we integrate y j+1 , . . . , y k−1 in fYj ,...,Yk (y j , . . . , y k ).
In the following expression of the joint density, we indicate variables and factors to be
integrated

fYj ,Yj+1 ,...,Yk−1 ,Yk (y j , y j+1 . . . , y k−1 , y k )

| {z }
variables to be integrated

[FX (y j )] j−1 [1 − FX (y k )]n−k

 n! f X (y j ) f X (y j+1 ) · · · f X (y k−1 ) f X (y k ) . (A)
| {z } (j − 1)! (n − k)!
factors to be integrated

By taking k − 1  j + (k − j − 1), we can write our variables of integration y j+1 , . . . , y k−1 as

y j+1 , . . . , y j+(k− j−1) . Clearly, we have k − j − 1 variables.
WE integrate y j+1 in the interval (y j , y j+2 ), then y j+1 in the interval (y j , y j+3 ), . . ., and
finally y j+(k−j−1) in the interval (y j , y j+(k− j) ) by using the result
integration of y j+k−j−2
z }| {
∫ y j+k−j ∫ y j+k− j−1 ∫ y j+3 ∫ y j+2  
··· f X (y j+1 ) · · · f X (y j+k− j−1 )dy j+1 dy j+2 · · · dy j+k− j−2 dy j+k−j−1
yj yj yj yj
| {z }
integration of y j+1
| {z }
integration of y j+2

[FX (y k ) − FX (y j )]k− j−1

 . (B)
(k − j − 1)!
It is not difficult to check that
∫ y j+2
f X (y j+1 )dy j+1  FX (y j+2 ) − FX (y j ),
yj

y j+3 [FX (y j+3 ) − FX (y j )]2

∫
f X (y j+2 )[FX (y j+2 ) − FX (y j )]dy j+2  ,
yj 2

12
 and
y j+4 [FX (y j+3 ) − FX (y j )]2 [FX (y j+4 ) − FX (y j )]3
∫
f X (y j+3 ) dy j+3  .
yj 2 (2)(3)
Now, observing the trend of the above three integrals, we can safely conclude that
y j+i+1 [FX (y j+i ) − FX (y j )]i−1 [FX (y j+i+1 ) − FX (y j )]i
∫
f X (y j+i ) dy j+i  ,
yj (i − 1)! i!
for i  1, 2, . . . , k − j − 1.
Statistical Inference, August 26, 2020

Finally, integrating y j+1 , . . . , y k−1 in (A) by using (B), we get the joint density of Yj and
Yk , 1 ≤ j < k ≤ n as

If X1 , X2 , . . . , X n is a random sample from a population with continuous pdf f X (x) with

support on (a, b). Then, the joint density of Yj and Yk , 1 ≤ j < k ≤ n is

n!
fYj ,Yk (y j , y k ) 
(j − 1)!(k − j − 1)!(n − k)!
× [FX (y j )] j−1 [FX (y k ) − FX (y j )]k− j−1 [1 − FX (y k )]n−k f X (y j ) f X (y k ),
a < y j < y k < b.

The smallest and largest order statistics are of special importance, as are certain func-
tions of order statistics known as the sample median and range. If n is odd then the
sample median is the middle observation, Yk where k  (n + 1)/2 if n is even, then it
is considered to be any value between the two middle observations Yk and Yk+1 where
k  n/2, although it is often taken to be their average. The sample range is the difference
of the smallest from the largest, R  Yn − Y1 .

Chapter 6: Formulas

1 Joint density of Y1 , . . . , Yn

If X1 , X2 , . . . , X n is a random sample from a population with continuous pdf f X (x), then

the joint pdf of the order statistics Y1 , . . . , Yn is

fY1 ,Y2 ,...,Yn (y1 , y2 , . . . , y n )  n! f X (y1 ) f X (y2 ) · · · f X (y n ),

13
 where a < y1 < y2 , · · · < y n < b.

2 Marginal density of Yk

If X1 , X2 , . . . , X n is a random sample from a population with continuous pdf f X (x). Then

Statistical Inference, August 26, 2020

the pdf of the kth order statistic Yk , 1 ≤ k ≤ n, is given by

n!
fYk (y k )  [FX (y k )]k−1 [1 − FX (y k )]n−k f X (y k )
(k − 1)! (n − k)!

if a < y k < b and zero otherwise.

For continuous random variables, the pdf’s of the minimum and maximum, Y1 and Yn ,
which are special cases of the above result are

fY1 (y1 )  n[1 − FX (y1 )]n−1 f X (y1 ), a < y1 < b

and

fYn (y n )  n[FX (y n )]n−1 n f X (y n ), a < y b < b,

respectively.

3 CDF of Y1 and Yn

For discrete and continuous random variables, the CDF of the minimum of the sample is

FY1 (y1 )  1 − [1 − FX (y1 )]n .

For the maximum,

FYn (y n )  [FX (y n )]n .

14
 4 Joint density of Yj and Yk

If X1 , X2 , . . . , X n is a random sample from a population with continuous pdf f X (x) with

support on (a, b). Then, the joint density of Yj and Yk , 1 ≤ j < k ≤ n is

n!
fYj ,Yk (y j , y k ) 
(j − 1)!(k − j − 1)!(n − k)!
Statistical Inference, August 26, 2020

× [FX (y j )] j−1 [FX (y k ) − FX (y j )]k− j−1 [1 − FX (y k )]n−k f X (y j ) f X (y k ),

a < y j < y k < b.

5 Joint density of Y1 and Yn

If X1 , X2 , . . . , X n is a random sample from a population with continuous pdf f X (x) with

support on (a, b). Then, the joint density of Y1 and Yn , is

fY1 ,Yn (y1 , y n )  n(n − 1)[FX (y n ) − FX (y1 )]n−2 f X (y1 ) f X (y n ),

a < y1 < y n < b.

15