1
STAT512/432 – 2011
Notes on order statistics of discrete random
variables
In STAT 512/432 we will almost always focus on the order statistics of con-
tinuous random variables. Despite this, these notes discuss order statistics, in
particular the maximum and the minimum, of n discrete random variables. We
start with some basic background theory.
Some background theory
Example: the uniform distribution
For a discrete uniform distribution, all possible values of the (discrete) random
variable have the same probability. In these notes we focus attention on the
discrete random variable Y that is equally likely to take each of the values
0, 1, 2, . . . , 9. That is,
1
Probability(Y = y) = , y = 0, 1, . . . , 9. (1)
10
The mean of Y is, clearly,
9
X 1
mean of Y = y = 4.5. (2)
y=0
10
A slightly more complicated calculation shows that the variance of Y is 8.25.
The cumulative distribution function of a discrete random variable
The cumulative distribution function F (y) of any discrete random variable Y is
the probability that the random variable takes a value less than or equal to y.
Thus using a standard notation,
X
F (y) = P r(Y = a). (3)
a≤y
The maximum of n discrete random variables
Definitions
The maximum of a number of a number if independent random variables is
frequently used in statistics. As an introduction to the relevant theory, we
now consider properties of the maximum of n iid discrete random variables
Y1 , Y2 , . . . , Yn , each having the same probability distribution with common cu-
mulative distribution function F (y). We denote this maximum by Ymax . Given
2
data y1 , y2 , . . . , yn , we write the observed (data) value of Ymax as ymax . There
can be ties: for example, if n = 5 and
y1 = 7, y2 = 9, y3 = 6, y4 = 9, y5 = 6,
then ymax = 9. In this case two of the observations tied at the maximum value.
Theory
Since the maximum of n quantities is less than or equal to any number y if and
only if all of these quantities are less than or equal to y, we have
n
Prob (Ymax ≤ y) = F (y) , (4)
n
Prob (Ymax ≥ y) = 1 − F (y − 1) , (5)
so that n n
Prob (Ymax = y) = F (y) − F (y − 1) . (6)
Example: the uniform distribution again
As an example, we use the theory above to consider properties of the maximum
of n random variables, each having the above uniform distribution considered
above. The cumulative distribution function F (y) of a random variable having
the above uniform distribution is easily seen to be given by
y+1
F (y) = , y = 0, 1, 2, . . . , 9. (7)
10
Then from (6) and (7), the probability that Ymax = Y is given by
n
y+1 y n
Pmax (y) = − , x = 0, 1, 2, . . . , 9. (8)
10 10
Mean and variance of a maximum of n random variables from the
above uniform distribution
From the above theory, the mean value µmax of Xmax , the maximum of n random
variables from the uniform distribution considered above is given by
9 n
X y+1 y n
µmax = y − . (9)
y=0
10 10
This simplifies, after some algebra, to
n n n
1 2 9
µmax = 9 − − − ··· − . (10)
10 10 10
In Homework 3 you will be asked to verify this calculation.
3
As a check on this calculation, the case n = 1 gives
1 2 9 1 + 2 + ··· + 9
µmax = 9 − − − ··· − =9− = 4.5. (11)
10 10 10 10
One can also find the value of the variance of Ymax , but we do not do this here.
One immediate conclusion that can be drawn from (10) is that, as n → ∞,
the mean of Ymax approaches 9. It can also be shown that the variance mean
of Ymax approaches 0 as n → ∞. Both these conclusions “make sense.” The
following table indicates the rate at which these occur.
value of n mean of Ymax variance of Ymax
5 7.79175 1.928782
10 8.508466 0.624477
20 8.866059 0.142473
The minimum of n discrete random variables
Properties of the minimum Ymin of n independently and identically distributed
random variables can be found in a manner similar to that for which properties
of a maximum were found. Using the notation above, we get
n
Prob (Ymin ≥ y) = 1 − F (y − 1) , (12)
n
Prob (Ymin ≥ y + 1) = 1 − F (y) , (13)
so that n n
Prob (Ymin = y) = 1 − F (y − 1) − 1 − F (y) . (14)
From (14) one can find the mean and the variance of Ymin . We do not give
the details here, and note only that in the case of the uniform distribution
considered above, the mean of Ymin is
1 n 2 9
( ) + ( )n + · · · + ( )n . (15)
10 10 10
Note that as n → ∞, this mean approaches 0. This makes sense.
The general order statistic of n discrete random variables
Ymin and Ymax are examples of order statistics: specifically, Ymin is the first
order statistic and Ymax is the nth order statistic. One can also define the
second, third, ... , (n − 1)th order statistics. However the theory for these for
a discrete random variable becomes quite complicated, so we do not consider it
here.
