Mathematical Statistics I

Module for Math 150.1

an
lim
Di
UP
s,
ic
at
em
h
at
M
of
te
itu
st
In
Chapter 5: Creating New Univariate
Distributions

5.1 Transformations
Transformation is a technique of constructing other distributions. We create a new ran-

an
lim
dom variable, hence creating another distribution, by considering functions of the original
random variable.

Di
UP
For a discrete random variable X with pmf fX (x), then the pmf of the random variable
s,

Y = g(X) is
ic

X
fY (y) = fX (x).
at

x∈g −1 (y)
em

Example 1. If X ∼ Unif{1, 2, . . . , n} and Y = 2X + 1, then Y ∼ Unif{3, 5, 7, . . . , 2n + 1}.

h
at

Example 2. If X ∼ Unif{−10, −9, . . . , 9, 10} and Y = |X|, then Y has pmf

M

(
of

1
21
, y=0
fY (y) = .
te

2
21
, y ∈ {1, 2, . . . , 10}
itu

For continuous random variables, we use the cdf instead of the pdf.
st
In

Theorem 5.1 (CDF Technique) Let X is be a continuous random variable with pdf
fX (x) and cdf FX (x). Also, let g(x) be an invertible function and Y = g(X).

1. If g(x) is monotonically increasing, then

FY (y) = P[Y ≤ y] = P[g(X) ≤ y] = P[X ≤ g −1 (y)] = FX (g −1 (y)).

2. If g(x) is monotonically decreasing, then

FY (y) = P[Y ≤ y] = P[g(X) ≤ y] = P[X ≥ g −1 (y)] = 1 − FX (g −1 (y)).

2
Chapter 5. Creating New Univariate Distributions 3

Example 3. Find the pdf of Y = X 2 , where X ∼ Unif[−1, 1].

Solution: Since Y = X 2 , then Y can only take on values on [0, 1]. Let y ∈ [0, 1], then
√ √
FY (y) = P[Y ≤ y] = P[X 2 ≤ y] = P[− y ≤ X ≤ y]
√ √
√ √ 1+ y 1− y √
= FX ( y) − FX (− y) = − = y
2 2
1
Thus, fY (y) = FY0 (y) = √ 1(0,1) (y).
2 y
Remark. Common Transformations (and how the new distribution is named)
• Y = θX, θ 6= 0 : scaled
• Y = X 1/τ , τ > 0, τ 6= 1 : transformed
• Y = X −1 : inverse

an
• Y = X 1/τ , τ < 0, τ 6= −1 : inverse transformed

lim
• Y = eX : log

Di
• Y = X ± c, c 6= 0: translated / shifted UP
Theorem 5.2 Let X be a continuous random variable with pdf fX (x) and cdf FX (x). Let
s,

Y = θX, where θ > 0. Then

ic
at

y  1 y 
em

FY (y) = FX and fY (y) = fX

θ θ θ
h

Remark. The parameter θ is a scale parameter for the random variable Y .

at
M

Example 4. If X ∼ N (µ, σ 2 ) and Y = θX, θ > 0. What is the distribution of Y ?

of

Solution: Based on the previous theorem,

te

(  2 ) (  2 )
itu

1 y  1 1 1 yθ − µ 1 1 y − µθ
fY (y) = fX = √ exp = √ exp .
st

θ θ θ σ 2π 2 σ σθ 2π 2 σθ
In

Thus, Y ∼ N (µθ, (σθ)2 ).

Example 5. If X has the Pareto Type II (also called Lomax) distribution P a(α, θ), where
α, θ > 0. Then its pdf and cdf are as follows
α
αθα

θ
fX (x) = and FX (x) = 1 − , x>0
(x + θ)α+1 x+θ
Suppose Y = (1 + r)X where r > −1. What is the distribution of Y ?
Solution: Since 1 + r > 0, then
  !α  α
y θ (1 + r)θ
FY (y) = FX =1− x =1−
1+r 1+r
+θ x + (1 + r)θ

Therefore, Y ∼ P a(α, (1 + r)θ).

Chapter 5. Creating New Univariate Distributions 4

Next, we look at how raising to a power affects the distribution of a random variable.

Theorem 5.3 Let X be a continuous random variable with pdf fX (x) and cdf FX (x) with
FX (0) = 0. Let τ > 0.

• if Y = X 1/τ , then FY (y) = FX (y τ ) and fY (y) = τ y τ −1 fX (y τ )

• if Y = X −1/τ , then FY (y) = 1 − FX (y −τ ) and fY (y) = τ y −τ −1 fX (y −τ )

Example 6. Suppose X has the exponential distribution with parameter 1. Determine

the cdf of the inverse, transformed and inverse transformed exponential distribution with
scale parameter θ > 0.
Solution: X ∼ Exp(1), thus FX (x) = 1 − e−x . Let θ > 0.

an
h  i  

lim
If Y = θX −1 , then FY (y) = 1 − 1 − exp − yθ = exp − yθ (Inverse Exponential)

Di
τ
If Y = θX 1/τ , τ > 0, then FY (y) = 1 − exp − yθ


(Weibull Distribution)
h τ i UP
If Y = θX −1/τ , τ > 0, then FY (y) = exp − yθ (Inverse Weibull Distribution)
s,

Definition 5.4 The incomplete gamma function with parameter α > 0 is given by
ic

ˆ x
at

1
em

Γ(α; x) = tα−1 e−t dt

Γ(α) 0
h
at

while the gamma function is defined by

M

ˆ ∞
of

Γ(α) = tα−1 e−t dt

0
te
itu

Example 7. Find the pdf’s of the scaled, inverse, transformed and inverse transformed
st

Gamma distribution with pdf

In

xα−1 e−x
fX (x) =
Γ(α)
Solution: Note that X ∼ Γ(α, 1).  α
y α−1 e−y/θ 1 1
• Scaled Gamma [Y = θX]: fY (y) = α = y α−1 e−y/θ
θ Γ(α) Γ(α) θ
Thus, Y ∼ Γ(α, 1/θ).
θα e−θ/y
• Inverse Gamma: fY (y) = α+1
y Γ(α)
τ uα e−u  y τ
• Transformed Gamma: fY (y) = where u =
yΓ(α) θ  τ
τ uα e−u θ
• Inverse Transformed Gamma: fY (y) = where u =
yΓ(α) y
Chapter 5. Creating New Univariate Distributions 5

Lastly, we look at how exponentiation affects the distribution of certain random variables.

Theorem 5.5 Let X be a continuous random variable with pdf fX (x) and cdf FX (x) with
fX (x) > 0 for all real x. Let Y = exp X. Then, for y > 0,
1
FY (y) = FX (ln y) fY (y) = fX (ln y)
y

Example 8. Lognormal distribution

Let X ∼ N (µ, σ 2 ) and Y = exp(X). Then
"  2 #
1 1 ln y − µ
fY (y) = √ exp − .
yσ 2π 2 σ

an
5.2 Mixed Random Variables

lim
Di
There exists certain random variables that are neither discrete nor continuous, but are a
UP
mixture of both. These random variables are called mixed random variables. In particular,
a mixed random variable has a continuous part and a discrete part. We shall illustrate such
ics,

random variables via some examples.

at
em

Example 9. Consider the random variable X having cdf

h

(
x2 , 0 ≤ x < 21
at

FX (x) = .
M

1, x ≥ 12
of

Below is the graph of the cdf

te
itu
st
In

Note that even though X(Ω) = [0, 21 ] (an interval), FX is not continuous. Hence X is not
a continuous random variable.
Chapter 5. Creating New Univariate Distributions 6

Definition 5.6 Let X be a random variable with cdf F (x). X is called a mixed random
variable, and its distribution a mixed distribution, if F (x) has the form
k
X
F (x) = αi Fi (x),
i=1

k
X
with 0 < αi < 1, αi = 1, k ≥ 2, and Fi (x) is cdf of either a continuous or discrete
i=1
random variable. The αi ’s are called the weights of the Fi ’s.

In general, the cdf FX of a mixed random variable X can be written as the convex combi-
nation of the cdf’s of a continuous rv and a discrete rv., i.e.,

an
FX (x) = αC(x) + βD(x)

lim
where α, β ≥ 0 and α + β = 1, and C is a cdf of some continuous rv while D is the cdf of

Di
some discrete rv. UP
s,

In our previous example, we see that

ic
at

(
1
x2 , 0 ≤ x < 2
em

αC(x) = and
1 1
4
, x≥ 2
h
at

(
1
0, 0 ≤ x <
M

2
βD(x) = .
3 1
, x≥
of

4 2
1
te

Since C and D are cdf’s then lim C(x) = lim D(x) = 1. Hence we take α = 4
and
x→+∞ x→+∞
itu

β = 34 . Finally,
st

(
1
4x2 , 0 ≤ x <
In

2
C(x) = and
1
1, x≥ 2
(
1
0, 0 ≤ x < 2
D(x) = .
1
1, x ≥ 2

Here, we can clearly see that C is a cdf of a continuous rv, while D is a cdf of a discrete rv.

Remark. The pdf of a mixed random variable exists, but it uses what we call the “Dirac
Delta Function” and is beyond the coverage of our course.

Theorem 5.7 Let X be a mixed random variable with cdf F (x). Then there exists α, β > 0
with α + β = 1, a continuous cdf C(x) and a discrete (step-function) cdf D(x) such that

F (x) = αC(x) + βD(x) for all x.

Chapter 5. Creating New Univariate Distributions 7

Moreover, if f (x) is the pdf of C(x) and p(x) is the pmf of D(x) then for any n ∈ N :
ˆ +∞ X
n
E[X ] = α xn f (x) dx + β xnk p(xk )
−∞ xk

Example 10. Find the mean of the mixed random variable from Example 9.
Solution: Take f (x) as the pdf of C(x), i.e., f (x) = C 0 (x). Thus
(
1
8x, 0 ≤ x < 2
f (x) = .
0, otherwise

Similarly, let p(x) be the pmf of D(x), then p(x) = 1{ 1 } (x).

2

Hence,

an
ˆ

lim
1  
2 1
E[X] = α x f (x) dx + β p(1/2)
2

Di
0
ˆ 1  
1 2 2 3 1 UP 11
= 8x dx + (1) = .
4 0 4 2 24
ic s,

5.3 Truncation
at
em

A truncated distribution is a conditional distribution that is obtained by restricting the

h
at

domain of some other known distribution. Truncation is usually used on a normal distri-
M

bution.
of
te

Definition 5.8 Let X be a random variable with cdf Fx (x). Let a, b be in the domain of
itu

FX such that a < b and FX (a) 6= FX (b). The cdf of the truncated random variable X given
st
In

that a < X ≤ B is given by

FX (x) − FX (a) FX (x) − FX (a)
F (x|a < X ≤ b) = =
P(a < X ≤ b) FX (b) − FX (a)

Theorem 5.9 Let X be a continuous random with cdf F (x) and pdf f (x). Suppose a, b is
in the range of X such that a < b and F (a) 6= F (b). Then the function

f (x)
g(x) = 1[a,b] (x)
F (b) − F (a)
is a pdf. Furthermore, the above is the pdf of the truncated random variable X given that
a ≤ X ≤ b.
Chapter 5. Creating New Univariate Distributions 8

Example 10. Let X ∼ N (µ, σ 2 ). Now suppose that we condition on X ∈ [a, b]. Find the
mean and variance of the truncated distribution.
b−µ a−µ



Solution: The probability that X ∈ [a, b] is given by Φ σ
−Φ σ
, where Φ is the cdf
of a standard normal rv. Then the pdf of the truncated rv is given by
n
2 o
√1 exp −1 x−µ
f (x) σ 2π 2 σ
f (x|a ≤ X ≤ b) = b−µ

a−µ

= b−µ

a−µ

, a ≤ x ≤ b.
Φ σ −Φ σ Φ σ −Φ σ
To obtain the raw moments of the truncated distribution, we first solve its mgf.
ˆ b
 tX etx φ(x) dx
a

m(t) = E e |a ≤ X ≤ b =
Φ b−µ a−µ



− Φ

an
σ σ
Note that
ˆ b ˆ b ˆ b

lim
1 1 2
n o
x−µ 2 −1
e 2σ2 [
tx tx −1 ( ) x−(σ 2 t+µ)] −(σ 2 t+µ)2 +µ2
e φ(x) dx = √ e e2 σ dx = √ dx

Di
a σ 2π a σ 2π a
ˆ b −1  x−(σ2 t+µ)2 2
UP
1 −1
[ µ2 −(σ 2 t+µ)2
]
= √ e 2σ2 e2 σ
dx
σ 2π a
ˆ b
s,

2 2
 
2
−1 x−(σ t+µ)
1
ic

σ 2 t2
= eµt+ 2 √ e 2 σ
dx
at

a σ 2π
em

b − σ2t − µ a − σ2t − µ
    
2 2
µt+ σ 2t
= e Φ −Φ
h

σ σ
at

Thus,
M

2 2
h  2   2 i
µt+ σ 2t b−σ t−µ
e Φ σ
− Φ a−σσ t−µ
of

m(t) =
Φ b−µ − Φ a−µ



te

σ σ
itu

b−µ

a−µ

!
φ − φ
So E [X|a ≤ X ≤ b] = m0 (0) = µ − σ σ σ
st

b−µ

a−µ

.
Φ σ −Φ σ
In

Likewise,

E X 2 |a ≤ X ≤ b = m00 (0)
 
"
# "
#
b−µ a−µ b−µ a−µ
φ0 − φ0



φ −φ
= σ 2 + µ2 + σ 2 σ
b−µ

σ
a−µ

− 2µσ σ
b−µ

σ
a−µ


Φ σ
−Φ σ
Φ σ
−Φ σ
Note that:
φ0 (x) = xφ(x).
∴ V ar(X|a ≤ X ≤ b) = E X 2 |a ≤ X ≤ b − (E [X|a ≤ X ≤ b])2
 
 
b−µ

b−µ
a−µ

a−µ
" b−µ

a−µ

#2 
 φ − φ σ φ −φ
= σ2 1 + σ σ σ
− σ σ
.
Φ b−µ − Φ a−µ b−µ a−µ





 σ σ
Φ σ
−Φ σ

Chapter 5. Creating New Univariate Distributions 9

Exercises.
X
1. Let X ∼ Exp(1) and Y = with θ > 0. Find the distribution of Y .
θ
2. Let X ∼ P oi(λ) and Y = cX, c > 0. Find the distribution of Y .

3. Let X be a standard Cauchy random variable, i.e., its pdf is given by

1
f (x) = , x ∈ R.
π(1 + x2 )

Let T = X −1 . Find the distribution of T .

4. Suppose X is the equally weighted mixture Y ∼ U ({0, 1, 2, 5}) and Z ∼ U [0, 4].
Find the mean and variance of X.

an
lim
5. Let X ∼ N (µ, σ 2 ). Let Y = X|X < b and Z = X|X > a. Find the mean and

Di
variance of Y and Z.
UP
6. Let X ∼ U (0, b), b > 0. Define Y = X|X < a with 0 < a < b. Show that
Y ∼ U (0, a).
ics,
at
h em
at
M
of
te
itu
st
In