STAT8310 Statistical Theory

2021

Topic 4.2
Transformations

STAT8310 2021 Topic 4.2 1

 Change of Variable
• Consider a rv X with pdf or pf fX (x).

• Let Y be some function of X, say Y = g(X). What is the

distribution of Y ?

• This is called a change of variable or transformation.

The pf of a transformed discrete rv

• There are no problems associated with transforming discrete
random variables.

• Consider a discrete random variable X with probability function

fX (x).

STAT8310 2021 Topic 4.2 2

 • Let the random variable Y = g(X).
• The inverse transformation function is

X = g −1 (Y ) = ψ(Y ) .

• If g is a monotonic function over the range of X for which

fX (x) > 0, then Y has probability function

fY (y) = fX (ψ(y)) .

Example: Suppose X ∼ P (λ), i.e.


 e−λ λx x = 0, 1, 2, . . .
x!
fX (x) =
0 otherwise

and Y = X 2 . Over x ∈ {0, 1, 2, . . .}, the transformation

STAT8310 2021 Topic 4.2 3

 g(x) = x2 is monotonic. We have

y = x2 = g(x)
√
x = y = ψ(y)

and the pf of Y is
 −λ √y
 e √λ y = 0, 1, 4, 9, . . .
fY (y) = ( y )!
0 otherwise

• If g is a non-monotonic function, then Y has probability function

X
fY (y) = fX (ψ(y)) .
x:g(x)=y

Since g is not 1–1, there may be several values of x giving rise to

the same value of g (x) .

STAT8310 2021 Topic 4.2 4

 Example: Suppose X has the discrete uniform distribution:

1 x = −2, −1, 0, 1, 2
5
fX (x) =
0 otherwise

and Y = X 2 . Over x ∈ {−2, −1, 0, 1, 2}, the transformation

g(x) = x2 is not monotonic. We have

y = x2 = g(x)
√
x = ± y = ψ(y)

STAT8310 2021 Topic 4.2 5

 and the pf of Y is

1


 5 y=0
fY (y) = 2
y = 1, 4
5

0 otherwise


The pdf of a transformed continuous rv

• How do we find the pdf of the random variable Y defined by
Y = g (X) , where g is a function?
• e.g. Let X ∼ U [0, 1] . i.e.

 1 ; 0≤x≤1
fX (x) =
 0 ; elsewhere

• Let Y = X 2 . What is the pdf of the rv Y ?

STAT8310 2021 Topic 4.2 6

 • For continuous rvs we consider the cdf rather than the pdf.
• When X ∼ U [0, 1],

 0 ; x<0


FX (x) = x ; 0≤x≤1


1 ; x>1


• The cdf of Y is defined by

FY (y) = P (Y ≤ y)

• But Y = X 2 . Thus
2


FY (y) = P X ≤ y
√
= P (X ≤ y)

STAT8310 2021 Topic 4.2 7

 • Hence, for y > 0,
√
FY (y) = FX ( y)
 √
 0 ; y<0


√ √
= y ; 0≤ y≤1
√


1 ; y>1


 √y ;

0≤y≤1
=
 1 ; y > 1,
√
since y ≥ 0, ∀y.
• Obviously, when y < 0, FY (y) = 0.
• The pdf of Y is obtained by differentiation:

STAT8310 2021 Topic 4.2 8

 d
fY (y) = {FY (y)}
dy

d
 dy (0) = 0 ; y≤0


d √ 1


= dy y = √
2 y ; 0<y≤1

 d

dy (1) = 0 ; y>1

i.e. 
1

 √ ; 0<y≤1
fY (y) = 2 y
0 ; otherwise



Checks:
1. fY (y) ≥ 0, ∀y
2. FY (1) = 1, FY (−∞) = 0.

STAT8310 2021 Topic 4.2 9

 • Another example. Suppose X has pdf

 e−x ; x > 0
fX (x) =
 0 ; x≤0

and we want the pdf of Y = X 3 .

• The range of Y is (0, ∞) . Thus, when y > 0,

FY (y) = P (Y ≤ y)
3


=P X ≤y
 1

= P X ≤ y3 ,

since X is positive.

STAT8310 2021 Topic 4.2 10

 • Hence
 1
FY (y) = FX y 3

 1 − e−y 31 ; y>0
=
 0 ; y≤0

and so  1
1 − 32 −y 3
3y e ; y>0

fY (y) =
 0 ; y≤0
• Checks:
1. fY (y) ≥ 0

2. FY (∞) = 1, FY (−∞) = FY (0) = 0.

∴ fY (y) satisfies the conditions to be a valid pdf.

STAT8310 2021 Topic 4.2 11

 Note: fY (y) goes to ∞ as y goes to 0. Again, this is not a
problem.

General Case: Change of Variable

Monotonic increasing case

• Suppose that y = g (x) is monotonic increasing.

• Let Y = g (X). Since g is monotonic increasing, its inverse, say
ψ, exists.
• Thus X = g −1 (Y ) = ψ (Y ).

STAT8310 2021 Topic 4.2 12

 • Now
FY (y) = P (Y ≤ y)
= P {g (X) ≤ y}
 −1

= P X ≤ g (y)
= P {X ≤ ψ (y)}
= FX (ψ (y)) .

• Hence
d
fY (y) = FY (y)
dy
d
= FX (ψ (y))
dy
= ψ ′ (y) FX
′
(ψ (y))
= ψ ′ (y) fX (ψ (y)) .

STAT8310 2021 Topic 4.2 13

 Monotonic decreasing case

• A similar formula exists for functions which are monotonic

decreasing.
• Suppose Y = g (X) where g (X) is monotonic decreasing. Then
again g has an inverse ψ and

FY (y) = P (Y ≤ y)
= P (g (X) ≤ y)
 −1

= P X ≥ g (y)
= P {X ≥ ψ (y)}
= P {X > ψ (y)}
= 1 − P {X ≤ ψ (y)}
= 1 − FX (ψ (y))

STAT8310 2021 Topic 4.2 14

 • Thus
d
fY (y) = {1 − FX (ψ (y))}
dy
= −ψ ′ (y) fX (ψ (y)) .

Checks: fY (y) is positive. Since Y = g (X) is monotonic

decreasing, then so will the inverse function ψ (Y ). Hence, ψ ′ (y)
will be negative, and −ψ ′ (y) will be positive.

Monotonic functions – unified formula

A single formula for monotonic change-of-variable function can be

written.
• If g is monotonic increasing,

fY (y) = ψ ′ (y) fX (ψ (y))

STAT8310 2021 Topic 4.2 15

 • If g is monotonic decreasing,

fY (y) = −ψ ′ (y) fX (ψ (y))

• Combining the two equations, if Y = g (X) is monotonic, we have

fY (y) = |ψ ′ (y)| fX (ψ (y))

The lognormal distribution

Let X be a normal random variable with mean µ and variance σ 2 .

Consider the transformation Y = eX . Then X = log(Y ). Since
y = ex is monotonic increasing, we have, for y > 0,
1 1 − 2σ12 (log(y)−µ)2
fY (y) = · √ e .
y 2πσ 2

STAT8310 2021 Topic 4.2 16

 The random variable Y is said to be a lognormal random variable
with parameters µ and σ 2 . The pdf of lognormal distribution is

 √ 1 e− 2σ12 (log(y)−µ)2 , y > 0
fY (y) = 2πσ 2 y
 0, otherwise.

Examples: Linear transformations

Let Y = aX + b. Suppose the support of fX (x) (i.e. {x; fX (x) > 0})
is [c, d] . The support of fY (y) is therefore

y ∈ [ac + b, ad + b] .

STAT8310 2021 Topic 4.2 17

 We have
y = g (x) = ax + b

y−b
x= = ψ (y)
a

′ 1
ψ (y) =
a

fY (y) = |ψ ′ (y) |fX (ψ (y))

  
 | 1 |fX y−b
a a ; y ∈ [ac + b, ad + b]
=
 0 ; elsewhere.

STAT8310 2021 Topic 4.2 18

 Example 1. Exponential distribution

Suppose that X is exponentially distributed with mean 1/λ. Then


 λe−λx ; x > 0
fX (x) =
 0 ; x≤0

where λ > 0.
Let Y = λX. Then the inverse function is given by
Y
ψ (Y ) =
λ
with derivative ψ ′ (Y ) = λ1 .

STAT8310 2021 Topic 4.2 19

 Since the transformation is monotonic,
 
1 y
fY (y) =   fX
λ λ

1
= · λe−λ(y/λ)
λ
= e−y for y > 0

i.e.

 e−y ; y>0
fY (y) =
 0 ; y≤0

i.e. If X is exponentially distributed with mean 1/λ, then Y = λX

has a standard exponential distribution.

STAT8310 2021 Topic 4.2 20

 Example 2. Normal distribution Suppose X has pdf
1 − 2σ12 (x−µ)2
fX (x) = √ e
2πσ 2

Put
X −µ
Y = .
σ
Now

y = (x − µ) /σ

x = ψ (y) = µ + σy

ψ ′ (y) = σ

STAT8310 2021 Topic 4.2 21

 and
fY (y) = |σ| fX (µ + σy)
(  2 )
σ 1 µ + σy − µ
=√ exp −
2πσ 2 2 σ
 2
1 y
= √ exp −
2π 2
1 − 1 y2
=√ e 2 .
2π
i.e. Y = (X − µ)/σ has a standard normal distribution.

Transforming pdfs
Suppose we wish to transform a continuous random variable with a
given pdf into a random variable with another given pdf. How do we

STAT8310 2021 Topic 4.2 22

 do this?
As an example, consider the standard exponential distribution. Let
X be a rv with pdf

 0 ; x<0
fX (x) =
 e−x ; x ≥ 0

Then 
 0 ; x<0
FX (x) =
 1 − e−x ; x≥0

Put
Y = g(X) = 1 − e−X

As x goes from 0 to ∞, y = 1 − exp(−x) goes from 0 to 1. The

transformation has the same functional form as the cdf. It is not by

STAT8310 2021 Topic 4.2 23

 accident that we have chosen this transformation.

Now
y = g(x) = 1 − e−x
x = − ln(1 − y) = ψ(y)
1
ψ ′ (y) =
1−y
and we have
fY (y) = |ψ ′ (y)| fX (ψ(y))
1
= · eln(1−y)
1−y
1
= · (1 − y)
1−y
=1.

STAT8310 2021 Topic 4.2 24

 Hence 
 1; 0<y<1
fY (y) =
 0; otherwise
i.e. Y is uniformly distributed on (0, 1). The cdf FX (x) was used to
transform the random variable X and the resulting random variable
was found to be uniformly distributed on [0, 1].
Theorem
Let X be a continuous rv with monotonic increasing cdf F (x). Let
Y = F (X). Then Y is uniformly distributed on [0, 1].
Proof
Suppose the support of X is (a, b) with a < b. Then the range of

STAT8310 2021 Topic 4.2 25

 Y = F (X) is (0, 1) , since F (x) is a probability. Now

FY (y) = P (Y ≤ y)
= P (F (X) ≤ y)

Since F (x) is monotonic increasing, its inverse exists and is

monotonic increasing. Thus, if y ∈ [0, 1] ,

−1


FY (y) = P X ≤ F (y)
−1


=F F (y)
= y.

If y < 0, FY (y) = 0 since the range of Y = F (X) is (0, 1) . Also, if

STAT8310 2021 Topic 4.2 26

 y > 1, FY (y) = FY (1) = 1. Thus

 0 ; y<0


FY (y) = y ; 0≤y≤1


1 ; y>1


and so 
 1 ; 0≤y≤1
fY (y) =
 0 ; elsewhere
i.e. Y ∼ U (0, 1). 

STAT8310 2021 Topic 4.2 27

 Transformation from U (0, 1) to an arbitrary
distribution: the inverse cdf transformation
• Suppose Y ∼ U (0, 1). Can we transform Y so that the resulting
random variable W will have a given pdf h(w) and cdf H(w)?
• If Y = FX (X) , with FX the cdf of X, then Y ∼ U (0, 1). Let’s
reverse the argument.
• Let W = H −1 (Y ). Now from Y ∼ U (0, 1) we have

 0 ; y<0


FY (y) = y ; 0≤y≤1


1 ; y>1


STAT8310 2021 Topic 4.2 28

 • Since H −1 (w) is monotonic increasing,
FW (w) = P (W ≤ w)
= P (H −1 (Y ) ≤ w)
= P (Y ≤ H(w))
= FY (H(w))
= H(w) ,
since 0 ≤ H(w) ≤ 1, ∀w. Thus W has the desired cdf.

Importance of this result

• Often in computer simulation we need to simulate random values
from specific distributions.
• There are many algorithms available for generating pseudo- or
quasi-random uniform values – the mixed congruential random

STAT8310 2021 Topic 4.2 29

 generator is an example.
• These values can then be easily transformed into
pseudo-random values from a specified distribution by
application of the inverse cdf transformation.
Example: Suppose we wish to simulate an exponential random
variable with mean 1/λ.
The pdf is 
 λe−λw ; w>0
fW (w) =
 0 ; w≤0
and the cdf is

 1 − e−λw ; w>0
FW (w) =
 0 ; w≤0

STAT8310 2021 Topic 4.2 30

 Suppose we have at our disposal numbers y1 , y2 , . . . , yn which appear
to be observations on random variables Y1 , Y2 , . . . , Yn which are
U (0, 1) .

−1
If W = FW (Y ), where FW (w) is given above, then

Y = FW (W )
= 1 − e−λW ,
e−λW = 1 − Y,
−λW = log (1 − Y ) ,
1
W = − log (1 − Y ) ,
λ
and W will have the cdf FW (w).

STAT8310 2021 Topic 4.2 31

 Note: If Y ∼ U (0, 1) , then (1 − Y ) ∼ U (0, 1). So we could also
use W = − λ1 log Y . This saves a computational step, which is useful
for really huge simulations.

We could thus generate observations which appeared to come from

the given exponential distribution by using
1
wj = − log (1 − yj ) , j = 1, . . . , n
λ
or
1
wj = − log yj , j = 1, . . . , n.
λ
One or other of these might be problematic. For example, the linear
congruential technique generates numbers of the form j/N, where N
is large and 0 ≤ j ≤ N − 1. Thus some of the yj′ s might be 0 and the

STAT8310 2021 Topic 4.2 32

 logs of these would be −∞.

Usefulness of the Technique

• If we wish to use the inverse cdf to generate a random variable
−1
with a given cdf FX , then we need to be able to calculate FX .
• This is easy for some cdfs, but even for quite common cdfs it is
not practical.
• For example, if we wished to transform to standard normality,
the cdf Φ has to be calculated numerically, and it is even more
difficult to calculate the inverse Φ−1 .

STAT8310 2021 Topic 4.2 33

 Further examples

Example 1.

Let X ∼ U (0, 1) and let Y = tan {π (X − 1/2)} . What is the pdf of

Y?

Note that as X goes from 0 to 1, π (X − 1/2) goes from −π/2 to π/2.

The transformation is thus 1–1, with inverse transform found by
expressing X in terms of Y : If

y = tan {π (x − 1/2)}

STAT8310 2021 Topic 4.2 34

 then
1
x = ψ (y) = 1/2 + tan−1 y
π

′ 1
ψ (y) = .
π (1 + y 2 )
Thus
1
fY (y) = 2
.
π (1 + y )

Thus Y has the Cauchy distribution. We can also get a Cauchy

random variable by taking the ratio of two independent standard
normal random variables.

STAT8310 2021 Topic 4.2 35

 Example 2.

Let X ∼ U (0, 1) and let Y = sin {π (X − 1/2)} . What is the pdf of

Y?

The transformation is again 1–1, with inverse transform found by

expressing X in terms of Y. If

sin {π (x − 1/2)} = y,

then
1
x = ψ (y) = 1/2 + sin−1 y
π
1
ψ ′ (y) = p
π 1 − y2

STAT8310 2021 Topic 4.2 36

 and so
1


 p ; −1 < y < 1
fY (y) = π 1 − y2 .
0 ; otherwise



Alternative formula (optional)

• Sometimes the formula

fY (y) = |ψ ′ (y)| fX (ψ (y))

is a little awkward to evaluate, or is better evaluated by other

methods.
• For example, it may be difficult to evaluate ψ ′ (y) , or the
function g (x) may be recognised easily in the form of fX (x).

STAT8310 2021 Topic 4.2 37

 • In the former case, we may use the fact that if y = g (x) , then
dx
ψ ′ (y) =
dy
1
=
dy
dx
1
= ′
g (x)
1
= ′ ,
g (ψ (y))
although sometimes the last step is not necessary (as the term
1
may end up being ‘cancelled’).
g (x)
′

STAT8310 2021 Topic 4.2 38

 • As an example, consider the case
1 − 2σ12 (x−µ)2
fX (x) = √ e ,
2πσ 2

and the transformation

X −µ
Y = g (X) = .
σ
′ 1 x−µ 2


Now g (x) = and the term σ in fX (x) is recognised as
σ
2
y . Thus
1 1 1 2
fY (y) = √ e− 2 y
1 2πσ 2
σ
1 1 2
= √ e− 2 y .
2π

STAT8310 2021 Topic 4.2 39

 Non-monotonic transformations
• If the transformation is not 1–1, we have to be very careful.
• Consider for example the case where X ∼ U [−2, 2] and we want
the pdf of Y = X 2 . Now

 1 ; −2 ≤ x ≤ 2
4
fX (x) =
 0 ; elsewhere.

STAT8310 2021 Topic 4.2 40

 • Thus, for −2 ≤ x ≤ 2,

FX (x) = P (X ≤ x)
Zx
1
= du
4
−2
h u ix
=
4 −2
1
= (x + 2)
4
and 
 0 ; x < −2


FX (x) = 1
4 (x + 2) ; −2 ≤ x ≤ 2 .


1 ; x>2


• The transformation Y = X 2 is not 1–1 on [−2, 2]

STAT8310 2021 Topic 4.2 41

 4
3
2
y
1
0

−2 −1 0 1 2
x

y = x2
• By definition

FY (y) = P (Y ≤ y)
2


=P X ≤y

STAT8310 2021 Topic 4.2 42

 √
But X 2 ≤ y iff |X| ≤ y. Thus {Y ≤ y} is the same as the event
 √ √
− y≤X ≤ y .
• Hence, when y ∈ (0, 4) ,
FY (y) = P (Y ≤ y)
√ √
= P (− y ≤ X ≤ y)
√ √
= P (X ≤ y) − P (X < − y)
√ √
= FX ( y) − FX (− y)
1 √ 1 √
= ( y + 2) − (− y + 2)
4 4
• Thus 
 0 ; y≤0


√
y
FY (y) = 2 ; 0<y≤4


1 ; y>4


STAT8310 2021 Topic 4.2 43

 and, so 
1

 √ ; 0<y≤4
fY (y) = 4 y
 0

; elsewhere.
Checks:
1. fY (y) ≥ 0
0 2
2. FY (0) = 2 = 0 , FY (4) = 2 = 1.

General formula for non 1–1 transformations

• Suppose that the range of X can be broken into intervals on
which a function g is 1–1.
• Denote these intervals by Ij , j = 1, 2, . . .. Note that the union of
the intervals may be the whole real line, so that there may be an
infinite number.
STAT8310 2021 Topic 4.2 44
 • Denote by ψj the inverse of g on Ij , and let Bj = g (Ij ) , i.e. let

Bj = {g (x) ; x ∈ Ij } .

• Let Y = g (X) . Then, using a result in the previous section, we

have
X  ′ 
fY (y) = ψj (y) fX (ψj (y))
j:y∈Bj
X fX (ψj (y))
= .
|g (ψj (y))|
′
j:y∈Bj

No proof is given but the proof is not difficult. It can be done

from first principles fairly easily.

STAT8310 2021 Topic 4.2 45

 Example 1

Suppose X has pdf

 x+1

; −1 < x < 3
fX (x) = 8
 0 ; otherwise

and let Y = X 2 . Let I1 = (−1, 0) and I2 = (0, 3) . (We’ve done this

because of the transformation.) Then the inverses are
√
x=− y on I1

and
√
x= y on I2 .

STAT8310 2021 Topic 4.2 46

 Also, B1 = (0, 1) while B2 = (0, 9) . Thus, when y ∈ (0, 1) , y ∈ B1
and y ∈ B2 , and so
 √ √
− y+1 y+1

1
fY (y) = √ +
2 y 8 8
1
= √ ,
8 y

while on y ∈ (1, 9) , y ∈ B2 only, so that

√
1 y+1
fY (y) = √ ×
2 y 8
√
y+1
= √ .
16 y

STAT8310 2021 Topic 4.2 47

 Thus  1
 √ ; 0<y≤1
8 y







 √


fY (y) = y+1
√ ; 1<y<9


 16 y







0 ; otherwise.


You should check that this is a valid pdf.

STAT8310 2021 Topic 4.2 48