Functions of Random

Variables
Method of Distribution Functions
X
1
,,X
n
~ f(x
1
,,x
n
)
U=g(X
1
,,X
n
) Want to obtain f
U
(u)
Find values in (x
1
,,x
n
) space where U=u
Find region where Uu
Obtain F
U
(u)=P(Uu) by integrating
f(x
1
,,x
n
) over the region where Uu
f
U
(u) = dF
U
(u)/du
Example Uniform X
Stores located on a linear city with density
f(x)=0.05 -10 x 10, 0 otherwise
Courier incurs a cost of U=16X
2
when she delivers to a
store located at X (her office is located at 0)
1600 0
80
) (
) (
1600 0
40 4 4
05 . 0 05 . 0 ) ( ) (
4 4
4
16
2 / 1
4
4
2
= =
=

'
+

'

'
+

'

= = =

= = =

u
u
du
u dF
u f
u
u u u
dx u U P u F
u
X
u
u U
u
X u X u U
U
U
u
u
U
Example Sum of Exponentials
X
1
, X
2
independent Exponential(U)
f(x
i
)=U
-1
e
-xi/U
x
i
>0, U>0, i=1,2
f(x
1
,x
2
)= U
-2
e
-(x1+x2)/U
x
1
,x
2
>0
U=X
1
+X
2

? A

) , 2 ( ~ 0
1
1 1
) (
1
1
1 1
1
1
1 1
) (
,
/
2
/
2
/ / / /
2
/ ) (
0
2
/
0
2
/ ) ( /
0
2
0
/ /
0
2 1
/ /
0 0
2
2 1 2 2 1
2 1 2 1
2 2 2 2 2
2
1 2 2 1
2
U F E
U
U U U U
U U U
U U
U
U U U U U
U U U U
U U U U
= = > =

'
+

'

'
+

'

= =
= =
= =
+
= = + =

Gamma U u ue
e
u
e e u f ue e
dx e dx e dx e e
dx e e dx dx e e u U P
X u X u X u X X u U
x u X u X X u U
u
u u u
U
u u
x u x
u
x
u
x u x
u
x u
x x
u
x x
u x u
Method of Transformations
X~f
X
(x)
U=h(X) is either increasing or decreasing in X
f
U
(u) = f
X
(x)|dx/du| where x=h
-1
(u)
Can be extended to functions of more than one random variable:
U
1
=h
1
(X
1
,X
2
), U
2
=h
2
(X
1
,X
2
), X
1
=h
1
-1
(U
1
,U
2
), X
2
=h
2
-1
(U
1
,U
2
)

= =
= =
2 2 1 1 2 1 2 1
1
2
2
1
2
2
1
1
2
2
1
2
2
1
1
1
) , ( ) ( | | ) , ( ) , (
| |
1
du u u f u f J x x f u u f
dU
dX
dU
dX
dU
dX
dU
dX
dU
dX
dU
dX
dU
dX
dU
dX
J
U
Example
f
X
(x) = 2x 0 x 1, 0 otherwise
U=10+500X (increasing in x)
x=(u-10)/500
f
X
(x) = 2x = 2(u-10)/500 = (u-10)/250
dx/du = d((u-10)/500)/du = 1/500
f
U
(u) = [(u-10)/250]|1/500| = (u-10)/125000
10 u 510, 0 otherwise
Method of Conditioning
U=h(X
1
,X
2
)
Find f(u|x
2
) by transformations (Fixing X
2
=x
2
)
Obtain the joint density of U, X
2
:
f(u,x
2
) = f(u|x
2
)f(x
2
)
Obtain the marginal distribution of U by
integrating joint density over X
2

=
2 2 2
) ( ) | ( ) ( dx x f x u f u f
U
Example (Problem 6.11)
X
1
~Beta(E=2,F=2 X
2
~Beta(E=1,F=lIndependent
U=X
1
X
2
Fix X
2
=x
2
and get f(u|x
2
)

1 0 )) ln( 1 ( 18
) ln( 18 18 0 18 ) ln( 18 18
18
18 ) ( ) | ( ) (
1 0 1 18 3
1
) / 1 )( / ( 6 ) ( ) | ( ) , (
0
1
) / 1 )( / ( 6 ) | (
/ 1 /
1 0 3 ) ( 1 0 ) 1 ( 6 ) (
2 2
1
2
2
2 2
1
2
2
1
2 2 2
2
2
2
2
2
2 2 2 2 2
2
2
2 2 2
2
1
2 1 2 1
2
2
2 2 1 1 1 1
+ =
= =

'
+

'

= =

'
+

'

= = =
=
= = =
= =

u u u u u
u u u u x u ux dx
x
u
u dx x f x u f u f
x u
x
u
u x
x
x u x u x f x u f x u f
x u
x
x u x u x u f
x
dU
dX
x U X x X U
x x x f x x x x f
u u u
U
Problem 6.11
0
1
2
3
4
5
6
7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
u
D
e
n
s
i
t
y

o
f

U
=
X
1
X
2
f(u)
f(u|x2=.25)
f(u|x2=.5)
f(u|x2=.75)
Method of Moment-Generating Functions
X,Y are two random variables
CDFs: F
X
(x) and F
Y
(y)
MGFs: M
X
(t) and M
Y
(t) exist and equal for |t|<h,h>0
Then the CDFs F
X
(x) and F
Y
(y) are equal
Three Properties:
Y=aX+b M
Y
(t)=E(e
tY
)=E(e
t(aX+b)
)=e
bt
E(e
(at)X
)=e
bt
M
X
(at)
X,Y independent M
X+Y
(t)=M
X
(t)M
Y
(t)
M
X1,X2
(t
1
,t
2
) = E[e
t1X1+t2X2
] =M
X1
(t
1
)M
X2
(t
2
) if X
1
,X
2
are indep.
Sum of Independent Gammas

'
+

'

=
= = = =
=
= =
=

= =

+ +
=

=
F E
F F F
F
F E
E
E E
E
, ~
) 1 ( ) 1 ( ) 1 (
) ( ) ( ) (
,..., 1 ) 1 ( ) (
nt) (independe ,..., 1 ) , ( ~
1 1
) ... (
1
1 1
1
1 1
n
i
i
n
i
i
X X
tX tX X X t tY
Y
n
i
i
X
i i
Gamma X Y
t t t
t M t M e e E e E e E t M
X Y
n i t t M
n i Gamma X
n
i
i
n
n
n n
i
i
.
. .
Linear Function of Independent Normals

'
+

'

,
|
+ =
)
`

|
,
|
+
)
`

|
,
|
+
= = = =
=
=
)
`

|
,
|
+ =
=

= = =
=
=
+ +
=
n
i
i i
n
i
i i
n
i
i i
n
i
i i
n
i
i i
n n
n n
n X X
X ta X ta X a X a t tY
Y
i
n
i
i i
i
i X
i i i
a a Normal X a Y
t a
t a
t a
t a
t a
t a
t a M t a M e e E e E e E t M
a X a Y
n i
t
t t M
n i Normal X
n
n n n n
i
1
2 2
1 1
2
1
2 2
1
2 2 2
1
2
1
1 1
1
) ... (
1
2 2
2
, ~
2
exp
2
exp
2
exp
) ( ) ( ) (
constants fixed } {
,..., 1
2
exp ) (
nt) (independe ,..., 1 ) , ( ~
1
1 1 1 1
W Q
W
Q
W
Q
W
Q
W
Q
W Q
.
. .
Distribution of Z
2
(Z~N(0,1))
2
1
2
1
0
/ 1
2
1
2
2 / 1 2 / 1
2 / 1
0
2 1
2
/
1 2 / 1
0
2 1
2
/
2 / 1 2
2 1
0
2 / 1 2 / 1 2
2
2 1
0
2
2 1
2 /
2 /
) 2 , 2 / ( ~ t independen mutually ,...,
) 2 / 1 (
) (
: Notes
) 2 , 2 / 1 ( ~
) 2 1 ( ) 2 1 ( 2
2
1
2 1
2
) 2 / 1 (
2
1
2
1
2
1
2
1
2
5 . 0 5 . 0
2
1
Let
0) about (symmetric
2
1
2
2
1
2
1
) (
2
1
) ( ) 1 , 0 ( ~
2
2 2
2 2
2
2
n
n
i
i n
y
t
u
t
u
t
z
t
z
t
z
z tz
Z
z
Z
n Gamma Z Z Z
dy e y
Gamma Z
t t
t
du e u du e u dz e
du u dz u
u du
dz
u z z u
dz e dz e dz e e t M
z e z f N Z
G F E
T
F E
G F E
T
T
T T T T
T T T
T
E F E
= =
= I
I =
= =
= =

I = = =
= = = = =
= = =
=

'
+

'

'
+

'

'
+

'

Distributions of and S
2
(Normal data)
X



_ a


_ a
data sampled the of s difference the of function a is So
) 1 ( 2
) 1 ( 2
) 0 )( 0 ( 2 ) 1 ( ) 1 (
) 1 ( 2
1
2
) 1 ( 2
1
2
) 1 ( 2
1
2
) 1 ( 2
1
) 1 ( 2
1
) (
) 1 ( 2
1
: of tion representa e Alternativ
1
: Variance Sample
0 : Note
,..., 1
1 1
: Mean Sample
d Distribute tly Independen and Normal ) , ( ~ ,...,
2
2
2
2 2
1 1 1
2
1
2
1 1 1
2
1
2
1 1
2 2
1 1
2
1 1
2 2
2
1
2
2
1 1 1 1
1 1
1
2
1
S
S
n n
S n n
S n n S n n
n n
X X X X X X n X X n
n n
X X X X X X n X X n
n n
X X X X X X X X
n n
X X X X
n n
X X
n n
S
S
n
X X
S
X X X n X X X
n i
n
a X a X
n n
X
X
NID NID X X
n
i
n
j
j i
n
j
j
n
i
i
n
i
n
j
j i
n
j
j
n
i
i
n
i
n
j
j i j i
n
i
n
j
j i
n
i
n
j
j i
n
i
i
n
i
i
n
i
i
n
i
i
n
i
i
i
n
i
i i
n
i
i
n
i
i
n
=

= +

=
)
`

|
,
|
+

=
)
`

|
,
|
+

=
= +

=
= = =
= = =
'
+

'

= =

= = = =
= = = =
= =
= = = =
=
= = = =
= =
=
W Q
Independence of and S
2
(Normal Data) X
? A


)) ( exp( )) ( exp(
)) ( exp( )) ( exp(
)] ( ) ( exp[
) ( ) ( exp ) ( ) , (
} exp{
2
2
0 exp ) ( ) 2 , 0 ( ~
} 2 exp{
2
2
2 exp ) ( ) 2 , 2 ( ~
) , ( ~ ,
2 1 2 2 1 1
2 1 2 2 1 1
2 1 2 2 1 1
1 2 2 2 1 1 2 1 ,
2 2
2 2
2
1 2
2 2
2 2
2
2 1
2
2 1
2 1
t t X E t t X E
t t X t t X E
t t X t t X E
X X t X X t E e E t t M
t
t
t M N X X D
t t
t
t t M N X X T
NID X X
ind
D t T t
D T
D
T
+
= + =
+ +
= + + = =
=
)
`

|
,
|
+ = =
+ =
)
`

|
,
|
+ = + =
+
W
W
W
W Q
W
Q W Q
W Q
Independence of T=X
1
+X
2
and D=X
2
-X
1
for Case of n=2
X
Independence of and S
2
(Normal Data) P2 X


) ( ) (
2
2
exp
2
2
2 exp
2
2
2
2
2 exp
2
) 2 2 (
) ( exp
2
) (
) ( exp
2
) (
) ( exp
)) ( exp( )) ( exp(
)) ( exp( )) ( exp(
2 1
2
2
2 2
1
2
1
2
2
2 2
1
2
1
2 1
2
2
2
1 2 1
2
2
2
1
2
2 1 2 1
2
2 1
2
2 1
2
2 1
2
2 1
2 1 2 2 1 1
2 1 2 2 1 1
t M t M
t t
t
t t
t
t t t t t t t t
t t t t
t t
t t
t t
t t
t t X E t t X E
t t X t t X E
D T
ind
=

'
+

'

'
+

'

+ =

'
+

'

+ +
=

'
+

'

+ + + +
+ + + =

'
+

'
+
+ +

'
+

'

+ =
+
= + =
W W
Q
W W
Q
W
Q
W
Q
W
Q
Independence of T=X
1
+X
2
and D=X
2
-X
1
for Case of n=2
Thus T=X1+X2 and D=X2-X1 are independent Normals and & S
2
are independent
X
Distribution of S
2
(P.1)

_ a




2 /
1 ) ( ) 1 ( ) ( ) 1 (
2
2
2
2
2
2
1
2
2
1
2
1
2
2
1
2 2
2
2
1
2
2
1
2
2
1
2
2
1
2
1
2 2
) 2 1 ( ) ( ) ( ) (
: t independen are and Now,
) 1 (
0
1
2
1
2
1
1 1
) 2 , 2 / ( ~
~ ) 1 , 0 ( ~ ) , ( ~
2
1
2 2
2
2
2
2
2 2
n
X
X n S n X n S n
n
i
i
n
i
i
n
i
i
n
i
i i
n
i
i
n
i
i
n
i
i
n
n
i
i
i
i
i i
t M t M t M t M
S X
X n S n
X n X X
X X X X n X X
X X X X X X
X X X X
X
n Gamma
X
Z N
X
Z NID X
n
i
i

+
=
= =
=
= = =
=
= = =

=
)
`

|
,
|
+ + =
)
`

|
,
|
+ + =
+ +
= + = =

'
+

'

'
+

'

=
=

Q
W W
Q
W W
Q
W
Q
W
Q
W
Q Q
W
Q Q
W
Q
W
Q
W W
Q
G
W
Q
G
W
Q
W Q
Distribution of S
2
P.2





2
1
2
2
2 / ) 1 ( ) 2 / 1 ( ) 2 / (
2 / 1
2 /
) (
1
) 1 (
2 / 1
) (
2
1
2
2
2
1
2
2
1
2
2
2
2 /
1 ) ( ) 1 ( ) ( ) 1 (
2
2 ,
2
1
~
) 1 (
) 2 1 ( ) 2 1 (
) 2 1 (
) 2 1 (
) (
) 2 1 ( ) ( ~
) 1 , 0 ( ~
1
,
1
~
: : consider Now,
) 2 1 ( ) ( ) ( ) (
: t independen are and Now,
2
2
2
1
2
2
2
2
2
2
1
2 2
2
2
2
2
2 2

= =

'
+

'

= =

= =
=

'
+

'

'
+

'

= = =

= = =
=
=

n
n n
n
X n
X
S n
X n
X
X
X
n
i
n
i
X X
n
X
X n S n X n S n
n
Gamma
S n
t t
t
t
M
M
t M
t t M
X n
N
X n
n
X
X
Z
n n n
N X
X n
t M t M t M t M
S X
n
i
i
n
i
i
G
W
G
W
Q
W
Q
W
Q
W
Q
W
W W Q Q Q
W
Q
W
Q
Q
W
W
W
Q
Q
W W
Q
W W
Q
Summary of Results
X
1
,X
n
random sample from N(Q,W
2
) population
In practice, we observe the sample mean and sample variance (not
the population values: Q, W
2
)
We use the sample values (and their distributions) to make
inferences about the population values

ons) distributi - F and for t, on presentati
.ppt on ng conditioni of method using derivation (See
~
) 1 (
) 1 (
) 1 (
/
/
t independen are ,
~
) 1 (
1
, ~
1
2
1
2
2
2
2
1
2
1
2
2
2
1
2
2
2
1

= = =

'
+

'

=

n
n
n
n
i
i
n
i
i
n
i
i
t
n
Z
n
S n
n
X
n S
X
t
S X
X X
S n
n
X X
S
n
N X
n
X
X
G
W
W
Q
Q
G
W W
W
Q
Order Statistics
X
1
,X
2
,...,X
n
Independent Continuous RVs
F(x)=P(Xx) Cumulative Distribution Function
f(x)=dF(x)/dx Probability Density Function
Order Statistics: X
(1)
X
(2)
... X
(n)
(Continuous can ignore equalities)
X
(1)
= min(X
1
,...,X
n
)
X
(n)
= max(X
1
,...,X
n
)
Order Statistics




) ( )] ( 1 [
)] ( 1 [
)] ( 1 [
] )] ( 1 [ 1 [
) (
) (
: Minimum of pdf
)] ( 1 [ 1 ) ( 1 ) ,..., ( 1
: Minimum of CDF
) ( )] ( [
) (
)] ( [
)] ( [
) (
) (
: Maximum of pdf
)] ( [ ) ( ) ,..., (
: Maximum of CDF
1 1
) 1 (
1
1 1 ) 1 (
) 1 (
1 1 ) (
1 1 ) (
) (
x f x F n
dx
x F d
x F n
dx
x F d
dx
x X dP
x g
x F x X P x X P x X x X P x X P
X
x f x F n
dx
x dF
x F n
dx
x F d
dx
x X dP
x g
x F x X P x X P x X x X P x X P
X
n n
n
n
n n
n n
n
n
n
n
n n n
n


=

=

=

=
= > > = > > =
= = =

=
= = =
.
.
Example
X
1
,...,X
5
~ iid U(0,1)
(iid=independent and identically distributed)
|
,
|
=
=
|
,
|
=
=
|
,
|

=

,
|
>

=
o.w. 0
1 0 ) 1 ( 5 ) 1 )( 1 ( 5
) ( : Minimum
o.w. 0
1 0 5 ) 1 ( 5
) ( : Maximum
o.w. 0
1 0 1
) (
1 1
1 0
0 0
) (
4 4
1
4 4
x x x
x g
x x x
x g
x
x f
x
x x
x
x F
n
Order Stats - U(0,1) - n=5
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
x
p
d
f
f(x)
gn(x)
g1(x)
Distributions of Order Statistics
Consider case with n=4
X
(1)
x can be one of the following cases:
Exactly one less than x
Exactly two are less than x
Exactly three are less than x
All four are less than x
X
(3)
x can be one of the following cases:
Exactly three are less than x
All four are less than x
Modeled as Binomial, n trials, p=F(x)
Case with n=4


)) ( 1 ( ) ( ) ( 12 ) ( ) ( 12 ) ( ) ( 12 ) (
) ( 3 ) ( 4
) ( ) ( 4 ) ( 4
)] ( 1 [ )] ( [
4
4
)] ( 1 [ )] ( [
3
4
)] ( 1 [ 1
)] ( 1 [ )] ( [
4
4
)] ( 1 [ )] ( [
3
4
)] ( 1 [ )] ( [
2
4
)] ( 1 [ )] ( [
1
4
2 3 2
3
4 3
4 4 3
0 4 3
) 3 (
4
0 4 3
2 2 3 1
) 1 (
x F x F x f x f x F x f x F x g
x F x F
x F x F x F
x F x F x F x F x X P
x F
x F x F x F x F
x F x F x F x F x X P
= =
=
+ =
=

'
+

'

'
+

'

=
=

'
+

'

'
+

'

'
+

'

'
+

'

=
General Case (Sample of size n)
|
,
|

=


= V


=


elsewhere 0
... ) ( )... ( !
) ,..., (
: statistics order all of on distributi Joint
) ( ) ( )] ( 1 [ )] ( ) ( [ )] ( [
)! ( )! 1 ( )! 1 (
!
) , ( : 1
l) multinomia (uses stats order and of on distributi Joint
1 ) ( )] ( 1 [ )] ( [
)! ( )! 1 (
!
) (
1 1
1 ,..., 12
1 1
1
n n
n n
j i
j n
j
i j
i j
i
i
j i ij
th th
j n j
j
x x x f x f n
x x g
x f x f x F x F x F x F
j n i j i
n
x x g n j i
j i
n j x f x F x F
j n j
n
x g
Example n=5 Uniform(0,1)
1 0 ) ( 20 ) , ( : 5 , 1
5 ) 1 ( ] 1 [ ] [
! 0 ! 4
! 5
) ( : 5
) 1 ( 20 ) 1 ( ] 1 [ ] [
! 1 ! 3
! 5
) ( : 4
) 1 ( 30 ) 1 ( ] 1 [ ] [
! 2 ! 2
! 5
) ( : 3
) 1 ( 20 ) 1 ( ] 1 [ ] [
! 3 ! 1
! 5
) ( : 2
) 1 ( 5 ) 1 ( ] 1 [ ] [
! 4 ! 0
! 5
) ( : 1
1 0 ) ( 1 ) (
5 1
3
1 5 5 1 15
4 5 5 1 5
5
3 4 5 1 4
4
2 2 3 5 1 3
3
3 2 5 1 2
2
4 1 5 1 1
1
= = =
= = =
= = =
= = =
= = =
= = =
= =





x x x x x x g j i
x x x x g j
x x x x x g j
x x x x x g j
x x x x x g j
x x x x g j
x x x F x f
Distributions of all Order Stats - n=5 - U(0,1)
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
x
p
d
f
f(x)
g1(x)
g2(x)
g3(x)
g4(x)
g5(x)