Professional Documents
Culture Documents
By
Dr.Richard Tuyiragize
School of Statistics and Planning
Makerere University
March 1, 2022
1 The multivariate pdf
If the probability distribution function (pdf) of a p-variate random vector, denoted as
′
X = x1 , x2 , · · · , xn is defined as the joint multivariate pdf of its components random
variables, then, we say that f (x1 , x2 , · · · , · · · , xp ) of f ((X)) are values assumed by the ran-
dom vector of the p random components.
In order to efficiently analyze the multivariate pdf, the random vector is divided into groups
or blocks called partitions. Partitioning can be considered simply as a data management
tool, which helps to organize the transfer of information between the main matrix and the
sub matrices. This is because the basic manipulations can be applied to the smaller blocks
just like the main matrix.
Marginal distribution
A partition may consist of one or more component random variables. For two partitions,
consider;
x1
x2
..
.
X1
′
X = xr =⇒ X =
X2
xr+1
.
..
xp
The marginal pdf of a partition X 1 is defined
as the joint marginal pdf of its component
random variables i.e. x1 , x2 , · · · , xr such that
X 1 ∼ f (X 1 ) = f x1 , x2 , · · · , xr
Z xr+2 Z xr+4 Z xp
Thus; f (X 1 ) = ... f (X) dxr+1 dxr+2 . . . dxp
xr+1 xr+3 xp−1
That is, the original pdf of the p-variate random vector after integrating out the undesired
partition.
Z x2 Z x4 Z xr
The marginal pdf of the partition X 2 ; f (X 2 ) = ... f (X) dx1 dx2 . . . dxr
x1 x3 xr−1
STA3120 1 Email:trgzrich@yahoo.com
Conditional distribution
For two random variables, X and Y with a joint pdf of f (XY ), we define the conditional
distribution of Y /X as,
f (XY )
f (Y /X) = f (X)
, where f (X) is the marginal pdf of X
Question
Solution
Z 1 Z 1 Z 1 Z 1
f (wz) = 16wxyz dxdy = 16wz xy dxdy
0 0 0 0
Z 1 1 1
1 2 1 2
f (wz) = 16wz y x = 8wz. y = 4wz
0 2 0 2 0
4wz, 0 ≤ w, z ≤ 1
f (w, z) =
0, elsewhere
Similarly,
f (wxyz) 16wxyz
f (xy/wz) = = = 4xy
f (wz) 4wz
4xy, 0 ≤ xy ≤ 1
f (xy/wz) =
0, elsewhere
STA3120 2 Email:trgzrich@yahoo.com
Stochastic independence
′
For a p-variate random vector X = x1 , x2 , · · · , xn with a pdf f (X). The component
random variables are said to be stochastically independent if;
Question
e−(x+y+z) , xyz ≥ 0
f (xyz) =
0, elsewhere
Show that the three variables are independent.
Solution
Z Z Z ∞ Z ∞
∞ ∞
−(x+y+z)
f (x) = f (xyz) dydz = e dydz = e . − e . − e = e−x
−x −z
−y
y z y z 0 0
Thus;
f (x) = e−x , f orx > 0
f (y) = e−y , f orx > 0
f (z) = e−z , f orx > 0
∴ f (x).f (y).f (z) = e−x .e−y .e−z = e−(x+y+z) = f (xyz)
1 1 x−µ 2
f (x) = √ e− 2 ( σ ) −∞<x<∞
σ 2π
STA3120 3 Email:trgzrich@yahoo.com
′
Consider the components of a p-variate random vector X = x1 , x2 , · · · , xp as being
identically independent univariate normal random variables. Then xj ∼ N (µj , σj2 ) such that
xj −µj 2
1 − 12 ( )
f (xj ) = √ e σj
− ∞ < xj < ∞
σj 2π
n
Y
f (x) = f (x1 ).f (x2 ). · · · . · · · .f (xp ) = f (xj ) j = 1, 2, 3, · · · , p
i=1
1 1 x1 −µ1 2 1 1 x2 −µ2 2 1 x −µ
− 1 ( p p )2
f (x) = √ e − 2 ( σ 1 ) . √ e − 2 ( σ2 ) . · · · . · · · . √ e 2 σp
σ1 2π σ2 2π σp 2π
p
x1 −µ1 2
− 12
P
1 ( σ1
)
f (x) = p e j=1
p
Y
(2π) 2 σj
i=1
σ11 0 ··· 0 ··· 0
0 σ22 · · · 0 ··· 0
Σ=
.. .. .. .. .. ..
. . . . . .
0 0 ··· 0 ··· σpp
p p
Y 1
Y
Let |Σ| = σ12 .σ22 . · · · .··· .σp2 = σj2 −→ |Σ| = 2 σj
i=1 i=1
p
x1 −µ1 2
− 21
P
1 ( σ1
)
Then, f (x) = p 1 e
j=1
(2π) 2 |Σ| 2
p
xj −µj 2 xj −µj x −µ ′
= ( x1σ−µ ) + ( x2σ−µ
1 2
) + · · · + ( xpσ−µ
2 2 p 2
P
But also, ( σj
) 1 2 p
) =( σj
)( jσj j )
j=1
STA3120 4 Email:trgzrich@yahoo.com
( x1σ−µ 1
1
)
( x2 −µ2 )
σ2
x1 − µ 1 x2 − µ 2 xp − µ p
⇒ ( )( )···( ) ..
σ1 σ2 σp .
xp −µp
( σp )
1
σ12
0 ··· 0 ··· 0
x1 − µ 1
1
x2 − µ 2
0 ··· 0 ··· 0
σ22
⇒ (x1 − µ1 x2 − µ2 · · · xp − µp )
..
.
. .. .. .. .. .. .
. . . . . .
1
xp − µ p
0 0 ··· 0 ··· σp2
′
⇒ (x − µ) Σ−1 (x − µ)
1 1 ′ −1 (x−µ)
∴ f (x) = p 1 e− 2 (x−µ) Σ ⇒ X ∼ Np (µ, Σpxp )
(2π) |Σ|
2 2
p
P
Y1 = a11 X1 + a12 X2 + · · · + a1p Xp = a1j Xj = a1j X
j=1
Pp
Y2 = a21 X1 + a22 X2 + · · · + a2p Xp = a2j Xj = a2j X
j=1
..
.
p
P
Yq = aq1 X1 + aq2 X2 + · · · + aqp Xp = aqj Xj = aqj X
j=1
In matrix for:
STA3120 5 Email:trgzrich@yahoo.com
a11 a12 · · · · · · a1p
Y1 X1
Y 2
a21 a22 · · · · · · a2p X 2
.. = . ⇐⇒ Y = AX
. .. .. . . . . .. ..
. . . . .
Yq Xq
ap1 ap2 · · · · · · aqp
The new random vector Y which is a vector of linear combinations of the components
of vector X is called a Linear compound of the original random vector X
′
The linear compound Y has a q-variate distribution given by Y ∼ Nq (Aµ, AΣA ),
where µ and σ are parameters of X ∼ Np (µ, Σ).
Proof
For mean vector Y , by definition µY = E(Y );
µY = E(AX). since Y = AX
= AE(X)
= Aµ
′
Varaince Covariance matrix, ΣY = E[Y − E(Y )][Y − E(Y )]
′
ΣY = E[AX − Aµ][AX − Aµ]
′
E[A(X − µ][A(X − µ]
′ ′ ′
Recall that for two matrices, A and B, (AB) = B A
′ ′
→ ΣY = E[A(X − µ)][(X − µ) A ]
′ ′ ′
= AE(X − µ)(X − µ) A = AΣX A
Example
0 1 0
Given X ∼ N = ,
1 0 9
STA3120 6 Email:trgzrich@yahoo.com
Solution
Y1 2 −1 X1
= → Y = AX
Y2 0 3 X2
′
Since, Y ∼ N (Aµ, AΣA )
2 −1 0 −1
µY = Aµ = =
0 3 1 3
′
′ 2 −1 1 0 2 −1 13 −27
Also, ΣY = AΣA = =
0 3 0 9 0 3 −27 81
−1 13 −27
Then, Y ∼ N ,
3 −27 81
′
From property 1 if q = 1, the A reduces to a 1xp simple row vector a = a1 , a2 , · · · , ap
′ ′ ′
and the Y = a X is distributed as N1 (a µ, a Σa).
′
where a X = a1 X1 + a2 X2 + · · · + ap Xp
′
i.e. Y being a linear function of the components X1 , X2 , · · · , Xp and a being an arbi-
trary vector of constants.
STA3120 7 Email:trgzrich@yahoo.com
σ11 σ12 · · · ··· σ1p
σ21 σ22 · · · · · · σ2p 1
0
′
a Σa = 1, 0, · · · , 0 . = σ11
.. .. . . .. .. ..,
. . . . .
0
σp1 σp2 · · · · · · σpp
′
where a = 0, 1, · · · , 0 , then
′
a X = X2
′
a µ = µ2
′
a Σa = σ22
Generalizations
Each component Xj of the multivariate random vector X has a univariate normal distribu-
tion Xj ∼ N (µj , σj )
The joint marginal distribution for any two components Xj and Xk is a bivariate normal
distribution.
Xj µj σjj σjk
∼N ,
Xk µk σkj σkk
For j, k, and l
Xj µj σjj σjk σjl
Xk ∼ N µk , σkj σkk σkk
Xl µl σlj σlk σll
Example
0 3 2 1 4
2 2 5 2 0
Given, X ∼ N4
1 , 1
3 3 1
1 4 0 1 2
STA3120 8 Email:trgzrich@yahoo.com
Partitioning multivariate normal random vector
Let X ∼ Np (µ, Σ). If we correspondingly partition X, µ and Σ as
x1 µ1
x2 µ2
. .
. .
. .
X1 µ1
X = xr =⇒ X = µ = µr =⇒ µ =
X 2 µ2
xr+1 µr+1
. .
.. ..
x µp
p
σ11 · · · σ1r σ1r+1 · · · σ1p
.. ..
.
..
.
..
.
..
.
..
. .
σr1 · · · σrr σrr+1 · · · σrp Σ11 Σ12
Σ= =
σr+11 · · · σr+1r σr+1r+1 · · · σr+1p Σ21 Σ22
. . . . . .
.. .. .. .. .. ..
σp1 · · · · · · ··· · · · σpp
X 1 ∼ Nr (µ1 , Σ11 )
X 1 /X 2 = x2 ∼ Nr (µ1.2 , Σ11.2 )
where
µ1.2 = µ1 + Σ12 Σ22 (x2 − µ2 )
Note that µ1.2 and Σ11.2 are respectively called the conditional mean vector and conditional
variance covariance matrix of X 1 /X 2 = x2
STA3120 9 Email:trgzrich@yahoo.com
Independence of partitions
For bivariate cases, two random variables X1 and X2 are said to be independent if their
covariance is zero i.e.
Generally
The partitions X 1 and X 2 of a multivariate normal random vector X are said to indepen-
dent if their covariance is zero.
Example
0 3 2 0
Given, X ∼ N3 2 , 2 1 0
1 0 0 4
Solution
STA3120 10 Email:trgzrich@yahoo.com
X 1 ∼ N (µ1 , Σ11 )
x1 0 3 2
∴ ∼N ,
x2 2 2 1
X 1 /X 2 = x2 ∼ Nr (µ1.2 , Σ11.2 )
where
0 0 −1 0
µ1.2 = µ1 + Σ12 Σ22 (x2 − µ2 ) = µ1.2 = + 4 (x3 − 1) =
2 0 2
3 2 0 −1 3 2
Σ11.2 = Σ11 − Σ12 Σ−1
22 Σ21 = - 4 0 0 =
2 1 0 2 1
4. Independence
0
Σ(p−r)xr = Σrx(p−r) = 0; Σ12 = and Σ21 = 0 0
0
Question
0 1 2 0 0
0 0 1 0 0
Let, X ∼ N4
0 , 0
0 1 3
0 0 0 2 4
By way of partitioning or otherwise, state or find;
X1
1. The marginal distribution of
X2
X1 X3 x3
2. The conditional distribution of given =
X2 X4 x4
X1 X3
3. Determine whether and are independent.
X2 X4
Answers
STA3120 11 Email:trgzrich@yahoo.com
X1 0 1 2
1. ∼ N2 ,
X2 0 0 1
X1 X3 x3 0 1 2
2. given = ∼ N2 ,
X2 X4 x4 0 0 1
0 X1 X3
3. Σ12 = and Σ21 = 0 0 ∴ and are independent.
0 X2 X4
STA3120 12 Email:trgzrich@yahoo.com