Professional Documents
Culture Documents
Multivariate Distributions
1-1
Characteristic Function
Rp
is
X (t ) = E (e t X ) =
i
t x f (x ) dx ,
Rp ,
where
Multivariate Distributions
Properties of cf:
1-2
X (0) = 1,
if
|X (t )| 1
is absolutely integrable (
|(x )|dx
then
f (x )
(2 )p
it
x (t ) dt . X
if X
= (X1 , X2 , . . . , Xp )
then for t
...
Multivariate Distributions
For X1 , . . . , Xp independent RV and t
1-3
= (t1 , t2 , . . . , tp )
p
is:
X (t ) = X1 (t1 ) . . . X (tp ).
For X1 , . . . , Xp independent RV, t
is:
X1 +...+X (t ) = X1 (t ) . . . X (t ).
p p
The characteristic function allows to recover all the cross-product moments of any order: have
jk 0, k = 1, . . . , p ,
j
p
= (t1 , . . . , tp ) .
t =0
we
X11 . . . Xp
= j1 +...+j i
X (t ) j1 . . . t j t1 p
Multivariate Distributions
X
1-4
R1
fX (x )
1 2
exp
x2
2
X (t ) = = =
since
1 2
tx exp x
2
dx
2 exp
exp
t2
2
1 2
(x it )2
2
dx
exp
t2
2 exp
i2 = 1 and
t) (x 2
i
dx
= 1.
Multivariate Distributions
1-5
Theorem
The distribution of X
Rp
Rp .
Rp
This theorem says that we can determine the distribution of X in by specifying all the one-dimensional distributions of the linear
combinations
p j =1
t j Xj
=t
X,
= (t1 , t2 , . . . , tp ) .
Multivariate Distributions
1-6
Summary: Moments
X (t ) = E (e it X ).
The distribution of a p -dimensional random variable X is completely determined by all one-dimensional distributions of
t X,t
Rp
(Theorem of Cramer-Wold).
Multivariate Distributions
1-7
Cumulants
For a random variable X with density f and nite moments of order
X (t ) = E (e itX )
j
has a derivative
ij
The values
j log{X (t )} = j , t j t =0 > 1)
= 1, . . . , k . j does X + a.
The cumulants are natural parameters for dimension reduction methods, in particular the Projection Pursuit method.
Multivariate Distributions
1-8
The relation between the rst k moments m1 , . . . , mk and the cumulants is given by
m1 m2
. . . 1 0
...
m1
...
.. . . . .
k = (1)k 1
. . .
mk
1
0
mk 1
...
k k
1 2
m1
Multivariate Distributions
1-9
Suppose that k
= 1,
then
1 = m1 .
For k
=2
we obtain
m1
1 1 0
2 =
m2
m1
2 = m2 m1
Multivariate Distributions
For k
1-10
m1
=3
3 =
m2 m3
m1 m2
= = =
m1
m1
m2
+ m3
0 1
m1
Multivariate Distributions
In a similar fashion we nd the moments from the cumulants:
1-11
m1 m2 m3 m4
= 1
2 = 2 + 1 3 = 3 + 32 1 + 1 2 2 4 = 4 + 43 1 + 32 + 62 1 + 1
A very simple relationship can be observed between the semi-invariants and the central moments
= m1 as dened 4 = 4 32 2.
before. We have, in
k = E (X )k , where fact, 2 = 2 , 3 = 3 ,
Multivariate Distributions
Skewness
1-12
and kurtosis
3 = 4 =
E (X E (X
)3 / 3 )4 / 4
The skewness and kurtosis determine the shape of onedimensional distributions. The skewness of a normal distribution is 0 and the kurtosis equals 3. The relation of these parameters to the cumulants is given by:
3 = 4 =
MVA: HumboldtUniversitt zu Berlin
3 2 4 2 2
3/2
Multivariate Distributions
1-13
Transformations
X
fX
pdf of Y
= 3X ?
X
= u (Y )
Rp Rp = ui (y ) yj
J =
fY (y )
xi yj
= abs(|J |)fX {u (y )}
Multivariate Distributions
1-14
Example
(x1 , . . . , xp ) = u (y1 , . . . , yp )
Y
= 3X X =
1 3Y
= u (y ) J =
1 3
0 .. .
0 abs(|J |)
1 3
1 3
Multivariate Distributions
1-15
= AX + b ,
X
nonsingular
= A1 (Y b) J = A 1
fY (y )
Multivariate Distributions
X
1-16
= (X1 , X2 ) R2
with density fX (x ) 1 1 1
= fX (x1 , x2 ) =
0 0
A=
= AX + b =
1
X1
+ X2 X1 X2 A1 =
1 1 2
|A| = 2,
abs(|A|
)= ,
2 1 2
1 1 1 1
fY (y )
1 2
fX
(y1 + y2 ), (y1 y2 )
2
Multivariate Distributions
1-17
Summary: Transformations
= u (Y ),
has pdf fY (y )
= abs(|J |) fX {u (y )}, (y ) J = u . y
i j
where
Multivariate Distributions
1-18
Multinormal Distribution
The pdf of a multinormal is (assuming that 1
f (x )
= |2 |1/2 exp (x ) 1 (x ) .
2
Np (, ) = , = > 0.
Expected value is EX
Multivariate Distributions
1-19
(x ) 1 (x ) = d 2
If X
Np (, ),
= (X ) 1 (X )
is
normal sample
7 7
contour ellipses
X2 2
-1
-2
-3
3 X1
-3
1
-2
-1
X2 2
3 X1
3 2
and
1.5
1.5 4 .0
MVAcontnorm.xpl
Multivariate Distributions
1-21
is
= k < p,
1 k
exp
(2 )k /2 (1 k )1/2
= G-inverse
(x ) (x )
2
Multivariate Distributions
1-22
Np (, )
is
f (x )
= |2 |1/2 exp (x ) 1 (x ) .
2
The contour curves of a multinormal are ellipsoids with half-lengths proportional to eigenvalues of
i ,
where
denote the
. Np (, )
to a
Y X
= ) Np (0, Ip ). Vice versa, one can create Np (, ) from Y Np (0, Ip ) via X = 1/2 Y + .
1/2 (X
Multivariate Distributions
1-23
< p)
then
it denes a singular normal distribution. The density of a singular normal distribution is given by
(2 )k /2 (1 k )1/2
where
exp
(x ) (x ) ,
2
Multivariate Distributions
1-24
Limit Theorems
Central Limit Theorem describes the (asymptotic) behaviour of
sample mean
X1 , X2 , . . . , Xn , i.i.d with Xi
(, )
for
The
n (x
) Np (0, )
CLT can be easily applied for testing. Normal distribution plays a central role in statistics.
MVA: HumboldtUniversitt zu Berlin
-3
-2
=5
(left) and n
= 35
(right).
MVAcltbern.xpl
=5
(left) and
= 85
(right).
MVAcltbern2.xpl
Multivariate Distributions
1-27
a consistent estimator of
P .
x is asymptotically normal:
n 2 (x
) Np (0, Ip )
Xi
N (, 2 )
N (0, 1)
Multivariate Distributions
1-28
distribution.
C1 P (
u1/2 , x
+
for
n n
u1/2
C1 ) 1
-2
-1
0 x
The standard normal cdf and the empirical distribution function for
= 100.
MVAedfnormal.xpl
-2 x
The standard normal cdf and the empirical distribution function for
= 1000
MVAedfnormal.xpl
-2
-1
0 x
MVAedfbootstrap.xpl
Multivariate Distributions
1-32
Fn
= n 1
n I (x x ) i =1 i
Xi
F i Fn
sup
n(x
x)
<u P
n(x
) <u
a.s.
Construction of Condence Intervals possible! The unknown distribution of x can be approximated by the known distribution of
x .
Multivariate Distributions
1-33
Transformation of Statistics
= (f1 , . . . , fq ) : Rp Rq p are real valued functions which are dierentiable at R , then f (t ) is asymptotically normal with mean f () and covariance D D, i.e., L n{f (t ) f ()} Nq (0, D D ) for n ,
If
n (t
) Np (0, )
and if f
where
D= (p q )
fj ti
(t )
t =
This theorem can be applied e.g. to nd the variance stabilizing transformation.
Multivariate Distributions
1-34
Example
Suppose
{Xi }n i =1 (, );
We have by CLT for n
=
n(x
0 0
1 0.5
0.5 1
= 2.
) N (0, ).
x2 ? x 1 + 3x 2 consider f = (f1 , f2 )
2 = x1 x2 ,
x2 1
with
f1 (x1 , x2 )
f2 (x1 , x2 )
= x1 + 3x2 ,
= 2.
Multivariate Distributions
Then f
() =
0 0
1-35
and
D = (dij ),
dij
fj xi
x =
1 2
1
2x1
1 3
=
x =0
1 3
1
3
1 3
1 2
1 D
7 2
7 2
13
D
This yields
D D
x2 x 1 + 3x 2
x2 1
N2
0 0
7 2
7 2
13
Multivariate Distributions
1-36
n(x
is
(, ) asymptotically N (0, )
then
(, )
then
x
.
u1/2
Multivariate Distributions
1-37
) Np (0, ), then this holds also for a function f (t ), i.e., n{f (t ) f ()} is asymptotically normal.
n (t