Probability 2 - Notes 13

Continuous n-dimensional random variables
The results for two random variables are now extended to n random variables.
Definition Random variables X1 , ...Xn are said to be jointly continuous if they have a joint
p.d.f. which is defined to be a function fX1 ,X2 ,...,Xn (x1 , ..., xn ) such that for any measurable set A
contained in ℜn ,
Z

P((X1 , ..., Xn ) ∈ A) =

Z

...

(x1 ,...,xn )∈A

fX1 ,X2 ,...,Xn (x1 , ..., xn )dx1 ...dxn

It is convenient to write the joint p.d.f. using vector notation as fX (x), where X and x are
n-vectors with ith entries Xi and xi respectively.
To obtain a marginal p.d.f. simply integrate out the variables which are not required.
Example fX,Y,Z (x, y, z) = 3 for 0 < x < z, 0 < y < z and 0 < z < 1. Then
fX,Y (x, y) =
fX,Z (x, z) =
fY,Z (y, z) =

R1

max(x,y) 3dz = 3(1 − max(x, y))

Rz
0

Rz
0

for 0 < x < 1 and 0 < y < 1.

3dy = 3z for 0 < x < z < 1.

3dx = 3z for 0 < y < z < 1.

Using the (marginal) joint p.d.f. for fX,Z (x, z), fX (x) =
Using the (marginal) joint p.d.f. for fY,Z (y, z), fY (y) =
Using the (marginal) joint p.d.f. for fX,Z (x, z), fZ (z) =

R1
x

R1
y

Rz
0

3zdz = 23 (1 − x2 ) for 0 < x < 1.

3zdz = 23 (1 − y2 ) for 0 < y < 1.
3zdx = 3z2 for 0 < z < 1.

Conditional p.d.f We can define the conditional p.d.f. for one set of random variables given
another set, so for 1 ≤ m < n,
fXm+1 ,...,Xn |X1 ,...,Xm (xm+1 , ..., xn |x1 , ..., xm ) =

fX1 ,...,Xn (x1 , ..., xn )
fX1 ,...,Xm (x1 , ..., xm )

Example Consider the example above and condition on one random variable. We will consider
two out of the three cases. For each 0 < z < 1,

fX,Y |Z (x, y, |z) =

1
3
= 2
2
3z
z

for 0 < x < z, 0 < y < z. So X,Y |Z = z are independent random variables each with U(0, z)
distribution. We say that they are conditionally independent.
For each 0 < x < 1,

1

. n) it is easily seen (by integrating over the appropriate sets) that r P(0 Xi j ∈ A0i j j = 1. z|x) = 3 3 2 2 (1 − x ) = 2 (1 − x2 ) for 0 < y < z and x < z < 1... ... .. n) are mutually independent. So the conditional distribution of Y |X = x... z) = 3 1 = 3z z for 0 < y < z... n) are mutually independent for all such events. of any subset of the Xi is obtained by integrating out the other variables it immediately follows that r fXi1 ...... Z = z depends only on z and is U(0.. ir and all r = 2. Since the p. y) = 3 1 = 3(1 − max(x..Xn (x1 ... xi ] for dxi > 0 small. In addition if events 0 Xi ∈ A0i (i = 1.. .Xir (xi1 ..... Also for each 0 < x < 1 and 0 < y < 1. Independence Definition. xn . so that the events 0 Xi ∈ A0i (i = 1. ... y). xir ) = ∏ fXi j (xi j ) j=1 for all x11 .Z (y|x. n.. Xn are said to be (mutually) independent if fX1 .Y (z|x... if we take Ai = (xi − dxi . xir for all possible subsets i1 .. Again we will consider two of the three cases.. . y)) for max(x. y) < z < 1. fY |X.. . then 2 . . 1).f.Z|X (y. ... . Then for any events 0 Xi ∈ A0i (i = 1. z). Now consider conditioning on two random variables. fZ|X..d. For each 0 < x < z < 1..fY. Hence Z|X = x.. y)) (1 − max(x.. r) = ∏ P(0 Xi j ∈ A0i j ) j=1 for all possible subsets of the n events.. xn ) = ∏ni=1 fXi (xi ) for all x1 . .Y = y ∼ U(max(x. . n jointly continuous random variables X1 .

z) = kxy2 = kx × y2 × 1 for 0 < x < 1. Xn are independent with X j ∼ N(µ j .. If X1 .... Xn are independent. h n i MY (t) = E et ∑ j=1 X j = E ³ The result that Y ∼ Gamma " # n n n ³ t ´− ∑ j=1 α j t ´−α j ³ tX j = 1 − e = M (t) = 1 − ∏ ∏ Xj ∏ θ θ j=1 j=1 j=1 n θ. . is obtained by multiplying the individual p.dxn j=1 It immediately follows that X1 .dxn u P( Xi ∈ A0i n 0 i = 1. fY (y) = c2 y2 for 0 < y < 1 and fZ (z) = c2 for 0 < z < 1.d.. is the ’likelihood’ in statistics). Properties. Then we can use property 3 to show that ³ ´ Y = ∑nj=1 a j X j ∼ N ∑nj=1 a j µ j . X1 .. xn ) = ∏ni=1 gi (xi ) for some function gi . .f.Y. h t ∑nj=1 a j X j " i MY (t) = E e =E # n j=1 j=1 n = 2 2 n ∏ eta j X j = ∏ MX j (a jt) n 2 n 2 2 ∏ eµ j (a jt)+(σ (a jt) /2) = et ∑ j=1 a j µ j +(t /2) ∑ j=1 a j σ j j=1 3 .. y.d. for any functions hi for which the expectations exist.Y. X1 . c2 . .f.f.f. Examples.. . σ2j ).. cn with ∏ni=1 ci = 1.. c2 = 3. 0 < z < 1. When this condition holds then the marginal p. are mutually independent..g.Z (x.. fXi (xi ) = ci gi (xi ) for a suitable choice of c1 .. Xn are independent then the joint p. α j ).. . integrates to 1 and c1 c2 c3 = 1... . E [∏ni=1 hi (Xi )] = ∏ni=1 E[hi (Xi )]. ∑nj=1 α j ...’s the joint p.. xn )dx1 . Then we can use property 3 to show that Y = ∑nj=1 X j ∼ Gamma θ. Hence c3 = 1. Xn are independent then.f’s are easily obtained. .d... The ranges are independent and the joint p. . ∑nj=1 α j ´ then follows from the uniqueness of the m. 1. 3...f.. c1 .Xn (x1 .d... splits as indicated... Xn are independent³ with X j ∼´Gamma(θ.. 3.0 fX1 . Hence X. ∑nj=1 a2j σ2j . Hence independence for the X 0 s is equivalent saying that all events 0 Xi ∈ A0i .. (i = 1... You can ’spot’ independence in the same way as for two random variables.f.g.. c3 from the results that each marginal p.d. We can find the constant k. This provides useful results for the m. . 2........ where c1 c2 c3 = 1. .d. n) = ∏ P( i=1 Xi ∈ A0i ) u r ∏ fXi j (xi j )dx1. n).f.v. X1 . Z are independent and fX (x) = c1 kx for 0 < x < 1. fX. . If X1 . (see examples)...’s together (for jointly continuous r. 2. Xn are independent iff the ranges are independent and fX1 . 1. 0 < y < 1.Xn (x1 .. kc3 = 2 and c1 = 31 and hence k = 6.

.yn ) . . yn ) = fX1 .g.tn ) = E e∑ j=1 t j X j = E[et X ] Properties... You can find D by rewriting the constraints on the ranges of x1 . X2 ..g. n (i. So fX1 . yn .. ... X3 are independent Exp(θ). Transformations of variables Let X1 .. . The joint m. x2 > 0 and x3 > 0 become y1 > 0.Yn (y1 .g. ∂h1 (y1 .. ir }.. ∂y1 . .f. 0 < y1 < y2 < y3 < ∞.d. Let X and t be n-vectors (column vectors) with jth entries X j and t j respectively. n) which have continuous partial derivatives..e.... ¯ ¯ .. ....Xn (h1 (y1 . of a subset Xi1 ... Find the joint p.. .d. y2 − y1 > 0 and y3 − y2 > 0. ∂hn (y∂y1 .. . If X1 . y3 ) = θ e × ¯¯¯¯ −1 1 ¯¯ 0 −1 1 ¯¯ The ranges x1 > 0.³ ´ The result that Y ∼ N ∑nj=1 a j µ j .. Note that the joint m.Y2 ........ xn ) which has support S contained in ℜn .e... .... yn ))× ¯¯ . ...yn ) n) ¯¯ . . . ... ... Then ¯¯ ¯¯ ∂h1 (y1 ... .Xir of the X 0 s is obtained by setting t j = 0 for all j not in the set {i1 .. Example. .f........Y3 (y1 ... Xn is now defined and its properties given. . x2 > 0 and x3 > 0.....f. The joint m. y2 ... fX1 . .f.. MX (0) = 1). ..f. The inverses are X1 = Y1 ... for n random variables X1 .. 1. Then h n i T MX (t) = MX1 ... . hn (y1 . x3 ) = θ3 e−θ(x1 +x2 +x3 ) for x1 > 0.Xn (x1 .... x2 . Hence using the result above: ¯¯ ¯¯ ¯¯ 1 ¯¯ 0 0 ¯¯ ¯¯ 3 −θy3 0 ¯¯¯¯ = θ3 e−θy3 fY1 ... 2.....X3 (x1 . . X2 = Y2 −Y1 and X3 = Y3 −Y2 .. . equals one when t j = 0 for all j = 1. Y2 = X1 + X2 and Y3 = X1 + X2 + X3 .yn ) ¯¯ ∂yn ¯¯ .. for Y1 = X1 . . ∑nj=1 a2j σ2j then follows from the uniqueness of the m. Xn be n jointly continuous random variables with joint p.. Xn are independent then 4 .X2 .f..y ∂y1 n ¯¯ ¯¯ ¯¯ ¯¯ ¯¯ ¯¯ ¯¯ ¯¯ for (y1 . ...Xn (t1 ....... xn in terms of y1 .Yn ) (for i = 1.g. yn ). ¯¯ ∂hn (y1 ... X1 . The joint moment generating function. fY1 . Consider random variables Yi = gi (X1 . i. n which is a one to one mapping from S to D with inverses Xi = hi (Y1 .. . yn ) ∈ D.. Xn ) for i = 1..

If we set ti = 0 for i 6= j..g. Proof.. 2. .f. . ..g. .... Zn are independent N(0...f.g. X j −µ σ for j = 1.. which is the m.. (so one determines the other). then Z1 .g. If Z1 .g..g. If MX (t) = ∏nj=1 g j (t j ) for some functions g j . Zn are independent N(0... n.. Xn are independent with the p.... Xn are independent... Hence MY (t) = E[et TY h T i h T T i ] = E et AZ = E e(A t) Z = MZ (AT t) = e(1/2)(A T t)T (AT t) T AAT t = e(1/2)t T n j=1 5 2 = e(1/2)t t = ∏ et j /2 .f.. to obtain some important results in statistics. 1) random variables. then Y is an n-vector (entries Y1 ... .. Z1 . and the joint m. h n i ´ n n ³ MZ (t) = E e∑ j=1 t j (X j −µ)/σ = ∏ E[et j (X j −µ)/σ ] = ∏ e−µt j /σ MX j (t j /σ) n = ∏ ³ j=1 e−µt j /σ eµ(t j /σ)+(σ 2 /2)(t ´ 2 j σ) j=1 j=1 n 2 = ∏ et j /2 j=1 2 Hence by property 4 of the joint m.. . 1).h ∑nj=1 t j X j " i MX (t) = E e # n n ∏ et j X j = ∏ MX j (t j ) =E j=1 j=1 3.f.f. 1) and Y = AZ with Z the n-vector with jth entry Z j and A an n × n orthogonal matrix (i. If X1 . n. Therefore MX j (t j ) = n n n j=1 j=1 j=1 g j (t j ) g j (0) and hence MX (t) = ∏ g j (t j ) = ∏ g j (0)MX j (t j ) = ∏ MX j (t j ) Hence from property 3 of the joint m.. of the N(0.. . 2 T Proof. of X j determined by the m. 1) distribution.f.f. .g. then we obtain the m. for X j . Now MZ (t) = ∏nj=1 MZ j (t j ) = ∏nj=1 et j /2 = e(1/2)t t .. Xn are independent N(µ.. X1 .d.e. . AT A = AAT = I where I is the n × n identity matrix). .f. There is a unique relationship between the joint p.f. Zn are independent with MZ j (t j ) = et j /2 . hence MX j (t j ) = g j (t j ) ∏i6= j gi (0). then X1 .. n gives 1 = ∏ni=1 gi (0). Z1 . Also setting ti = 0 for i = 1.Yn ) of independent N(0. 1. Zn are Proof.. .. Hence from the uniqueness property of the m. j = 1.. 4. Use of the joint m..g.. 1)..d. σ2 ) and if Z j = independent N(0.. MX j (t j ) = g j (t j ) g j (0) .f.

. 1). 1) independent of U = ∑nj=2 Y j2 = ∑nj=1 (Z j − Z)2 = ∑nj=1 (X j − X)2 /σ2 ∼ χ2(n−1) . . √1 . Note: This provides the basis for the t and χ2 tests met in Fundamentals of Statistics 1. 1) then E[etY ] = (1 − 2t)−1/2 . 1) variables will also be used to prove results in Statistical Modelling 1.. and the uniqueness of the m. n. Xn are independent N(µ... .. If X1 . .. 1) then Y1 and U = ∑nj=2 Y j2 are independent with Y1 ∼ N(0..U (s. 1) and from result (3) Y1 = nZ = n(X − µ)/σ ∼ N(0. ³ Use result (2) with A the n×n matrix with first row √ nZ. Also from result (1) Z1 . Let Z j = (X j − µ)/σ for j = 1. Now # " h i n n n 2 2 2 sY1 +t ∑ j=2 Y j tY j sY1 sY1 = E[e ] ∏ E[etY j ] MY1 . Then Y1 = n n j=1 j=1 √1 .. Y1 ∼ N(0.f.Yn are independent N(0. √1 .g..f.. Y1 .f...Yn are independent N(0.. Also √1 . √1 n n n ´ ∑ Y j2 = YT Y = (AZ)T (AZ) = ZT AT AZ = ZT Z = ∑ Z 2j Therefore n 2 ∑ (Z j − Z) j=1 n = ∑ 2 Z 2j − nZ j=1 n = ∑ j=1 Y j2 −Y12 n = ∑ Y j2 j=2 √ √ Then from result (2) Y1 .. We use the result proved earlier that if Y ∼ N(0. 1) independent of U ∼ χ2(n−1) .Hence using property 4 of the joint m...... Theorem. If Y1 . 1) and U ∼ χ2(n−1) . .. 6 Z= . . √ n(X − µ)/σ ∼ N(0. Zn are independent N(0. Orthogonal transformations of independent N(0. .. .g.g..Yn are independent N(0. σ2 ) then ∑nj=1 (X j − X)2 /σ2 ∼ χ2(n−1) . 1) independent of √ √ Proof..g. √1 n n n ´ ³ .. . 3. 1)... 2 Proof.t) = E e = E e ∏e n 2 = MY1 (s) ∏ E[etY j ] = es 2 /2 j=2 j=2 j=2 n 2 ∏ (1 − 2t)−1/2 = es /2(1 − 2t)−(n−1)/2 j=2 Hence using property 4 of the joint m.f and the uniqueness of the m. Then nZ = n(X − µ)/σ and ∑nj=1 (Z j − Z)2 = ∑nj=1 (X j − X)2 /σ2 .