228 PR~BABlLITY AND &ANDOM VARIABLES

~.
o r x FIGURE 6-32

Differentiating with respect to x. we obtain

lex)
f(x I x > t) = 1 _ F(t) x> t (6-223)

The product l(x I x > t) dx equals the probability that the system fails in the interval
(x, x + dx). assuming that it functions at time t.

EX \MPLE 6-44 ~ If f(x) = ce-CX, then F(/) = 1 - e-cr and (6-223) yields
ce-CX
l(x I x> t} = -e--
ct = f(x - t)

This shows that the probability that a system functioning at time t fails in the interval
(x, x + dx) depends only on the difference x - t (Fig. 6-32). We show later that this is
true only if f (x) is an exponential density. ....

CONDmONAL FAILURE RATE. The conditional density f (x I x > I) is a function of

x and t. Its value at x = t is a function only of t. This function is denoted by pet) and
is called the conditionalfailure rate or, the hazard rate of the system. From (6-223) and
the definition of hazard rate it follows that

{3(t) = f(t I x> t) = 1 !~(t) (6-224)

!he product {3(/) dt is the probability that a system functioning at time I fails in the
interval (t, t + dt). In Sec. 7-1 (Example 7-3) we interpret the function {3(t) as the
expected failure rate.

EXAi\IPLL fl--l:' ~ (a) If f(x) = ce-cx, then F(t) = 1 - e- ct and

ce-cr
{3(t) = 1 _ (1 _ e-ct) = c
(b) ,If f(x) = c2xe -CX, then F(x) = 1 - cxe-CX - e-cx and
c2 te- ct c?-t
{3(t) = ctrCI + rCI = 1 + ct
From (6-224) it follows that
F'(t) R'(t)
{3(t) = 1 - F(t) = - R(t)
We shall use this relationship to express the distribution of x in terms of the function
 CHAPTER 6 lWO RANDOM VAlUABLES 229

pet). Integrating from 0 to x and using the fact that In R(O) = 0, we obtain

-fox pet) dt = In R(x)

Hence

R(x) =I - F(x) = exp { -fox {3(t) dt}

And since f (x) = F' (x), this yields

f(x) = {3(x) exp { -fox {3(t) dt} (6-225)

EXAMPLE 6-46 ~ A system is called memoryless if the probability that it fails in an interval (t, x),
assuming that it functions at time t, depends only on the length of this interval. In other
MEMORYLESS words, if the system works a week, a month, or a year after it was put into operation, it
SYSTEMS is as good as new. This is equivalent to the assumption that f(x I x> t) = f(x - t) as
in Fig. 6-32. From this and (6-224) it follows that with x = t:
pet) = f(t I x> t) = f(t - t) = f(O) = c
and (6-225) yields f(x) = ce- cx • Thus a system is memoryless iff x has an exponential
density. ~

EXAMPLE 6-47 ~ A special form of pet) of particular interest in reliability theory is the function
pet) = ct b- 1

This is a satisfactory approximation of a variety of failure rates, at least near the origin.
The corresponding f(x) is obtained from (6-225):

f(x) = cx b- J exp { _ c;b } (6-226)

This function is called the WeibuLl density. (See (4-43) and Fig. 4-16.) ~

We conclude with the observation that the function {3(t) equals the value of the
conditional density f(x I x > t) for x = t; however, pet) is not a density because its
area is not one. In fact its area is infinite. This follows from (6-224) be'cause R(oo) =
1- F(oo) =0.

INTERCONNECTION OF SYSTEMS. We are given two systems SI and S2 with times

to failure x and y, respectively, and we connect them in parallel or in series or in standby
as in Fig. 6-33, forming a new system S. We shall express the properties of S in terms
of the joint distribution of the random variables x and y~

Parallel: We say that the two systems are connected in parallel if S fails when both
systems fail. Denoting by z the time to failure of S, we conclude that z = t when the
230 PlOBABILITY AND RANDOM VARIABLES

Parailct Series Stand-by

s ~ s
z
~ 5

Y
Yt

'L
o z
(0)
X o w
(b)
x
s

o s
(c)
X

FIGURE 6-33

larger of the numbers x and y equals t. Hence [see (6-77)-(6-78)]

z = max(x, y) F~(z) = Fx,(z, z)
If the random variables x and yare independent. F~(z) = F.t(z)Fy(z).

Series: We say that the two systems are connected in series if S fails when at least one
of the two systems fails. Denoting by w the time to failure of S. we conclude that w = t
when the smaller of the numbers x and y equals t. Hence [see (6-80)-(6-81)]
w = min(x, y) F... (w) = F;lI(w) + Fy(w) - Fx,(w, w)
If the random variables x and y are independent,

where Px(t), P,(t). and p... (r) are the conditional failure rates of systems Sl. ~. and S,
respectively.

Standby: We put system Sl into operation, keeping s,. in reserve. When SI fails, we put
s,. into operation. The system S so formed fails when S2 fails. If tl and t2 are the times
of operation of Sl and S2. tl + 12 is the time of operation of S. Denotin.g by s the time to
failure ~ system S, we conclude that
s=x+y
The distribution of s equals the probability that the point (x, y) is in the triangular
shaded region of Fig. 6-33c. If the random variables x and y are independent, the density
ofs equals

h(S) = l' fx(t)f,(s - t) dt

as in (6-45).
 CHAPT.ER 6 TWO IWIDOM VAJUABLBS 231

6-7 CONDITIONAL EXPECTED VALUES

An"""""" theorem (5-55) to conditional UwU"lU'''', we obtain the conditional

E{g(y) IM)dy
be used to define the of y.
Using a limit argument as in (6-205). we can also define the conditional mean
E {g (y) 1x}. In particular.

77,lx = E{Ylx} = [Y/(Y Ix)dy (6-228)

is die conditional mean of y assuming x = x. and

conditional variance.
shall illustrate these ...cu'"W"'UUl~" tIlroullh example.

/x,(x, y) = {~ 0< Iyl < x < 1

otherwise
(6-230)

Determine E{xly} and E{ylx}.

SOLUTION
6-34 shows. /xy(x, y) = 1 and zero elsewhere. Hence

/x(x) = L: O<x<l

/y(y) = 11-IYI
Iyl < 1

This gives
/Xy(x. y) 1
/x!y(x, y) = /y(y) = 1 -Iyl 0 < Iyl < x < 1 (6-231)

FIGURE (;.34
232 PROBABILITY AND RANDOM VA1UA8L'I!S

and
~ ( I) = fxy(x.y) = -
1
Jylx y x 0< Iyl < x < 1 (6-232)
fx(x) 2x
Hence

E{x I y} = J xfxl)'(x I y) dx
t
= i lYI (1 _
x 1
Iyl) dx = (1 - lyD
X211
"2 Iyl

1 - lyl2 1 + Iyl
= 2(1-lyl) = -2- Iyl < 1 (6-233)

E{Ylx} = J yfYlx(Y Ix)dy = I x y

-x 2x dy
1 y21X
= 2x "2 -x = 0 o< x < 1 (6-234)

For a given x, the integral in (6-228) is the center of gravity of the masses in the
vertical strip (x, x + dx). The locus of these points, as x varies from -00 to 00, is the

1:
function

q>(x) = yf(y Ix) dy (6-235)

known as the regression line (Fig. 6-35).

Note If the random variables x and y are functionally related, that is, ify = g(x), then the probability masses
on thexy plane are on the line y = g(x) (see Pig. 6-Sb); hence E(y Ix) = g(x).

Galton's law. The term regression has its origin in the following observation attributed
'to the geneticist Sir Francis Galton (1822-1911): "Population extremes regress toward
their mean." This observation applied to parents and their adult children implies that
children of tall (or short) parents are on the average shorter (or taller) than their parents.
In statistical terms be phrased in terms of conditional expected values:
Suppose that the random variables x and y model the height of parents and their
children respectively. These random variables have the same mean and variance, and
they are positively correlated.:

l1x = TJx = 11 Ux = u)' =U r>O

y
::.~

}~
. - - t -......r cp(x)
;,~;

X,'~x+dxx

:;i FIGURE 6-35

 CHAPTER 6 1WO RANDOM VARIABLES 233

o x FIGURE 6-36

According to Galton's law. the conditional mean E {y I x} of the height of children whose
parents height is x, is smaller (orlarger) than x if x > 11 (or x < 11):
<x if x> 11
E{y Ix} = q>(x) { >x if x<l1
This shows that the regression line q>(x) is below the line y = x for x > 11 and above
this line if x < 11 as in Fig. 6-36. If the random variables x and y are jointly normal,
then [see (6-236) below] the regression line is the straight line q>(x) = rx. For arbitrary
random variables, the function ({l(x) does not obey Galton's law. The term regression is
used, however, to identify any conditional mean.

D\:Al\lPLE G-49 ~ If the random variables x and y are normal as in Example 6-41, then the function
x -111
E{ylx} = 712 +ru2--
U1
(6-236)

is a straight line with slope rU21 Ul passing through the point (111, 112). Since for normal
random variables the conditional mean E {y I x} coincides with the maximum of I (y Ix),
we conclude that the locus of the maxima of all profiles of I(x, y) is the straight line
(6-236).

1:1:
From theorems (6-159) and (6-227) it follows that

E{g(x,y) 1M} = g(x, y)/(x,y I M)dxdy (6-237)

~

This expression can be used to determine E {g (x, y) I x}; however, the conditional density
I(x, y I x) consists of line masses on the line x-constant. To avoid dealing with line
masses, we shall define E{g(x, y) I xl·as a limit:
As we have shown in Example 6-39, the conditional density t(x, y Ix < x <
x + AX) is 0 outside the strip (x,x + AX) and in this strip it is given by (6-203) where
Xl = =
X andx2 = X+Ax. It follows, therefore, from (6-237) withM {X < x !:: X+AX}
that

E{g(x. y) Ix < x!:: x 1 r+Ax

00
+ AX} = -oo}z g(a, y) F;c(x
I(a y)da
+~) _ Fz(x) dy
As AX -+ O. the innerintegral tends to g(x, y)/(x, y)ll(x). Defining E{g(x, y) Ixl as
234 PROBABIUTY AND RANDOM VAlUABLES

the limlt of the above integral. we obtain

E(g(x, y) Ixl = I: g(x, y)/(y Ix)dy (6-238)

I:
We also note that

E{g(x,y) Ix} = g(x,y)/(ylx)dy (6-239)

because g(x, y) is a function of the random variable y, with x a parameter; hence its
conditional expected value is given by (6-227). Thus
E{g(x,y) Ix} = E{g(x,y) Ix) (6-240)
One might be tempted from the above to conclude that (6-240) follows directly
from (6-227); however, this is not so. The functions g(x, y) and g(x, y) have the same
expected value, assuming x = x, but they are not equal. The first is a function g(x, y) of
the random variables x and y, and for a specific ~ it takes the value g[x(~), y(~)]. The
second is a function g(x, y) of the real variable x and the random variable y, and for a
specific ~ it takes the value g[x, y(~)] where x is an arbitrary number.

Conditional Expected Values as Random Variables

=
The conditional mean of y, assuming x x, is a function rp(x) =
E{y Ix) of x given
by (6-235). Using this function, we can construct the random variable q:I(x) = E (y Ixl
as in Sec. 5-1. As we see from (5-55), the mean of this random variable equals

E{rp(x))

Since f (x, y) =
= I: rp(x)/(x)dx = 1: 1:
I (x) I (y Ix), the last equation yields
I(x) yl(y \x)dydx

E{E{y I x}) = 1:I: yl(x, y) dx dy = E{y) (6-241)

This basic result can be generalized: The conditional mean E (g(x, y) I x} of

g(x, y), assuming x = x, is a function of the real variable x. It defines, therefore, the
function E {g(x, y) I x} of the random variable x. As we see from (6-159) and (6-237),
the mean of E{g(x, y) I x} equals

I: I:I(x) g(x, y)/(ylx)dydx

But ¢.e last integral equals E{g(x, y)}; hence

= 1:1: g(x, Y)f(x,~y)dxdy

E{E{g(x, y) I x}} = E{g(x, y)} (6-242)

We note, finally. that

E{g} (X)g2(y) Ixl = E{gl (X)g2(Y) Ixl = gl (X)E{g2(Y) Ixl (6-243)

E{gl(X)g2(Y)} = E{Efgl(X)g2(Y) Ix}} = E{gl(X)E{g2(Y) Ix}} (6-244)
 OiAPrBR 6 1WO RANDOM VAIUABLI!S 235

,EXAl\IP i -" t ~ Suppose that the random variables x and y are N(O, 0, O'f, O'f. r). As we know
E{x2 } 30'~
FurthermOlre, f (y Ix) is a normal

Using (6-244), we shall show that

E{xy} = rO'J0'2 E{xly2} = E{xl}E{r} + 2E2{xy} (6-246)

Proof-

E{xy} = E{xE{y

E{xlyZl = E{x2 E{yZ

(1,2
= 30')4 r 22-
0'(

and the proof is complete [see also (6-199»). ~

PROBLEMS
are independent, identically (U.d,) random variables with common
h(X) =e-1 U(y)
the p.d.f. of the following x + y, (b) x - y, (c)
min(x, y). (f) max(x. y).
6-2 x and y are independent and uniform in the interval (0, a). Find the p.d.f. of (a) x/y,
(b) y/(x + y), (c) Ix - YI:
6-3 The joint p.d.f. of the random variables x and y is given by
lin the shaded area
f~1(X, y) = { 0 otherwise

Let z = x + y. Find Fz(z) and fz{z).

-1

, FIGU.RE~3
236 PROBA8JLlTY AN'£) 1WIDOM VAlUABLES

6-4 The joint p.d.f. of x and y is defined as

6x x ~ 0, y ~ 0, x + y ~ 1
IIl,(x, y) = { 0 otherwise
Define z = x - y. Fmd the p.dl. of z.
6-5 x and y are independent identically distributed normal random variables with zero mean and
common variance ql, that is. x .... N(O, ql), y ,... N(O, q2) and Ill} (x, y) = III (x)I,(y).
=
Find thep.d.f. of(a)z "';X2 + T. (b) w = Xl + y2, (e) u x - y. =
6-6 The joint p.d.f. of x and y is given by
2(1 - x) 0 < x ~ I, 0 ~ Y ~ 1
Iz,(x, y) = { 0 otherwise

Detennine the probability density function of z = xy.

6-7 Given

{
X +Y 0 ~ x ~ 1, 0 ~ y ~ 1
IIl'l(x, Y) = 0 otherwise

= =
Show that (a) x + y has density II(z) 1}, 0 < Z < 1, II(z) %(2 - z), 1 < z < 2.
and 0 elsewhere. (b) xy has density 12(z) = 2(1 - z), 0 < Z < 1, and 0 elsewhere. (e) y/x
has density I,(z) = (1 + 7.)/3.0 < z < 1, 13(z) = (1 + z)f3z3. % > 1. and 0 elsewhere.
(d) y - x has density 14(z) = 1 -lzl.lzl < 1. and 0 elsewhere.
6·8 Suppose x and y have joint density
I 0 ~ oX ~ 2, 0 ~ 1 ~ 1, 21 ~ x
h,(X, y) = { 0 otherwise
Show that z = x + Ybas density
(lf3)Z 0 < %< 2
IIl,(x, y) = { 02 - (2/3)% 2 < z < 3
elsewhere
6-9 x and y are uniformly distributed on the triangular region 0 ~ 1 ~ x ~ 1. Show that (a) z =
=
x/y has density ,,(z) l/zl, z ~ 1. and It(z) = 0, otherwise. (b) Detennine the density
ofxy.
6-10 x and y are unifonnly distributed on the triangular region 0 < x ~ y ~ x + y ~ 2. Find the
p.d.f. of x + y and x - y.
6·11 x and y are independent Gamma random variables with common parameters a and p. Find
the p.d.f. of (a) x + Y. (b) x/y. (e) xf(x + y),
6-12 x and y are independent unifonnly distributed random variables on (O,d). Plnd the joint
p.d.f. ofx + y and x - y.
6-13 x and yare independent Rayleigh random variables with common parameter q2. Detennine
the density of x/yo
6·14 The random variables x and y are independent and z = x + y. Fmd I,(Y) if
IIl(x) = ee-UU(x) Ir.(z) = c?-tf"U(z)
6-1S The random variables x and y are independent and y is uniform in the interval (0, 1). Show
=
that. if z x + y. then
 CHAI'TER 6 lWO RANDOM VARIABLES 237

6·16 (a) The function g(x) is monotone increasing and y = g(x). Show that
F,,(X) if Y > g(x)
F"y(x.y) = {
Fy(Y) if Y < g(x)
(b) Find Fxylx. y) if g(x} is monotone decreasing.
6·17 The random variables x and y are N(O, 4) and independent Find fl(1.} and Ft(x) if (a) z =
2x + 3y. and (b) z = x/yo
6·18 The random variables x and y are independent with

f,,(x) = :2 e - x2 (J.ct2 U(X) f,(y) = {Ol/TtJl- y2 lyl < 1

.. Iyl> 1
Show that the random variable z = xy is N(O, ( 2 ).
6-19 .The random variables x and yare independent with Rayleigh densities
fAx} = ;2e-xl/2a2u(x) frey) = ;2e-y2!2~2UlY)
(a) Show that ifz = x/yo then
2a2 1.
ft (z) =P (1.2 + a2/ fil)2 U (z) {i)

(b) Using (i), show that for any k > 0,

k2
PIx ~ ky) = kl + a 2/ft2
6-20 The random variables x and y are independent with exponential densities
fAx) = ae-"U(x) f,(Y) = fte-~YU(y)

Find the densities of the following random variables:

x
(a) 2x + y (b) x - y (e) - (d) rnax(x, y) (e) min(x, y)
y
6-21 The random variables x and yare independent and each is uniform in the interval (0. a). Fmd
the density of the random variable z = Ix - YI.
6·22 Show that (a) the convolution of two normal densities is a normal density, and (b) the
convolution of two Cauchy densities is a Cauchy density.
6-23 The random variables x and yare independent with respective densities x2 (m) and x2 (n).
Show that if (Example 6-29)
x/m
Z=-
Yin
This distribution is denoted by F(m, n) and is called the Snedecor F distribution. It is used
in hypothesis testing (see frob. 8·34).
6-24 Express F, .. (z, w) in tenns of Fx.,(x, y) ifz = max(x. y), W = min(x, y).
6-25 Let x be the lifetime of a certain electric bulb, and y that ofits replacement after the failure of
the first bulb. Suppose x and y are independent with common exponential density function
with parameter 1. Find the probability that the combined lifetime exceeds 2>... What is the
probability that the replacement outlasts the original component by A?
6-26 ]I; and yare independent unifonnly distributed random variables in (0. 1). Let
w = max(x, y) z = min(x, y)
Fmd thep.di. of (a) r = w -z, (b) s = w+z.
238 PROBABILrrY:AND RAl'IDOM VARIABLES

6·27 Let x and y be independent identically distributed exponential rahdom variables with common
=
parameter A. Fmd the p.d.f.s of (a) z y /max(x, y). (b) w = x/min (x, 2y).
6·28 If x and y are independent exponential random variables with common parameter A. show
that x/ex + y) is a uniformly distributed random variable in (0, 1).
6·29 x and y are independent exponential random variables with common parameter A. Show that

z = minCK. y) and w = max(x. y) - min(x. y)

are independent random variables.
6·30 Let x and y be independent random variables with common p.d.f. ix(x) = IrOto:x Ot- I •
o < x < ~, and zero otherwise (0: ~ 1). Let z = min(x, y) and w = max(x, y). (a) Frod the
p.d.f. ofx + y. (b) Find the joint p.d.f. of z and w. (c) Show that z/w and w are independent
random variables.
6·31 Let x and y be independent gamma random variables with parameters (al.~) and (0:2, {J).
respectively. (a) Determine the p.d.f.s of the random variables x + y, x/y, and x/ex + y).
(b) Show that x + y and x/yare independent random variables. (c) Show that x + y and
x/ex + y) are independent gamma and beta random variables, respectively. The converse to
(b) due to Lukacs is also true. It states that with x and y representing nonnegative random
variables, if x + y and x/yare independent, then x and y are gamma random variables with
common (second) parameter p.
6·32 Let x and y be independent normal random variables with zero mean and unit variances.
= =
(a) Find the p.d.f. ofx/lyl as well as that of Ixl/lyl. (b) Let u x + y and v r + y2. Are
u and v independent?
6-33 Let x and y be jointly normal random variables with parameters J.Lx, J.L" 0-;, q;,and r. Find
a necessary and sufficient condition for x + y and x - y to be independent
6·34 x and y are independent and identically distributed D<lrmal random variables with zero mean
and variance q2. Define

u = --=x?=-=r= v= xy
..jX2 + y2 ..jX2 +y2
(a) Find the joint p.dJ. J•.,(u. v) of the random variables u and v. (b) Show that u and v are
independent normal random variables. (c) Show that [(x - y)2 - 2fllJx2 + y2 is also a
normal random variable. Thus nonlinear functions of normal random variables can lead to
normal random variables! (This result is due to Shepp.)
6-35 Suppose z has an F distribution with (m. n) degrees of freedom. (a) Show that liz also has
an F distribution with (n, m) degrees of freedom. (b) Show that mz/(mz + n) has a beta
distribution.
6·36 Let the joint p.d.f. of x and y be given by

O<Y~X~OO
otherwise
Define z = x + Y. w = x - y. Find the joint p.d.f. of z and w. Show that z is an exponential
random variable.
6-37 Let
2e-<x+Y) 0< x < y< 00
ix,(x, y) ={0 th' .
o erwJse

Define z=x+y, w=y/x. Determine tbejoint p.d.f. ofz and w. Are z and w independent
random variables?
6-38 The random variables x and (} are independent and (} is uniform. in the interVal (-7r, 7r).
Show thatifz = xcos(wt + 8}. then

!,(z) = .!.1-IZ1 !:x(Y) dy + .!. [

f:x(Y) dy
7r -00 -/y2 - Zl 7r 1,1 -/]2 - Zl
6-39 The random variables x and y are independent. x is N (0. u 2 ). and y is uniform in the interval
(0. 7r). Show thatifz = x + a cosY. then
1~
"z- 7ru,,27r
F ( ) - 1·
0
e-~-fH:oJ1)2/262 d y

6-40 The random variables x and y are of discrete type, independent, with Pix = n} = a••
Ply = n) = b•• n = 0.1, .... Show that, ifz = x + y, then
/I

P{z = n} = ~ 4tb.._t. n =0.1.2•...

h.O
6-41 The random variable x is of discrete type taking the values x. widt Pix = x.} = P" and the
random variable y is of continuous type and independent of x. Show that if z = x + y and
w = "y, then

•
6-42 x and y are independent random variables with geometric p.m.f.
Pix = k} = pq" k = 0, 1.2, " . Ply = m} = pq'" m = 0, 1.2•...
Find the p.m.f. of (a) x + y and (b) x - y.
6-43 Let x and y be independent identically distributed nonnegative discrete random variables
with
Pix = k} = Ply = k} = Pi k = 0,1.2, ...
Suppose

P{x= klx+y=k} = P{x= k-Ilx+y=k} = k+1 1 k~0

Show that x and y are geometric random variables. (This result is due to Chatterji.)
6-44 x and y are independent, identically distributed binomial random variables with parameters
n and p. Show that z = x + y is also a binomial random variable. Find its parameters.
6-45 Let x and y be independent random variables with common p.m.f.
P(x = k) = pq" k = 0.1,2, ... q = P- 1
(0) Show that min(x, y) and x - y are independent random variables. (b) Show that z =
min(x. y) and w = max(", y) - min (x, y) are independent random variables.
6-46 Let" and y be independent Poisson random variables with parameters ~I and A.,. respectively.
Show that the conditional density function of x given x + y is binomial.
6-47 The random variables Xl and X2 are joindy normal with zero mean. Show that their density
can be written in the form

!(XI,X2)
I
= 27r,jXexp {I c- Xt} .
-2 X I
c= [""II ""12]
IJ.2I JI.22
240 PROBABJUI'Y AND RANt10M VARIABL£S

6;48 Show that if the random variables x and y are nonnaI and independent, then

P{xy<O} = G(~) +GC:) -20(::)0(::)

6·49 The random variables" and y are N (0; ( 2 ) and independent Show that if z = Ix - yj, then
E{z} = 20"/";;;, E(Z2} = 2u 2 •
6·50 Show that if x and y are two independent exponential random variables with fx(x) ==
e-.rU(x), f,(y) = e-YU(y), and z = (x - y)U(x - y), then E(z} = 1/2.
6·51 Showthatforanyx,yrealorcomplex(a) E{xy}l2::: E{lxj2}E{JYI2};(b)(triangleinequality)
v'E{lx + Y12} ::: v'E{lxI2} + E{lYI2}.
6-52 Show that, if the correlation coefficient r.r1 = I, then y = ax + b.
6-53 Show that, if E{x2 } = E{y2} = E{xy}, then X = y.
6-54 The random variable n is Poisson with parameter Aand the random variable x is independent
of n. Show that, if z = nx and

6-55 Let x represent the number of successes and y the number of failures of n independent
Bernoulli trials with p representing the probability of success in anyone trial. Find the
=
distribution ofz = x - y. Show that E{z} = n(2p - 1), Var{z} 4np(1 - p).
6-56 x and yare zero mean independent random variables with variances u~ and u;, respectively,
that is, x ,.., N (0, ut), y '" N (0, ui). Let

z=ax+by+c c;#O

(a) Find the characteristic function Cl>~(u) of z. (b) Using Cl>t(u) conclude that z is also a
normal random variable. (c) Fmd the mean and variance of z.
6-57 Suppose the Conditional distribution of " given y = n is binomial with parameters nand
PI. Further, Y is a binomial random variable with parameters M and 1'2. Show that the
distribution of x is also binomial. Find its parameters.
6·58 The random variables x and y are jointly distributed over the region 0 < JC < Y < 1 as

kX
fxy(x, y) = { 0
< < <I
0 x y
th •
o erwtse
for some k. Determine k. Find the variances of x and y. What is the covariance between :II:
andy?
6-59 x is a Poisson random variable with parameter A and y is a normal random variable with
mean J.t and variance u 2 • Further :II: and y are given to be independent. (a) Find the joint
characteristic function of x and y. (b) Define z = :II: + y. Find the characteristic function of z.
6-60 x and y are independent exponential random variables with common-:parameter A. Find
(a) E[min(x, y)], (b) E[max(2x, y)].
6-61 The joint p.d.f. of x and y is given by

6x
{ 0
x > 0, y > 0, 0 < x +y ~ I
f., (x, y) = otherwise

Define z = x - y. (a) FInd the p.d.f. of z. (b) Finel the conditional p.d.f. of y given x.
(c) Detennine Varix + y}.
6-62 Suppose xrepresents the inverse of a chi-square random variable with one degree of freedom,
and the conditional p.d.f. of y given x is N (0, x). Show that y has a Cauchy distribution.
 CHAPTBR6 1WORANDOMVARIABLES 241

6-63 For any two random variables x and y.let (1; =Var{X).o} = Var{y) and (1;+1 =VarIx + y).
(a) Show that
a.t +y < 1
ax +0",. -

(b) Moce generally. show that for P ~ 1

{E(lx + ylP)}l/p
-:-=-~...,..:.;,-:--..:..:-::::::-..,..-:-,:-:-:- < 1
{E(lxIP»l/P + (E(lyl')lIlp -

6-64 x and y are jointly normal with parameters N(IL",. IL" 0":. a;. Pxy). Find (a) E{y I x = x}.
and (b) E{x21 Y= y}.
6-65 ForanytworandomvariablesxandywithE{x2 } < oo,showthat(a)Var{x} ~ E[Var{xIY}].
(b) VarIx} = Var[E{x Iy}] + E[Var{x I y}].
6·66 .Let x and y be independent random variables with variances (1? and ai, respectively. Consider
the sum
z=ax+(1-a)y
Find a that minimizes the variance of z.
6-67 Show that, if the random variable x is of discrete type taking the valuesxn with P{x = x.1 =
p" and z = 8(X, y), then
E{z} = L: E{g(x", Y)}Pn fz(z) = L: fz(z I x.)Pn
n n

6-68 Show that, if the random variables x and y are N(O, 0, (12, (12, r), then

(a) E{/l"(Ylx)} = (1J2n"~2-r2) exp { --2a-=-/-(;x-~-r""'2)}

(b) I
E{/.. (x)/,(Y)} = ~
2Jta 2 4- r
6·69 Show that if the random variables ~ and yare N(O. O. at, ai, r) then
e
E{lxyll = -n"2 i0 •
arcsIn
0"10"2
20"1 a2
- J.L d/-L+ - -
n"
20"1 (12
= --(cosa
n"
.
+asma)

wherer = sina and C = r(1I(12.

(Hint: Use (6-200) with g(x, y) = Ixyl.)
6-70 The random variables x and y are N(3, 4, 1,4,0.5). Find I(y Ix) and I(x I y).
6-71 The random variables x and y are uniform in the interval (-1, 1 and independent. Fmd the
conditional density f,(r I M) of the random variable r = x2 + y2, where M = {r ::s I}.
(i·72 Show that, if the random variables x and y are independent and z = x + Y1 then 1% (z Ix) =
ly(2. -x).
6-73 Show that, for any x and y, the random variables z = F.. (x) and w = F, (y I x) are independent
and each is uniform in the interval (0. 1).
6-74 We have a pile of m coins. The probability of heads of the ith coin equals PI. We select
at random one of the coins, we toss it n times and heads shows k times. Show that the
probability that we selected the rth coin equals
~(l- p,),,-k
pW - PI),,-t + ... + p~(l- P1II)H-k
(i.75 Therandom variable x has a Student t distribution len). Show that E(x2} =n/(n - 2).
6·76 Show that if P~(t~ = fll(t Ix'> e), ~Ct 11 > t) and fJ~li) =kJJ,(t), then 1- F.. (x)
, [1 -,F,(X)]k.
6-77 Show that, for any x, y, and e > 0,
1
P{I!(-YI > s}:s s1E{lX-YI1}

6-78 Show that the random variables x and Yate independent iff for any a and b:
E{U(a - x)U(b - y)) = E{U(a - x)}E{U(b - y»
6·79 Show dlat
 CHAPTER

7
SEQUENCES
OF RANDOM
VARIABLES

7-1 GENERAL CONCEPI'S

A random vector is a vector
x = [x..... ,XII) (7-1)

whose components Xi are random variables.

The probability that X is in a region D of the n-dimensional space equals the
probability masses in D:

P(X e D} = L I(X)dX X = [Xl. .... x lI ] (7-2)

where
I(x) -- I(X" )- all F(Xlo ... ,X/I)
••• ,X/I - aXl •••• , aX,. (7-3)

is the joint (or, multivariate) density of the random variables Xi and

.
F(X) = F(xlo ••• ,XII) = P{Xt ~ Xlo •••• x,. !: XII} (7-4)
is their joint distribution.
If we substitute in F(Xl .... , Xn) certain variables by 00, we obtain the joint
distribUtion of the remaining variables. If we integrate I(xl, ... , XII) with respect to
certain variables, we obtain the joint density of the remaining variables. For example
F(XloX3) = F(Xl, OO,X3, (0)

/(x}, X3) = roo r

)-00)"-'
l(xlo X2, X3, X4) dX2 dX4
(7-S)

243
244 PROBAB/UTY AND RANDOM VARIABLES

'Note We haVe just identified various. functions in terms of their independent variables. Thus / (Xl, X3) is -;
joint density oflhe random variables "I and x3 and it is in general dif{erenrfrom thejoiDtdenSity /(X2. x.) of
the random variables X2 and X4. Similarly. the density /; (Xi) of the random variable Xi will often be denoted
by f(~i).
-
TRANSFORMATIONS. Given k functions
gl(X) •...• gk(X)

we form the random variables

YI = gl (X), ... ,Yk = gJ;(X) (7-6)
The statistics of these random variables can be determined in terms of the statistics of X
al! in Sec. 6-3. If k < n, then we could determine first the joint density of the n random
variables YI •... , Yk. ".1:+1> ••• , X'I and then use the generalization of (7-5) to eliminate
the x's. If k > n, then the random variables YII+I, ••• Yle can be expressed in terms of
I

Ylt ... y". In this case, the masses in the k space are singular and can be detennined in
I

terms of the joint density ofy ...... Y". It suffices, therefore, to assume that k = n.
=
To find the density !,,(YI, ...• YIl) of the random vector Y [YII ... ,YII] for a
specific set of numbers YI, ••. , YII' we solve the system
gl (X) = Ylo ...• g,,(X) = Yn (7-7)
If this system has no solutions, then fY(YI • ... ,Yn) = O. If it has a singlt~ solution
X = [Xl, ..•• x,,), then
f(y ) - !x(XI, ... ,X,,) (7-8)
'Y I, ···.Y" - IJ(xlo ... , X,,) I
where

J(Xl,·· .• Xn )= .............. . (7-9)

og" og"
aXI ax"
is the jacobian of the transformation (7-7). If it has several solutions. then we add the
corresponding terms as in (6-1l5).

Independence
The random variables Xl> ••• , XII are called (mutually) indepen&nt if the events
(XI ~ XI}, ••• , {x" ~ XII} are independent. From this it follows that

F(Xlo ..•• x,,) = F(Xt)· •• F(xn )

(7-10)

EXAi\IPLE 7-1 .. Given n independent random variables Xi with respective densities Ii (XI), we funn
the random variables
. Yk = XI + ... + XJ; k = 1, ... , n
 CHAPTER 7 SEQUENCES OF RANDOM VARIABLES 24S

We shall determine the joint density of Yt. The system

Xl = Yl. Xl + X2 = Yl •..•• Xl + ... + Xn = Yn
has a unique solution
Xk = Yk - Yk-I

and its jacobian equals 1. Hence [see (7-8) and (7-1O)J

!y(Yl ••. . , Yn) = 11 (Yl)!2(Yl - Yt)· .. J~(Yn - Yn-l) (7-11)

.From (7-10) it follows that any subset of the set Xi is a set of independent random
variables. Suppose, for example. that
I(xi. X2. X3) = !(Xl)!(X2)!(X3)
=
Integrating with respect to Xl, we obtain !(XI. Xz) !(XI)!(X2). This shows that the
random vr¢ables Xl and X2 are independent. Note, however. that if the random variables
Xi are independent in pairs. they are not necessarily independent. For example. it is
possible that
!(X., X2) = !(Xl)!(XZ) !(x .. X3) = !(Xl)!(X3) !(X2. X3) = !(X2)!(X3)
but !(Xl. Xz. X3) # !(Xl)!(X2)!(X3) (see Prob. 7-2),
Reasoning as in (6-21). we can show that if the random variables Xi areindependent,
then the random variables

are also independent

INDEPENDENT EXPERIMENTS AND REPEATED TRIALS. Suppose that

Sn = SI x .. · X Sn
is a combined experiment and the random variables Xi depend only on the outcomes ~,
of S,:
i = l ..... n
If the experiments Sj are independent, then the random variables Xi are independent [see
also (6-22)]. The following special case is of particular interest. :
Suppose that X is a random variable defined on an experiment S and the experiment
sn
is performed n times generating the experiment = S X ••• x S. In this experiment.
we defil)e the random variables Xi such that
i = 1, ... ,n (7-12)
From this it follows that the distribution F; (Xi) of Xi equals the distribution Fx (x) of the
random variable x. Thus, if an experiment is performed n times. the random variables Xi
defined as in (7-12) ate independent and they have the same distribution Fx(x). These
random variables are called i.i.d. (independent. identically distributed).
246 PROBABIUTY ANDRANDOMVARIAB4iS

EX \\IPLE 7-2 ... The oider statistics of the random variables X; are n random variables y" defined
as follows: For a specific outcome ~, the random variables X; take the values Xi (n.
ORDER Ordering these numbers. we obtain the sequence
STATISTICS
Xrl (~) :s ... :s X/~ (~) ::: ••• :s Xr• (~)
and we define the random variable Yle such that
YI (n = 141 (~) :s ... ~ Yk(~) = 14t(~) ::: ... ::: Ylf(~) = Xr. (~) (7-13)
We note that for a specific i, the values X; (~) of X; occupy different locations in the above
ordering as ~ cbanges.
We maintain that the density lie (y) of the kth statistic Yle is given by

Ik(Y) = (k _ 1)7~n _ k)! F;-I(y)[l- Fx(y)]n-Ie Ix(Y) (7-14)

where Fx (x) is the distribution of the Li.d. random variables X; and Ix (x) is their density.

Proof. As we know
I,,(y)dy = Ply < Yle ~ Y +dy}

=
The event B {y < y" ~ y + dy} occurs iff exactly k - 1 of the random variables Xi are less
than y and one is in the interval (y. y + dy) (Fig. 7-1). In the original experiment S, the events
AI = (x ~ y) A1 = (y < x ~ y + dy} As = (x > y + dy)
fonn a partition and
P(A.) = F.. (y) P(A~ = I .. (y)dy P(A,) = 1 - Fx(y)

In the experiment SR, the event B occurs iff A I occurs k - 1 times. A2 occurs once, and As OCCUIS
= = =
n - k times. With k. k - I, k1 1, k, n - k. it follows from (4-102) that
n! k-I ..-I<
P{B} = (k _ 1)11 !(n _ k)! P (AJ)P(A1)P (A,)

and (7-14) results.

Note that
II(Y) = n[1- F.. (y)]"-I/.. (Y) In(Y) = nF;-I(y)/s(y)
These are the densities of the minimum YI and the maximum y" of the random variables Xi,
Special Case. If the random variables Xi are exponential with parameter J..:
hex) = ae-AJrU(x) FAx) = (1- e-AJr)U(x) c

then
!I(Y) = nAe-lItyU(y)

that is, their minimum y, is also expoqential with parameter nA.

x,.
"'I
)E )( )(
y
)E
1k y+ dy
)(
1..

FIGURE 7·1
 ClfAPTER 7 SEQUBNCf!S OF RANDOM VARIABLES 247

£XA,\IPLE 7-3 ~ A system consists of m components and the time to failure of the ith component is
a random variable Xi with distribution Fi(X). Thus
1 - FiCt) = P{Xi > t}
is the probability that the ith component is good at time t. We denote by n(t) the number
of components that are good at time t. Clearly,
n(/) = nl + ... + nm
where
n·- { I Xi>t E{nd = 1 - Fi(l)
, - 0 Xj<t
Hence the mean E{n(t)} = l7(t) ofn(t) is given by
17(/) = 1- FI(t) + ... + 1 - Fm(t)
We shall assume that the random variables Xi have the same distribution F(t). In this
case,
7](1) = m[1 - F(t}]
Failure rate The difference 11(/) - 11(t + dt) is the expected number offailures
in the interval (I. t + dl). The derivative -11' (I) = ml(/) of -17(1) is the rate offailure.
The ratio
fi(l) = _ 17'(1) =
I(t) (7-15)
1 - F(t)
71(t)
is called the relative expectedlailure rate. As we see from (6-221), the function fi(t)
can also be interpreted as the conditional failure rate of each component in the sys-
tem. Assuming that the system is put into operation at 1 = O. we have n(O) = m; hence
17(0) = E{n(O)} =m. Solving (7-15) for 71(1), we obtain

17(t) = m exp { -lo' fi(1:) dT }

E\:r\~IPLE 7-4 ~ We measure an object of length 17 with n instruments of varying accuracies. The
results of the measurements are n random variables
MEASURE-
MENT E{Vi} = 0
ERRORS where Vi are the measurement errors which we assume independent with :{CEO mean. We
shall determine the unbiased, minimum variance, linear estimation of TJ. This means the
following: We wish to find n constants ai such that the sum
+ ... + all X"
j) = alx\
is a random variable with mean E{i}} = a\E{xd + ... +a"E{x,,} = 17 and its variance
V = atat+ ... +a;a; .
is minimum. Thus our problem is to minimize the above sum subject to the constraint
(7-16)
248 PROBABIUTY AND RANDOM VARIABLES

To solve this problem, we note that

V= alert + ... + a;er; - A(al + ... +.a" - 1)

for any A (Lagrange multiplier). Hence V is minimum if

av
-
2
=2aj er. -A=O
A
ai=-2
aai ' 20';
Inserting into (7-16) and solving for A, we obtain
A 1
- - V - --:-----...,.
2- - 1/ert + ... + 1/er;
Hence
A Xt!erf + ... + x,,/er:
11 = 1/er? + ... + 1/er;
Illustration. The voltage E of a generator is measured three times. We list here the results4
Xi of the measurements, the standard deviations er; of the measurement errors, and the.
estimate t of E obtained from (7-17):
Xi = 98.6, 98.8, 98.9 = 0.20, 0.25, 0.28
eri

t = xt/O.04 + x2/0.0625 + x3/0.0784 = 98.73

1/0.04+ 1/0.0625 + 1/0.0784

Group independence. We say that the group Gx of the random variables XI ••••• x" is!
independent of the group G y of the random variables YI ••••• Yk if

l(xl, ...• X"' Yit ...• Yk} = I(xi • ...• XII)I(YI, ... , Yk} (7-18~

By suitable integration as in (7-5) we conclude from (7-18) that any subgroup of Gx iJ

independent of any subgroup of G y • In particular, the random variables Xi and Yi ar~
independent for any i andj.
Suppose that S is a combined experiment Sl x S2. the random variables Xi depend
only on the outcomes of S" and the random variables Yj depend only on the outcomes
of S2. If the experiments SI and S2 are independent, then the groups Gx and Gy are
independent.
We note finally that if the random variables Zm depend only on th~ random variables
Xi of Gx and the random variables Wr depend only on the random variables Yj of GJ .
then the groups Gt and G w are independent.
Complex random variables The statistics of the random variables

ZI = XI + jy..... , Z" = x" + jy"

are determined in terms of the joint density I(xi • ... , X". YI •••• , YII) of the 2n randolT
variables Xi and Yj. We say that the complex random variables Zi are independent if

I(Xl.··., X". Ylt.·., Yn) = I(x., Yl)'" I(x", YII) (7-19'

 CHAPTER 7 SSQUENCES OF RANDOM VAlUABLES 249

Mean and Covariance

Extending (6~159) to n random variables, we conclude that the mean of g(Xlt ..• , XII)
equals

1 1
-00
00 •••
00

-00
g(x], ...• x,. )f(Xl •... , XII) dXI ••• dXII (7-20)

If the random variables Z, = Xi + hi are complex. then the mean of g(zi •...• zn)
I equals

. From this it follows that (linearity)

E{algl(X) + ... +a".gm(X)} = aIE{gl(X)} + ... +amE(gm(X)}
for any random vector X real or complex.

CORRELATION AND COVARIANCE MATRICES. The covariance C'J of two real ran-
dom variables X; and Xj is defined as in (6-163). For complex random variables
eij = E{(X; - 7U)(xj - J7j)} = E{x;xj} - E{x,}E{xj}
by definition. The variance of Xi is given by
U{ = Cti = ElIXt - 71112} = EUXtI2 } -IE(x,}l2
The random variables Xi are called (mutually) JUlCorreiated if C'j = 0 for every
I # j. In this case, if
X=XI + ... +x" (7-21)

[\ \\IPI r "":-5 ~ The random variables

1 II
v= -L)X; _1)2
n - 1 1=1
are by definition the sample mean and the sample variance. respectively. of Xi. We shall
show that. if the random variables Xi are unco.ttelated with the same mean E {Xi} = "
u1 u
and variance = 2• then
E{X) = J7 (7-22)

and
(7-23)

Proof. The first equation in (7-22) follows from the linearity of expected values and the second
from (7-21):
1 II

E{i} =;; 2: E{x;} = 71

I_I
250 PROBABJUrY ANrl RANDOM VARIABLES

To prove (7-23); we observe that

1
E{(X; -71)(1 - 7])} = -E(x/ - 7])[(XI - '1) + ... +.(x" - 7])]}
n
l' q2
= -E(Xi
n
-71)(Xj -71)} = -
n
because the random variables Xi and Xj are uncor:related by assumption. Hence
_ 2 2 2 q2 0'2 n- 1 2
E{(xi - x) } =E{{(x, - 71} - (I - 71)] } = q + -n - 2-
n
= --q
n
This yields

-
E{v} =-1- L" E{(x, -I)21 = -
n--
n-l
- 0 '2
n-l n-l n
i_I

and (7-23) results.

Note that if the random variables XI are independent identically distributed (Li.d.) with
E{lXi - 7114} = 1J.4. then (see Prob. 7-21)

21(
=-
0'0
n
n-3,,) 1J.4 - - - q
n-l

If the random variables Xl ••••• Xn. are independent, they are also uncorrelated.
This follows as in (6-171) for real random variables. For complex random variables the
=
proof is similar: If the random variables ZI = Xl +jy1and Z2 X2 +j Y2 are independent,
then I(x" X2. Ylt Y2) = I(x" Yl)f(X2. h). Hence

1 1
00 •• •
00
zlzif(Xh Xl. Ylt n) dxl dYI dX2 dn

I: I: I: J:
-00 -00

= zt/(xJ, YI) dXl dYI Z2/(X2, n) dX2 dYl

This yields E{Zlzi} = E(zl }E{zi} therefore, ZI and Z2 are uncorrelated.

Note, finally, that if the random variables XI are independent, then
E{gt(xI)'" gn(xn)} = E{gl(XI)}'" E{gn(xn)} (7·24)
Similarly, if the groups Xl ••••• Xn and YI ••••• Yle are independent, then
E{g{Xlo ...• Xn)h(yl ••••• fie)} = E{g(Xls"" xn)}E{h(YI •• · .• Ylc)}

The correlation matrix. We introduce the matrices

Rn = [~1.1'~~~'~~~1 Cn = [~1~'~:~'~~~1 a
Rnl '" Rnn Cnl ••• C/I/I
where
RIj = E{Xixjl = Rjl Cij = R'j - 7Ji7Jj = Cj j

The first is the correlation matrix of the random vector X = [Xl ••••• Xn] and the
~ its covariance matrix. Clearly.
Rn = E{X'X*}
 CKAPTIlR 7 SEQUENCES OF RANDOM VARIABLES 251

where X' is the transpose of X (column vector). We shall discuss the properties of the
matrix Rn and its determinant ~". The properties of Cn are similar because Cn is the
correlation matrix of the "centered" random variables Xi - 71i.

~ The matrix RII is nonnegarive definite. This means that

Q = I:aiajR;j = ARnAt ~ 0 (7-25)

i.j
where At is the complex conjugate transpose of the vector A = [at. ... , an].

Proof. It follows readily from the linearity of expected values

E{lalxl + ... +QIlx,,1 2} = LQ;ajE{x;xj} (7-26)

I.j

If (7-25) is strictly positive, that is, if Q > 0 for any A#- O. then RIO is called positive definite. J
The difference between Q ~ 0 and Q > 0 is related to the notion of linear dependence. ~

DEFINITION t> The random variables Xi are called linearly independent if

E{lalXI + ... +anxnI2} > 0 (7-27)
for any A #0. In this case [see (7-26)], their correlation matrix Rn is positive
definite. ~

The random variables Xi are called linearly dependent if

(7-28)
for some A # O. In this case, the corresponding Q equals 0 and the matrix Rn is singular
[see also (7-29)].
From the definition it follows iliat, if the random variables Xi are linearly indepen-
dent, then any subset is also linearly independent.

The correlation determinant. The det~nninant ~n is real because Rij = Rji' We shall
show that it is also nonnegative
(7-29)
with equality iff the random variables Xi are linearly dependent. The familiar inequality
~2 = RllR22 - R~2 ::: 0 is a special case [see (6-169)].
Suppose, first, that the random variables Xi are linearly independent. We maintain
that, in this case, the determinant ~n and all its principal minors are positive
(7-30)

IWe shall use the abbreviation p.d. to indicate that R" satisfies (7-25). The distinction between Q ~ 0 and
Q > 0 will be understood from the context.
252 PROBAllIL1TY ANOAANDOMVARlABW

Proof. 'This is true for n = 1 because III = RII > O. Since the random variables of
any subset of the set (XI) are linearly independent. we can assume that (7-30) is true for
k !: n - 1 and we shall show that Il.n > O. For this purpose. we form the system
Rlla. + ... + Rlnan = 1
R2Ja! + ... + R2nan = 0 (7-31)
Rnlal +···+RllI/an =0
Solving for a.. we obtain al = Iln_l/ Il n• where Iln-l is the correlation determinant of
the random variables X2 ••••• Xn' Thus al is a real number. Multiplying the jth equation
by aj and adding. we obtain

Q =~ * = al =
L.Ja,ajRij
.
Iln-l
- -
lln
(7-32)
I .J

In this, Q > 0 because the random variables X, are linearly independent and the left side
of (7-27) equals Q. Furthermore. lln-l > 0 by the induction hypothesis; hence lln > O.
We shall now show that, if the random variables XI are linearly dependent, then
lln = 0 (7-33)

Proof. In this case, there exists a vector A =F 0 such that alXI + ... + allxn = O. Multi-
plying by xi and taking expected values. we obtain
alRIl + ... + an Rill = 0 i = 1• ...• n
This is a homogeneous system satisfied by the nonzero vector A; hence lln = O.
Note. finally, that
llll !: RuRn ... Rlln (7-34)
with equality iff the random variables Xi are (mutually) orthogonal, that is, if the matrix
Rn is diagonal.

7·2 CONDITIONAL DENSITIES,

CHARACTERISTIC FUNCTIONSt
AND NORMALITY
Conditional densities can be defined as in Sec. 6-6. We shall discuss various extensions
of the equation f<Y Ix)=f(x, y)lf(x). Reasoning as in (6-205), we conclude that
the conditional density of the random variables Xn ••••• Xk+1 ass~g Xt. •••• XI is
given by ,
I(Xt ••••• Xk ••••• X,,)
I(xn • .... xk+ll Xj:o ... , Xl) = f( ) (7-35)
XI,··· ,Xk
The corresponding distribution function is obtained by integration:
F(xlI' ••• ,Xk+1 I Xk • .... XI)

= t'- .: '1~!+1 f(a ll ••••• ak+! IXk • ••• , Xl) daHl' .. dan (7-36)
i-co -00
 CIiAPTEIt 7 SEQUENCES OF RANDOM VARlABLBS 253

For example.
!(Xl IX2, X3) -- !(xJ,
f(
X2, X3) _
) -
dF(XI IX2. X3)
d
X2,X3 Xl
Chain rule From (7-35) it follows that
!(x" ... , x,,) = !(XII I XII-I • •..• XI) ... !(xli xl)f(XI) (7-37)

~ We have shown that [see (541)] if x is a random variable with distribution F(x).
then the random variable y = F (x) is uniform in the interval (0, 1). The following is a
generalization.
Given n arbitrary random variables Xi we form the random variables
(7-38)
We shall show that these random variables are independent and each is uniform in the
interval (0. 1).
SOLUTION
The random variables Yi are functions of the random variables Xi obtained with the
transformation (7-38). ForO::: Yi ::: 1. the system
YI = F(Xl) Yl = F(xllxl) •. "'YII = F(x li lXII-I ..... XI)

bas a unique solution XI • •••• Xn and its jacobian equals

8Yl 0 0 o
aXI
8)'l
o

-8Yn ................. -8Yn

ax) 8xII
This determinant is triangular; hence it equals the product of its diagonal elements
aYk
~ = !(Xk Ixk-It ...• Xl)
(]Xk
Inserting into (7-8) and using (7-37), we obtain
!(y )_ !(x" .... XII ) _ 1
I.· ... yn - !(XI)!(X2Ix,) ... !(xll lxn-1o .... Xt)-
$

in the n-dimensional cube 0 ::: Yi ::: 1, and 0 otherwise. ~

I:
From (7-5) and (7-35) it follows that

!(xll x 3) = !(XI,X2I x 3) dx2

!(X,I X4) = [I: !(Xl I X2,X3. X4,)!(X2,X3 IX4) dX2 dX3

Generalizing. we obtain the following rule for removing variables on the left or on
the right of the conditional line: To remove any number of variables on the left of the
254 'PROBA81UTYANDRANDOMVARIABLES

,conditional line, we integrate with respeot to them. To remove any number of variables
, to the right of the line, we multiply by their conditional density with respect to thJ
remaining variables on the right, and we integrate the product. The following specia.
case is used extensively (Chapman-Kolmogoroff equation, see also Chapter 16):

!(x, I X3) = .L: !(x, I X2. X3)!(X21 X3) dX2 (7-39)

Discrete type The above rule holds also for discrete type random variableS
provided that all densities are replaced by probabilities and all integrals by sums. WtA
mention as an example the discrete form of (7-39): If the random variables x" X2, X3I
take the values a;, bk, Cn respectively, then
P(x, = aj I X3 = Ci} = L: PIx, = aj Ibk, Cr }P{X2 = bit. Icr } (7-40
It.

CONDITIONAL EXPECTED VALUES. The conditional mean of the random variabl~

g(x" ... , xn ) assuming M is given by the integral in (7-20) provided that the deI1sit~
!(x" ... , xn) is replaced by the conditional density !(x" ... , Xn I M). Note, in partic"'l
ular, that [see also (6-226)J

(7-41~

This is a function of X2, ... , Xn; it defines, therefore, the random variable E{xil X2,'
: .. , xn}. Multiplying (7-41) by !(X2, .•. , xn) and integrating. we conclude that

E{E{x, IX2, •••• Xn}} = E(xd (7-42)

Reasoning similarly, we obtain

= E{E{x, I X2, X3, X4}}
E{xil X2, X3}

= 1: E{x,lx2,X3.X4}!(X4Ix2.X3)dx4

, This leads to the following generalization: To remove any number of variables on the
(7-43)

right of the conditional expected value line, we multiply by their conditional density
with respect to the remaining variables on the right and we integrate the product. For

1:
example,

E{x,l x3} = E{Xllx2,X31!(x2I x 3)dx2 (7-44)

and for the discrete case [see (7-40)]

E(XI I cr } = L: E{XI I ble. Cr }P{X2 = blr.l cr } (7-45)
It.

EXAMPLE 7-7 .. Given a discrete type random variable n taking the values 1, 2, ... and a sequence
of random variables XIt. independent of n, we form the sum

(7-46)
 CHAPTER 7 SEQUENCES OF RANDOM VARIABLSS 255

This sum is a random variable specified as follows: For a specific ~,n(~) is an integer
and sen equals the sum of the numbers Xk(n for k from 1 to n(~). We maintain that if
the random variables XI: have the same mean, then
E(s} = l1E{n} where E {Xk} = 11 (7-47)
Clearly, E{Xkl n = n} = E{Xk} because Xk is independent ofn. Hence
E(sln =n} = E {tXk In = n} = tE(Xk} = l1 n
k=1 k=J
From this and (6-239) it follows that
E{s} = E{E{s InH = E{17n}
and (7-47) results.
We show next that if the random variables Xk are uncorrelated with the same
variance (12, then
(7-48)
Reasoning thus, we have
n n
E{s21 n = n} = L L E{XiXk} (7-49)
{ .. I k=l

where
{
(12 + 772 i =k
E{XiXk} = 772 i :F k

The double sum in (7-49) contains n terms with i = k and n2 - n terms with i :F k;
hence it equals
«(12 + 112)n + 772 (n1 - n) = 772n2 + (12n
This yields (7-48) because
E{S2} = E{E{s2In}} = E{111n2 + (11n}
Special Case. The number n of particles emitted from a substance in t seconds is
a Poisson random variable with parameter )..t. The energy Xk of the kth particle has
a Maxwell distribution with mean 3kT/2 and variance 3k1 T 1 /2 (see .Prob. 7-5), The
sum sin (7-46) is the total emitted energy in t seconds. As we know E{n} = At,
E(n2} = )..2t2 + At [see (5-64)]. Inserting into (7-47) and (7-48), we obtain

E(s} = 3kT)..t (12 _ 15k2 T 2>..t ~

2 s - 4

Characteristic Functions and Normality

The characteristic function of a random vector is by definition the function
4>(0) = E{e}OX'} = E{e}(0I1XI+"'+OI,oSo>} = ClJUO) (7-50)
256 PROBABllJTY AND RANDOM VARIABLES

where

As an application, we shall show that if the random variables XI are independent

with respective densities Ii (Xi), then the density fz(z) of their sum z = Xl + ' . , + XII
equals the convolution of their densities
(7-51)

Proof. Since the random variables XI are independent and ejw;Xi depends only on X;, we
conclude that from (7-24) that
E {e}«(/)IXI + ' +CllnX.)} = E{ejwIXI} ... E{ejI»,tXn }
Hence
(7-52)
where <Pi (a» is the characteristic function ofxj. Applying the convolution theorem for
Fourier transforms. we obtain (7-51).

EXAl\lPLE 7-8 ~ (a) Bernoulli trials: Using (7 -52) we shall rederive the fundamental equation (3-13),
We define the random variables Xi as follows: Xi = 1 if heads shows at the ith trial and
Xi = 0 otherwise. Thus
PIx; = 1} = P{h} = P P{XI = O} = prE} = q (7-53)
The random variable z = XI + .. , + XII takes the values 0,1.. '" n and {z = k} is the
event {k heads in n tossings }, Furthermore.
II

<Pz(a» = E{e jOlZ } = L P{z =k}e jk,., (7-54)

k=O

The random variables Xi are independent because Xj depends only on the outcomes of
the ith trial and the trials are independent. Hence [see (7-52) and (7-53)]

<Pz(a» = (pe ftD + q)1I = ~ (~) l'ejktDq"-k

Comparing with (7-54), we conclude that

P{z = k} = P{k heads} = (;) ~I:qll-k (7-55)

(b) Poisson theorem: We shall show that if p « I, th~n

-np ( )1:
P{z = k} ::: e np
k!
as in (4-106). In fact, we shall establish a more general result. Suppose that the randorr
variables XI are independent and each takes the value 1 and 0 with respective probabilitiel
Pi and qi = 1 - Pi' If Pi « 1, then
eP/(?-l) ~ 1 + PI (e j (/) - 1) = Ple j ,., +ql = <Pi(a»
 CHAI'TSR 7 SEQUENCES OF RANDOM VARIABLES 257

With z = XI + ... + XII' it follows from (7-52) that

4>~«(I» :::: ePI(~"-I) • • • eP.(~"-I) = eQ(~-I)
where a = PI + ... + Pn. This leads to the conclusion that [see (5-119)] the random
variable z is approximately Poisson distributed with parameter a. It can be shown that
the result is exact in the limit if
Pi -+ 0 and PI + ... + PrJ -+- a as n -+- 00

NORMAL VECTORS. Joint normality of n random variables Xi can be defined as lD

(6-23): Their joint density is an exponential whose exponent is a negative quadratic. We
give next an equivalent definition that expresses the normality of n random variables in
termS of the normality of a single random variable.

DEFINITION ~ The random variables Xi are jointly normal iff the sum

alxl + ... + aI/x" = AX' (7-56)

is a normal random variable for any A.

We shall show that this definition leads to the following conclusions: If the random
variables Xi have zero mean and covariance matrix C, then their joint characteristic
function equals
4>(0) = exp{ -! GeO' } (7-57)

Furthermore, their joint density equals

I(X) = 1 exp{ -!xe- I X'} (7-58)

J(21t:)lIa
where a is the determinant of C.
Proof. From the definition of joint normality it follows that the random variable

W = WIXI + ... + W"x,. = 'lX' (7-59)

is normal. Since E{xj} = 0 by assumption. this yields [see (7-26)]

E(w} =0 E(w2} = L w/O)jCIj = 0';

(.j

Setting '1 =0 and W == 1 in (5-100). we obtain

E{e J"'} == exp [-1]

This yields

E { e jQX1 } = exp { - ~ L Cl>!WjCIj } (7-60)

'.J
as in (7-51). The proof of (7-58) follows from (7-57) and the Fourier inversion theorem.
Note that if the random variables X; are jointly normal and uncorrelated, they are indepen-
dent. Indeed, in this case, their covariance matrix is diagonal and its diagonal elements equal O'l.