168 PlOBABlLITY ANQR'ANDOMVARWILBS

(b) The moments f.I.,. of x are functions of v. Using (i). show that

f.I.n(v) =
n(n -1)
2
1-0 f.I."_2(fJ)d~

5-49 Show that, ifxis an integer-valued random variable with moment function r(z) as in (5,,113).
then

P{x = k} = - 1
2:n' -It
1" r(eJCt»e- lbI dfJ)

S-SO A biased coin is tossed and the first outcome is noted. The tossing is continued until the
outcome is the complement of the first outcome. thus completing the first run. Let x denote
the length of the first run. Find the p.mJ of x, and show that

E{x} = E.q +!p

s..Sl A box contains N identical items of which M < N are defective ones. A sample of size n
is taken. from the box, and let x represent the number of defective items in this sample.
(a) Fmd the distribution function of x if the n samples are drawn with replacement.
(b) If the n samples are drawn without replacement, then show that

P(x = k} = (~)(~~) max(O, n + M - N):5 k!:: min(M. N)

Find the mean and variance of x. The distribution in (b) is known as the hypergeomelric
distribution (see also Problem 3-5). The lottery distribution in (3-39) is an example of this
distribution.
(e) In (b), let N -+ 00. M -+ 00, such that MIN -+ P.O < P < 1. Then show that the
bypergeometric random variable can be approximated by a Binomial random variable with
parameters n and P. provided n « N.
s..Sl A box contains n white and m black marbles. Let x represent the number of draws needed
fur the rth white marble.
(a) If sampling is done with replacement, show that x has a negative binomial distribution
with parameterS r and p = nl(m + n). (b) If sampling is done without replacement, then
show that

Pix =k} = (k - 1) (',,:=~) k =',r+ l •..• ,m+n

r- 1 (m:-)
(e) For a given k and r. show that the probability distribution in (b).tends to a negative
binomial distribution as n + m -+ 00. Thus. for large population size. sampling with or
without replacement is the same.
 CHAPTER

6
TWO
RANDOM
VARIABLES

6-1 BIVARIATE DISTRIBUTIONS

We are given two random variables x and y, defined as in Sec. 4-1, and we wish to
detennine their joint statistics, that is, the probability that the point (x, y) is in a specified
region) Din thexj plane. The distribution functions Fx (x) and F,(y) of the given random
variables determine their separate (marginal) statistics but not their joint statistics. In
particular, the probability of the event
{x === x} n {y === y} = {x === x. y === y}
cannot be expressed in terms of Fx (x) and F,(y). Here, we show that the joint statistics
of the random variables x and y are completely determined jf the probability of this event
is known for every x and y ..

Joint Distribution and Density

The joint (bivariate) distribution Fx,(x, y) or, simply. F (x, y) of two ~dom variables
x and y is the probability of the event -
{x === x, y === y} = {(x, y) e DI}
where x and y are two arbitrary real numbers and DI is the quadrant shown in Fig. 6-1a:
F(x, y) = PIx === x, y === y} (6-1)

I The region D is arbitrary subject only co !be mild condition that it can be expressed as a COUD1ab1e UDion or
intersection of xecrangles.

169
170 PROBABILITY AND IWIDOM VAlUABLES

Yz

(a) (b) (e)

FIGURE6-l

PROPERTIFS
1. The function F (x. ]) is such that
F(-oo,]) =0, F(x. -00) = 0, F(oo,oo) = 1

Proof. As we know, P {x = -oo} = P{1 = -oo} = O. And since

{x = -00,1 $ y} c {x = -oo} {x ~ x, y =-oo} C {y = -oo}
the first two equations follow. The last is a consequence of the identities
{x $ -00,1 $ -oo} =S peS) =1
2. The event {Xl < X $ X2. 1 $ y} consists of all points (x,1) in the vertical half-strip
Dz and the event {x $ X. ]1 < Y$ Yll consists of all points (x,1) in the horizontal
half-strip D3 of Fig. 6-lb. We maintain that
{XI < X ~ X2, Y $ y} = F(X2. y) - F(XI, y) (6-2)
{x $ X, Yl < Y $ Yll = F(x,)'2) - F(x, YI) (6-3)

Proof. Oearly, for X2 > XI

{x $ X2. 1 $ y} = {x $ Xl> 1 $ y} U {XI < x !5 X2, Y $ y}
The last two events are mutually exclusive; hence [see (2-10)]
P{x $ x2.1 $ y} = PIx $ XI.1 $ y} + P{XI < X $ X2.1 $ y}
and (6-2) results. The proof of (6-3) is similar.

3. P{XI < X $ X2.]1 < 1 $ Yl} = F(X2.)'2) - F(x.,)'2)

- F(X2, YI) + F(x" Yl) (6-4)

This is the probability that (x, y) is in the rectangle D4 ofF18. 6-1c.

Proof. It follows from (6-2) and (6-3) because

{Xl < X $ X2, Y$ n} = {Xl < X $ X2, Y $ Yl} U {Xl < X $ X2. YI < 1 $ Yl}
and the last two events are mutually exclusive.
 CHAPTER 6 1WO RANDOM VARIABLes 171

JOINT DENSITY. The jOint density of x and y is by definition the function

I(x,y) -_ 02 F(x, y)
8xay (6-5)
From this and property 1 it follows that

F(x,y) = l~J:oo j(a.fJ)dadfJ (6-6)

JOINT STATISTICS. We shall now show that the probability tbat the point (x, y) is in a
region D of the xy plane equals the integral of I(x, y) in D. In other words,

P{(x,y) e D} = il I(x.y)dxdy (6-7)

where {(x, y) e D} is the event consisting ofall outcomes ~ sucb that the point [x(n, y(~)]
is in D.

Proof. As we know, the ratio

F(x + ax, y + Ay) - F(x, y + Ay) - F(x + Ax. y) + F(x, y)
AxAy
tends to aF(x, y)/axoy as Ax -+ 0 and Ay -+ O. Hence [see (6-4) and (6-5)]
Pix < x::: x + Ax, y < y ::: y + Ay} ~ I(x, y) Ax Ay (6-8)
We have thus shown that the probability that (x, y) is in a differential rectangle equals
I(x, y) times the area Ax Ay of the rectangle. This proves (6-7) because the region D
can be written 88 the limit of the union of such rectangles.

MARGINAL STATISTICS. In the study of several random variables, the statistics of

each are called maIginal. Thus Fx (x) is the marginal distribution and Ix (x) the marginal
density of x. Here, we express the marginal statistics of x and y in tenns of their joint
statistics F(x. y) and I(x, y).
We maintain that

1:
Fx(x) = F(x. (0) F,(y) = F(oo, y) (6-9)

lAx) = I: I(x,y)dy ly(Y) = I(x, y)dx (6-10)

Proof. Clearly, (x ::: oo} = {y ::: oo} = S; hence

{x::: x} = {x ~ x, y ::: oo} {y ::: y} = [x::: 00, y ::: y}
The probabilistics of these two sides yield (6-9).
Differentiating (6-6), we obtain / Z
• Y
aF~X'Y)=j
aF(X,Y)=j'/(X,fJ)dfJ
ax -00 Y-oo I(a.y)da (6-11)

Setting y = 00 in the first and x = 00 in the second equation, we obtain (6-10) because
172 PROBABILITY AND RANDOM VARlABLES

[see (6-9n
Ix(x) = aF(x.oo) f) (x) = BF(oo, y)
ax By

EXISTENCE THEOREM. From properties 1 and 3 it follows that

F(-oo,y) =0 F(x, -00) = 0 F(oo,oo) =1 (6-12)
and
F(X2,)'2) - F(xt. Y2) - F(X2. Yl) + F(XI, Yl) ::: 0 (6-13)
for eyery Xl < X2, Yt < Hence [see (6-6) and (6-8)]

J: I:
)'2.

I(x, y) dx dy = 1 I(x, y) ::: 0 (6-14)

Conversely, given F(x. y) or I(x, y) as before, we can find two random variables
x and y, defined in some space S, with distribution F(x, y) or density I(x, y). This can
be done by extending the existence theorem of Sec. 4-3 to joint statistics.

Probabllity Masses
The probability that the point (x, y) is in a region D of the plane can be interpreted as
the probability mass in this region. Thus the mass in the entire plane equals 1. The mass
in the balf-pl~e x::: x to the left of the line Lx of Fig. 6-2 equals Fx(x). The mass in
the balf-plane y ::: y below the line L, equals Fy(y). The mass in the doubly-shaded
qUadrant {x::: x, y ::: y} equals F(x, y).
Finally, the mass in the clear quadrant (x > x, y > y) equals
P{x> x, y > y} = 1- Fx(x) - Fy(Y) + F(x, y) (6-15)
The probability mass in a region D equals the integral [see (6-7)]

LJ I(x, y)dxdy

If, therefore, I (x, y) is a bounded function, it can be interpreted as surface mass density.

FIGURE 6-2
 CHAPTER 6 1WO RANDOM VARIABLES 173

~ Suppose that

f() 1 _(Xl+yZ){2c1~
x,y = -lC<7
22 e (6-16)

We shall find the mass m in the circle x 2 + y2 ~ a2 • Inserting (6-16) into (6-7) and using
the transformation
x=rcos8 y=rsin8
we obtain

INDEPENDENCE ~ Two random variables x and y are called (statistically) independent if the events
{x E A} and {y E B} are independent [see (2-40)], that is. if
PIx EA. y E B} = P{x E AJP{y E B} (6-18)

where A and B are two arbitrary sets on the x and y axes, respectively.
Applying this to the events {x ~ x} and {y ~ y}. we conclude that, if the random
variables x and y are independent. then
F(x, y) = F:c(x)F)'(y) (6-19)
Hence
f(x. y) = fx(x)fy(y) (6-20)
It can be shown that, if (6-19) or (6-20) is true, then (6-18) is also true; that is. the
random variables x and y are independent [see (6-7)]. ~

EXA\IPLL h-2 ~ A fine needle of length 2a is dropped at random on a board covered with parallel
lines distance 2b apart where b > a as in Fig. 6-3a. We shall show that the probability p
BUFFON'S that the needle intersects one of the lines equals 2a/nb.
NEEDLE In tenns of random variables the experiment just discussed can be phrased as: We
denote by x the distance from the center of the needle to the nearest line and by (J the
angle between the needle and the direction perpendicular to the lines. We assume that
the random variables x and (J are independent. x is uniform in the interval (0, b). and e
is unifonn in the interval (O,n/2). From this it follows that ::
12 Tr
f(x.8) = fx(x)fs(8) = 'b;; 0<8 <-
- - 2
and 0 elsewhere. Hence the probability that the point (x. e) is in a region D included in
the rectangle R of Fig. 6-3b equals the area of D times 2/Trb.
The needle intersects the lines if x < a cos 8. Hence p equals the shaded area of
Fig. 6-3b times 2/Trb: .

P = PIx < acose}

n
210
= -b 0
2/71
acos8d8 =2a
-
nb
174 PROBABILITY AND RANDOM VARIABLES

2b

()

(a) (b)

nGURE6·3

This can be used to determine experimentally the number 7t using the relative frequency
interpretation of p: Ifthe needle is droppedn times and it intersects the lines ni times, then
nl 2a 2an
-=::!p=- hence 7t=::!- ...
n 7tb bnj

> If the random variables x and y are independent, then the random variables
z = g(1') w = h(y}
are also independent.

Proof. We denote by Az the set of points on the x axis such that g(x) ~ z and by Bw the set of
points on the y axis such that h(y) :s w. Clearly,
{z :s z} = {x e Az} {w :s w} = {y e B",} (6·21)
Therefore the events {z :5 z} and {w :5 w} are independent because the events {x e A z} and
{y e Bw} are independent. ~

INDEPENDENT EXPERIMENTS. As in the case of events (Sec. 3-1), the concept of

independence is important in the study of random variables defined on product spaces.
Suppose that the random variable x is defined on a space SI consisting of the outcomes
{~I} and the random variable y is defined on a space S2 conSisting of the outcomes {~}.
In the combined ~periment SIX ~ the random variables x and y are such that
(6-22)
In other words, x depends on the outcomes of SI only, and y depends on the outcomes
of S2 only.

~ If the ex.periments Sl and ~ are independent, then the random variables x and y are
independent.

Proof. We denote by Ax the set {x :s xl in SI and by By theset{y ~ y} in S2' In the spaceSI x Sl'
{x :s x} = A.. X S2 (y :5 y) = SI X B,
 CHAPJ'ER. 6 1WO RANDOM VAlUABLES 175

From the independence of the two experiments, it follows that [see (3-4)] the events Ax x ~ and
S. x By are independent. Hence the events (x :::: xl and {y :::: y} are also independent. ~

JOINT ~ We shall say that the random variables x and y are jointly normal if their joint density
NORMALITY is given by

I(x, y) = A exp {_ 1 (X - 7JI)2 _ 2r (x - 7JI)(y - 112) + (y - 712)2)}

2(1 - r2) O't <71 <72 <71
(6-23)
This function is positive and its integral equals 1 if

A= 1 Irl < 1 (6-24)

21r<71<72~
Thus I (x, y) is an exponential and its exponent is a negative quadratic because Ir I < 1.
The function I (x, y) will be denoted by
(6-25)
As we shall presently see, 711 and 712 are the expected values ofx and y, and O't and <7/
their variances. The significance of r will be given later in Example 6-30 (correlation
coefficient).
We maintain that the marginal densities of x and y are given by

(6-26)

Proof. To prove.this, we must show that if (6-23) is inserted into (6-10), the result is
(6-26). The bracket in (6-23) can be written inth~ form

(...) = (X -711 _ r Y -712)2 + (1- r2) (y - 7}2)2

0'1 0'2 <7:}
Hence

where
(y -172)0'1
17 = 171 + r...;;-.-:..:..:.....-;;.
0'2
The last integral represents a normal random variable with mean IL and variance
(1- r2)0'f. Therefore the last integral is a constant (independent of x and y) B =
J21r(l - r1)O't. Therefore
I,(y) = ABe(Y-'f/2)2f14?
And since I.,(Y) is a density. its area must equal!, This yields AB = 1/<72$, from
which we obtain A = 1/21r<7t0'2Jl - r2 proving (6-24), and the second equation in
(6-26). The proof of the first equation is similar. ~
176 PROBABILtI'Y AND RANDOM VARIABLES

Notes 1. From (6-26) it follows that if two random variables are jointly nonnal. they are also marginally
nonnal. However, as the nex.t example shows, Ihe cOllverse is 1101 trU4.
2. Joint nonna]ity can be defined also as follows: 1Wo random variableS x and y are jointly normal if
the sum ax + by is normal for every a and b [see (7-56»).

EX \i\IPLE 6-3 ~ We shall construct two random variables x and y that are marginally but not jOintly
normal. Toward this, consider the function
I(x, y) = 1.r(x)/y(y)[I + p{2F.r(x) -1}{2Fy(Y) -I}] Ipl < 1 (6-27)
where 1:X<x) and Iy(y) are two p.d.fs with respective distribution functions FAx) and
FyU;).1t is easy to show that I(x, Y) ~ 0 for all x, y, and

[00 r: lex, y)dxdy =1

which shows that (6-27) indeed represents a joint p.d.f. of two random variables x and
y. Moreover, by direct integration

1-00
+00
I(x, y)dy = 1.r(X) + p(2Fx (x) -l)/x(X) 11 '"2
-1
udu
= lAx)
where we have made use of the substitution u = 2Fy(Y) - 1. Similarly,

[:00 I(x. y)dx = ly(Y)

implying that Ix(x) and ly(Y) in (6-27) also represent the respective marginal p.d.f.s of
x and y, respectively.
In particular. let l.r(x) and ly(Y) be normally distributed as in (6-26). In that case
(6-27) represents a joint p.d.f. with normal marginals that is Dot jointly normal. ~

Circular Symmetry
We say that the joint density of two random variables x and y is circularly symmetrical
if it depends only on the distance from the origin. that is, if
I(x. y) = g(r) r = ..jx2 + y2 (6-28)

THEORE!\I
. (d
-.. - .... If the random variables x and y are circularly symmetrical and independent, then
they are normal with zero mean and equal variance.

Proof. From (6-28) and (6-20) it follows that

g(..jx2 + y2) = /:,,(x)/,(y) (6-29)

Since
ag(r) dg(r) ar and aT x
ax- = ""'(i;- ax ax = -;
 CHAPTf!R.6 1WO RANDOM VAlUABLllS 177

we conclude. differentiating (6.29) with respect to x. that

x , ,
;8 (r) = /,,<x)/ylY)
Dividing both sides by x8(r) = x/Ax)/) (y). we obtain
1 g'(r) 1 f:<x)
(6-30)
-; g(r) = :; fA (x)

The right side of (6-30) is independent of y and the left side is a function of r = Vx 2 + y2. This
shows that both sides are independent of x and y. Hence
18'(r)
---
r 8(r)
=a =constant
and (~28) yields
(6-31)
Thus the random variables x and yare normal with zero mean and variance (12 = -l/a. ....

DISCRETE TYPE RANDOM VARIABLES. Suppose the random variables x and y are
of discrete type taking the values of Xi and Yle with respective probabilities
P(x = xd = Pi Ply = YIe} = qle • (6-32)
Their joint statistics are determined in terms of the joint probabilities
P{x = X,. Y = YIe} = Pile (6-33)
Qearly.

1,1e
because, as i and k take all possible values, the events {x = Xi, Y = YIe} are mutually
exclusive, and their union equals the certain event.
We maintain that the marginal probabilities Pi and q" can be expressed in terms
of the joint probabilities Pile:
Pi = LPile (6-34)
Ie
This is the discrete version of (6-10).

Proof. The events {y = YIe} form a partition of S. Hence as k ranges over all possible
values. the events {x =XI, Y= YIe} are mutually exclusive and their union equals {x = Xi} •
. This yields the first equation in (6.:34) [see (2-41)]. The proof of the second is similar.

POINT MASSES. If the random variables x and y are of discrete type taking the values
Xiand Yt. then the probability masses are 0 evexywbere except at the point (Xi, YIe). We
have, thus. only point masses and the mass at each point equals Pile [see (6-33)J. The
probability PI = P {x = Xj} equals the sum of all masses Pile on the line X = Xi in
agreement with (6-34). .
178 PR.OBABlUTY AND RANDOM VARIABLES

If i == 1•...• M and k == 1•...• N. then the number of possible point masses on the
plane equals M N. However. as Example 6-4 shows. some of these masses might be O.

EX \i\IPLE 6-4 ~ (a) In the fair-die experiment;'" equals the number of dots shown and y equals twice
this number:
xc/;) == i y(/;) = 2i i = 1....• 6
In other words, Xl = i, Yk = 2k, and
Pik = P(x = i, Y == 2k} == {!6 i =k

o i ~k
Thus there are masses only on the six points (i, 2i) and the mass of each point equals
1/6 (Fig. 6-4a).
(b) We toss the die twice obtaining the 36 outcomes /; ik and we define x and y
such that x equals the first number that shows, and y the second
i,k=I ..... 6.
Thus Xi == i, Yk == k. and Pik == 1/36. We have, therefore, 36 point masses (Fig. 6-4b)
and the mass of each point equals 1/36. On the line X == i there are six points with total
mass 1/6.
(c) Again the die is tossed twice but now

Y(/;ik) =i +k
In this case, x takes the values 0, 1•...• 5 and y the values 2, 3, ... , 12. The number
of possible points 6 x 11 == 66; however. only 21 have positive masses (Fig. 6-4c).
Specifically. if x = O. then y =2, or 4•...• or 12 because if x =0, then i =k and
y = 2i. There are. therefore, six mass points in this line and the mass of each point
equals 1/36. If x = 1, then y = 3, or 5•... , or 11. Thus, there are, five mass points on
the line X = 1 and the mass of each point equals 2/36. For example, if x = 1 and y = 7,
then i = 3, k = 4, or i = 4, k = 3; hence Pix = I, y = 7} == 2/36. ~

y y y <> 1/36
12 • 6 • • • • • • 12
• •
• 2136
10 • 5 • • • • • • 10 ..
• • • •
8 • 4 • • • • • • 8
• • • • •
6 • 3 • • • • • • 6
• • •
4 • 2 • • • • • • 4
•
2 • • • • • • • 2

1 2 3 4 5 6 x 2 3 4 5 6 x 1 2 3 4 5 x
(a) (b) (c)

FIGURE 6-4
 CHAPTER 6 1WO RANDOM VARlA8LES 179

y
x= cosz y=sinz
g(x)
y

o Xi x

(a) (b) (c)

FIGUREfi-5

LINE MASSES. These cases lead to line masses:

1. If x is of discrete type taking the values Xi and y is of continuous type, then all
probability masses are on the vertical lines X = Xj (Fig. 6-Sa). In particular, the
mass between the point Yl and )/2 on the line x = Xi equals the probability of the
event
= Xi. YI ~ Y ~)/2)
[x
2. Ify = g(x), then all the masses are on the curve Y = g(x). In this case, F(x, y)
can be expressed in tenns of F% (x). For example, with x and y as in Pig. 6-Sb.
F(x, y) equals the masses on the curve y = g(x) to the left of the point A and
between B and C equal FAx) - Fx (X2)' Hence
F(x. y) = Fx(xt) + Fx(X3) - Fx(xv y =g(Xl) = g(X2) = g(X3)
3. If x = g(z) and y = h(z). then all probability masses are on the curve x = get),
Y = h(t) specified parametrically. For example, if g(z) = COSt, h(z) = sinz. then
the curve is a circle (Fig. 6-5c). In this case, the joint statistics of x and y can be
expressed in terms of Fz(z).

If the random variables x and y are of discrete type as in (6-33) and independent,
then
Pill = PiPII (6-35)
This follows if we apply (6-19) to the events {x = Xi} and {y = YIIl. This is the discrete
version of (6-20).

EX Ai\ IPL Ie (1-5 .. A die with P{fj} = Pi is tossed twice and the random variables x and y are such
that .

Thus x equals the first number that shows and y equals'the second; hence the random
variables x and y are independent This leads to the conclusion that
Pi/c = Pix = i, y = k} = PIPJ: ~
ISO PROBABILITY AND RANDOM VARIABLIiS

6~2 ONE FUNCTION OF TWO

RANDOM VARIABLES
Given two random variables x and y and a function g(x, y). we form a new random
variable z as
z=g(x,Y) (6-36)
Given the joint p.d.f. Ixy(x, y), how does one obtain 11.(1.), the p.d.f. of z? Problems of
this type are of interest from a practical standpoint. For example, a received signal in
a communication scene usually consists of the desired signal buried in noise, and this
formulation in that case reduces to z = X + y. It is important to know the statistics of the
incoming sigilaJ. for proper receiver design. In this context, we shall analyze problems
of·the type shown in Fig. 6-6. Referring to (6-36), to start with,
Ft(z) = P{z(~) ~ z} = P{g(x, y) ~ z} = P(x, y) e D z }
= fr (
}X,YEDz
Ixy(x. y) dx dy (6·37)

where Dz in thexy plane represents the region where the inequality g(x, y) ~ z is satisfied
(Fig. 6-7).

x+y
max(x,y) x-y

min(x. yJ xly

max(x,y) FIGUREU

Dz
g(x. Y)"'z
• x

FIGURE 6·7
 CHAP1'BR6 lWORANDOMVARlAllLES 181

Note that Dz need not be simply connected. From (6-37), to determine Fz(z) it is
enough to find the region Dr. for every z, and then evaluate the integral there.
We shall illustrate this method to detennine the statistics of various functions of x
andy.

EXA\IPLE 6-6 ~ Let z = x + y. Determine the p.d.f. fz(z),

From (6-37),

F1.(z) = P{x+y::: z} = [ 1x=-oo

),=-00
Z->
fxy(x,y)dxdy (6-38)

since the region Dr. of the xy plane where x + y ::: z is the shaded area in Fig. 6-8 to
the left of the line x + y ::: z. Integrating over the horizontal strip along the x axis first
(inner integral) followed by sliding that strip along the y axis from -00 to +00 (outer
integral) we cover the entire shaded area.
We can find 11.(z) by differentiating Fz(z) directly. In this context it is useful to
recall the differentiation rule due to Leibnitz. Suppose

Fz(z) = l b(Z)

o(t.)
J(x. z) dx (6-39)

Then
f1.(z) = dFz{z) = db(z) /(b(z). z) _ da(z) /(a(z). z) + r(t.) alex. z) dx (6-40)
dz dz dz Ja(t.) az
Using (6-40) in (6-38) we get

/1.(z) = I: (:z l:y fxy(x. y) dX) dy

= 1 00 (
-00 1· /Xy(z - y. y) - 0 + Loo
r-)' atx)'az'
(x y»)
dy

= I: fxy(z - y. y) dy

Alternatively. the integration in (6-38) can be carried out first along the y axis
(6-41)

followed by the x axis as in Fig. 6-9 as well (see problem set).

FIGURE 6·8
182 P1tOBABIUTY AND lIANDOM VARIABLeS

yO

FIGURE 6-,

If x and y are independent, then

I~y(x, y) = Iz(x)/y(y) (6-42)
and inserting (6-42) into (6-41) we get

11.(1.) = l~-oo Ix(z - y)/,(y)dy = [-00 Ix (x)/,(1. -x)dx (6-43)

This integral is the convolution of the functions Ix(1.) and 1,(1.) expressed two different,
ways. We thus reach the following conclusion: If two random variables are independent,
then the density of their sum equals the convolution of their densities.
As a special case, suppose that Ix (x) =0 for x < 0 and I,(Y) =0 for y =< O. then
we can make use of Fig. 6-10 to determine the new limits for DI..
In that case

or
Iz(Z) = r (a81. 11.-7
1,..0 )
%=00 Ix,(x, y) dx dy

'= {1t. /x,(1. - y, y)dy 1. > 0

(6-44)
o z:::o
y

ftGURE6-10
 CHAPTl!R6 TWORANDOMVARIABLI!S 183

On the other hand, by considering vertical strips first in Fig. 6-10, we get

Fz(z) = 1z l~-x
x=O y-o
Ixy(x, y)dydx

or

f1.(z) = 11.x=o Ixy(x, z - x) dx

= {[.' /,(x)/,(z -x)dx
(6-45)

if x a.nd y are independent random variables. ~

EX \i\lPLE 6-7 .. Suppose x and y are independent exponential random variables with conunon pa-
rameter A. Then
(6-46)
and we can make use of (6-45) to obtain the p.d.f. of z = x + y.

11.(z) = lot A2e-1."'e-1.(1.-xl dx = A2e->.r.l1. dx

= ZA2e-l.1. U (1.) (6-47)

As Example 6-8 shows, care should be taken while using the convolution formula for
random variables with finite range. ~

EX \i\lPLE 6-R .. x and yare independent uniform random variables in the common interval (0,1).
Determine 11.(z), where z = x + y. Clearly,
z=x+y=*O<z<2
and as Fig. 6-11 shows there ~e two cases for which the shaded areas are quite different
in shape, and they should be considered separately.

y y

z- 1

x x

0 ... %<1 1«0%<2

FIGURE 6-11
 For O-:s r. < I,
Fz(z) = 1~ l::-Y ldxdy = l Z
1.2
(z - y)dy = -2' o < r. < 1 (6-48}
y=O ~-O y-o

For 1 :s z < 2, notice that it is easy to deal with the unshaded region. In that case,

Fz (z)=I-P{z>z}=l- t
1,=1.-1 x"'<%-y
11 Idxdy

=1- 1 1

y=%-1
(1 - z + y) dy = 1 - -~
(2-Z)2
2
1 :s r. < 2 (6-49)

Thus

I~(z) = dFz(r.) = {z 0 :s z < 1 (6-50)

dz 2- z 1:s z < 2 I

By djrect convolution of Ix (x) and ly(Y), we obtain the same result as above. In fact,
for 0 .=: z < 1 (Fig. 6-12a)

/z(z) = J IAr. - x)/,(x) dx = 1~ 1dx = 1. (6-51)

1~1. - x) 1:Ar.-x)/p)

x 1.

(0) 0 "'z< I

Ip) 1:Ar.- x) 1:Jr. - x)/,<x)

(b) 1-0::<2

o . 2
(c)
 CI:W'TBIl6 1WORANDOMVAltL\BLSS 185

and for 1 ~ 1. < 2 (Fig. 6-12b)

Iz(z) = 11z-1
Idx = 2-z (6-52)

Fig. 6-12c shows Iz(1.), which agrees with the convolution of two rectangular waveforms
as well. ~

EX·\l\ IPLC 6-9 ~ Letz = x -yo Determine It.(z).

From (6-37) and Fig. 6-13
z=x-y
Fz(1.) = P{x - y ~ 1.} =
(00
Jy=-oo
1'~=-oo
+)' f~y(x, y) dx dy
and hence
dFz(z)
Iz(1.) = - d -
1.
= 1 00

-00
IJty(1. + y, y) dy (6-53)

If x and y are independent, then this formula reduces to

ft.(z) = L f;e(z + Y)/y(y) dy = !:A-'l.) ® f,Cy) (6-54)

which represents the convolution of f;e(-z) with 1,(1.).

As a special case, suppose
fx(x) =0 x < 0,
In this case, Z can be negative as well as positive, and that gives rise to two situations
that should be analyzed separately, since the regions of integration for 1. :! 0 and z < 0
are quite different.
For z ~ 0, from Fig. 6-14a

FIG~6-13
186 PROBABJLlTY AND RANDOM VARIABLES

(a) (b)

< 0, from Fig. 6-14b

{1r.
After differentiation, this gives

/x,(1. + y, y) dy 1.:::' 0
/~(1.) = 00
(6-55)
d)' 1. < 0
~l:

z = x/y. Detennine
have
z=x/y
(6-56)
The inequality xI y ~ 1. can be rewritten as x ~ y1. if Y > 0, and x ~ yz if Y < O. Hence
the event {x/y ~ z} in (6-56) needs to be conditioned by the event A = (y > O} and its
compliment A. Since A U A = S. by the partition theorem, we have
P{x/y ~ z} == P{x/y ~ Z n (A U A)}
== P{x/Y~z.y <OJ
= P{x ~ yz. Y < O}
6-15a shows the area first tenn, and Fig. 6·15b
corresponding to the second teon
Integrating over these two regions, we get

Fl.(z)= f oo
Jy=fJ
l YZ
x=-oo
fxy(x,Y)dXdY +!. [,
,..-00 x-y~
/xy(x,y)dxdy (6-58)
 CHAPTER 6 1WO RANDOM VARIABLES 187

y y

(a) P(x :s yz, y > 0) (b) P(x =::: yz. y < 0)

FIGURE 6-15

x FIGURE 6-16

Differentiation gives

h(?) = roo yfx.y(Yz. y)dy +

10
r -yfx.y(Yz. y)dy
Loo
= I: lylfx,(Yz. y) dy

Note that if x and y are non-negative random variables, then the area of integration
(6-59)

reduces to that shown in Fig. 6-16.

This gives

F~(z) = i lY~ fxy{x,y)dxdy

1y=o x=o
or

fz(z) = i y/xy(Yz, y) dy
1,-0 (6-60)

<II(
EX \'\IPIX (,-11 ~ x and y are jointly normal random variables with zero mean and

fxy(x, y) =
1 e
- [;;6; (S--k.!t+5)]
3(1-r) -I -1"2 -2 (6-61)
- 21r 0'\ 0'2../f'="'T2
Show that the ratio z = x/y has a Cauchy density centered at TO'I/0'2.
188 PROBABILITY ~ND lWfI)OM VARIA8I.BS

SOLUTtON'
Inserting (6-61) into (6-59) and using the fact that hy(-x, -y) = fxy(x, y), we obtain

f~('l..) = 2
2:1l'O't0'2.Jl - 'r2
10
00
ye-Y'2/2ao2 dy = ---==== ~
:Il'O't0'2.Jl - r2
where

Thus

(6-62)

which represents a Cauchy random variable centered at rO'110'2. Integrating (6-62) from
-00 to z. we obtain the corresponding distribution function to be

1 1 a2Z - ral
Ie' ( )
rz Z + - arctan --=--===
= -2:1l' a ..Ji'"=r2
(6-63)
t

As an application, we can use (6-63) to determine the probability masses mit m2,
m3. and m4 in the four quadrants of the xy plane for (6-61). From the spherical symmetry
of (6-61), we have

But the second and fourth quadrants represent the region of the plane where x Iy < O.
The probability that the point (x, y) is in that region equals. therefore. the probability
that the random variable z = x/y is negative. Thus
1 1 r
m2 +m4 = P(z ~ 0) = Fz(O) = -2 - -arctan ~
:Il' -vl-r~
, and

If we define a = arc tan r I~, this gives

1 a 1 a
m 1 = m3 = 4 + 2:1l' m2 = m4 = 4 - 2:1l' (6-64)
..
Of course, we could have obtained this result by direct integration of (6-61) in each
quadrant. However. this is simpler.

~ Let x and y be independent gamma random variables with x'" G(m, a) and y ....
G(n, a). Show that z = xj(x + y) has a beta distribution.

Proof. /Xy(x. y} = /x(x)f'1(Y)

= 1 ,x"'-I ,,-I -(}t+,l/a (6-65)

a"'+np(m)r(n) y e
 QL\PJ'SR. 6 1WO RANDOM VARIABLES 189

Note that 0 < z < I, since x and y are non-negative random variables

Fz(t} = P(z ~ z) =p (_X_

x+y
~ z) = p (x ~ y_z_)
I-t

= 1"0
[1Z/(I-zl
0 f",(x,y)dxdy

where we have made use of Fig. 6-16. Differentiation with respect to t gives

k(z) =[ (1 ~ Z)2 fJly(Jz/(l - z), y) dy

= [_y_
o (1 - z)2 czH'+nr(m)r(n)
1 (-E-)_-I y"-I.-'1/(I-zP dy
)- z

_ 1 ,t"-I roo ...... -I.-,Icr(I~) d

- am.... r(m)r(II) (1 - t)_+1 Jo Y y

zm-I(l-t},,-I [ 81+,,-1 -d
r(m+n) ___ I(l )"-1
= r(m)rCn) 0 " e"= r(m)r(n) 4-Z

_J-Z--1(l- Z),,-1 0 < z; < 1

= { ~(m,n) (6-66)
o otherwise
which represents a beta distribution. ~

I \ \i\IPLI h-13 ... Letz =r + y2. Determine /,('1.).

We have

F,('l.) = P{x2 + y2 ~ 'l.} = rr

J J,,2+r~~
/JlY(x, y) dx dy

But. r +y1 ~ 'l. represents the area of a circle with radius ..[i. and hence (see Fig. 6-17)

F~('l.) = 1 .Ji. 1.Jz::;i /xy(x,y)dxdy

>--.Ji. z=-.J,-r
y
.jZ'

•
-fi
FlGURE6-l7
190 l'IlOBABILITY AND lWIDOM VARJABLSS

As an illustration. consider Example 6-14.

EX,\.i\IPLL 10-14 ... x and y are independent nonnal random variables with zero mean and common
variance u 2• Determine h(Z) for z = xl + r.
SOLUTION
Using (6-67), we get

/z(Z) = 1 .fi I
--Ii 2";1. - y2
( 2· --2e(Z-.I'+,)(JH
1
21ru
..2 2 2) dy
e-Z/2t12
=--
:11'0'2
1
0
-1i 1
";1. - )'2
e-z/2t12
dy=--
:ll'U 2
171/l ';;'cos6 d6
0 ';;'oos6
= ~2e-Z(JH2u(Z) (6-68)

where we have used the substitution y = .;;. sin 6. From (6-68), we have the following:
Ifx and y are independent zero mean Gaussian random variables with common variance
u 2, then x2 + y2 is an exponential random variable with parameter 2q2. ~

EX \ \11'1 J h-15 ... Let z = ";X2 + y2. Find /7.(z).

SOLUTION
From Fig. 6-17, the present case corresponds to a circle with radius Z2. Thus

Fz(z) = l' 1..;;9

pc-t z_-Jz2_y2
/z,(x, y) dx dy

(6-69)

In particular, jf x and y are zero mean independent Gaussian random variables as in the
previous example, then c

h(1.) =2 r 1.
Jo ";Z2 -
2 e-<z2-):l+y2)/2u2dy
y2 21ru2

= .l:!:...e-z2/2t12
:ll'U2
r
Jo
1
";Z2 - y2
dy = ~e-z2/2t11
:ll'U 2 J
o
r 12 lCOSe de
zcosS
(6-70)

which represents a Rayleigh distribution. Thus, if w x + iy, where x and y are =

real independent normal random variables with zero mean and equal variance, then
 CHAPTER 6 '!WO RANDOM VARIABLES 191

the random variable Iwl = ,/x2 + y2 has a Rayleigh density. w is said to be a complex
Gaussian random variable with zero mean, if its real and imaginary parts are independent.
So far we have seen that the magnitude of a complex Gaussian random variable has
Rayleigh distribution. What about its phase
() = tan-I (~) (6.71)

Clearly, the principal value of 6 lies in the interval (-](/2, 7C /2). If we let u = tan (J =
y/x. then from Example 6·11, u has a Cauchy distribution (see (6-62) with 0'1 = 0'2,
r =0)

-oo<u<oo

As a result, the principal value of (J has the density function

1 1 1/7C
f9(9) = IdO/dul ",(tan 9) = (l/sec29) tan20 + 1

= {1/7C -7C/2 < 0 < 7C/2

(6-72)
o otherwise
However, in the representation x+ jy = re j8 , the variable 6 lies in theintervaI (-7C, 7C).
and taking into account this scaling by a factor of two, we obtain
1/27C -]( < 0 < 7C
fe(O) = {o .
otherwlse
(6-73)

To summarize. the magnitude and phase of a zero mean complex Gaussian random
variable have Rayleigh and uniform distributions respectively. Interestingly, as we will
show later (Example 6-22), these two derived random variables are also statistically
independent of each other! ~

Let us reconsider Example 6·15 where x and y are independent Gaussian random
variables with nonzero means /1-x and /1-y respectively. Then z = ";X2 + y2 is said to be
a Rician random variable. Such a scene arises in fading multipath situations where there
is a dominant constant component (mean) in addition to a zero mean Gaussian random
variable. The constant component may be the line of sight signal and the zero mean
Gaussian random variable part could be due to random multipath components adding up
incoherently. The envelope of such a signal is said to be Rician instead flf Rayleigh.

EXAi\IPLE 6-1(; ~ Redo Example 6·15, where x and y are independent Gaussian random variables with
nonzero means /1-x and /1-y respectively.
SOLUTION
Since
}; (x y)
xy.
= _1_e-l(x-J.'~)2+(y-J.'»2J/2CT2
27C0'2

J
substituting this into (6-69) and letting y = z sin 0, /1- = /1-; + /1-~, /1-x = /1- cos tP.
192 PROBA.81UTY AND RANDOM VARIABLES

(6-74)

where
10(1]) ~ _1_121r e"coa(8-~) dB = .!.11r e"cos8 dO
2rr 0 1l' 0

is the modified Bessel function of the first kind and zeroth order. ~

Order Statistics
In general. given any n-tuple Xl. X2 •...• XII' we can rearrange them in an increasing
order of magnitude such that
X(I) ::: x(2) ::: ••• ::: X(II)

wherex(l) = min (x\, X2 •••• , x ).andX(2)is the second smallest value amongxlt X2 •••••
ll

x'u and finally x(n) = max(xJ. X2 •••• , XII)' The functions min and max are nonlinear op-
erators. and represent special cases of the more general order statistics. If Xl. X2, •••• XII
represent randolp .variables, the function XCk) that takes on the value X(k) in each pos-
sible sequence (Xl, X2 • •••• XII) is known as the kth-order statistic. {x(1). X(2) •.•• ,X(II)}
represent the set of order statistics among n random variables. In this context
R = K(n) - x(1) (6-75)
represents the range, and when n = 2, we have the max and min statistics.
Order statistics'is useful when relative magnitude of observations is of importance.
When worst case scenarios have to be accounted for. then the function max(·) is quite
useful. For example, let XJ, X2, ••• ,:x,. represent the recorded flood levels over the past
n years at some location. If the objective is to construct a dam to prevent any more
flooding, then ~he height H of the proposed dam should satisfy the inequality
(6-76)
with some finite probability. In that case, the p.d.f. of the random variable on the right
side of (6-76) can be used to compute the desired height. In another case, if a bulb
manufacturer wants to determine the average time to failure (f.L) of its bulbs based on a
sampleofsizen, the sample mean (Xl +X2 + ... +Xn)/n can be used as an estimate for
/1-. On the other hand, an estimate based on the least time to failure has other attractive
features. This estimate min(x}, X2, •••• Xn) may not be as good as the sample mean in
terms of their respective variances, but the min(·) can be computed as soon as the first
bulb fuses, whereas to compute the sample mean one needs to wait till the last of the lot
extinguishes.
 CHAPTER 6 twO RANDOM VARIABLES 193

~ Let z = max(x, y) and w = min (x. y). Determine fz(z) and fw{w).
z::::: max(x.y) x> y
w::::: min(x, y) z = max(x. y) = { yX x Sy
(6-77)

we have (see (6-57)]

Fz(z) = P{max(x. y) S z}
= P {(x S Z, x > y) U (y S z. x S y)}
= PIx s z. x> y} + Ply ~ z, x S y}
since {~ > y} and {x S y} are mutually exclusive sets that form a partition. Figure 6-18a
and 6-1Sb show the regions satisfying the corresponding inequalities in each term seen
here.
Figure 6-1Sc represents the total region. and from there
Fz{z) = P{x ~ z, y s z} = Fxy{z. z) (6-78)

If x and y are independent, then

y y

=z

(a) Pix ~ <';, x > y)

(z, z)

(c)

FIGURE ,.18
194 l'ROBABJUTY AND RANDOM VARIABLES

y
x= w

, (a) Ply s: w, x >yj" (b)P[x $w, X s: y)

(c)

and hence
(6-79)
Similarly,

w = min(x, y) = {yx x> y

x:sy
(6-80)

Thus.
FIO(w) = P{min(x, y) :S w}
= Ply :S w, x > y} + PIx :S w, x :S y}
Once again, the shaded areas in Fig. 6-19a and 6·19b show the regions satisfying these
ineqUatities, and Fig. 6-19c shows them together.
From Fig. 6-19c,
Fl/I(w) = 1 - pew > w} = 1 - P{x > w, y > w}
=Fx(w) + F,(w) - Fx,(w, w) (6-81)

where we have made use of (6-4) with X2 = Y2 = 00, and XI = )'1 = W. ~

 CHAP'J'1'!R 6 1WO R.ANDOM VAlUABU!S 195

EX \\IPLL h-IS • Let x and y be independent exponential random variables with common paranleter
A. Define w = min(x, y). Fmd fw(w).
SOLUTION
From (6-81)
Fw(w) = Fx(w) + Fy(w) - Fx(w)Fy(w)
and hence
.
fw(w) = fx(w) + fy(w) - f.r(w)Fy(w) - Fx(w)fy(w)
But fx(w) = !,(w) = Ae-}.w. and F.r(w) = Fy(w) = 1 - e-}.w. so that
!w(w) = lAe}.w - 2(1 - e-}.wp..e-l. = lAe-21.wU(w)
W (6-82)

Thus min(x. y) is also exponential with parameter lA. ~

FX \ \11'1 L 11-19 • Suppose x and y are as given in Example 6-18. Define

min(x,y)
2:= -.....;......=..
max(x,y)
Altbough min(·)/max(·) represents a complicated function. by partitioning the whole
space as before, it is possible to simplify this function. In fact

z={xY/x/Y xx>~ YY (6-83)

As before, this gives

Fz(z) = P{x/y ~ z, x ~ y} + P{y/x ~ z. x> y}

= P{x ~ Y2:. x ~ y} + Ply ~ xz, x> y}
Since x and y are both positive random variables in this case. we have 0 < Z < 1. The
shaded regions in Fig. 6-2Oa and 6-20b represent the two terms in this sum.

y y

x x

(a) (b)

FIGUU6·20
196 PlWBAB1UTY AND RANOOM VARIABLES

from Fig. 6-20,

F~(z) = lo°O1)~ fx,,(x.y)dxdy+
l°Olxl fxy(x.y)dydx
x...o 0 y...()
Hence
ft(z) = 1o , 00
yfxy(YZ, y) dy + roo xfxy(x, Xl.) dx
Jo
=1 00
y(Jxy(YZ, y) + /xy(Y, yz») dy

=1 00
yA 2 (e-}..(YZ+Y) + e-}..(Y+)t») dy

= 2A2 roo ye-}"(I+z)y dy = 2 fue- u du

Jo (1 +%)2 0

2 0<7.<1
= { 0(1 +7.)2 (6-84)
otherwise

~ Let x and y be independent Poisson random variables with parameters Al and A2.
respectively. Let z = x + y. Determine the p.mJ. of z.
DISCRETE Since:x and y both take values (0,1,2, " .), the same is true for z. For any n =
CASE 0, 1, 2, ... , {x +Y= n} gives only a finite number of options for:x and y. In fact, if :x = 0,
then y must be n; ifx = I, then y must be n - I, and so on. Thus the event (x + y = n}
is the union of mutually exclusive events At = {x = k, Y= n - k}, k = 0 -+ n.

PI' =n} = P{x+y = n} = P W{x= k.y = n- k})

n
= L P{x = k, Y = n - k} (6-85)

If x and y are also independent, then

P{x = k, Y = n - k} = P{x = k)P{y = n - k}
and hence

P{z=n} =
"
LP{x = k}P{y = n -k}
k=O
n ')..k lll-k e-(1.1+l2) II n'
_ " -}..I -1 -}..2 "'2 - " . }.h..;-k
-~e k!e (n-k)'- n! ~kl(n-k)! I
_
- e
-(}..I+}..d (AI + A2)" ,
n = 0, 1, 2~ ...• 00 (6-86)
n!
Thus z represents a Poisson random variable with parameter AI + A2, indicating that sum
of.independent Poisson random variables is a Poisson random variable whose parameter
is the sum of the parameters of the original random variables.
 CHAI'I'ER 6 1WO RANDOM VARIABLES 197

As Example 6-20 indicates. this procedure is too tedious in the discrete case. As we
shall see in Sec. 6-5, the joint characteristic function or the moment generating function
can be used to solve problems of this type in a much easier manner. ~

6·3 TWO FUNCTIONS OF TWO

RANDOM VARIABLES
In the spirit of the previous section, let us look at an immediate generalization. Suppose
x and yare two random variables with joint p.d.f. ixy(x, y). Given two functions g(x, y)
and h (x, y). define two new random variables
z = g(x. y) (6-87)
w = h(x,y) (6-88)
How does one determine their joint p.d.!. izw(z, w)? Obviously with itw(z. w) in hand,
the marginal p.d.f.s iz(z) and iw(w) can be easily detennined.
The procedure for determining izw(z. w) is the same as that in (6-36). In fact for
given numbers z and w,
Fzw(z. w) = P{z(~) ~ z. w(~) ~ w} = P{g(x, y) ~ z, hex. y) ~ w}

= P{(x, Y) E D t . w } = J!
(x.)' IE D....
/xy(x, y) dx dy (6-89)

where D Z .1I) is the region in the xy plane such that the inequalities g(x, y) ~ z and
hex. y) ~ ware simultaneously satisfied in Fig. 6-21.
We illustrate this technique in Example 6-21.

I EX,\l\IPLE h-21 ~ Suppose x and y are independent uniformly distributed random variables in the
=
interval (0,6). Define z = min(x, y), W max.(x, y). Determine iZII)(z, w).
SOLUTION
Obviously both z and w vary in the interval (0,6). Thus
Fzw(z. w) = 0 jf z < 0 or w < 0 (6-90)
F~w(z, w) = P{z ~ z. w ~ w} = P{mm(x,y) ~ z, max (x, y) ~ w} (6-91)

FIGURE 6·21