48 CONDITIONING BY AN EVENT 177 }.8 Conditioning by an Event Anexperiment produces two random variables, X and Y, We lear that the outcome (x,y) is anelement ofan event, B. We use the information (x, y) € B to construct a new probability ‘model. If X and ¥ are discrete, the new model is a conditional joint PME, the ratio of the joint PMF to P(B). If X and ¥ are continuous, the new model is a conditional joint PDE lefined as the ratio ofthe joint PDF to P{B]. The definitions of these functions follow from the same intuition as Definition 1.6 for the conditional probability of an event. Section 4.9 ‘considers the special case of an es ent that corresponds to an observation of one of the two random variables: either B = {X = x), or B = {Y = efinition 4.9 Conditional Joint PMF For discrete random variables X and ¥ and an event, B with P[B] > 0, the conditional Joint PMF of X and ¥ given B is Pxvie (ty) = PIX =x, ¥ = y|B] The following theorem is an immediate consequence of the definition, teorem 4.19 For any event B, a regi of the X. ¥ plane with P{B] ~ 0, Pry oy) Prva (ey) ={ ~ PLB] ° otherwise eB. Example 4.13 Random variables X and Y have the joint PMF Py. yx, ») si at ois as shown. Let # denote the event X-+Y <4. Find the 1 Conditional PMF of x and ¥ given B. Event B = ((1,1),(2, 1), 2,2), 3.1] consists ofall points 4+ Pyyip(e.y) (ty) Such that x + = 4. By adding up the probabilities of all outcomes in #, we find . PIB) = Pry (1.1) 2,1) Pry ‘The conditional PMF Py. yy (x, ») is shown on the let. 191 >rob- iperty 1, and ibrero, ) there ne 4.145)178 CHAPTER4 PAIRS OF RANDOM VARIABLES ase of two continuous random variables, we have the following definition of the 1 probability mo Inthe condition Definition 4.10 Conditional Joint PDF sai Given am event B with P(B] > 0, the conditional joint proba Yis lity density function of X wy» HY) (ye B. frye y= PIB 0 otherwise. Example 4.14 x and ¥ are random variables with joint PDF rted={ 9 cihares. 5 pu= ff ke as { f | ish Corresponding to Theorem 4.12, we have Theorem 4.20 Conditional Expected Value For random variables X and ¥ and an event B of non expected value of W = g(X. Y) given Bis probability, the conditional piscre: E1wia)= YD ets 90Pana 9 Continuous: E(W1B) ‘Another notation for conditional expe a] Theorem Exa Ex48 CONDITIONING BY AN EVENT 179 411 Conditional variance The conditional variance of the random variable W = g(X, ¥) is Var (W1B) [= uwa)? ia] Another notation for conditional varian computational shorteu is ofp. The following formula isa convenient W181 = £[ 0218] nyo? TBEAAS Continuing Exarpo 4.13, tnd the condtona expected value andthe conditional var ance of W = X +Y given the event 2 = (x4) < 4) bial om Example 4.13 that Px yg(x,») has four points with nonasso proba. Peay M2) (1.3), and 2,2), Tele probabilies are 3/7, 3/14 ter nas respectively. Therefore, E De tyre rete 7 cass: +35 tat eg ot (489) ig isi Similarly, e (wa) + ) (490) meray BI =8 ‘sp 1f : Toayigettonal variances Var 2 = E1W2|a)-ce(W8)? = (131/14) (41/142 = 153/196 RATE. Continuing Example 4.14, ind the condional expected value of W = xy siven the event B = (x +¥ > 4} i B For the event B shown in the adjacent graph, Example 4.14 Showed that the conditional PDF of X,Y given B ie 2S Outed with prob: (4.144) max(X1, X2). with the property peters 11, 01, 2, N <1. oy = 02 = 1, and etry of a sombrero, nen p = -0.9 there » 1. PDF, we define 4.145)180 CHAPTERS PAIRS OF RANDOM VARIABLES From Theorem 4.20, erxviai=[" f 492 393) 2 oy) (A) From Example 4.8, random variables L and T have joint PMF Prrll.t) [t= 40sec t= 60sec Tpage [01S ~ 0 : EZ ! (4.95) 2 pages | 0.3 02 1 = 3 pages | 0.15 od For random variable V = LT, we define the event A = (V > 80}. Find the condi tal PMF P.rja{lst) of Land T given A, What are E[V\A] and Var[V|A]? (B) Random variables X and Y have the joint PDF xy/4000 1 80), Find the conditional joint PDF fx.y\a(t.#) of X and ¥ given B. What are E[W|B) and Var{ WB) 4.9 Conditioning by a Random Variable In Secti n order to derive a new probability model for the 4.8, we use the partial knowledge that the outcome of an experiment (x, y) € B experiment. Now we turn our attention to the special case in which the partial knowledge consists of the value of one of the random variables: either B = (X = x) or B = Leaming (Y = y} changes our knowledge of random variables X, Y. We now have complete knowledge of ¥ and modified knowledge of X. From this information, we derive a modified probability model for X. The new model is eit onal PMF of X given Y or a conditional PDF of X given Y When X and ¥ are diserete, the conditional PMF and associated expected values represent a specialized notation for their counterparts, Py,yjg(x, y) and Elg(X, ¥)|B] in Section 4.8. By contrast, when X and Y are continuous, we cannot apply Section 4.8 directly because P[B] = PLY = y] = Oas discussed in Chapter 3. Instead, we define a conditional PDF as the ratio of the joint PDF to the marginal PDF a condi Defini Theo49° conor IGBYA RANDOM VARIABLE 181 Definition 4.12 Conditional PMF Forany event Y = y such that Py(y) > 0, the conditional PMF of X given Y = y is Puy (ely) = PIX =x1Y = y] ‘The following theorem contains the relationship between the joint PME of X and ¥ and the two conditional PMFs, Pyiy(x|y) and Py, x (yx Theorem 422 Forrandom variables X and ¥ with oint PME Px. y(x, y)- and and y such hat Px and Pr(y) > 0, Pry (x, 9) = Py (ly) Py (9) Prix (via) Px (x Proof Referring to Definition 4.12, efnition 1.6, and Theorem 4.3, w PIX =x Puy Gly) = P(X =a1r= y=" Px v(e.y) = Pry(aly) Py Example 4.17 3 eT el Random variables x and ¥ ha Px.y(e, y), a8 given in Example 4.13 and repeated in the accompanying graph, Find the conditional PME. wi Of given X = x for each x Sy the joint PME 72 apply Theorem 4.22, we first find the marginal PMF Py x). By Theorem 43 Sxl) = Eyes, Px.r(ty). For a given X = x, we sum the nonzero probsbiites along the vertical line x — x. That is, “4 x 4 x V/s +1/8 r Wx Px tx 21/124 1/12 x =| 4x 116+ 1/164 1/16-+ 1/16 4 x4, o otherwise, 0 otherwise. ‘Theorem 4.22 implies that for x € {1,2,3. 4 Pry (x Prix Ole) = ELEY «apy veg, yy Pete J, and brero. ) there jne 4.145)182 CHAPTER PAIRS OF RANDOM VARIABLES For each x € (1, 2.3.4), Pye (lx) is a diferent PME. Ly i 1.2) Saya { M2 XEth23h — pret 4 ye (2.34) prxi=| 8° pamea” — PrXOM =| Shanice x, the conditional PMF of ¥ is the discrete uniform (1, x) random variable. For each y € Sy, the conditional probability mass function of X. ability model of X.” We can use this model in any way tha have in the absence of knowledge of Y respect 10 Pxyy( ves us a new prob- it we use Py(x), the mode! we Most important, we can find expected values with ) just as we do in Chapter 2 with respect to Px (x). Theorem 423 Conditional Expected Value of a Function Y are discrete random variables. For any the conditional expected value of Flg(X. WY =y The conditional expected value of X given ¥ = y is a special case of Theorem 4. eix=a1= D Pay on 499) “Theorem 4.22 shows how to obtain the conditional PMF given the joint PMP, Px.y(x, In many practical situations, including the next example, we first obtain information about marginal and conditional probabilities. We can then use that information to build the ‘complete model. Example 4.18 In Example 4.17, we derived the following conditional PMFs: Py ix(oI1). Prix (2 Pyix@l3), and Pyix(yl4). Find ELY|X =x] for x = 1.2.3.4 Applying Theorem 4.23 with gs we calculate eWK=u=1 E(x =21 4.100) BY|X =3)=2. WX =4] 4.01 Now we consider the case in which X and Y are continuous random variables. observe (Y = y) and define the PDF of X given (Y = y}. We cannot use B = n Definition 4.10 because P[Y = y] = 0. Instead, we define density function, denote We Y=) ‘conditional probabil as fxyy (aly).4.9 CONDITIONING BY A RANDOM VARIABLE 183, Definition 4.13 Conditional PDF For y such that fy(y) > 0, the conditional PDF of X given (¥ = y)is Definition 4.13 implies finx (via) = 22) (4.102) Example 4.19 Retuming to Example 4.5, random variables X and ¥ have joint PDF y A fotey)=2 Osysrs) \ fey (y= _ 4103) otherwise eX For0 < x <1, find the conditional POF fy; (|). For 0 find the conditional POF fxyyaly For0 = x <1, Theorem 4.8 implies teed=f fertandy= f 2dy=2x 4.108) tional POF of ¥ given X is frvtt.y) _[ x O 0, the conditional PDF fxjy(xly) ives us a new probability ” b184 CHAPTERS PAIRS OF RANDOM VARIABLES Definition 4.14 Definition 4.15 model of X. We can use this model in any way that we use f(x), the model we have in the absence of knowledge of ¥. Most important, we can find expected values with respect, to fxiy(z|y) just as we do in Chapter 3 with respect to fy(x). More the conditional expected value of a function of the random variable X Conditional Expected Value of a Function om variables X an For continuous ran expected value of g(X, Y) given Y a ¥ and any y such that fy(y) > 0, she conditional Ele, YY = 91 = [~ ate fu ly) ae ‘The conditional expected value of 2 fen ¥ = y isa special case of Definition 4.14: FIX =y [7 xfuy oy a (4108 ‘When we introduced the concept of expected value in Chapters 2 and 3, we observed that E[X] is a number derived from the probability model of X. This is also true for E[X|B] The conditional expected value given an event is a number derived from the conditional probability model. ‘The situation is more complex when we consider E[X|Y = y}, the conditional expected value given a random variable. In this ease, the conditional expected value is a different number for each possible observation y © Sy. Therefore, E[X|Y is a deterministic function of the observation y. This implies that when we perform an experiment and observe Y = y, E[X|Y = y] is a function of the random variable Y. We use the notation E[X/¥] to denote this Function of the random variable ¥. Since a function of a random variable is another random variable, we conclude that E[X|Y] is a random variable? For some readers, the following definition may help to clarify this point Conditional Expected Value The conditional expected value E{X\Y] is a f Y= y then E(X\Y] = ELXIY = y] ition of random variable ¥ such that if Example 4.20 For random variables X and ¥ in Example 4.5, we found in Example 4.19 that the ‘conditional PDF of X given ¥ is perm fer) [Wl yest fa TT Te Re otherwise, (4.108) Find the condtional expected values ELXIY = yl.and E(x Given the condtonal POF fy (rly), we perform the itogration enxweayl= [~ afer any uo) (any TheorenTheorem 4.25 Theorem 4.26 49 CONDITIONING BY A RANDOM VARIABLE 185 Since EIX|Y = yl = (1+9)/2, EIXIY] = (1+ ¥)/2. An interesting property ofthe random variable E[X|¥ ists expected value ELELX|Y]} We find £{E[X|Y']] in wo steps: first we calculate g(y) = E[X|Y = y] and then we apply ‘Theorem 3.4 to evaluate Elg(¥)] tep process is known as iterated expectation Iterated Expectation E(E(XWN = E(X1 Proof We consider continuous random variables X and ¥ and apply Theorem 3.4 sewn [ exw =siora a. To obtain this formula from Theorem 3.4, we have used ELX|Y = y) in place of g(x) and f(y) in Equation (4.108) for EUX|Y = Place of f(x). Next, we substitute the right side eceveirm= [~ (f xfnweeinae) fron dy ats) d reversing the order of integration, we obtain Rearranging terms inthe double inte; ete [2 [7 ferarn son avd 4. Newt, we apply The 4.2 a Theorem 48 oer ht he ine itz simpy me evewun= [steeds aus ofthis formula is the definition of E[X]. A similar the theorem for discrete random variables. ‘The proof is compl ELE [g(X)|Y1] = E[g(X)) eucormeteteceinn=[" Etewove nord. — nto and if Y is discrete, we have a similar expression, F(eCO)= EL OI = EOP 1Py()186 CHAPTER PAIRS OF RANDOM VARIABL: Theorem 4.26 decomposes the calculation of E[g(X)] into two steps: the calculation of Elg(X)\Y = y), followed by the averaging of E[g(X)|Y = y] over the distribution of Y ‘This is another example of iterated expectation. In Section 4.11, we will se thatthe iterated expectation can both facilitate understanding as well as simplify calculations. Example 4.21 At noon on a weekday, we begin recording new call attempts at a telephone switch Let X denote the arrival time of the ist call, as measured by the number of seconds after noon. Let ¥ denote the arrival time of the second call. In the most common model used in the telephone industry, X and Y are continuous random variables with joint POF [> Oss 0 calls/second is the average arrival rate of telephone calls. Find the narginal POFS f(x) and fy(9) and the conditional PDFS f(y) and fix ls). For <0, f(s) —0. Forx > 0, Theorem 4.8 gives f(x Quiz ya [Be ay ane 4119 Referring to Appendix A.2, we see that X Is an exponential random variable with expected value 1/2. Given X = x, the conditional PDF of ¥ is 4.120) otherwise ‘To interpret this result, let U = Y — X denote the interarrival te, the time between the arrival of the frst and second calls. Problem 4.10.15 asks the reader to show that ‘ven X = x, U has the same PDF as X. That is, U is an exponential (2) random Nariable. Nowwe can find the marginal PDFot Y. Fory <0, fy(») = 0. Theorem 4.8 implies pone] Beara ave yz0 a rO=] 9 otherwise ss ¥ isthe Erlang (2,2) random variable (Appendix A). Given ¥” = y, the consitional POF of X is fier (e.y) | Wy Osx property 08 b=0, b=0, Paato=1 02 pat b=1, 128) 0 otherwise otherwise. ‘i oi aN (1 What the probabitiy model for what isthe conditional A and B? Write the join Pip expected value ELB\A — 9} Pa.0(@, b) as a table 1 PESO whavis the condiional ey) rp ag what is the conditional PMF Pain(ai0)? variance VarlA|B = O}of a? (8) The POF of random variable x ‘nd he conditional PDF ofrandom variable ¥ given X are a? O Oand z < 0 are quite different > 0, from Fig. 6-14a FIGURE 6-13PERM > ete = v/y. Determine /-) FIGURE 6-14 and for z <0, from Fig, 6-14b [> foslx. yd ay Afier differentiation, this y.ydy 220 (659) We have F.(2) = Pixly 52} (656) The inequality x/y < z ean be rewritten as x < yz ify > 0, and x the event (x/y < 2) in (6-56) needs to be conditioned by the e compliment 4. Since AU A = S,by yzity <0. Heng rent A = {y > 0} andis partition theorem, we have Pixjy = Pixjy <2 (AUA)] 0)+Pl/yszy <0) = Pix syzy> 0) + Pixz yey <0) es F corresponding to the second term in (6 over these two regions, We 6-15a shows the area corresponding to the first term, and Fig. 6-15b shows tl fonrraedys f° [” fotenaayrwonanoom vaRiAnLes 187 FIGURE 616 Differentiation gives (6-59) Note that if x and y are non-negative random variables, then the area of integration es to that shown in Fig. 6-16. x y)dedy (6-60) < D> x andy are jointly normal random variables with zero mean and 1 ( ~GnowwI— ‘Show that the ratio 2 = x/y has a Cauchy density centered at r188 pRowasLsy AND RANOOM VARIABLES SOLUTION Inserting (6-61) into (6-59) and usi Thus ting (6-62) from dat ray, inction to be Which represents a Cauchy random variable cen ve to z, we obtain the corresponding distribution arctan (663) owt 4 ermine the probability masses mi, ms As an application, we can use (6-63) 10 de From the spherical symmety ims, and mg inthe four quadrants ofthe xy plane for (6-61 of (6-61), we have mn of the plane where x/y <0 EEN But the second and fourth quadrants represent the reg ls, therefore, the probabil The probability that the point (x, y) is in that region equa table 2 = x/y is negative. Thus Be x? + y’ that the random va ma +g = P(t <0) = F.0) = and 1d (my + ma) = 5 + > arctan If we define a = are tan r/ VT athe ave obtained this result by direct integration of (6-61) in ea Of course, we could ha quadrant. However, this is simpler. D> Let x and y be independent gamma random variables with x ~ G(m.a) andy Gina). Show that z = x/(x + y) has a beta distribution, Proof. fahts) = LAOH t __ \ r>0 y>0 ar (mEUU) bee = +9" Determine /-(2) Note that << 1, since x and y are non-negative random variables which represents a beta distbution. We have = ff fotenaray < ¢ represents the area ofa circle with radius JZ, and hence (see FIGURE 617190 prosasiiry AND RANDOM VARIANLE This Ndy 667 < As an illustration, consider Example 6-14. PSEMNEY x and y are indepe variance o, Determin: dent normal random variables with zero mean and common fc) forz =x? +y SOLUTION Using (6-67), we get 66 where wave used the substitution y = /Z sind. From (6-68), we have the following Ifx and y are independent 2er0 mean Gaussian random variables with common variance co”, then x? +? is an exponential random variable with parameter 20°. Redo Example 615, where x andy are independent Gaussian random varibles with SOLUTION192. rronasiry AxD RANDOM VARIABLES “ NIT se) da z= max(x, y) w= minx, 9) : (4) 67) whe non erento om 40 is the modified Bessel function of the fst kind and zeroth order. Order Statistics In eneral, given any n-tuple x, Xa... %,y We can rearrange them in an increasing ‘of magnitude such that where xy) = min(xy, 2, ---,¥a)and Xi) isthe second smallest value among}, X2 e_and finally Xia) = MAX(X}, X3.«.» Xq)« The functions min and max are nonlinear op trators, and represent special cases ofthe more general order statistics. If, ¥2. +X represent random variables, the function xq) that takes on the value a) in each pos Sible sequence (41, 42,..-. ta) is known as the kth-order statistic. (X41. Xa. +++ Xl represent the set of order statistics among mt random variables, tn this context REX represents the range, and when n = 2, we have the max and min statistics ‘Order statistics is useful when relative magnitude of observations is of importance When worst case scenarios have to be accounted for, then the function ma) is quit useful, For examy x, represent the recorded flood levels over the pa mn yeats at some location. If the objective is to construct a dam to prevent any mart flooding, then the height H of the proposed dam should satisfy the inequality HY > max(xy,%3)-+6%0) with some finite probability. In that case, the p.d-—. of the random variable on the rig Side of (6-76) can be used to compute the desired height. In another case, if a ball nanufacturer wants to determine the average time to failure (j) of its bulbs based oad Sample of size n the sample mean (Xs ++X2+-++++,)/7 can be used as an estimate On the other hand, an estimate based on the least time to failure has other attract features. This estimate mint; ,) may not be as good as the sample mean terms of their respective variances, but the min.) can be computed as soon as the fi bulb fuses, whereas to compute the sample mean one needs to wait tll the last ofthe extinguishes.