This action might not be possible to undo. Are you sure you want to continue?

Prof. V. Venkata Rao

2

CHAPTER 2

U

Probability and Random Variables

2.1 Introduction

At the start of Sec. 1.1.2, we had indicated that one of the possible ways of classifying the signals is: deterministic or random. By random we mean unpredictable; that is, in the case of a random signal, we cannot with certainty predict its future value, even if the entire past history of the signal is known. If the signal is of the deterministic type, no such uncertainty exists. Consider the signal x ( t ) = A cos ( 2 π f1 t + θ ) . If A , θ and f1 are known, then (we are assuming them to be constants) we know the value of x ( t ) for all t . ( A , θ and f1 can be calculated by observing the signal over a short period of time). Now, assume that x ( t ) is the output of an oscillator with very poor frequency stability and calibration. Though, it was set to produce a sinusoid of frequency f = f1 , frequency actually put out maybe f1' where f1' ∈ ( f1 ± ∆ f1 ) .

Even this value may not remain constant and could vary with time. Then, observing the output of such a source over a long period of time would not be of much use in predicting the future values. We say that the source output varies in a random manner.

Another example of a random signal is the voltage at the terminals of a receiving antenna of a radio communication scheme. Even if the transmitted

2.1

Indian Institute of Technology Madras

Principles of Communication

Prof. V. Venkata Rao

(radio) signal is from a highly stable source, the voltage at the terminals of a receiving antenna varies in an unpredictable fashion. This is because the conditions of propagation of the radio waves are not under our control.

But randomness is the essence of communication. Communication theory involves the assumption that the transmitter is connected to a source, whose output, the receiver is not able to predict with certainty. If the students know ahead of time what is the teacher (source + transmitter) is going to say (and what jokes he is going to crack), then there is no need for the students (the receivers) to attend the class!

Although less obvious, it is also true that there is no communication problem unless the transmitted signal is disturbed during propagation or reception by unwanted (random) signals, usually termed as noise and interference. (We shall take up the statistical characterization of noise in Chapter 3.)

However, quite a few random signals, though their exact behavior is unpredictable, do exhibit statistical regularity. Consider again the reception of radio signals propagating through the atmosphere. Though it would be difficult to know the exact value of the voltage at the terminals of the receiving antenna at any given instant, we do find that the average values of the antenna output over two successive one minute intervals do not differ significantly. If the conditions of propagation do not change very much, it would be true of any two averages (over one minute) even if they are well spaced out in time. Consider even a simpler experiment, namely, that of tossing an unbiased coin (by a person without any magical powers). It is true that we do not know in advance whether the outcome on a particular toss would be a head or tail (otherwise, we stop tossing the coin at the start of a cricket match!). But, we know for sure that in a long sequence of tosses, about half of the outcomes would be heads (If this does not happen, we suspect either the coin or tosser (or both!)).

2.2

Indian Institute of Technology Madras

Principles of Communication

Prof. V. Venkata Rao

Statistical

regularity

of

averages

is

an

experimentally

verifiable

phenomenon in many cases involving random quantities. Hence, we are tempted to develop mathematical tools for the analysis and quantitative characterization of random signals. To be able to analyze random signals, we need to understand random variables. The resulting mathematical topics are: probability theory, random variables and random (stochastic) processes. In this chapter, we shall develop the probabilistic characterization of random variables. In chapter 3, we shall extend these concepts to the characterization of random processes.

2.2 Basics of Probability

We shall introduce some of the basic concepts of probability theory by defining some terminology relating to random experiments (i.e., experiments whose outcomes are not predictable).

2.2.1. Terminology

Def. 2.1: Outcome

The end result of an experiment. For example, if the experiment consists of throwing a die, the outcome would be anyone of the six faces, F1 ,........, F6

Def. 2.2: Random experiment

An experiment whose outcomes are not known in advance. (e.g. tossing a coin, throwing a die, measuring the noise voltage at the terminals of a resistor etc.)

Def. 2.3: Random event

A random event is an outcome or set of outcomes of a random experiment that share a common attribute. For example, considering the experiment of throwing a die, an event could be the 'face F1 ' or 'even indexed faces' ( F2 , F4 , F6 ). We denote the events by upper case letters such as A , B or

A1 , A2 ⋅ ⋅ ⋅ ⋅

2.3

Indian Institute of Technology Madras

Principles of Communication

Prof. V. Venkata Rao

Def. 2.4: Sample space

The sample space of a random experiment is a mathematical abstraction used to represent all possible outcomes of the experiment. We denote the sample space by

S.

S

Each outcome of the experiment is represented by a point in

and is

called a sample point. We use s (with or without a subscript), to denote a sample point. An event on the sample space is represented by an appropriate collection of sample point(s).

Def. 2.5: Mutually exclusive (disjoint) events

Two events A and B are said to be mutually exclusive if they have no common elements (or outcomes).Hence if A and B are mutually exclusive, they cannot occur together.

Def. 2.6: Union of events

The union of two events A and B , denoted A ∪ B , {also written as

(A

+ B ) or ( A or B )} is the set of all outcomes which belong to A or B or both.

**This concept can be generalized to the union of more than two events.
**

Def. 2.7: Intersection of events

The intersection of two events, A and B , is the set of all outcomes which belong to A as well as B . The intersection of A and B is denoted by ( A ∩ B ) or simply ( A B ) . The intersection of A and B is also referred to as a joint event

**A and B . This concept can be generalized to the case of intersection of three or
**

more events.

Def. 2.8: Occurrence of an event

An event A of a random experiment is said to have occurred if the experiment terminates in an outcome that belongs to A .

2.4

Indian Institute of Technology Madras

5 Indian Institute of Technology Madras . S 2. For example. it is likely that ⎜ A ⎟ will fluctuate quite badly. Def. Def. denoted by P ( A ) . we expect.11: The relative frequency definition: Suppose that a random experiment is repeated n times. Venkata Rao Def. ⎜ A ⎟ may not deviate from ⎝ n ⎠ 2. ⎜ A ⎟ to tend to a definite limiting ⎝ n ⎠ value. V.2 Probability of an Event The probability of an event has been defined in several ways.9: Complement of an event The complement of an event A . If the event A occurs nA times. If n is the order of 100. and ii) the classical definition. is defined as ⎛n ⎞ P ( A ) = lim ⎜ A ⎟ n → ∞ ⎝ n ⎠ ⎛ nA ⎞ ⎜ n ⎟ represents the fraction of occurrence of A in n trials.Principles of Communication Prof. 2.10: Null event The null event. But ⎝ n ⎠ ⎛n ⎞ as n becomes larger and larger. 2. is an event with no sample points. denoted by A is the event containing all points in S but not in A . let the experiment be that of tossing a coin and A the event ⎛n ⎞ 'outcome of a toss is Head'.1) ⎛n ⎞ For small values of n . denoted φ . ⎝ ⎠ (2.2. then A B = φ and vice versa). Two of the most popular definitions are: i) the relative frequency definition. 2. Thus φ = (note that if A and B are disjoint events. then the probability of A .

. V. 2.Principles of Communication Prof. for i ≠ j and (2. say ten percent and as n becomes larger and larger. it is possible for us to derive some additional relationships: i) If A B ≠ φ .3. In the classical approach. namely.3c) (Note that in Eq. we require the probability measure (to the various events on the sample space) to obey the following postulates or axioms: P1) P ( A ) ≥ 0 P2) P ( S ) = 1 P3) ( AB ) = φ .2) where it is assumed that all outcomes are equally likely! Whatever may the definition of probability. the symbol + is used to mean two different things.. 2. then P ( A) = NA N (2. Using Eq. If N A of those outcomes are favorable to the occurrence of the event A .12: The classical definition: The relative frequency definition given above has empirical flavor. 2. we 2 1 ⎛n ⎞ expect ⎜ A ⎟ to converge to .5a) 2. then P ( A + B ) = P ( A ) + P ( B ) − P ( A B ) (2. An be random events such that: a) Ai A j = φ ..3(c). Venkata Rao 1 by more than.6 Indian Institute of Technology Madras . the probability of the event A is found without experimentation. then P ( A + B ) = P ( A ) + P ( B ) (2..3a) (2.4) ii) Let A1 . This is done by counting the total number N of the possible outcomes of the experiment.. to denote the union of A and B and to denote the addition of two real numbers). 2 ⎝ n ⎠ Def. A2 ...3b) (2.

4. An are said to be mutually exclusive (Eq.. Venkata Rao b) A1 + A2 + . Note: A1 .. 2.7 Indian Institute of Technology Madras . Using the former. it represents the probability of B occurring... The occurrence of the event 'the capacitor on the second draw' is very much dependent on what has been drawn at the first instant. Such dependencies between the events is brought out using the notion of conditional probability ..6 and 2.7 is left as an exercise. (2.5b).5a) and exhaustive (Eq. 2.. (2. ⋅ ⋅ ⋅.7) A very useful concept in probability theory is that of conditional probability. 2. 2. + An = S..5b) (2. P ( A ) = P ( A A1 ) + P ( A A2 ) + . it is quite likely that the occurrence of the event B is very much influenced by the occurrence of the event A ..Principles of Communication Prof. we have ⎛ nAB ⎞ P ( AB ) = lim ⎜ ⎟ n → ∞ ⎝ n ⎠ ⎛n ⎞ P ( A ) = lim ⎜ A ⎟ n → ∞ ⎝ n ⎠ 2.6) Then. The conditional probability P ( B | A ) can be written in terms of the joint probability P ( A B ) and the probability of the event P ( A ) .. To give a simple example.. iii) P ( A ) = 1 − P ( A ) The derivation of Eq. denoted P ( B | A ) . let a bowl contain 3 resistors and 1 capacitor. + P ( A An ) where A is any event on the sample space. This relation can be arrived at by using either the relative frequency definition of probability or the classical definition. In a real world random experiment. given that A has occurred. V. A2 .

9 expresses the probability of joint event AB in terms of conditional probability. Similar relation can be derived for the joint probability of a joint event involving the intersection of three or more events.10b) Eq. P (B ) ≠ 0 P (B ) (2.9. we have P ( A | B) = P ( AB ) P (B ) .8(a) and 2.8b) Eq.8(b) can be written as P ( A B ) = P (B | A) P ( A) = P (B ) P ( A | B ) (2.9) In view of Eq.8(a) as P (B | A) = Similarly P (B ) P ( A | B ) P ( A) . 2. P ( A) ≠ 0 (2. V.10(a) or 2. As P ( B | A ) refers to the probability of B occurring. Venkata Rao where nAB is the number of times AB occurs in n repetitions of the experiment. For example P ( A BC ) can be written as 2.10a) P ( A | B) = P ( A) P (B | A) . we can also write Eq. P ( A) ≠ 0 ⎟ = P ( A) ⎟ ⎟ ⎠ n → ∞ ⎛ nAB ⎜ = lim ⎜ n n → ∞ ⎜ nA ⎜ n ⎝ or (2. given that A has occurred. we have Def 2.13: Conditional Probability P ( B | A ) = lim nAB nA ⎞ ⎟ P ( AB ) .8 Indian Institute of Technology Madras . Eq. 2. 2.8a) P ( A B ) = P (B | A) P ( A) Interchanging the role of A and B . 2. say P ( B | A ) and the (unconditional) probability P ( A ) .Principles of Communication Prof. P (B ) ≠ 0 (2. 2.10(b) is one form of Bayes’ rule or Bayes’ theorem.

knowledge of occurrence of A tells no more about the probability of occurrence B than we knew without that knowledge. whereas if they are disjoint.1 Let A1 . B .12) Eq. the events A and B are said to be statistically independent.Principles of Communication Prof.13.. then P ( A B ) = P ( A ) P ( B ) . Then. 2. An be n mutually exclusive and exhaustive events and B is another event defined on the same space. A2 . The notion of statistical independence can be generalized to the case of more than two events. Alternatively...13) That is. Show that P Aj | B = ( ) i = 1 ) ( ) ∑ P (B | Aj ) P ( Aj ) P B | Aj P Aj n ( (2. 2...14) Either Eq.9 Indian Institute of Technology Madras . C are independent when 2... Ak are said to be statistically independent if and only if (iff) the probability of every intersection of k or fewer events equal the product of the probabilities of its constituents. then P ( A B ) = 0 .14 can be used to define the statistical independence of two events. Thus three events A .. A2 .13 or 2....11) Exercise 2. Venkata Rao P ( A BC ) = P ( A B ) P (C | AB ) = P ( A ) P ( B | A ) P (C | AB ) (2. A set of k events A1 . Suppose the events A and B are such that P (B | A) = P (B ) (2.12 represents another form of Bayes’ theorem. Another useful probabilistic concept is that of statistical independence. V. 2. then P ( A B ) = P ( A) P (B ) A and B satisfy the (2... Note that if A and B are independent. if Eq..

Also arrival of one person is independent of the other. much against the wishes of the parents on both the sides. both are somewhat lacking in punctuality and their arrival times are equally likely to be anywhere in the interval 11 to 12 hrs on that day. Venkata Rao P ( A B ) = P ( A) P (B ) P ( AC ) = P ( A ) P (C ) P ( BC ) = P ( B ) P (C ) and P ( A BC ) = P ( A ) P ( B ) P (C ) We shall illustrate some of the concepts introduced above with the help of two examples. a) b) Picture the sample space Let the event A stand for “P1 and P2 meet”. 2. c) Find the probability that the lovebirds will get married and (hopefully) will live happily ever after. shown in Fig. a) The sample space is the rectangle. Mark this event on the sample space. 2.m.10 Indian Institute of Technology Madras . Example 2. Unfortunately. before leaving in a huff never to meet again. after seeing each other for some time (and after a few tiffs) decide to get married. However. on the ensuing Friday (looks like they are not aware of Rahu Kalam or they don’t care about it).Principles of Communication Prof. They agree to meet at the office of registrar of marriages at 11:30 a.1 Priya (P1) and Prasanna (P2).1(a). V. both are also very short tempered and will wait only 10 min.

2.1(b): The event A of Example 2. V. Fig.1 b) The diagonal OP represents the simultaneous arrival of Priya and Prasanna. as shown in the figure.11 Indian Institute of Technology Madras .Principles of Communication Prof. 2. indicating the possibility of P1 and P2 meeting.1 2.1(a): S of Example 2. meeting between P1 and P2 would take place if P2 arrives within the interval a to b . is shown in Fig.1(b). The event A . Assuming that P1 arrives at 11: x . 2. Venkata Rao Fig.

marked 1 and 2. Are A and B independent? We know that P ( B | A ) = P ( AB ) P ( A) . T1 H2 and 4 P ( A) = 3 4 1 1 P (B | A) = 4 = 3 3 4 Intuitively. it represents the outcome T1 T2 .12 Indian Institute of Technology Madras . therefore. ( T1 indicates toss of coin 1 resulting in tails.2: Let two honest coins.. Let the event A be 'not H1 H2 ' and B be the event 'match'. be tossed together. (Treating each of these outcomes as an event. similarly T2 etc. H1 H2 ). given 'not H1 H2 ’ the 3 toss would have resulted in anyone of the three other outcomes which can be 2. H1 T2 . i.) We shall treat that all these outcomes are equally likely. we find that these events 4 are mutually exclusive and exhaustive). that is the probability of occurrence of any of these four outcomes is 1 . H1 H2 . Venkata Rao c) Probability of marriage = Shaded area 11 = Total area 36 Example 2. The four possible outcomes are T1 T2 . the result P ( B | A ) = 1 is satisfying because. 1 . Find P ( B | A ) . Hence P ( A B ) = H1 T2 ’. T1 H2 . The event A comprises of the outcomes ‘ T1 T2 . (Match comprises the two outcomes T1 T2 . A B is the event 'not H1 H2 ' and 'match'.Principles of Communication Prof. V.e.

The six faces of the die can be treated as the six sample points in S . 2.Principles of Communication Prof. which falls on the real line segment [a1 . has a probability of As P ( B ) = 1 . X ( si ) . to each si ∈ S .13 Indian Institute of Technology Madras . that is.3. that is.2. This implies that the outcome T1 T2 given 3 1 1 and P ( B | A ) = . say X . 3 1 .1 Distribution function: Taking a specific case. as shown in Fig. a2 ] . let the random experiment be that of throwing a die. 2 3 2. V.2: A mapping X ( ) from S to the real line. namely 'not H1 H2 '. Venkata Rao treated to be equally likely. The figure shows the transformation of a few individual sample points as well as the transformation of the event A .3 Random Variables Let us introduce a transformation or function.2. Fig. X assigns a real number. whose domain is the sample space (of a random experiment) and whose range is in the real line. 2. A and B are dependent events. 2.

2. But. for convenience. let S consist of four sample points. will be as shown in Fig. then the events on the sample space will become transformed into appropriate segments of the real line.. FX ( whose domain is the real line and whose range is the interval [0. Venkata Rao Fi = si . 2.3. . with 1 . then 8 8 2 the distribution function FX ( x ) . FX ( x ) is the probability of the event. it could be any ) is a function other symbol and we may use FX ( α ) . P ( s3 ) = and P ( s4 ) = . i = 1. denoted by FX ( FX ( x ) = P ⎡{s : X ( s ) ≤ x}⎤ ⎣ ⎦ ) which is given by (2. Once the transformation is induced. (Note that.. Then we can enquire into the probabilities such as P ⎡{s : X ( s ) < a1}⎤ ⎣ ⎦ P ⎡{s : b1 < X ( s ) ≤ b2 }⎤ ⎣ ⎦ or P ⎡{s : X ( s ) = c}⎤ ⎣ ⎦ These and other probabilities can be arrived at.. provided we know the Distribution Function of X. 4 . we use x as the argument of FX ( ) . If X ( si ) = i − 1. 1] ). FX ( a1 ) etc. comprising all those sample points which are transformed by X into real numbers less than or equal to x .15) That is.) Evidently. .Principles of Communication Prof. i = 1.. s1 to s4 . Let X ( si ) = i . 6 . As an example of a distribution function (also called Cumulative Distribution Function CDF). 3. V. 2.5. 4 each with sample point representing an event with the probabilities P ( s1 ) = P ( s2 ) = 1 1 1 .. 2.14 Indian Institute of Technology Madras .

if and only if P ⎡{s : X ( s ) = a}⎤ = Pa ⎣ ⎦ (2. Properties iv) and v) follow from the fact {s : X ( s ) ≤ b} ∪ {s : b < X ( s ) ≤ a} = {s : X ( s ) ≤ a} Referring to the Fig. In other words.3: An example of a distribution function FX ( ) satisfies the following properties: i) ii) iii) iv) v) FX ( x ) ≥ 0. V. In general.15 Indian Institute of Technology Madras .5 ) = 1 1 at the point .3. note that FX ( x ) = 0 for x < − 0.5 whereas FX ( − 0. then FX ( a ) ≥ FX ( b ) The first three properties follow from the fact that FX ( ) represents the probability and P ( S ) = 1. there is a discontinuity in FX of magnitude Pa at a point x = a . 2.16) 2. 2. Venkata Rao Fig. − ∞ < x < ∞ FX ( − ∞ ) = 0 FX ( ∞ ) = 1 If a > b .5 . then ⎡FX ( a ) − FX ( b ) ⎤ = P ⎡{s : b < X ( s ) ≤ a}⎤ ⎣ ⎦ ⎣ ⎦ If a > b .Principles of Communication Prof. there is a discontinuity of 4 4 x = − 0.

F X ( x ) is continuous to the right. B = {s3 . s1 to s6 .5 ] = P ( s4 ) is not known as s4 is not an event on the sample space. every transformation from S into the real line need not be a random variable. 2. let S consist of six sample points. and has a step of size Pa at point a if and if Eq.) However. (The term “random variable” is somewhat misleading because an RV is a well defined function from S into the real line.16 Indian Institute of Technology Madras . lim P ⎡a < X ( s ) ≤ a + ∆ ⎤ = lim ⎡FX ( a + ∆ ) − FX ( a ) ⎤ ⎣ ⎦ ⎣ ⎦ ∆ → 0 ∆ → 0 We intuitively feel that as ∆ → 0 . B and C can be obtained. Hence.Principles of Communication Prof. We see that the probabilities for the 6 2 6 various unions and intersections of A . Functions such as X ( ) for which distribution functions exist are called Random Variables (RV). s4 . F X { } ( a+ ) − FX ( a ) = 0 . 2. P ( B ) = and P (C ) = . For example. the limit of the set s : a < X ( s ) ≤ a + ∆ is the null set and can be proved to be so. with ∆ > 0 . V. s2 }. Venkata Rao The properties of the distribution function are summarized by saying that Fx ( TP PT ) is monotonically non-decreasing. Then the distribution function fails to exist because P [s : 3.16 is satisfied. The only events that have been identified on the sample space are: A = {s1 . where a + = ∆ → 0 lim (a + ∆ ) That is. {s : X ( s ) ≤ x} should be an event in the sample space with some assigned probability. In other words. s5 } and C = {s6 } and their probabilities are P ( A ) = 2 1 1 . Consider.5 < x ≤ 4. for any real x . Let the transformation X be X ( si ) = i . 1 TP PT Let x = a . is continuous to the right at each point x 1.

4. i = 3. s2 } . The PDF. s1 to s6 .4.17b) The distribution function may fail to have a continuous derivative at a point x = a for one of the two reasons: i) ii) the slope of the Fx ( x ) is discontinuous at x = a Fx ( x ) has a step discontinuity at x = a The situation is illustrated in Fig.2 Let identified S be a sample space with six sample points. 2. 3 2 6 Let Y ( ) be the transformation.3. Sketch FY ( y ) . i = 6 ⎩ Y ( si ) Show that Y ( ) is a random variable by finding FY ( y ) . 2.2 Probability density function Though the CDF is a very fundamental concept. P ( B ) = and P (C ) = . V. Venkata Rao Exercise 2. that is fX ( x ) = or FX ( x ) = d FX ( x ) dx (2. s5 } and C = {s6 } with P ( A ) = 1 1 1 . 5 ⎪3 . in practice we find it more convenient to deal with Probability Density Function (PDF). B = {s3 .17 Indian Institute of Technology Madras . ⎧1.Principles of Communication Prof. The events on S are the same as above. s4 . f X ( x ) is defined as the derivative of the CDF. i = 1. 2.17a) −∞ ∫ f (α) d α X x (2. namely. A = {s1 . 2 ⎪ = ⎨2 .

V. we include an impulse Pa δ ( x − a ) in the PDF.3. In the first case.Principles of Communication Prof.18 Indian Institute of Technology Madras . for the CDF shown in Fig. This impulse function cannot be taken as the limiting case of 2⎦ 8 ⎣ an even function (such as 1 2 ε → 0 1 ⎛x⎞ ga ⎜ ⎟ ) because. (Note that FX ( x ) is continuous to the right. we resolve the ambiguity by taking f X to be a derivative on the right. f X ( x ) has an impulse of weight 1 8 at x = 1 2 as 1⎤ 1 ⎡ P ⎢ X = ⎥ = . 2. 2. ε ⎝ε⎠ 1 2 lim 1 −ε 2 ∫ f X ( x ) dx = lim ε → 0 1 2 ∫ 1 ⎛ 1⎞ 1 δ ⎜ x − ⎟ dx ≠ 8 ⎝ 2⎠ 16 −ε 2.18. fX ( x ) = 1 ⎛ 1⎞ 1 ⎛ 1⎞ 1 ⎛ 3⎞ 1 ⎛ 5⎞ δ⎜ x + ⎟ + δ⎜ x − ⎟ + δ⎜ x − ⎟ + δ⎜ x − ⎟ 4 ⎝ 2⎠ 8 ⎝ 2⎠ 8 ⎝ 2⎠ 2 ⎝ 2⎠ (2. 2. if there is a discontinuity in FX at x = a of magnitude Pa . FX ( x ) has a discontinuous slope at x = 1 and a step discontinuity at x = 2 . Venkata Rao Fig. the PDF will be. That is. For example.18) In Eq.) The second case is taken care of by introducing the impulse in the probability domain.4: A CDF without a continuous derivative As can be seen from the figure.

then X corresponds to a discrete variable.Principles of Communication Prof.3 Joint distribution and density functions Consider the case of two random variables. FX ( x ) .19 Indian Institute of Technology Madras . 1 ≤ x < 3 ⎪8 2 2 ⎩ Such an impulse is referred to as the left-sided delta function. is a continuous function of x for all x . say X and Y . Y ( s ) ≤ y }⎤ ⎣ ⎦ 1 TP PT (2. given by FX . we shall discuss some examples of these three types of variables. 2. We can induce more than one transformation on a given sample space. 8 1 1 ⎧2 ⎪8 . This ensures. it is convenient to denote the variable simply by X . we have i) ii) fX ( x ) ≥ 0 ∞ (2. Later on in this lesson. lim ε → 0 1 −ε 2 ∫ f X ( x ) dx = 1 . V.3. then X 1 is a continuous random variable. 2. a random variable can be classified as: i) continuous (ii) discrete and (iii) mixed.19a) = 1 −∞ ∫ f ( x ) dx X (2. Venkata Rao 1 2 However. If we induce k such transformations. As FX is non-decreasing and FX ( ∞ ) = 1. If FX ( x ) is a TP PT staircase.19b) Based on the behavior of CDF.20) As the domain of the random variable X ( ) is known. We say that X is a mixed random variable if FX ( x ) is discontinuous but not a staircase.Y ( x . They can be characterized by the (two-dimensional) joint distribution function. y ) = P ⎡{s : X ( s ) ≤ x . − 2 ≤ x < 2 ⎪ FX ( x ) = ⎨ ⎪3 . If the CDF. then we have a set of k co-existing random variables.

Fig. 2. In other words.Y ( x .Y ( x . y ) is the probability associated with the set of all those sample points such that under X . V.Y ( x1 .Y ( − ∞ . FX . y ) ≥ 0 . −∞ < y < ∞ FX . let A be the set of all those sample S such that X ( s ) ≤ x1 . ∞ ) = 1 FX .Y ( x1 . Similarly. Y ( s )} corresponding to FX . Properties of the two dimensional distribution function are: i) ii) iii) iv) v) FX .Y ( ∞ . 2. Venkata Rao That is. FX . ∞ ) = FX ( x ) 2. if B is comprised of all those S such that Y ( s ) ≤ y1 . under Y . y ) = FX .20 Indian Institute of Technology Madras . then F ( x1 . y1 ) is the probability associated with the set of all sample points whose transformation does not fall outside the shaded region in the two dimensional (Euclidean) space shown in Fig. the transformed values will be less than or equal to y .Y ( ∞ . y ) = FY ( y ) FX .Y ( x . y1 ) Looking at the sample space points s ∈ S.Principles of Communication Prof.Y ( x . y1 ) is the probability sample points s ∈ associated with the event AB .5. − ∞ ) = 0 FX . their transformed values will be less than or equal to x and at the same time. −∞ < x < ∞.5: Space of { X ( s ) .

21b) The notion of joint CDF and joint PDF can be extended to the case of k random variables.Y ∞ f X ( x1 ) = x ∞ ⎫ ⎤ d ⎧ 1⎡ ⎪ ⎪ ⎨ ∫ ⎢ ∫ f X . Hence. y 2 ) ≥ FX .Y ( x2 . β) d β d α X . y ) = ∂2 FX .21a) or FX . then FX .Y ( x1 .Y ( x .21 Indian Institute of Technology Madras . fY ( y ) = ∞ −∞ ∫ f ( x . f X ( x ) and fY ( y ) . Hence it is common to refer FX ( x ) as the marginal 2. β ) dβ X . the statistics of any individual variable is called the marginal.Y 1 or −∞ ∫ f ( x . y ) ∂x ∂y (2. y ) dy X . where k ≥ 3 . FX .23b) (In the study of several random variables.Y (2. Given the joint PDF of random variables X and Y . Venkata Rao vi) If x2 > x1 and y 2 > y1 . V. f X ( x1 ) = fX ( x ) = d FX ( x ) dx ∞ ∞ = x = x1 −∞ ∫ f ( x . ∞ ) = FX ( x1 ) . y1 ) We define the two dimensional joint PDF as f X . β ) d β⎥ d α ⎬ d x1 ⎪− ∞ ⎢ − ∞ ⎥ ⎪ ⎦ ⎩ ⎣ ⎭ (2.22) Eq.Y ( x . y ) dx X .23a) Similarly.Y ( x2 . We know that. y ) = − ∞− ∞ ∫ ∫ f ( α .Y (2.Y ( α . it is possible to obtain the one dimensional PDFs. y1 ) ≥ FX . β ) d α dβ X . FX ( x1 ) = x1 − ∞− ∞ ∫ ∫ f ( α .Y y x (2. That is.Y ( x .Principles of Communication Prof.22 involves the derivative of an integral.Y ( x1 . 2.

Y ( x .) 2. we have the pair of equations. An analogous relation can be defined for fY | X ( y | x ) . y1 ) fY ( y1 ) (2. y ) .3. but fixed. We might also be interested in the PDF of X given a specific value of y = y1 .Y ( x .26a) = 1 and −∞ ∫ f ( x | y ) dx X |Y (2.25d) f X . FY ( y ) and fY ( y ) are similarly referred to.Y ( x . Once we understand the meaning of conditional PDF. y ) fX ( x ) (2. that is.Principles of Communication Prof.26b) 2.22 Indian Institute of Technology Madras . f X |Y ( x | y ) = fY | X ( y | x ) = f X .Y ( x . given y = y1 . y ) = f X |Y ( x | y ) fY ( y ) = fY | X ( y | x ) f X ( x ) The function f X |Y ( x | y ) may be thought of as a function of the variable x with variable y arbitrary. y ) fY ( y ) f X . f X |Y ( x | y ) ≥ 0 ∞ (2. Venkata Rao distribution function of X and f X ( x ) as the marginal density function.25b) (2.4 Conditional density Given f X . This is called the conditional PDF of X . it satisfies all the requirements of an ordinary PDF. That is.25a) and or (2.Y ( x . Accordingly.24) where it is assumed that fY ( y1 ) ≠ 0 . V.25c) (2. we might as well drop the subscript on y and denote it by f X |Y ( x | y ) . denoted f X |Y ( x | y1 ) and defined as f X |Y ( x | y1 ) = f X . we know how to obtain f X ( x ) or fY ( y ) .

y .3. Let X and Y be independent. f X .. iff f X .. f X .5 Statistical independence In the case of random variables.. 2. we have f X |Y ( x | y ) fY ( y ) = f X ( x ) fY ( y ) or f X |Y ( x | y ) = f X ( x ) (2. Y .Y ( x . the k -dimensional joint PDF factors into the product i = 1 ∏ f X i ( xi ) k (2.3. z ) = f X ( x ) fY ( y ) fz ( z ) (2. Venkata Rao 2. We call k random variables X1 .28) and three random variables X . Example 2. 2.30a) (2.3 A random variable X has ⎧ 0 .27) Hence.. y ) = f X ( x ) fY ( y ) Making use of Eq. Z are independent. . two random variables X and Y are statistically independent. . 2. iff.25(c).29) Statistical independence can also be expressed in terms of conditional PDF.23 Indian Institute of Technology Madras . x > 10 ⎩ i) ii) Find the constant K Evaluate P ( X ≤ 5 ) and P ( 5 < X ≤ 7 ) 2. X k statistically independent iff.. Z ( x . X 2 .. Then.30(a) and 2.Principles of Communication Prof.. We shall now give a few examples to illustrate the concepts discussed in sec.30(b) are alternate expressions for the statistical independence between X and Y . y ) = f X ( x ) fY ( y ) (2.Y . 0 ≤ x ≤ 10 ⎪100 K .Y ( x .30b) Similarly. the definition of statistical independence is somewhat simpler than in the case of events. V. x < 0 ⎪ FX ( x ) = ⎨ K x 2 .. fY | X ( y | x ) = fY ( y ) Eq.

02 x . 2 ii) the given PDF can be written as ⎧a e b x . Example 2. V.25 ⎝ 100 ⎠ P ( 5 < X ≤ 7 ) = FX ( 7 ) − FX ( 5 ) = 0. we require That is.24 Indian Institute of Technology Madras . i) ii) iii) Determine the relation between a and b so that f X ( x ) is a PDF. ∫ a e − b x dx = 0 −∞ ∫ ae −bx dx = 2 ∫ a e − b x dx = 1 . Venkata Rao iii) What is f X ( x ) ? i) ii) FX ( ∞ ) = 100 K = 1 ⇒ K = 1 . ⎩ x < 0 0 ≤ x ≤ 10 x > 10 Note: Specification of f X ( x ) or FX ( x ) is complete only when the algebraic expression as well as the range of X is given. In order for f X ( x ) to ∞ i) represent a legitimate PDF. Determine the corresponding FX ( x ) Find P [1 < X ≤ 2] .4 Consider the random variable X defined by the PDF fX ( x ) = a e −bx . 0 ∞ ∞ 1 . ⎩ x < 0 x ≥ 0. − ∞ < x < ∞ where a and b are positive constants. As can be seen f X ( x ) ≥ 0 for − ∞ < x < ∞ . ⎧0 ⎪ = ⎨0.Principles of Communication Prof. ⎪0 . ⎪ fX ( x ) = ⎨ − b x ⎪a e .24 fX ( x ) = d FX ( x ) dx . hence b = 2 a . 2. 100 ⎛ 1 ⎞ P ( x ≤ 5 ) = FX ( 5 ) = ⎜ ⎟ × 25 = 0.

Venkata Rao For x < 0 . Take a specific value of x = 2 FX ( 2 ) = −∞ ∫ f (x) d x X ∞ = 1 − ∫ fX ( x ) d x 2 But for the problem on hand. 2 1 − bx e . y ) fX ( x ) 2. we have FX ( x ) = b bα 1 e d α = eb x . otherwise ⎩ Find (a) (b) i) fY | X ( y | 1) and ii) fY | X ( y | 1. y ) = ⎨ 2 ⎪0 . x > 0 2 We can now write the complete expression for the CDF as ⎧ ⎪ ⎪ FX ( x ) = ⎨ ⎪1 − ⎪ ⎩ iii) FX ( 2 ) − FX (1) = 1 bx e . ∫ f X ( x ) d x = 2 ∞ −2 −∞ ∫ f (x) d x X Therefore.Y ( x . 0 ≤ y ≤ 2 ⎪ Let f X .25 Indian Institute of Technology Madras . 2 2 −∞ ∫ 2 x Consider x > 0 . 2 x < 0 x ≥ 0 1 −b e − e− 2 b 2 ( ) Example 2.5 ⎧1 . 0 ≤ x ≤ y. V. FX ( x ) = 1 − 1 − bx e .Principles of Communication Prof.5 ) Are X and Y independent? (a) fY | X ( y | x ) = f X .Y ( x .

5 ≤ y ≤ 2 fY | X ( y | 1. for any given x . otherwise 4 the dependence between the random variables X and Y is evident from the statement of the problem because given a value of X = x1 . y ≥ x . 1.5 ) = 2 = ⎨ 1 ⎩0 . 2. V. (i) ∫2 dy x 2 1 = 1− x . Also we see that fY | X ( y | x ) depends on x whereas if X and Y were to be independent. 0 ≤ x ≤ 2 2 fY | X ( y | 1) = 1 2 = ⎧1 . The maximum value taken by y is 2 . 1 ≤ y ≤ 2 ⎨ 1 ⎩0 . Venkata Rao f X ( x ) can be obtained from f X . then fY | X ( y | x ) = fY ( y ) . Hence. otherwise 2 (ii) b) 1 ⎧2 . Y should be greater than or equal to x1 for the joint PDF to be non zero. fX ( x ) = Hence.26 Indian Institute of Technology Madras .Y by integrating out the unwanted variable y over the appropriate range.Principles of Communication Prof. in addition.

27 Indian Institute of Technology Madras .4 Transformation of Variables The process of communication involves various operations such as modulation.6) Fig. 2. filtering etc. detection.6: PDFs for the exercise 2. y ) Show that ⎧ y . We will now develop the necessary theory for the statistical characterization of the output random variables. 0 ≤ y ≤ 10 ⎪100 ⎪ fY ( y ) = ⎨ ⎪ 1 − y . given the transformation and the input random variables.Y ( x .3 For the two random variables X and Y .3 Find a) b) f X . the following density functions have been given. 10 < y ≤ 20 ⎪ 5 100 ⎩ 2. (Fig.Principles of Communication Prof. we typically generate new random variables from the given ones. Venkata Rao Exercise 2. V. In performing these operations. 2. 2.

. = g ( xn ) = .Principles of Communication Prof.... To find fY ( y ) .28 Indian Institute of Technology Madras . x1 . y = g ( x1 ) = ....1 Let Y = g ( X ) .4.. This can be obtained with the help of the following theorem (Thm. 2.... By this we mean the number Y ( s1 ) associated with any sample point s1 is Y ( s1 ) = g ( X ( s1 ) ) Our interest is to obtain fY ( y ) . 2... . Proof: Consider the transformation shown in Fig.... for the specific y . we will show that fY ( y ) = g ' ( x1 ) f X ( x1 ) + . Venkata Rao 2. 2.. V.. solve the equation y = g ( x ) .... + g ' ( xn ) f X ( xn ) + . obtained from X by the transformation Y = g ( X ) . ... Theorem 2.... f X ( x ) .. x2 and x3 ..31) where g ' ( x ) is the derivative of g ( x ) .1).. We see that the equation y 1 = g ( x ) has three roots namely.. (2.7. Denoting its real roots by xn .1 Functions of one random variable Assume that X is the given random variable of the continuous type with the PDF.. Let Y be the new random variable.

for any given y 1 . Therefore. x3 < x ≤ x3 + d x3 where d x1 > 0 . d x2 = P [ x3 < X ≤ x3 + d x 3 ] = f X ( x 3 ) d x3 . 2.7: X − Y transformation used in the proof of theorem 2. From the above. it follows that P [ y1 < Y ≤ y1 + d y ] = P [ x1 < X ≤ x1 + d x1 ] + P [ x2 + dx2 < X ≤ x2 ] + P [ x3 < X ≤ x 3 + d x3 ] This probability is the shaded area in Fig. Venkata Rao Fig.Principles of Communication Prof. this set consists of the following intervals: x1 < x ≤ x1 + d x1 . d x1 = dy g ' ( x1 ) P [ x2 + d x2 < x ≤ x2 ] = f X ( x2 ) d x2 . we need to find the set of values x such that y1 < g ( x ) ≤ y1 + d y and the probability that X is in this set. d x3 > 0 .29 Indian Institute of Technology Madras .1 We know that fY ( y ) d y = P [ y < Y ≤ y + d y ] . 2. x2 + d x2 < x ≤ x2 . V. As we see from the figure. d x 3 = dy g ' ( x2 ) dy g ' ( x3 ) 2. P [ x1 < X ≤ x1 + d x1 ] = f X ( x1 ) d x1 .7. but d x2 < 0 .

Let us find fY ( y ) . where a is a constant. 0 ) . by canceling d y from the Eq. x1 ) . δ ( y − y1 ) of area FX ( x1 ) − FX ( x0 ) . and x ranges from the interval ( − 1.30 Indian Institute of Technology Madras . elsewhere ⎩ and a = − 1 Then. Venkata Rao We conclude that fY ( y ) d y = g ' ( x1 ) f X ( x1 ) dy + g ' ( x2 ) f X ( x2 ) dy + g ' ( x3 ) f X ( x3 ) dy (2. x ≤ 1 ⎪ Let f X ( x ) = ⎨ ⎪ 0 . Note that if g ( x ) = y1 = constant for every x in the interval ( x0 .Principles of Communication Prof.6 Y = g ( X ) = X + a . then we have P [Y = y1 ] = P ( x0 < X ≤ x1 ) = FX ( x1 ) − FX ( x0 ) . 2. Hence fY ( y ) contains an impulse. 1) . V. 2. We shall take up a few examples. there is a unique x satisfying the above transformation. 2. Hence. For a given y . that is FY ( y ) is discontinuous at y = y1 .32. fY ( y ) d y = g '(x) fX ( x ) d y and as g ' ( x ) = 1. we have fY ( y ) = f X ( y − a ) ⎧1 − x .32) and Eq. Example 2. We have g ' ( x ) = 1 and x = y − a . we have the range of y as ( − 2.31 follows. fY ( y ) = 1 − y + 1 As y = x − 1.

− 2 ≤ y ≤ 0 ⎪ Hence fY ( y ) = ⎨ 0 . Venkata Rao ⎧1 − y + 1 . there is a unique b x . then 1 ⎛y⎞ fX ⎜ ⎟ .31 Indian Institute of Technology Madras . we have X = 1 Y . V. Let us find fY ( y ) . where b is a constant. we have fY ( y ) = x ⎧ ⎪1 − . As g ' ( x ) = b .6 As can be seen.8: FX ( x ) and FY ( y ) of example 2. otherwise ⎩ and b = − 2 . Again for a given y .7 Let Y = b X . 2. b ⎝b⎠ 2. Solving for X . Example 2.8. the transformation of adding a constant to the given variable simply results in the translation of its PDF. 2.Principles of Communication Prof. Fig. elsewhere ⎪ ⎩ FX ( x ) and FY ( y ) are shown in Fig. 0 ≤ x ≤ 2 Let f X ( x ) = ⎨ 2 ⎪ 0 .

Let us find fY ( y ) . 2. otherwise f X ( x ) and fY ( y ) are sketched in Fig.9: f X ( x ) and fY ( y ) of example 2. 2.Principles of Communication Prof.8.4 Let Y = a X + b . a ⎝ a ⎠ If f X ( x ) is as shown in Fig.32 Indian Institute of Technology Madras . a > 0 . where a and b are constants. 2. − 4 ≤ y ≤ 0 ⎣ ⎦ 0 .8 Y = a X 2 . compute and sketch fY ( y ) for a = − 2 . Example 2. V.9. Show that fY ( y ) = 1 ⎛y − b⎞ fX ⎜ ⎟. and b = 1. Venkata Rao ⎧1 ⎪ fY ( y ) = ⎨ 2 ⎪ ⎩ y⎤ ⎡ ⎢1 + 4 ⎥ . Fig.7 Exercise 2.2.

otherwise Let a = 1 . otherwise ⎩ We shall compute and sketch fY ( y ) . Example 2. y ≥ 0 a ⎟⎥ ⎠⎦ . If y ≥ 0 . b) 2.9 Consider the half wave rectifier transformation given by ⎧0 . 2. y ≥ 0 ⎪ = ⎨ 2π y ⎝ 2⎠ ⎪ 0 . X ≤ 0 Y =⎨ ⎩X . If y < 0 . − < x < Let f X ( x ) = ⎨ 2 2 2 ⎪ 0 . X > 0 a) Let us find the general expression for fY ( y ) 1 3 ⎧1 ⎪ .33 Indian Institute of Technology Madras . and f X ( x ) = ⎛ x2 ⎞ exp ⎜ − . −∞ < x < ∞ ⎜ 2 ⎟ ⎟ 2π ⎝ ⎠ 1 (Note that exp ( α ) is the same as e α ) Then fY ( y ) = 1 2 y ⎡ ⎛ y⎞ ⎛ y ⎞⎤ ⎢ exp ⎜ − 2 ⎟ + exp ⎜ − 2 ⎟ ⎥ 2π ⎣ ⎝ ⎠ ⎝ ⎠⎦ ⎧ 1 ⎛ y⎞ exp ⎜ − ⎟ . V. then it has two solutions. then the equation y = a x 2 has no real solution.Principles of Communication Prof. otherwise ⎩ Sketching of f X ( x ) and fY ( y ) is left as an exercise. x1 = x2 = − y . Hence fY ( y ) = 0 for y < 0 . Venkata Rao g ' ( x ) = 2 a x .31 yields a y and a fY ( y ) ⎧ 1 ⎡ ⎛ y⎞ ⎛ ⎪ ⎢f X ⎜ ⎟ + fX ⎜ − ⎜ a⎟ ⎜ ⎪ ⎠ ⎝ ⎣ = ⎨ 2a y ⎢ ⎝ a ⎪ ⎪ 0 ⎩ y ⎞⎤ ⎟⎥ . and Eq.

Venkata Rao a) Note that g ( X ) is a constant (equal to zero) for X in the range of ( − ∞ . y = 0 ⎪ 3 ⎪1 = ⎨ . As Y = X for x > 0 . fY ( y ) f X ( x ) and fY ( y ) are sketched in Fig. − <x< Specifically. 0 < y ≤ 2 ⎪2 .9 2. we have fY ( y ) = f X ( y ) for y > 0 . otherwise ⎪0 ⎪ ⎩ Then. 0 ) . otherwise b) 1 3 ⎧1 ⎪ . 2. there is an impulse in fY ( y ) at y = 0 whose area is equal to FX ( 0 ) . y ≥ 0 where w ( y ) = ⎨ ⎩0 .34 Indian Institute of Technology Madras .10. elsewhere ⎩ ⎧1 ⎪4 δ(y ) . let f X ( x ) = ⎨ 2 2 2 ⎪0 . Hence ⎧f ( y ) w ( y ) + FX ( 0 ) δ ( y ) ⎪ fY ( y ) = ⎨ X ⎪0 . V. fY ( y ) = 0 for y < 0 . Fig.Principles of Communication Prof. Hence. 2. As Y is nonnegative. otherwise ⎩ ⎧1 .10: f X ( x ) and fY ( y ) for the example 2.

a continuous RV is transformed into a mixed RV. Y .35 Indian Institute of Technology Madras . Hence the PDF of Y has only two impulses. Let us write fY ( y ) as fY ( y ) = P1 δ ( y − 1) + P−1 δ ( y + 1) where a) P− 1 = P [ X < 0 ] and P1 [ X ≥ 0 ] b) Taking f X ( x ) of example 2. In this case.9. V. X ≥ 0 a) b) Let us find the general expression for fY ( y ) .11: f X ( x ) and fY ( y ) for the example 2.10 ⎧− 1. 2.10 Note that this transformation has converted a continuous random variable X into a discrete random variable Y .Principles of Communication Prof. we have P1 = has the sketches f X ( x ) and fY ( y ) . Example 2.11 4 4 Fig. We shall compute and sketch fY ( x ) assuming that f X ( x ) is the same as that of Example 2. Y assumes only two values. X < 0 Let Y = ⎨ ⎩+ 1.9. 2. Fig. namely ±1. 2. Venkata Rao Note that X . 3 1 and P− 1 = .

Fig. 2. 2.5 Let a random variable X with the PDF shown in Fig.5 is applied as input to the X − Y transformation shown in Fig. Compute and sketch fY ( y ) .13. V.Principles of Communication Prof.6 The random variable X of exercise 2.12: (a) Input PDF for the transformation of exercise 2.12(a) be the input to a device with the input-output characteristic shown in Fig.12(b). Venkata Rao Exercise 2. Compute and sketch fY ( y ) . 2.5 (b) Input-output transformation Exercise 2. 2. 2.36 Indian Institute of Technology Madras .

That is. . X takes the values − 2. Example 2.. 3 with probability 1 . 4. fY ( y ) = 6 6 i = 1 ∑ δ(x − i ) . 6 2 b) If. however.. then P [Y = y k ] = Pk + Pm .11 Let Y = X 2 ... If however. − 1. . ..37 Indian Institute of Technology Madras . 1. 1 .2 Functions of two random variables Given two random variables X and Y (assumed to be continuous type). ∑ 3 6 b) 1 6 i = −2 δ ( x − i ) . 2. find fY ( y ) . respectively. V. then 6 Y takes the values 0... assuming the value Yk = g ( x k ) with probability Pk . If y k = g ( x ) for only one x = xk . two new random variables. 2.. 6 3 3 6 1 1 ⎡δ ( y ) + δ ( y − 9 ) ⎤ + ⎡δ ( y − 1) + δ ( y − 4 ) ⎤ ⎣ ⎦ 3⎣ ⎦ 6 2. a) 1 If f X ( x ) = 6 If f X ( x ) = i = 1 ∑ δ ( x − i ) . 9 with probabilities That is. 1.. In this case.. 22 . . 6 ) with probability of values 12 .Principles of Communication Prof. find fY ( y ) .. then Y takes the 6 a) If X takes the values (1. 0. Venkata Rao We now assume that the random variable X is of discrete type taking on the value x k with probability Pk . Y = g ( X ) is also discrete. Y ) and W = h ( X . y k = g ( x ) for x = xk and x = xm . 62 with probability 1 1 . fY ( y ) = 1 1 1 1 . the RV.. Y ) 2. then P [Y = y k ] = P [ X = xk ] = Pk .. Z and W are defined using the transformation Z = g ( X ..4.

w ⎞ J⎜ ⎟ where ⎝ x. y ⎠ ∂x = ∂z ∂y ∂w ∂y ∂ z ∂w ∂ z ∂w − ∂x ∂y ∂y ∂x That is.Y ( x1 .W ( z . y ) = w1 . solve the system of equations g ( x . We shall assume that the transformation is one-to-one. h ( x . 2.33a) (2. y ⎠ ∂z ∂x ⎛ z. We shall now state the theorem which relates fZ . w ) .W ( z .38 Indian Institute of Technology Madras . y1 ) ⎛ z. y ) = w1 .Y ( x . we require the Jacobian of the transformation. 2. w ) = f X .33. w ) and f X . V. Venkata Rao Given the above transformation and f X .33b) then there is a unique set of numbers. For this. h ( x . For a more general version of this theorem. y1 ) satisfying Eq. given g ( x .Principles of Communication Prof. y ) . denoted ⎛ z. y ) .W ( z . for x and y . (2. refer [1]. we would like to obtain fZ . Theorem 2. w ⎞ J⎜ ⎟ = ∂w ⎝ x.2: To obtain fZ . ( x1 . Then. That is. fZ .Y ( x . the Jacobian is the determinant of the appropriate partial derivatives. y ) = z1 . y ) = z1 . w ⎞ J⎜ ⎟ ⎝ x1 . w ) .W ( z .1. y1 ) be the result of the solution.34) Proof of this theorem is given in appendix A2. y1 ⎠ (2. Let ( x1 .

y ≤ 1 2 and (b) fZ ( z ) .Principles of Communication Prof. x ≤ 1 2 1 .Y ( x .14: (a) The space where f X . 2. we obtain x = 1 (z + w ) 2 and y = 1 ( z − w ) . fX ( x ) = fY ( y ) = 1 .W is non-zero 2. Venkata Rao Example 2. If Z = X + Y and W = X − Y . Fig. V.W ( z . y ) is non-zero. w ) a) From the given transformations. 2. We see that the mapping is one-to-one. Fig.39 Indian Institute of Technology Madras .14(a) 2 depicts the (product) space A on which f X .Y is non-zero (b) The space where fZ .12 X and Y are two independent RVs with the PDFs . let us find (a) fZ .

W From Fig. w can take values only in the − z to z .W ( z . The Jacobian of the transformation 1 1 ⎛ z. −2 ≤ z ≤ 0 4 2. for a given z ( z ≥ 0 ). w ⎞ J⎜ = − 2 and J ( ⎟ = 1 −1 ⎝ x. 2. b) fZ ( z ) = ∞ −∞ ∫ f (z.14(b). w ) d w Z . otherwise ⎩ ) = 2. y ⎠ Hence fZ . we have fZ ( z ) = −z ∫ 8 dw z = − 1 z.14(b). z.W ⎧1 1 4 = ⎪8 . V. w ) is non-zero) as follows: A space The line x = 1 B space 1 (z + w ) = 1 ⇒ w = − z + 2 2 1 ( z + w ) = − 1 ⇒ w = − ( z + 2) 2 1 (z − w ) = 1 ⇒ w = z − 2 2 1 (z − w ) = − 1 ⇒ w = z + 2 2 The line x = − 1 The line y = 1 The line y = − 1 The space is B is shown in Fig.40 Indian Institute of Technology Madras . 0 ≤ z ≤ 2 4 For z negative. we can see that. Hence fZ ( z ) = + z −z ∫ 8 dw 1 1 = 1 z. w ∈ B (z.Principles of Communication Prof. 2. Venkata Rao We can obtain the space B (on which fZ . w ) = ⎨ 2 ⎪0 .

Principles of Communication Prof.Y ( x . fR . Y ) is specified and what is required is fZ ( z ) .41 Indian Institute of Technology Madras . 2.W ( z . As the given transformation is from cartesian-to-polar coordinates. ϕ ) = ⎨ 2 π ⎝ 2⎠ ⎪ . Theorem 2. otherwise ⎩0 It is left as an exercise to find fR ( r ) . y < ∞ .13 Let the random variables R and Φ be given by. w ) is found from which fZ ( z ) can be obtained. − π < ϕ < π ⎪ Hence. y ) = exp ⎢ − ⎜ ⎟⎥ . V. Venkata Rao Hence fZ ( z ) = 1 z .2 can also be used when only one function Z = g ( X . ϕ ) . a conveniently chosen auxiliary or dummy variable W is introduced. fΦ ( ϕ ) and to show that R and Φ are independent variables. − ∞ < x. using the theorem fZ . 0 ≤ r ≤ ∞ . we can write x = r cos φ and y = r sin φ . Typically W = X or W = Y . and the transformation is one -to -one. z ≤ 2 4 Example 2. It is given that ⎝X⎠ ⎡ ⎛ x 2 + y 2 ⎞⎤ 1 f X . Φ ( r . 2π 2 ⎠⎦ ⎣ ⎝ Let us find fR . Φ ( r . ∂r ∂x ∂ϕ ∂x ∂r ∂y cos ϕ = − sin ϕ ∂ϕ r ∂y sin ϕ 1 cos ϕ = r r J = ⎧ r ⎛ r2 ⎞ exp ⎜ − ⎟ . R = X 2 + Y 2 and ⎛Y ⎞ Φ = arc tan ⎜ ⎟ where we assume R ≥ 0 and − π < Φ < π . To apply the theorem.

35 becomes fZ ( z ) = −∞ ∫ f ( z − w ) f (w ) d w X Y (2. −2 ≤ y ≤ 1 fY ( y ) = ⎨ 3 ⎪0 .35) If X and Y are independent. 2. 2.36(b). 2.W ( z . Let us introduce a dummy variable W = Y .Principles of Communication Prof. let us find P [ Z ≤ − 2] . Then.36a) (2.15. V.14 Let X and Y be two independent random variables. X = Z − W . −1 ≤ x ≤ 1 fX ( x ) = ⎨ 2 ⎪0 . we obtain fZ ( z ) as shown in Fig. w ) and fZ ( z ) = ∞ −∞ ∫ f (z − w . fZ .42 Indian Institute of Technology Madras .Y ( z − w . and Y = W . As J = 1 . fZ ( z ) is the convolution of f X ( z ) and fY ( z ) . with ⎧1 ⎪ . Venkata Rao Let Z = X + Y and we require fZ ( z ) . 2. otherwise ⎩ If Z = X + Y . w ) = f X .Y ∞ (2. w ) d w X . From Eq. otherwise ⎩ ⎧1 ⎪ . fZ = f X ∗ fY Example 2. then Eq.36b) That is. Carrying out the convolution.

y ≤ 1 ⎪ Let f X .Y ( x . we have w ∞ fZ ( z ) = −∞ ∫ w f X . x ≤ 1. V. y ) = ⎨ 4 ⎪0 .14 P [ Z ≤ − 2] is the shaded area = 1 . Venkata Rao Fig. 2. let us find an expression for fZ ( z ) .43 Indian Institute of Technology Madras .15: PDF of Z = X + Y of example 2.15 Let Z = X . we have X = ZW Y = W As J = 1 . Y Introducing the dummy variable W = Y . w ) dw ⎧1 + x y . elsewhere ⎩ Then fZ ( z ) = ∞ −∞ ∫ w 1 + zw 2 dw 4 2.Y ( zw . 12 Example 2.Principles of Communication Prof.

2. then 1 + zw 2 1 + zw 2 fZ ( z ) = ∫ w dw = 2 ∫ w dw 4 4 0 −1 1 1 = z⎞ 1⎛ ⎜1 + 2 ⎟ 4⎝ ⎠ ii) For z > 1 . Let fZ .16(b).Principles of Communication Prof.W is non-zero To obtain fZ ( z ) from fZ . we have to integrate out w over the appropriate ranges. w ) be non-zero if ( z . Fig.16: (a) The space where f X . w ) ∈ the given transformation.W ( z . y ) is non-zero if (x.Y ( x .2. Under x ≤ 1 and y ≤ 1 (Fig. we have fZ ( z ) = 2 − 1 z ∫ 0 1 + zw 2 1⎛ 1 1 ⎞ w dw = ⎜ 2 + ⎟ 4 4 ⎝z 2 z3 ⎠ 2. Venkata Rao f X .W ( z . i) Let z < 1.44 Indian Institute of Technology Madras . V.16a). B will be as shown in Fig. y ) ∈ A . where A is the product space B .Y is non-zero (b) The space where fZ . 2. we have fZ ( z ) = 2 iii) 1 z 1 + zw 2 1⎛ 1 1 ⎞ ∫ 4 w dw = 4 ⎜ z2 + 2 z3 ⎟ ⎝ ⎠ 0 For z < − 1. w ) .

Y ⎜ . V. y ≥ 0 . z ≤ 1 ⎛ 1 1 ⎞ ⎜ 2 + ⎟. In transformations involving two random variables. by making use of the distribution function approach.7 Let Z = X Y and W = Y . z > 1 2 z3 ⎠ ⎝z Exercise 2.16 The input to a noisy channel is a binary random variable with P [ X = 0] = P [ X = 1] = 1 .45 Indian Institute of Technology Madras .Principles of Communication Prof. such cases are handled better. otherwise Show that fZ ( z ) = 1 2π e 2 −z 2 . we may encounter a situation where one of the variables is continuous and the other is discrete. The output of the channel is given by Z = X + Y 2 2. as illustrated below. w ⎟ d w w ⎝w ⎠ X and Y be independent with fX ( x ) = 1 π 1 + x2 2 . Venkata Rao ⎧1 ⎪4 ⎪ Hence fZ ( z ) = ⎨ ⎪1 ⎪4 ⎩ z⎞ ⎛ ⎜1 + 2 ⎟ ⎝ ⎠ . − ∞ < z < ∞. Example 2. a) b) Show that fZ ( z ) = ∞ −∞ ∫ 1 ⎛z ⎞ f X . x ≤ 1 and y ⎧ ⎪y e− fY ( y ) = ⎨ ⎪0 ⎩ 2 .

(In this method.) We shall illustrate the distribution function method with another example.Y ( x . is a very basic method and can be used in all situations.1 or 2. we have FY ( z ) + FY ( z − 1) . Obtain fZ ( z ) . given f X . − ∞ < y < ∞ . we compute FY ( y ) and then obtain fY ( y ) = d FY ( y ) dy . The method of obtaining PDFs based on theorem 2. Venkata Rao where Y is the channel noise with fY ( y ) = fZ ( z ) .17 Let Z = X + Y .Principles of Communication Prof.46 Indian Institute of Technology Madras . for transformations involving more than one random variable. Example 2. 2. Find Let us first compute the distribution function of Z from which the density function can be derived. as illustrated by means of example 2. 1 2π e − y2 2 . Similarly. V.2 is called change of variable method. if Y = g ( X ) . Of course computing the CDF may prove to be quite difficult in certain situations.16. we have P [ Z ≤ z | X = 0 ] = FY ( z ) Similarly P [ Z ≤ z | X = 1] = P ⎡Y ≤ ( z − 1) ⎤ = FY ( z − 1) ⎣ ⎦ Hence FZ ( z ) = 1 1 d FZ ( z ) . As fZ ( z ) = 2 2 dz ⎛ z − 1 2 ⎞⎤ ( ) ⎟⎥ exp ⎜ − ⎜ ⎟⎥ 2 2π ⎝ ⎠⎦ 1 ⎡ ⎛ z2 ⎞ 1⎢ 1 ⎟+ exp ⎜ − fZ ( z ) = ⎜ 2 ⎟ 2 ⎢ 2π ⎝ ⎠ ⎣ The distribution function method. P ( Z ≤ z ) = P [ Z ≤ z | X = 0] P [ X = 0 ] + P [ Z ≤ z | X = 1] P [ X = 1] As Z = X + Y . y ) .

17. y ) d y ⎥ ⎥ ∂ z ⎢− ∞ ⎢−∞ ⎥⎥ ⎣ ⎦⎦ ⎣ = ⎡ ∂ ∫∞ d x ⎢ ∂ z ⎢ − ⎣ ∞ ∞ z− x −∞ ∫ ⎤ f X .37a) = −∞ ∫ f (x.Y ( x . 2. namely.Y ( x . V. Y ) lying in the shaded area shown in Fig. y ) d y ⎥ ∫∞ ⎢ − ∞ ⎥ − ⎣ ⎦ ∞ ∞ ⎡z − x ⎤⎤ ∂ ⎡ fZ ( z ) = ⎢ ∫ d x ⎢ ∫ f X . Venkata Rao P [Z ≤ z ] = P [ X + Y ≤ z ] = P [Y ≤ z − x ] This probability is the probability of ( X .47 Indian Institute of Technology Madras . FZ ( z ) = ⎡z − x ⎤ d x ⎢ ∫ f X .Y ( x . y ) d y ⎥ ⎥ ⎦ (2. 2.Y It is not too difficult to see the alternative form for fZ ( z ) .17: Shaded area is the P [ Z ≤ z ] That is. Fig. 2. z − x) d x X .Principles of Communication Prof.

mathematical expectation or simply expectation) of random variable X is defined as mX = E [ X ] = X = ∞ −∞ ∫ x fX ( x ) d x (2.Principles of Communication Prof. y ) d y X .37(b) is the same as Eq. V. 2. We now define a few of these averages and explore their significance. 2]. Details can be found in [1. Venkata Rao fZ ( z ) = ∞ −∞ ∫ f (z − y . Similarly. numbered 1. This can be generalized to the case of functions of n variables. the expected value of a function of X . Note that m X is a constant.37b) If X and Y are independent. we might be in a situation where the PDF is not available but are able to estimate (with reasonable accuracy) certain (statistical) averages of the random variable. However.5 Statistical Averages The PDF of a random variable provides a complete statistical characterization of the variable. 2. g ( X ) .Y (2.2 and 2 respectively are placed in bowl and are mixed. A player is to be 2.35.48 Indian Institute of Technology Madras . Some of these averages do provide a simple and fairly adequate (though incomplete) description of the random variable. This can be illustrated as follows: Three small similar discs.38) where E denotes the expectation operator. So far we have considered the transformations involving one or two variables.39) Remarks: The terminology expected value or expectation has its origin in games of chance. we have fZ ( z ) = f X ( z ) ∗ fY ( z ) . The mean value (also called the expected value. We note that Eq. 2. is defined by E ⎡g ( X ) ⎤ = g ( X ) = ⎣ ⎦ ∞ −∞ ∫ g (x) f (x) d x X (2.

which results in the mean value of Eq. 2. then E ⎡ g ( X ) ⎤ gives n-th central moment. Then. It seems reasonable to assume that the player has a '1/3 claim' on the 9 dollars and '2/3 claim' on three dollars. we obtain the n-th moment of the probability distribution of the RV.49 Indian Institute of Technology Madras . X . g ( X . Note that g ( x ) is such that g (1) = 9 and g ( 2 ) = 3 . If we take X to be (discrete) random variable with the PDF fX ( x ) = 1 2 δ ( x − 1) + δ ( x − 2 ) and g ( X ) = 15 − 6 X . that is. ⎣ ⎦ n E ⎡( X − m X ) ⎤ = ⎣ ⎦ ∞ −∞ ∫ (x − m ) X n fX ( x ) d x (2.Principles of Communication Prof. Y ) . Consider a function of two random variables. His total claim is 9(1/3) + 3(2/3). he will receive nine dollars. V. that is. then 3 3 ∞ ⎡ ⎤ E ⎣g ( X ) ⎦ = −∞ ∫ (15 − 6 x ) f ( x ) d x X = 5 That is. 2. If he draws the disc numbered 1. or five dollars. Venkata Rao blindfolded and is to draw a disc from the bowl.38) and the second moment ( n = 2 . the mathematical expectation of g ( X ) is precisely the player's claim or expectation [3]. For the special case of g ( X ) = X n . E ⎡Xn ⎤ = ⎣ ⎦ ∞ −∞ ∫ x n fX ( x ) d x (2. resulting in the mean square value of X ).42) We can extend the definition of expectation to the case of functions of k ( k ≥ 2 ) random variables. he will receive 3 dollars.40) The most widely used moments are the first moment ( n = 1. E ⎡X2⎤ = X2 = ⎣ ⎦ n ∞ −∞ ∫ x 2 fX ( x ) d x (2.41) If g ( X ) = ( X − m X ) . if he draws either disc numbered 2.

43. y ) d x dy + ∞ − ∞− ∞ ∫ ∫ (β y ) f ( x . E ⎡( X − m X ) ⎤ = ⎣ ⎦ ∞ −∞ ∫ (x − m ) f (x) d x X X = mX − mX = 0 Consider the second central moment 2 2 E ⎡( X − m X ) ⎤ = E ⎡ X 2 − 2 m X X + m X ⎤ ⎣ ⎦ ⎣ ⎦ From the linearity property of expectation. y ) d x dy X . we have the first central moment being always zero because.Y ∞ (2. y ) f (x. 2 2 E ⎡ X 2 − 2 mX X + mX ⎤ = E ⎡ X 2 ⎤ − 2 mX E [ X ] + mX ⎣ ⎦ ⎣ ⎦ 2 2 = E ⎡ X 2 ⎤ − 2 mX + mX ⎣ ⎦ 2 = X 2 − mX = X 2 − ⎡ X ⎤ ⎣ ⎦ 2 2. From Eq.1 Variance Coming back to the central moments. 2.43) An important property of the expectation operator is linearity. Y ) = α X + βY where α and β are constants.5. This result can be established as follows. Y ) ⎤ = ⎣ ⎦ ∞ − ∞− ∞ ∫ ∫ g (x. V. if Z = g ( X . that is. y ) d x dy X . y ) d x d y X .Y ( x . Venkata Rao E ⎡g ( X . we have E [Z ] = = ∞ − ∞− ∞ ∞ ∞ ∫ ∫ ( α x + β y ) f ( x .Y ∞ − ∞− ∞ ∫ ∫ ( α x ) fX .Y ∞ Integrating out the variable y in the first term and the variable x in the second term. then Z = α X + βY .50 Indian Institute of Technology Madras .Principles of Communication Prof. we have E [Z ] = ∞ −∞ ∫ α x f (x) d x + ∫ β y f (y ) d y X Y −∞ ∞ = α X + βY 2.

3 is useful in establishing similar inequalities involving random variables. ⎣ ⎦ ⎣ ⎦ Note: The kind of manipulations used in proving the theorem 2. Specifying the variance essentially constrains the effective width of the density function. The symbol σ 2 is generally used to denote the variance. If E ⎡ g ( X ) ⎤ exists.3: Let g ( X ) be a non-negative function of the random variable X . Venkata Rao The second central moment of a random variable is called the variance and its (positive) square root is called the standard deviation. E ⎡g ( X ) ⎤ = ⎣ ⎦ = ∞ −∞ ∫ g (x) f (x) d x X X X B ∫g (x) f (x) d x + ∫ g (x) f (x) d x A Since each integral on the RHS above is non-negative. we can write E ⎡g ( X )⎤ ≥ ⎣ ⎦ ∫g (x) f (x) d x X A If x ∈ A . (If necessary. This can be made more precise with the help of the Chebyshev Inequality which follows as a special case of the following theorem. Theorem 2. we use a subscript on σ 2 ) The variance provides a measure of the variable's spread or randomness. hence E ⎡g ( X )⎤ ≥ c ∫ f X ( x ) d x ⎣ ⎦ A But ∫ f X ( x ) d x = P [ x ∈ A] = P ⎡g ( X ) ≥ c ⎤ ⎣ ⎦ A That is. which is the desired result. 2. then g ( x ) ≥ c . E ⎡ g ( X ) ⎤ ≥ c P ⎡ g ( X ) ≥ c ⎤ . V. ⎣ ⎦ P ⎡g ( X ) ≥ c ⎤ ≤ ⎣ ⎦ E ⎡g ( X ) ⎤ ⎣ ⎦ c (2.Principles of Communication Prof.44) Proof: Let A = { x : g ( x ) ≥ c} and B denote the complement of A . then for every positive constant c .51 Indian Institute of Technology Madras .

We then have. Venkata Rao To see how innocuous (or weak. the k2 probability of X − m X ≥ 2 σ X does not exceed 1/4 or 25%. 2. where the required probability is concentrated.3. Chebyshev 2. 1 2 P ⎡( X − m X ) ≥ k 2 σ 2 ⎤ ≤ 2 X ⎣ ⎦ k In other words.52 Indian Institute of Technology Madras . ⎣ ⎦ Then Eq. perhaps) the inequality 2. By the same token. Naturally. That is. With k = 2 for example.45b) To establish 2. we expect X to occur within the range ( m X ± 2 σ X ) for more than 75% of the observations. 1 P ⎡ X − mX ≥ k σ X ⎤ ≤ 2 ⎣ ⎦ k which is the desired result. smaller is the width of the interval around m X . V.Principles of Communication Prof. Chebyshev inequality can be interpreted as: the probability of observing any RV outside ± k standard deviations off its mean value is no larger than 1 .44 is.45a) (2. we would take the positive number k to be greater than one to have a meaningful result. at most 100 million would be as tall as a 10 full grown Palmyra tree!) Chebyshev inequality can be stated in two equivalent forms: i) ii) 1 P ⎡ X − mX ≥ k σ X ⎤ ≤ 2 . let g ( X ) = ( X − m X ) and c = k 2 σ2 in theorem X 2 2. let g ( X ) represent the height of a randomly chosen human being with E ⎡ g ( X ) ⎤ = 1. smaller the standard deviation.44 states that the probability of choosing a person over 16 m tall is at most 1 ! (In a population of 1 billion. k > 0 ⎣ ⎦ k 1 P ⎡ X − mX < k σ X ⎤ > 1 − 2 ⎣ ⎦ k where σ X is the standard deviation of X .6m .45(a). (2.

y ) d x d y x y f X ( x ) fY ( y ) d x d y ∞ − ∞− ∞ ∞ ∫ ∫ ∫ = x fX ( x ) d x −∞ −∞ ∫ y f (y ) d y Y = m X mY 2. − ∞ < x < ∞ and α > 0 . Venkata Rao inequality thus enables us to give a quantitative interpretation to the statement 'variance is indicative of the spread of the random variable'.5. That is. Suppose X and Y are independent. we have λ X Y = E [ X Y ] − m X mY (2. if α f X ( x ) = 2 π 2 .53 Indian Institute of Technology Madras . normalized with respect to the product σ X σY is termed as the correlation coefficient and we denote it by ρ .Y ] = E ⎡( X − m X ) (Y − mY ) ⎤ ⎣ ⎦ (2. Y ) = ( X − m X ) (Y − mY ) in Eq.47) The correlation coefficient is a measure of dependency between the variables.Y ( x . λ X Y = cov [ X . α +x (This is called Cauchy’s PDF) 2. That is. then X = 0 but X 2 is not finite. 2.Principles of Communication Prof. Note that it is not necessary that variance exists for every PDF.46a) Using the linearity property of expectation.2 Covariance An important joint expectation is the quantity called covariance which is obtained by letting g ( X .43. ρX Y = E [ X Y ] − mX mY σ X σY (2. E[XY] = = ∞ ∞ − ∞− ∞ ∞ ∞ ∫ ∫ x y f X . V.46b) The cov [ X . We use the symbol λ to denote the covariance.Y ] . Then. For example.

When the random variables are independent. the outcome y is conditioned on the outcome x in such a manner that there is a greater possibility of (y − mY ) being of the same sign as ( x − mX ) . Assume X and Y to be independent. and sometimes negative with respect to mY . In the course of many trials of the experiments and with the outcome X = x1 . When the joint experiment is performed many times.Principles of Communication Prof. for X and Y be independent. On the other hand. if X and Y are uncorrelated. α being constant. number of trials tends to zero as the number of trials keep increasing. Two random variables X and Y are said to be orthogonal if E [ X Y ] = 0 . If the variables are neither independent nor totally dependent. Intuitively. That is. If ρ X Y ( or λ X Y ) = 0 .54 Indian Institute of Technology Madras . they are uncorrelated. this result is appealing. Similarly if the probability of ( x − m X ) and ( y − mY ) being of the opposite sign is quite large. we have ρ X Y = 0 and for the totally dependent (y = ± α x ) case. the sum of the numbers x1 ( y − mY ) would be very small and the quantity. Taking the extreme case of this dependency. then we expect ρ X Y to be negative. then Y = ± α x1 . then X Y = X Y . then ρ will have a magnitude between 0 and 1. That is. ρ X Y = 1. Suppose for example. then Y would occur sometimes positive with respect to mY . Then ρ X Y = ± 1 . given X = x1 . let X and Y be dependent. V. the fact that they are uncorrelated does not ensure that they are independent. As an example. we have λ X Y (and ρ X Y ) being zero. Then we expect ρ X Y to be positive. Venkata Rao Thus. sum . However. let X and Y be so conditioned that. and given X = x1 . let 2. the random variables X and Y are said to be uncorrelated.

if 2. That is Y = k1 X1 + k 2 X 2 where k1 and k2 are constants.48a) If X1 and X 2 are uncorrelated. V. XY = X3 1 1 = ∫ x3 d x = α α −∞ ∞ α −α ∫ 2 x3 d x = 0 2 As X = 0 . Then. 2 . That is. X Y = 0 which means λ X Y = X Y − X Y = 0 . then Y = x1 ! Let Y be the linear combination of the two random variables X1 and X 2 . i = 1.Principles of Communication Prof. we can show that 2 2 σY = k1 σ1 + k 2 σ2 + 2 ρ12 k1 k 2 σ1 σ 2 2 (2.55 Indian Institute of Technology Madras . But X and Y are 2 not independent because. − fX ( x ) = ⎨ α 2 2. Venkata Rao α α ⎧1 < x < ⎪ .48b) Note: Let X1 and X 2 be uncorrelated. That is. X i 2 We will now relate σY to the known quantities. if X = x1 . and let Z = X1 + X 2 and W = X1 − X 2 . 2 σY = E ⎡Y 2 ⎤ − ⎡E [Y ]⎤ ⎦ ⎣ ⎦ ⎣ 2 E [Y ] = k1 m1 + k 2 m2 2 E ⎡Y 2 ⎤ = E ⎡ k1 X12 + k 2 X 2 + 2 k1 k 2 X1 X 2 ⎤ ⎣ ⎦ ⎣ ⎦ With a little manipulation. ⎪0 . 2 2 Then σ2 = σW = σ1 + σ2 . the sum as well as the difference random Z 2 2 variables have the same variance which is larger than σ1 or σ 2 ! 2 The above result can be generalized to the case of linear combination of n variables. σ2 i = σ2 . Let E [ X i ] = mi . Let ρ12 be the correlation coefficient between X1 and X 2 . then 2 2 σY = k1 σ1 + k2 σ2 2 (2. otherwise ⎩ and Y = X 2 .

. 0 ≤ x ≤ 2 . 0 ≤ x ≤ 2 . 2 ≤ x ≤ 4 4 ≤ x Therefore. E[X] = 1 1 2 ∫8x dx + ∫8x dx 0 2 2 4 2. We shall now give a few examples based on the theory covered so far in this section.49 is quite obvious. . Venkata Rao Y = 2 σY = i = 1 ∑k n n i X i .56 Indian Institute of Technology Madras . then 2 i i = 1 ∑k σ2 + 2 ∑∑ k i k j ρi j σi σ j i i j i < j (2. x < 0 .18 Let a random variable X have the CDF ⎧0 ⎪x ⎪ ⎪8 FX ( x ) = ⎨ 2 ⎪x ⎪16 ⎪ ⎩1 .49) where the meaning of various symbols on the RHS of Eq.Principles of Communication Prof. 2. V. x < 0 . 2 ≤ x ≤ 4 4 ≤ x We shall find a) X and b) σ2 X a) The given CDF implies the PDF ⎧0 ⎪1 ⎪ ⎪8 fX ( x ) = ⎨ ⎪1 x ⎪8 ⎪0 ⎩ . Example 2.

− < x < fX ( x ) = ⎨ 2 2 ⎪0.Principles of Communication Prof. Venkata Rao = 1 7 31 + = 4 3 12 2 b) σ 2 = E ⎡ X 2 ⎤ − ⎡ E [ X ]⎤ X ⎦ ⎣ ⎦ ⎣ E ⎡X2⎤ = ⎣ ⎦ = σ 2 X 1 2 1 3 ∫8x dx + ∫8x dx 0 2 1 15 47 + = 3 2 6 2 2 4 47 ⎛ 31 ⎞ 167 = −⎜ ⎟ = 6 144 ⎝ 12 ⎠ Example 2. elsewhere ⎩0 2. 0 < y < 1 f X . E [Y ] = 1 2 − 1 2 ∫ cos ( π x ) d x = 1 2 2 = 0. otherwise ⎩ 2 Let us find E [Y ] and σY . 0 < x < 1. 2.0636 π 1 = 0. y ) = ⎨ .96 2 π Example 2.57 Indian Institute of Technology Madras .20 Let X and Y have the joint PDF ⎧ x + y . where 1 1 ⎧ ⎪1.19 Let Y = cos π X . From Eq. V.5 2 E ⎡Y ⎤ = ⎣ ⎦ 2 2 Hence σY = − 1 2 ∫ cos2 ( π x ) d x = 1 4 − 2 = 0.Y ( x .38 we have.

E [ X | Y ] . y ) dx dy = ∫ ∫ x y ( x + y ) dx dy 2 0 0 1 1 = 17 72 b) To find ρ X Y .Principles of Communication Prof. namely. E [Y ] .51) Similarly. The quantity.21 Let the joint PDF of the random variables X and Y be 2. We shall illustrate the calculation of conditional mean with the help of an example. E[X | Y = y] = E[X |Y ] = ∞ −∞ ∫ x f X |Y ( x | y ) dx (2. E ⎡g ( X ) | Y = y ⎤ = ⎣ ⎦ ∞ −∞ ∫ g ( x ) f ( x | y ) dx X |Y (2. Example 2.43. we can define the conditional variance etc.50) is called the conditional expectation of g ( X ) . If g ( X ) = X . given Y = y . then we have the conditional mean. we require E [ X Y ]. 2. We can easily show that E [ X ] = E [Y ] = 7 11 48 2 and E [ X Y ] = . V. σ2 = σY = X 144 12 144 1 11 Hence ρ X Y = − Another statistical average that will be found useful in the study of communication theory is the conditional expectation. E [ X ]. σ X and σY .58 Indian Institute of Technology Madras .Y ( x . Venkata Rao Let us find a) E ⎡ X Y 2 ⎤ and ⎣ ⎦ b) ρ X Y a) From Eq. we have E ⎡XY 2⎤ = ⎣ ⎦ ∞ − ∞− ∞ ∫ ∫ ∞ x y 2 f X .

f X |Y ( x | y ) = fY ( y ) = f X .Y ( x . 2. 0 < y < x f X . Venkata Rao ⎧1 ⎪ . outside ⎩ Let us compute E [ X | Y ] . Discrete random variables i) Binomial: Consider a random experiment in which our interest is in the occurrence or nonoccurrence of an event A . we are interested in the event A or its complement. 0 < x < 1. To find E [ X | Y ] .59 Indian Institute of Technology Madras . B . y ) fY ( y ) ∫x dx y 1 1 = − ln y . say. y ) ≡ ⎨ x .6 Some Useful Probability Models In the concluding section of this chapter. 0 < y < 1 1 1 x f X |Y ( x | y ) = = − . we require the conditional PDF. 2. We will begin our discussion with discrete random variables.1.Y ( x . V. Let the experiment be repeated n times and let p be the 2.Principles of Communication Prof. we shall discuss certain probability distributions which are encountered quite often in the study of communication theory. − ln y x ln y y < x < 1 Hence E ( X | Y ) = = ∫ x ⎢ − x ln y ⎥ d x ⎣ ⎦ y 1 ⎡ 1 ⎤ y −1 ln y Note that E [ X | Y = y ] is a function of y . ⎪0 . That is.6.

as trails are independent.. ..... given by 2.....18: Binomial RV for the special case n = 5 P ( A B A A B ) = P ( A ) P ( B ) P ( A ) P ( A ) P ( B ) . .. Taking a special case. say. 1.. ⎛ 5⎞ 2 P [ X = 3] = ⎜ ⎟ p3 (1 − p ) ⎝ 3⎠ Generalizing this to arbitrary n and k . The sample space (representing the outcomes of these five repetitions) has 32 sample points.... The sample points such as ABAAB . (Each sample point is actually an element of the five dimensional Cartesian product space)..Principles of Communication Prof. let n = 5 and k = 3 .. will map into real number 3 as shown in the Fig. 2.. k = 0.. 1..18... n . ‘number of occurrences of A in n trials'. ⎝3⎠ In other words. 2. The sample point s1 could represent the sequence ABBBB .. s1 .. s32 .. = p (1 − p ) p 2 (1 − p ) = p 3 (1 − p ) 2 ⎛5⎞ There are ⎜ ⎟ = 10 sample points for which X ( s ) = 3 .. Venkata Rao probability of occurrence of A on each trial and the trials are independent. 2. we have the binomial density. Fig.60 Indian Institute of Technology Madras .. A A A B B etc. X can be equal to 0. k = 3 . then we can write f X ( x ) .. Let X denote random variable.. V. for n = 5 .. . n . If we can compute P [ X = k ].

Let X be the random variable representing the number of errors per block. Then X is b (16.01 = 0. f X ( x ) ≥ 0 and ∞ −∞ ∫ fX ( x ) d x = i = 0 ∑ n Pi = i = ∑ ⎜i n ⎛n⎞ i n −i ⎟ p (1 − p ) 0⎝ ⎠ n = ⎡(1 − p ) + p ⎤ ⎣ ⎦ = 1 It is left as an exercise to show that E [ X ] = n p and σ2 = n p (1 − p ) .01 and that errors in various digit positions within a block are statistically independent. i) E [ X ] = n p = 16 × 0. 0. We write X is b ( n . The following example illustrates the use of binomial PDF in a communication problem.61 Indian Institute of Technology Madras . the algebra to derive them is laborious). Venkata Rao fX ( x ) = i = 0 ∑ P δ(x − i ) i n (2.22 A digital communication system transmits binary digits over a noisy channel in blocks of 16 digits. Example 2. V. Assume that the probability of a binary digit being in error is 0.Principles of Communication Prof. i) ii) Find the expected number of errors per block Find the probability that the number of errors per block is greater than or equal to 3.16. (Though X the formulae for the mean and the variance of a binomial PDF are simple.52) ⎛n⎞ n−i where Pi = ⎜ ⎟ p i (1 − p ) ⎝i ⎠ As can be seen. 2.01) . p ) to indicate X has a binomial PDF with parameters n and p defined above.

2. Note that Chebyshev inequality is not very A random variable X which takes on only integer values is Poisson distributed. we obtain.8 Show that for the example 2. Evidently f X ( x ) ≥ 0 and We will now show that E [ X ] = λ = σ2 X ∞ −∞ ∫ f X ( x ) d x = 1 because λm ∑0 m ! = eλ . Chebyshev inequality results in P ⎡ X ≥ 3 ⎤ ≤ 0.22.53) where λ is a positive constant. λm e = ∑ .002 Exercise 2.02 . V.1) (1 − p ) 2 16 i 16 − i ⎝ ⎠ = 0.Principles of Communication Prof. if ∞ fX ( x ) = m = 0 ∑ δ ( x − m) λm e− λ m! (2. m = ∞ Since.0196 ⎣ ⎦ tight. ii) Poisson: 0. m = 0 m! λ ∞ we have d eλ dλ ( ) = eλ = ∞ m λm − 1 1 ∑0 m ! = λ m = ∞ m = 1 ∑ ∞ m λm m! E[X] = m λm e− λ ∑ 1 m ! = λ eλ e − λ = λ m = Differentiating the series again.62 Indian Institute of Technology Madras . Venkata Rao ii) P ( X ≥ 3 ) = 1 − P [ X ≤ 2] = 1− i =0 ⎛ ⎞ ∑ ⎜ i ⎟ ( 0.

2. Hence σ2 = λ .19.54) Fig.19: Uniform PDF It is easy show that (b − a) a+b E[X] = and σ2 = X 2 12 2 Note that the variance of the uniform PDF depends only on the width of the interval ( b − a ) . 1) or ( 2. Venkata Rao E ⎡ X 2 ⎤ = λ 2 + λ .2. elsewhere ⎩ A plot of f X ( x ) is shown in Fig. whether X is uniform in ( − 1. (2.2.6. it has the same variance. X ⎣ ⎦ 2. 4 ) .2 Continuous random variables i) Uniform: A random variable X is said to be uniformly distributed in the interval a ≤ x ≤ b if. a ≤ x ≤ b ⎪ fX ( x ) = ⎨ b − a ⎪0 . ⎧ 1 . namely 1 . V. 3 ii) Rayleigh: An RV X is said to be Rayleigh distributed if.Principles of Communication Prof.63 Indian Institute of Technology Madras . Therefore.

55) A typical sketch of the Rayleigh PDF is given in Fig.) Fig.2. (2. V.12 is Rayleigh PDF. 2.20. ( fR ( r ) of example 2. Venkata Rao ⎧x ⎛ x2 ⎞ exp ⎜ − ⎪ ⎟.2. x ≥ 0 fX ( x ) = ⎨ b ⎝ 2 b⎠ ⎪ . elsewhere ⎩0 where b is a positive constant.20: Rayleigh PDF Rayleigh PDF frequently arises in radar and communication problems.64 Indian Institute of Technology Madras .Principles of Communication Prof. We will encounter it later in the study of narrow-band noise processes.

Note that if X is N m X . ⎝ ⎠ X 2. Venkata Rao Exercise 2. 1 TP PT In this notation. V. −∞ < x < ∞ exp ⎢− 2 σ2 ⎢ ⎥ X ⎣ ⎦ (2. We use the symbol X N ( m X . then E [ X ] = and iii) Gaussian By far the most widely used PDF. 2.56 is a valid PDF. The Gaussian PDF is symmetrical with respect to m X . σ2 ) to denote the Gaussian density1.65 Indian Institute of Technology Madras .56) where m X is the mean value and σ2 the variance.3. we show that X PT f X ( x ) as given by Eq. Then. the Gaussian PDF is X completely specified by the two parameters. 2. 1) . In appendix A2.9 a) Let f X ( x ) be as given in Eq. 2. Hint: Make 0 ∞ the change of variable x 2 = z . σ X . That is. 1) denotes the Gaussian PDF with zero mean and unit variance. As can be seen from the Fig. Show that ∫ f X ( x ) d x = 1. then Y = 2 ( ) ⎛ X − mX ⎞ ⎜ σ ⎟ is N ( 0. in the context of communication theory is the Gaussian (also called normal) density. specified by fX ( x ) = 1 2 π σX ⎡ ( x − m X )2 ⎤ ⎥. 2 πb 2 is Rayleigh distributed.21. m X and σ2 . N ( 0. x d x = b) Show that if X E ⎡X2⎤ = 2 b ⎣ ⎦ dz .Principles of Communication Prof.55.

2. P [ X ≥ a] = ∞ ∫ a 1 2 π σX ⎡ ( x − m X )2 ⎤ exp ⎢− ⎥dx 2 σ2 ⎢ ⎥ X ⎣ ⎦ This integral cannot be evaluated in closed form.2 at the end of the chapter. We have. Venkata Rao Fig. we have ⎝ σX ⎠ P [ X ≥ a] = ∞ a − mX σX ∫ 1 2π e − z2 2 dz ⎛ a − mX ⎞ = Q⎜ ⎟ ⎝ σX ⎠ where Q ( y ) = ∞ ∫ y ⎛ x2 ⎞ exp ⎜ − ⎟dx 2π ⎝ 2 ⎠ 1 (2. 2.66 Indian Institute of Technology Madras .Principles of Communication Prof. A small list is given in appendix A2. V.5 Consider P [ X ≥ a ] . 1) . Q( ) function table is available in most of the text books on communication theory as well as in standard mathematical tables.57) Note that the integrand on the RHS of Eq. 2.21: Gaussian PDF Hence FX ( m X ) = mX −∞ ∫ f (x) d x X = 0.57 is N ( 0. By making a change of variable ⎛ x − mX ⎞ z = ⎜ ⎟ .

We shall develop Gaussian random processes in detail in Chapter 3 and. Let Y have the PDF. V.23 A random variable Y is said to have a log-normal PDF if X = ln Y has a Gaussian (normal) PDF. y ≥ 0 exp ⎢− ⎪ fY ( y ) = ⎨ 2 π y β 2 β2 ⎢ ⎥ ⎣ ⎦ ⎪ 0 . a) b) c) Show that Y is log-normal Find E (Y ) If m is such that FY ( m ) = 0. The electrical noise in a communication system is often due to the cumulative effects of a large number of randomly moving charged particles. For a more precise statement and a thorough discussion of this theorem. Venkata Rao The importance of Gaussian density in communication theory is due to a theorem called central limit theorem which essentially states that: If the RV X is the weighted sum of N independent random components. find m . we shall make use of this theory in our studies on the noise performance of various modulation schemes. to the total. in Chapter 7. Let X = ln Y or x = ln y (Note that the transformation is one-to-one) a) 2. where each component makes only a small contribution to the sum. Hence the instantaneous value of the noise can be fairly adequately modeled as a Gaussian variable. then FX ( x ) approaches Gaussian as N becomes large.67 Indian Institute of Technology Madras . fY ( y ) given by.5 . each particle making an independent contribution of the same amount. otherwise ⎪ ⎩ where α and β are given constants. ⎧ ⎡ ( ln y − α )2 ⎤ 1 ⎪ ⎥.Principles of Communication Prof. you may refer [1-3]. regardless of the distribution of the individual components. Example 2.

V. y ) . Venkata Rao dx 1 1 = → J = dy y y Also as y → 0 .Y ( x . ∞ ) is 1. k1 = 2 π σ X σY 1 − ρ2 2. then P [ X ≤ ln m ] = 0. exp ⎢− 2 β2 ⎥ 2π β ⎢ ⎣ ⎦ 1 −∞ < x < ∞ Note that X is N ( α . y ) = + −2ρ ⎢ ⎥⎬ 2 2 k1 σX σY σ X σY ⎩ k2 ⎣ ⎦⎭ 1 (2.5 That is. y < ∞ given by. x → ∞ Hence f X ( x ) = ⎡ ( x − α )2 ⎤ ⎥. ⎧ 1 ⎡ ( x − m X )2 ( y − mY )2 ( x − mX ) ( y − mY ) ⎤⎫ exp ⎨− fX . we will consider the bivariate Gaussian PDF. Y ( x . β2 ) b) Y = E ⎡e X ⎤ = ⎣ ⎦ ⎡ − ( x − α) 2 1 x ⎢ e e 2β ∫ 2π β − ∞ ⎢ ⎣ ∞ 2 2 ⎤ ⎥dx ⎥ ⎦ = e β2 α+ 2 ⎡ x − ( α + β2 ) ⎤ ⎡∞ ⎤ ⎦ − ⎣ 1 ⎢ ⎥ 2 β2 e d x⎥ ⎢∫ ⎢− ∞ 2 π β ⎥ ⎣ ⎦ As the bracketed quantity being the integral of a Gaussian PDF between the limits ( − ∞ .58) where.5 .Principles of Communication Prof. − ∞ < x . x → − ∞ and as y → ∞ . we have Y = e α + β2 2 c) P [Y ≤ m ] = P [ X ≤ ln m ] Hence if P [Y ≤ m ] = 0.68 Indian Institute of Technology Madras . ln m = α or m = e α iv) Bivariate Gaussian As an example of a two dimensional density. f X .

y ) = f X ( x ) fY ( y ) iff ρ = 0 That is. 1 TP PT Note that the converse is not necessarily true. if the Gaussian variables are uncorrelated. unless X and Y independent. in general. V. σ2 ) and Y is N ( mY . 2. then Z is Gaussian. then the marginal density of X or Y is 2 Gaussian. We can construct examples of a joint PDF fX .2).5.22 gives the plot of a bivariate Gaussian PDF for the case of ρ = 0 and σ X = σY . P3) If Z = α X + βY where α and β are constants and X and Y are jointly Gaussian. Y and let fX are and fY be Gaussian. with respect to non-Gaussian variables (we have already seen an example of this in Sec. σY ) 1 X TP PT P2) f X Y ( x .5. then they are independent. That is not true. Y is jointly Gaussian.69 Indian Institute of Technology Madras .Principles of Communication Prof. X is N ( m X . Therefore fZ ( z ) can be written after computing mZ and σ2 with the help of the formulae given in section 2. Venkata Rao k 2 = 2 (1 − ρ2 ) ρ = Correlation coefficient between X and Y The following properties of the bivariate Gaussian density can be verified: P1) If X and Y are jointly Gaussian. Let fX and fY be obtained from fX . 2. which is not Gaussian but results in fX and fY that are Gaussian. Y . This does not imply fX . that is. Z Figure 2.

2. Similarly for ρ < 0. If ρ > 0 . σ2 = σY = 1 and X ρX Y = − 1 . 2. Example 2.Y) lying in the shaded region D shown in Fig. of course.22: Bivariate Gaussian PDF ( σ X = σY and ρ = 0 ) For ρ = 0 and σ X = σY . positive and (ii) ρ .Y resembles a (temple) bell. f X .24 2 Let X and Y be jointly Gaussian with X = − Y = 1 . negative.2. we have two cases (i) ρ .23. V. Let us find the probability of 2 (X. imagine the bell being compressed along the X = − Y axis so that it elongates along the X = Y axis.Principles of Communication Prof.70 Indian Institute of Technology Madras . the striker missing! For ρ ≠ 0 . with. Venkata Rao Fig.

24(b).24: (a) Region A and (b) Region B used to obtain region D A ⎤ − P ⎡( x .Principles of Communication Prof.23: The region D of example 2. Hence the required probability is. y ) ∈ B ⎤ ⎦ ⎣ ⎦ The required probability = P ⎡( x .71 Indian Institute of Technology Madras . y ) ∈ ⎣ For the region y ≥ − 1 2 A . Venkata Rao Fig. V. 2.24 Let A be the shaded region shown in Fig. Fig. we have y ≥ − x + 1 and for the region B . 2. 2 2.24(a) and B be the shaded region in Fig. 2. we have 1 x + 2 . 2.

Principles of Communication

Prof. V. Venkata Rao

X X ⎡ ⎤ ⎡ ⎤ P ⎢Y + ≥ 1⎥ − P ⎢Y + ≥ 2⎥ 2 2 ⎣ ⎦ ⎣ ⎦

Let Z = Y +

X 2

Then Z is Gaussian with the parameters, Z =Y + σ2 = Z = 1 1 X = − 2 2

1 2 1 2 σ X + σY + 2 . ρ X Y 4 2 1 1 1 3 + 1− 2. . = 4 2 2 4

Z+ 3 4 1 2 is N ( 0, 1)

⎛ 1 3⎞ That is, Z is N ⎜ − , ⎟ . Then W = ⎝ 2 4⎠

P [ Z ≥ 1] = P ⎡W ≥ ⎣

3⎤ ⎦

5 ⎤ ⎡ P [ Z ≥ 2] = P ⎢W ≥ ⎥ 3⎦ ⎣

Hence the required probability = Q

( 3) − Q⎛ ⎜ ⎝

5 ⎞ ⎟ 3⎠

( 0.04 − 0.001) =

0.039

2.72

Indian Institute of Technology Madras

Principles of Communication

Prof. V. Venkata Rao

Exercise 2.10

X and Y are independent, identically distributed (iid) random variables, each being N ( 0, 1) . Find the probability of X , Y lying in the region Fig. 2.25.

A shown in

Fig. 2.25: Region

A of Exercise 2.10

Note: It would be easier to calculate this kind of probability, if the space is a product space. From example 2.12, we feel that if we transform

(X,Y)

B.

into

(Z , W )

such that Z = X + Y , W = X − Y , then the transformed space

B

would be square. Find fZ ,W ( z , w ) and compute the probability ( Z , W ) ∈

2.73

Indian Institute of Technology Madras

Principles of Communication

Prof. V. Venkata Rao

Exercise 2.11

Two random variables X and Y are obtained by means of the transformation given below. X = ( − 2 loge U1 ) 2 cos ( 2 π U2 ) Y = ( − 2 loge U1 ) 2 sin ( 2 π U2 )

1 1

(2.59a) (2.59b)

U1 and U2 are independent random variables, uniformly distributed in the range 0 < u1 , u2 < 1 . Show that X and Y are independent and each is

N ( 0, 1) .

Hint: Let X1 = − 2 loge U1 and Y1 = Show that Y1

X1

is Rayleigh. Find f X ,Y ( x , y ) using X = Y1 cos Θ and

Y = Y1 sin Θ , where Θ = 2 π U2 . Note: The transformation given by Eq. 2.59 is called the Box-Muller transformation and can be used to generate two Gaussian random number sequences from two independent uniformly distributed (in the range 0 to 1) sequences.

2.74

Indian Institute of Technology Madras

Principles of Communication

Prof. V. Venkata Rao

Appendix A2.1 Proof of Eq. 2.34

The proof of Eq. 2.34 depends on establishing a relationship between the differential area d z d w in the z − w plane and the differential area d x d y in the x − y plane. We know that

fZ ,W ( z , w ) d z d w = P [ z < Z ≤ z + d z , w < W ≤ w + d w ]

**If we can find d x d y such that
**

fZ ,W ( z , w ) d z d w = f X ,Y ( x , y ) d x d y , then fZ ,W can be found. (Note that

the variables x and y can be replaced by their inverse transformation quantities, namely, x = g − 1 ( z , w ) and y = h − 1 ( z , w ) ) Let the transformation be one-to-one. (This can be generalized to the case of one-to-many.) Consider the mapping shown in Fig. A2.1.

Fig. A2.1: A typical transformation between the ( x − y ) plane and ( z − w ) plane in the z − w

Infinitesimal rectangle

ABC D

plane is mapped into the

parallelogram in the x − y plane. (We may assume that the vertex A transforms to A' , B to B' etc.) We shall now find the relation between the differential area of the rectangle and the differential area of the parallelogram.

2.75

Indian Institute of Technology Madras

with vertices P1 . A2.2 where V1 = ( P2 − P1 ) and V2 = ( P3 − P1 ) . Venkata Rao Consider the parallelogram shown in Fig. That is.2. y ) be the co-ordinates of P1 . y + d z⎟ = ⎜x + ∂z ∂z ⎝ ⎠ ⎛ ⎞ ∂x ∂y P3 = ⎜ x + dw . A = V1 × V2 2. the area A of the parallelogram is. y + d z⎟ ⎜ ⎟ ∂z ∂z ⎝ ⎠ ⎛ ⎞ ∂x ∂y dz. P3 and P4 . Fig.76 Indian Institute of Technology Madras . A2. V1 = V2 = ∂x ∂y dz i + dz j ∂z ∂z ∂x ∂y dw i + dw j ∂w ∂w where i and j are the unit vectors in the appropriate directions.2: Typical parallelogram Let ( x . P2 . A2. y + dw⎟ ∂w ∂w ⎝ ⎠ Consider the vectors V1 and V2 shown in the Fig. V.Principles of Communication Prof. Then the P2 and P3 are given by ⎛ ⎞ ∂ g− 1 ∂ h− 1 P2 = ⎜ x + dz. Then.

Y ( x .77 Indian Institute of Technology Madras . and i × j = − ( j × i ) = k where k is the unit vector perpendicular to both i and j . w ) = f X . Venkata Rao As i × i = 0 . y ) ⎛ z.W ( z . y ⎠ . w ⎠ That is. ⎛ x. y ⎞ fZ. V. j × j = 0 . Y ( x . y ) J ⎜ ⎟ ⎝ z. we have V1 × V2 = ∂x ∂y ∂y ∂x d z dw − ∂ z ∂w ∂ z ∂w ⎛ x. w ⎞ J⎜ ⎟ ⎝ x.Principles of Communication Prof. 2. y ⎞ A = J⎜ ⎟ d z dw ⎝ z. w ⎠ = fX .

1151 2.2743 1.00 0.10 3.1251 2.80 0.80 0.45 0.0548 3.2912 1.0228 4.30 0.00034 0.40 0.55 0.35 0.70 0.85 0.40 0.05 0.60 0. Venkata Rao Appendix A2.20 0.55 0.60 0.65 0.00 0.0808 2.0062 0.00007 0. Note that Q ( 0 ) = 0.0139 0.0668 3.35 0.4013 1.90 0.0047 0.0735 2.90 0.20 0.0179 0.70 0.15 0.10 0.45 0.0107 0.1977 1.80 0.60 0.50 0.00 0.0885 2.0322 3.50 0.00016 0.40 0.15 0.00048 0.0019 0.05 0.0287 3.2578 1.1587 2.5 .4801 1.00 0.1357 2.78 0.95 0.25 0.0446 3.30 0.95 0.20 0.3821 1.00023 0.10 0.1841 1.90 0.3632 1.10 0.30 0.90 4.65 0. V.90 0.0495 3.50 0.0256 3.40 0. y Q (y ) y Q (y ) y Q (y ) y Q (y ) 0.80 0.85 0.Principles of Communication Prof.3085 1.2420 1.1469 2.3264 1.2 Q( ) Function Table Q (α) = ∞ ∫ α 1 2π e 2 − x 2 dx It is sufficient if we know Q ( α ) for α ≥ 0 .00010 0.27 4.0035 0.00005 1.0968 2.0013 0.00069 0.0082 0.4602 1.25 0.0401 3.1711 1.4405 1.0156 2.75 0.2266 1. because Q ( − α ) = 1 − Q ( α ) .0606 3.0026 0.28 3.70 0.0010 10− 3 10− 3 2 10− 4 10− 4 2 10− 5 10− 6 3.30 0.3446 1.78 Indian Institute of Technology Madras .4207 1.20 0.50 0.75 0.2119 1.0359 3.00003 2.10 0.70 3.60 0.70 0.

2 ⎝ 2⎠ 2. the complementary error function which is ∞ erfc ( α ) = 1 − erf ( α ) = and the error function.Principles of Communication Prof.79 Indian Institute of Technology Madras . erf ( α ) = Hence Q ( α ) = 2 π 2 ∫e α 0 − β2 dβ π ∫e α − β2 dβ 1 ⎛ α ⎞ erfc ⎜ ⎟. Venkata Rao Note that some authors use erfc ( given by ) . V.

3.3.3.3) Then. A2.56. A2. 2. r = v 2 + y 2 and θ = tan−1 ⎜ ⎟ . ⎝v ⎠ and d x d y = r d r d θ . 2 −y Let I = −∞ ∫e dv = −∞ ∫e 2 dy. is a valid PDF by ∞ ( ) establishing ∞ −∞ 2 −v ∫ f (x) d x X ∞ 2 = 1 .2 and Eq.3 in Eq. A2.2) (A2. Then. σ2 is a valid PDF X We will show that f X ( x ) as given by Eq. we have the required result. Let v = 1 2π ∞ −∞ ∫e − v2 2 dv = 1 (A2.3.1. 2π ∞ I 2 = ∫ ∫e 0 0 − r2 2 r dr dθ 2π = 2 π or I = That is. I 2 ⎡ ∞ − v2 ⎤ ⎡ ∞ − y2 ⎤ 2 dv⎥ ⎢ ∫ e 2 d y⎥ = ⎢∫e ⎢− ∞ ⎥ ⎢− ∞ ⎥ ⎣ ⎦⎣ ⎦ ∞ = − ∞− ∞ ∫ ∫e ∞ − v2 + y2 2 dv d y ⎛y⎞ Let v = r cos θ and y = r sin θ . d v = Using Eq. V.80 Indian Institute of Technology Madras .1) x − mX σx dx σx (A2. Then.3. Venkata Rao Appendix A2. (Note that f X ( x ) ≥ 0 for − ∞ < x < ∞ ).Principles of Communication Prof. 2.3.3 Proof that N m X . (Cartesian to Polar coordinate transformation).

2nd edition. The Macmillan Co. ‘Probability. J.. ‘Principles of Communication Engineering’. A. John Wiley.. Random Variables and Stochastic Processes’.. Venkata Rao References 1) Papoulis. McGraw Hill (3rd edition). T. 1965. A. and Craig. 1991. M.81 Indian Institute of Technology Madras ... V. P P 2. ‘Introduction to Mathematical Statistics”. P P 2) Wozencraft. 1965. 3) Hogg. R. V. J. I.Principles of Communication Prof. and Jacobs.

Sign up to vote on this title

UsefulNot useful- PROBABILITY THEORY AND RANDOM PROCESSES
- Frequency Domain Processing
- Chapter 2 - Probability and Random Variables
- Principles of Communication
- Random Signals and Noise
- 1
- Digital Signal Processing(3)
- Probability Random Variables and Random Processes Part 1
- Probability and Statistics
- RF-Circuit-Design-Theory-and-Application-by-Ludwig-Bretchko-Solution-Manuel.pdf
- Digital Signal Processing - Computer Based Approach - Solution Manual
- Matlab Signal Processing
- Representation of Signals
- 1_8
- Principles of Communication
- Principles of Communication
- pulse code modulation sampling
- Principles of Communication
- Course Notesci
- Lecture
- Assignment 2 - Engineering Statistics.pdf
- STOCHPROC
- Maths Test
- 10.1.1.25
- Exam P/1 Sample Questions
- edu-2009-spring-p-ques
- final exam 2011-2012
- Probability and Stochastic Processes 2E, By Roy D. Yates , David J. Goodman
- 5 Random Variates
- Principles of Communication

Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

We've moved you to where you read on your other device.

Get the full title to continue

Get the full title to continue reading from where you left off, or restart the preview.