1_5 | Probability Theory | Probability Density Function

Principles of Communication

Prof. V. Venkata Rao

2

CHAPTER 2
U

Probability and Random Variables

2.1 Introduction
At the start of Sec. 1.1.2, we had indicated that one of the possible ways of classifying the signals is: deterministic or random. By random we mean unpredictable; that is, in the case of a random signal, we cannot with certainty predict its future value, even if the entire past history of the signal is known. If the signal is of the deterministic type, no such uncertainty exists. Consider the signal x ( t ) = A cos ( 2 π f1 t + θ ) . If A , θ and f1 are known, then (we are assuming them to be constants) we know the value of x ( t ) for all t . ( A , θ and f1 can be calculated by observing the signal over a short period of time). Now, assume that x ( t ) is the output of an oscillator with very poor frequency stability and calibration. Though, it was set to produce a sinusoid of frequency f = f1 , frequency actually put out maybe f1' where f1' ∈ ( f1 ± ∆ f1 ) .

Even this value may not remain constant and could vary with time. Then, observing the output of such a source over a long period of time would not be of much use in predicting the future values. We say that the source output varies in a random manner.

Another example of a random signal is the voltage at the terminals of a receiving antenna of a radio communication scheme. Even if the transmitted

2.1
Indian Institute of Technology Madras

Principles of Communication

Prof. V. Venkata Rao

(radio) signal is from a highly stable source, the voltage at the terminals of a receiving antenna varies in an unpredictable fashion. This is because the conditions of propagation of the radio waves are not under our control.

But randomness is the essence of communication. Communication theory involves the assumption that the transmitter is connected to a source, whose output, the receiver is not able to predict with certainty. If the students know ahead of time what is the teacher (source + transmitter) is going to say (and what jokes he is going to crack), then there is no need for the students (the receivers) to attend the class!

Although less obvious, it is also true that there is no communication problem unless the transmitted signal is disturbed during propagation or reception by unwanted (random) signals, usually termed as noise and interference. (We shall take up the statistical characterization of noise in Chapter 3.)

However, quite a few random signals, though their exact behavior is unpredictable, do exhibit statistical regularity. Consider again the reception of radio signals propagating through the atmosphere. Though it would be difficult to know the exact value of the voltage at the terminals of the receiving antenna at any given instant, we do find that the average values of the antenna output over two successive one minute intervals do not differ significantly. If the conditions of propagation do not change very much, it would be true of any two averages (over one minute) even if they are well spaced out in time. Consider even a simpler experiment, namely, that of tossing an unbiased coin (by a person without any magical powers). It is true that we do not know in advance whether the outcome on a particular toss would be a head or tail (otherwise, we stop tossing the coin at the start of a cricket match!). But, we know for sure that in a long sequence of tosses, about half of the outcomes would be heads (If this does not happen, we suspect either the coin or tosser (or both!)).

2.2
Indian Institute of Technology Madras

Principles of Communication

Prof. V. Venkata Rao

Statistical

regularity

of

averages

is

an

experimentally

verifiable

phenomenon in many cases involving random quantities. Hence, we are tempted to develop mathematical tools for the analysis and quantitative characterization of random signals. To be able to analyze random signals, we need to understand random variables. The resulting mathematical topics are: probability theory, random variables and random (stochastic) processes. In this chapter, we shall develop the probabilistic characterization of random variables. In chapter 3, we shall extend these concepts to the characterization of random processes.

2.2 Basics of Probability
We shall introduce some of the basic concepts of probability theory by defining some terminology relating to random experiments (i.e., experiments whose outcomes are not predictable).

2.2.1. Terminology
Def. 2.1: Outcome

The end result of an experiment. For example, if the experiment consists of throwing a die, the outcome would be anyone of the six faces, F1 ,........, F6
Def. 2.2: Random experiment

An experiment whose outcomes are not known in advance. (e.g. tossing a coin, throwing a die, measuring the noise voltage at the terminals of a resistor etc.)
Def. 2.3: Random event

A random event is an outcome or set of outcomes of a random experiment that share a common attribute. For example, considering the experiment of throwing a die, an event could be the 'face F1 ' or 'even indexed faces' ( F2 , F4 , F6 ). We denote the events by upper case letters such as A , B or
A1 , A2 ⋅ ⋅ ⋅ ⋅

2.3
Indian Institute of Technology Madras

Principles of Communication

Prof. V. Venkata Rao

Def. 2.4: Sample space

The sample space of a random experiment is a mathematical abstraction used to represent all possible outcomes of the experiment. We denote the sample space by

S.
S

Each outcome of the experiment is represented by a point in

and is

called a sample point. We use s (with or without a subscript), to denote a sample point. An event on the sample space is represented by an appropriate collection of sample point(s).
Def. 2.5: Mutually exclusive (disjoint) events

Two events A and B are said to be mutually exclusive if they have no common elements (or outcomes).Hence if A and B are mutually exclusive, they cannot occur together.
Def. 2.6: Union of events

The union of two events A and B , denoted A ∪ B , {also written as

(A

+ B ) or ( A or B )} is the set of all outcomes which belong to A or B or both.

This concept can be generalized to the union of more than two events.
Def. 2.7: Intersection of events

The intersection of two events, A and B , is the set of all outcomes which belong to A as well as B . The intersection of A and B is denoted by ( A ∩ B ) or simply ( A B ) . The intersection of A and B is also referred to as a joint event

A and B . This concept can be generalized to the case of intersection of three or
more events.
Def. 2.8: Occurrence of an event

An event A of a random experiment is said to have occurred if the experiment terminates in an outcome that belongs to A .

2.4
Indian Institute of Technology Madras

and ii) the classical definition.1) ⎛n ⎞ For small values of n . then the probability of A . But ⎝ n ⎠ ⎛n ⎞ as n becomes larger and larger.5 Indian Institute of Technology Madras . 2. 2.2 Probability of an Event The probability of an event has been defined in several ways.2. then A B = φ and vice versa). Thus φ = (note that if A and B are disjoint events. Def. let the experiment be that of tossing a coin and A the event ⎛n ⎞ 'outcome of a toss is Head'.9: Complement of an event The complement of an event A . ⎝ ⎠ (2. ⎜ A ⎟ may not deviate from ⎝ n ⎠ 2. For example. If the event A occurs nA times.11: The relative frequency definition: Suppose that a random experiment is repeated n times. is defined as ⎛n ⎞ P ( A ) = lim ⎜ A ⎟ n → ∞ ⎝ n ⎠ ⎛ nA ⎞ ⎜ n ⎟ represents the fraction of occurrence of A in n trials. V. Venkata Rao Def. Two of the most popular definitions are: i) the relative frequency definition. denoted by P ( A ) . If n is the order of 100.10: Null event The null event. denoted φ . it is likely that ⎜ A ⎟ will fluctuate quite badly. is an event with no sample points. S 2. ⎜ A ⎟ to tend to a definite limiting ⎝ n ⎠ value. Def. denoted by A is the event containing all points in S but not in A . 2.Principles of Communication Prof. we expect.

2) where it is assumed that all outcomes are equally likely! Whatever may the definition of probability. 2.. An be random events such that: a) Ai A j = φ ..6 Indian Institute of Technology Madras .. then P ( A + B ) = P ( A ) + P ( B ) − P ( A B ) (2. to denote the union of A and B and to denote the addition of two real numbers). This is done by counting the total number N of the possible outcomes of the experiment.. V.12: The classical definition: The relative frequency definition given above has empirical flavor. the symbol + is used to mean two different things. then P ( A) = NA N (2. the probability of the event A is found without experimentation.3b) (2. 2.. If N A of those outcomes are favorable to the occurrence of the event A . then P ( A + B ) = P ( A ) + P ( B ) (2.3(c). In the classical approach.. 2. we 2 1 ⎛n ⎞ expect ⎜ A ⎟ to converge to .3.. Using Eq.Principles of Communication Prof.5a) 2.4) ii) Let A1 . for i ≠ j and (2. say ten percent and as n becomes larger and larger.3c) (Note that in Eq.3a) (2. it is possible for us to derive some additional relationships: i) If A B ≠ φ . we require the probability measure (to the various events on the sample space) to obey the following postulates or axioms: P1) P ( A ) ≥ 0 P2) P ( S ) = 1 P3) ( AB ) = φ . A2 . namely. Venkata Rao 1 by more than. 2 ⎝ n ⎠ Def.

.. Note: A1 . This relation can be arrived at by using either the relative frequency definition of probability or the classical definition.5b). V.5a) and exhaustive (Eq. 2. In a real world random experiment.6) Then.. let a bowl contain 3 resistors and 1 capacitor. it is quite likely that the occurrence of the event B is very much influenced by the occurrence of the event A . we have ⎛ nAB ⎞ P ( AB ) = lim ⎜ ⎟ n → ∞ ⎝ n ⎠ ⎛n ⎞ P ( A ) = lim ⎜ A ⎟ n → ∞ ⎝ n ⎠ 2.. + An = S. The occurrence of the event 'the capacitor on the second draw' is very much dependent on what has been drawn at the first instant. (2..... Such dependencies between the events is brought out using the notion of conditional probability . An are said to be mutually exclusive (Eq. Using the former. 2. 2. it represents the probability of B occurring. To give a simple example.7 Indian Institute of Technology Madras .7) A very useful concept in probability theory is that of conditional probability. 2. (2.4. iii) P ( A ) = 1 − P ( A ) The derivation of Eq. denoted P ( B | A ) . Venkata Rao b) A1 + A2 + ...7 is left as an exercise. given that A has occurred.5b) (2. P ( A ) = P ( A A1 ) + P ( A A2 ) + . A2 .Principles of Communication Prof. ⋅ ⋅ ⋅. + P ( A An ) where A is any event on the sample space.6 and 2. The conditional probability P ( B | A ) can be written in terms of the joint probability P ( A B ) and the probability of the event P ( A ) .

For example P ( A BC ) can be written as 2. As P ( B | A ) refers to the probability of B occurring.8(a) as P (B | A) = Similarly P (B ) P ( A | B ) P ( A) . given that A has occurred. Eq.13: Conditional Probability P ( B | A ) = lim nAB nA ⎞ ⎟ P ( AB ) . P ( A) ≠ 0 (2. say P ( B | A ) and the (unconditional) probability P ( A ) . 2.8a) P ( A B ) = P (B | A) P ( A) Interchanging the role of A and B . P (B ) ≠ 0 P (B ) (2.10(a) or 2.9) In view of Eq.8 Indian Institute of Technology Madras .Principles of Communication Prof.10(b) is one form of Bayes’ rule or Bayes’ theorem. Venkata Rao where nAB is the number of times AB occurs in n repetitions of the experiment. we have P ( A | B) = P ( AB ) P (B ) .9. P ( A) ≠ 0 ⎟ = P ( A) ⎟ ⎟ ⎠ n → ∞ ⎛ nAB ⎜ = lim ⎜ n n → ∞ ⎜ nA ⎜ n ⎝ or (2.9 expresses the probability of joint event AB in terms of conditional probability. V. we can also write Eq. 2.10b) Eq. 2. P (B ) ≠ 0 (2.8(b) can be written as P ( A B ) = P (B | A) P ( A) = P (B ) P ( A | B ) (2. 2. Similar relation can be derived for the joint probability of a joint event involving the intersection of three or more events.10a) P ( A | B) = P ( A) P (B | A) .8(a) and 2. we have Def 2.8b) Eq. 2.

V.14) Either Eq..12 represents another form of Bayes’ theorem.. 2. Venkata Rao P ( A BC ) = P ( A B ) P (C | AB ) = P ( A ) P ( B | A ) P (C | AB ) (2.1 Let A1 . A set of k events A1 . A2 . 2.13) That is.. knowledge of occurrence of A tells no more about the probability of occurrence B than we knew without that knowledge. the events A and B are said to be statistically independent. 2. then P ( A B ) = 0 . B . whereas if they are disjoint. Note that if A and B are independent. Thus three events A . Another useful probabilistic concept is that of statistical independence.13.12) Eq.9 Indian Institute of Technology Madras .13 or 2. Alternatively.14 can be used to define the statistical independence of two events. C are independent when 2. if Eq. then P ( A B ) = P ( A) P (B ) A and B satisfy the (2.... The notion of statistical independence can be generalized to the case of more than two events. An be n mutually exclusive and exhaustive events and B is another event defined on the same space..... then P ( A B ) = P ( A ) P ( B ) . Suppose the events A and B are such that P (B | A) = P (B ) (2. Then. Ak are said to be statistically independent if and only if (iff) the probability of every intersection of k or fewer events equal the product of the probabilities of its constituents.. A2 .. Show that P Aj | B = ( ) i = 1 ) ( ) ∑ P (B | Aj ) P ( Aj ) P B | Aj P Aj n ( (2..11) Exercise 2..Principles of Communication Prof.

1(a). Also arrival of one person is independent of the other. V.m. a) The sample space is the rectangle. They agree to meet at the office of registrar of marriages at 11:30 a. c) Find the probability that the lovebirds will get married and (hopefully) will live happily ever after. 2. 2. Venkata Rao P ( A B ) = P ( A) P (B ) P ( AC ) = P ( A ) P (C ) P ( BC ) = P ( B ) P (C ) and P ( A BC ) = P ( A ) P ( B ) P (C ) We shall illustrate some of the concepts introduced above with the help of two examples. However. Mark this event on the sample space. before leaving in a huff never to meet again. a) b) Picture the sample space Let the event A stand for “P1 and P2 meet”. shown in Fig. much against the wishes of the parents on both the sides.1 Priya (P1) and Prasanna (P2). Example 2. both are also very short tempered and will wait only 10 min. on the ensuing Friday (looks like they are not aware of Rahu Kalam or they don’t care about it). Unfortunately.Principles of Communication Prof. both are somewhat lacking in punctuality and their arrival times are equally likely to be anywhere in the interval 11 to 12 hrs on that day.10 Indian Institute of Technology Madras . after seeing each other for some time (and after a few tiffs) decide to get married.

1(b). 2. Venkata Rao Fig. is shown in Fig.1 2.11 Indian Institute of Technology Madras . meeting between P1 and P2 would take place if P2 arrives within the interval a to b . Assuming that P1 arrives at 11: x . as shown in the figure.Principles of Communication Prof. The event A . 2.1(b): The event A of Example 2. V. Fig. 2.1(a): S of Example 2. indicating the possibility of P1 and P2 meeting.1 b) The diagonal OP represents the simultaneous arrival of Priya and Prasanna.

be tossed together. Venkata Rao c) Probability of marriage = Shaded area 11 = Total area 36 Example 2. The event A comprises of the outcomes ‘ T1 T2 . T1 H2 and 4 P ( A) = 3 4 1 1 P (B | A) = 4 = 3 3 4 Intuitively. given 'not H1 H2 ’ the 3 toss would have resulted in anyone of the three other outcomes which can be 2. i.) We shall treat that all these outcomes are equally likely. Are A and B independent? We know that P ( B | A ) = P ( AB ) P ( A) . H1 H2 ). marked 1 and 2. V.12 Indian Institute of Technology Madras . (Match comprises the two outcomes T1 T2 . H1 H2 . therefore. (Treating each of these outcomes as an event. Hence P ( A B ) = H1 T2 ’. the result P ( B | A ) = 1 is satisfying because.2: Let two honest coins. it represents the outcome T1 T2 . Find P ( B | A ) .e. we find that these events 4 are mutually exclusive and exhaustive). 1 . Let the event A be 'not H1 H2 ' and B be the event 'match'. A B is the event 'not H1 H2 ' and 'match'. The four possible outcomes are T1 T2 . T1 H2 . ( T1 indicates toss of coin 1 resulting in tails. that is the probability of occurrence of any of these four outcomes is 1 . similarly T2 etc. H1 T2 .Principles of Communication Prof..

1 Distribution function: Taking a specific case. a2 ] . let the random experiment be that of throwing a die. V. as shown in Fig. which falls on the real line segment [a1 . that is. whose domain is the sample space (of a random experiment) and whose range is in the real line. has a probability of As P ( B ) = 1 . to each si ∈ S .3 Random Variables Let us introduce a transformation or function. 2. A and B are dependent events.Principles of Communication Prof. The figure shows the transformation of a few individual sample points as well as the transformation of the event A . 2. namely 'not H1 H2 '. that is. Venkata Rao treated to be equally likely.2: A mapping X ( ) from S to the real line. The six faces of the die can be treated as the six sample points in S .2. Fig. X assigns a real number. 2.13 Indian Institute of Technology Madras . 2 3 2. 3 1 . This implies that the outcome T1 T2 given 3 1 1 and P ( B | A ) = . X ( si ) .3. say X .2.

comprising all those sample points which are transformed by X into real numbers less than or equal to x . If X ( si ) = i − 1. Let X ( si ) = i .3. FX ( whose domain is the real line and whose range is the interval [0. for convenience. 4 each with sample point representing an event with the probabilities P ( s1 ) = P ( s2 ) = 1 1 1 . (Note that. 4 . s1 to s4 . 2. with 1 . 2. 2. i = 1. . then the events on the sample space will become transformed into appropriate segments of the real line.14 Indian Institute of Technology Madras .. it could be any ) is a function other symbol and we may use FX ( α ) . we use x as the argument of FX ( ) .. let S consist of four sample points. As an example of a distribution function (also called Cumulative Distribution Function CDF). then 8 8 2 the distribution function FX ( x ) .) Evidently. Then we can enquire into the probabilities such as P ⎡{s : X ( s ) < a1}⎤ ⎣ ⎦ P ⎡{s : b1 < X ( s ) ≤ b2 }⎤ ⎣ ⎦ or P ⎡{s : X ( s ) = c}⎤ ⎣ ⎦ These and other probabilities can be arrived at. i = 1.15) That is. 1] ).Principles of Communication Prof. Venkata Rao Fi = si . P ( s3 ) = and P ( s4 ) = .. will be as shown in Fig. Once the transformation is induced. FX ( a1 ) etc.5.. V. 2. FX ( x ) is the probability of the event. 6 . 3. But. .. provided we know the Distribution Function of X. denoted by FX ( FX ( x ) = P ⎡{s : X ( s ) ≤ x}⎤ ⎣ ⎦ ) which is given by (2.

5 whereas FX ( − 0. then ⎡FX ( a ) − FX ( b ) ⎤ = P ⎡{s : b < X ( s ) ≤ a}⎤ ⎣ ⎦ ⎣ ⎦ If a > b . In general.Principles of Communication Prof. 2. then FX ( a ) ≥ FX ( b ) The first three properties follow from the fact that FX ( ) represents the probability and P ( S ) = 1. Venkata Rao Fig.16) 2. Properties iv) and v) follow from the fact {s : X ( s ) ≤ b} ∪ {s : b < X ( s ) ≤ a} = {s : X ( s ) ≤ a} Referring to the Fig. V.3: An example of a distribution function FX ( ) satisfies the following properties: i) ii) iii) iv) v) FX ( x ) ≥ 0.5 .15 Indian Institute of Technology Madras . there is a discontinuity in FX of magnitude Pa at a point x = a . In other words. − ∞ < x < ∞ FX ( − ∞ ) = 0 FX ( ∞ ) = 1 If a > b . if and only if P ⎡{s : X ( s ) = a}⎤ = Pa ⎣ ⎦ (2.3. there is a discontinuity of 4 4 x = − 0. note that FX ( x ) = 0 for x < − 0. 2.5 ) = 1 1 at the point .

2. s5 } and C = {s6 } and their probabilities are P ( A ) = 2 1 1 . Hence. Functions such as X ( ) for which distribution functions exist are called Random Variables (RV).5 ] = P ( s4 ) is not known as s4 is not an event on the sample space. every transformation from S into the real line need not be a random variable. where a + = ∆ → 0 lim (a + ∆ ) That is.) However. F X { } ( a+ ) − FX ( a ) = 0 . s1 to s6 . lim P ⎡a < X ( s ) ≤ a + ∆ ⎤ = lim ⎡FX ( a + ∆ ) − FX ( a ) ⎤ ⎣ ⎦ ⎣ ⎦ ∆ → 0 ∆ → 0 We intuitively feel that as ∆ → 0 . for any real x . Then the distribution function fails to exist because P [s : 3. Consider. (The term “random variable” is somewhat misleading because an RV is a well defined function from S into the real line. B = {s3 . The only events that have been identified on the sample space are: A = {s1 . In other words. the limit of the set s : a < X ( s ) ≤ a + ∆ is the null set and can be proved to be so. V. Let the transformation X be X ( si ) = i . B and C can be obtained. F X ( x ) is continuous to the right. is continuous to the right at each point x 1. and has a step of size Pa at point a if and if Eq. s2 }. let S consist of six sample points. For example. Venkata Rao The properties of the distribution function are summarized by saying that Fx ( TP PT ) is monotonically non-decreasing. s4 . 2. {s : X ( s ) ≤ x} should be an event in the sample space with some assigned probability.5 < x ≤ 4.16 is satisfied. 1 TP PT Let x = a . P ( B ) = and P (C ) = .Principles of Communication Prof.16 Indian Institute of Technology Madras . with ∆ > 0 . We see that the probabilities for the 6 2 6 various unions and intersections of A .

5 ⎪3 . 2. s2 } .3. 2. Sketch FY ( y ) . s4 . that is fX ( x ) = or FX ( x ) = d FX ( x ) dx (2. 2 ⎪ = ⎨2 . The PDF. s5 } and C = {s6 } with P ( A ) = 1 1 1 . namely. 2. A = {s1 . Venkata Rao Exercise 2. i = 1. V. i = 3. 3 2 6 Let Y ( ) be the transformation.17 Indian Institute of Technology Madras . f X ( x ) is defined as the derivative of the CDF.17b) The distribution function may fail to have a continuous derivative at a point x = a for one of the two reasons: i) ii) the slope of the Fx ( x ) is discontinuous at x = a Fx ( x ) has a step discontinuity at x = a The situation is illustrated in Fig. s1 to s6 . B = {s3 . P ( B ) = and P (C ) = . ⎧1.2 Probability density function Though the CDF is a very fundamental concept. The events on S are the same as above.2 Let identified S be a sample space with six sample points. 4. i = 6 ⎩ Y ( si ) Show that Y ( ) is a random variable by finding FY ( y ) . in practice we find it more convenient to deal with Probability Density Function (PDF).Principles of Communication Prof.4.17a) −∞ ∫ f (α) d α X x (2.

18) In Eq. for the CDF shown in Fig. Venkata Rao Fig. In the first case.4: A CDF without a continuous derivative As can be seen from the figure. (Note that FX ( x ) is continuous to the right. 2. fX ( x ) = 1 ⎛ 1⎞ 1 ⎛ 1⎞ 1 ⎛ 3⎞ 1 ⎛ 5⎞ δ⎜ x + ⎟ + δ⎜ x − ⎟ + δ⎜ x − ⎟ + δ⎜ x − ⎟ 4 ⎝ 2⎠ 8 ⎝ 2⎠ 8 ⎝ 2⎠ 2 ⎝ 2⎠ (2. f X ( x ) has an impulse of weight 1 8 at x = 1 2 as 1⎤ 1 ⎡ P ⎢ X = ⎥ = . ε ⎝ε⎠ 1 2 lim 1 −ε 2 ∫ f X ( x ) dx = lim ε → 0 1 2 ∫ 1 ⎛ 1⎞ 1 δ ⎜ x − ⎟ dx ≠ 8 ⎝ 2⎠ 16 −ε 2. FX ( x ) has a discontinuous slope at x = 1 and a step discontinuity at x = 2 .Principles of Communication Prof. we resolve the ambiguity by taking f X to be a derivative on the right. V. 2. we include an impulse Pa δ ( x − a ) in the PDF. 2.3. For example.) The second case is taken care of by introducing the impulse in the probability domain.18 Indian Institute of Technology Madras . That is. This impulse function cannot be taken as the limiting case of 2⎦ 8 ⎣ an even function (such as 1 2 ε → 0 1 ⎛x⎞ ga ⎜ ⎟ ) because.18. if there is a discontinuity in FX at x = a of magnitude Pa . the PDF will be.

given by FX . 1 ≤ x < 3 ⎪8 2 2 ⎩ Such an impulse is referred to as the left-sided delta function.19a) = 1 −∞ ∫ f ( x ) dx X (2. 8 1 1 ⎧2 ⎪8 . We say that X is a mixed random variable if FX ( x ) is discontinuous but not a staircase. is a continuous function of x for all x .20) As the domain of the random variable X ( ) is known. y ) = P ⎡{s : X ( s ) ≤ x .3. say X and Y . a random variable can be classified as: i) continuous (ii) discrete and (iii) mixed. lim ε → 0 1 −ε 2 ∫ f X ( x ) dx = 1 . then X 1 is a continuous random variable. − 2 ≤ x < 2 ⎪ FX ( x ) = ⎨ ⎪3 .19 Indian Institute of Technology Madras . FX ( x ) .3 Joint distribution and density functions Consider the case of two random variables. it is convenient to denote the variable simply by X . we have i) ii) fX ( x ) ≥ 0 ∞ (2. 2.19b) Based on the behavior of CDF. As FX is non-decreasing and FX ( ∞ ) = 1. Later on in this lesson.Y ( x . Venkata Rao 1 2 However. V. They can be characterized by the (two-dimensional) joint distribution function. If we induce k such transformations. we shall discuss some examples of these three types of variables. then X corresponds to a discrete variable. We can induce more than one transformation on a given sample space. then we have a set of k co-existing random variables. If FX ( x ) is a TP PT staircase. If the CDF. Y ( s ) ≤ y }⎤ ⎣ ⎦ 1 TP PT (2.Principles of Communication Prof. 2. This ensures.

y1 ) is the probability associated with the set of all sample points whose transformation does not fall outside the shaded region in the two dimensional (Euclidean) space shown in Fig.Y ( x1 . ∞ ) = 1 FX .Y ( x .Y ( ∞ . 2. under Y . let A be the set of all those sample S such that X ( s ) ≤ x1 . Properties of the two dimensional distribution function are: i) ii) iii) iv) v) FX . y ) ≥ 0 . y1 ) Looking at the sample space points s ∈ S. V. 2. then F ( x1 .Y ( x . −∞ < x < ∞. y ) is the probability associated with the set of all those sample points such that under X .Y ( x .5: Space of { X ( s ) .Y ( x . Venkata Rao That is.5. Similarly. y ) = FY ( y ) FX . ∞ ) = FX ( x ) 2. FX .Principles of Communication Prof. − ∞ ) = 0 FX . if B is comprised of all those S such that Y ( s ) ≤ y1 . Fig.Y ( x1 . Y ( s )} corresponding to FX .20 Indian Institute of Technology Madras . −∞ < y < ∞ FX . y ) = FX . FX .Y ( − ∞ . their transformed values will be less than or equal to x and at the same time. In other words.Y ( ∞ . y1 ) is the probability sample points s ∈ associated with the event AB . the transformed values will be less than or equal to y .

Given the joint PDF of random variables X and Y .Y ( x2 .22) Eq. f X ( x1 ) = fX ( x ) = d FX ( x ) dx ∞ ∞ = x = x1 −∞ ∫ f ( x . y 2 ) ≥ FX . FX ( x1 ) = x1 − ∞− ∞ ∫ ∫ f ( α .Principles of Communication Prof. Hence. y ) dx X .Y ( x . β ) dβ X .21 Indian Institute of Technology Madras . β ) d α dβ X . ∞ ) = FX ( x1 ) .Y y x (2.22 involves the derivative of an integral.21a) or FX .Y ( x1 .Y ( x1 . where k ≥ 3 . y1 ) We define the two dimensional joint PDF as f X .Y 1 or −∞ ∫ f ( x . 2. the statistics of any individual variable is called the marginal.Y ( x . β) d β d α X . V. it is possible to obtain the one dimensional PDFs.Y ∞ f X ( x1 ) = x ∞ ⎫ ⎤ d ⎧ 1⎡ ⎪ ⎪ ⎨ ∫ ⎢ ∫ f X .Y (2. y ) = − ∞− ∞ ∫ ∫ f ( α .Y (2. y1 ) ≥ FX . Hence it is common to refer FX ( x ) as the marginal 2.Y ( x2 . f X ( x ) and fY ( y ) .Y ( α .21b) The notion of joint CDF and joint PDF can be extended to the case of k random variables. y ) dy X . We know that. y ) ∂x ∂y (2. FX . fY ( y ) = ∞ −∞ ∫ f ( x . then FX . y ) = ∂2 FX . That is.23b) (In the study of several random variables. Venkata Rao vi) If x2 > x1 and y 2 > y1 .Y ( x .23a) Similarly. β ) d β⎥ d α ⎬ d x1 ⎪− ∞ ⎢ − ∞ ⎥ ⎪ ⎦ ⎩ ⎣ ⎭ (2.

we have the pair of equations.Principles of Communication Prof. That is. An analogous relation can be defined for fY | X ( y | x ) .25c) (2. We might also be interested in the PDF of X given a specific value of y = y1 .22 Indian Institute of Technology Madras .26b) 2.Y ( x . we might as well drop the subscript on y and denote it by f X |Y ( x | y ) .Y ( x . V.4 Conditional density Given f X . but fixed.25b) (2. Accordingly. y ) fX ( x ) (2.25d) f X . FY ( y ) and fY ( y ) are similarly referred to. y ) fY ( y ) f X . Venkata Rao distribution function of X and f X ( x ) as the marginal density function.Y ( x . given y = y1 . y ) .24) where it is assumed that fY ( y1 ) ≠ 0 .) 2. f X |Y ( x | y ) = fY | X ( y | x ) = f X . we know how to obtain f X ( x ) or fY ( y ) . This is called the conditional PDF of X . Once we understand the meaning of conditional PDF.Y ( x . denoted f X |Y ( x | y1 ) and defined as f X |Y ( x | y1 ) = f X .Y ( x .25a) and or (2. that is. y ) = f X |Y ( x | y ) fY ( y ) = fY | X ( y | x ) f X ( x ) The function f X |Y ( x | y ) may be thought of as a function of the variable x with variable y arbitrary. f X |Y ( x | y ) ≥ 0 ∞ (2.3.26a) = 1 and −∞ ∫ f ( x | y ) dx X |Y (2. it satisfies all the requirements of an ordinary PDF. y1 ) fY ( y1 ) (2.

Principles of Communication Prof. x < 0 ⎪ FX ( x ) = ⎨ K x 2 .Y . We shall now give a few examples to illustrate the concepts discussed in sec.. fY | X ( y | x ) = fY ( y ) Eq.25(c).30(b) are alternate expressions for the statistical independence between X and Y .Y ( x . 2... the definition of statistical independence is somewhat simpler than in the case of events.. Let X and Y be independent. . y ) = f X ( x ) fY ( y ) (2. iff f X .Y ( x . Z ( x .28) and three random variables X ..3..5 Statistical independence In the case of random variables. iff.3 A random variable X has ⎧ 0 . Venkata Rao 2. the k -dimensional joint PDF factors into the product i = 1 ∏ f X i ( xi ) k (2. Y . 0 ≤ x ≤ 10 ⎪100 K . two random variables X and Y are statistically independent. X k statistically independent iff..29) Statistical independence can also be expressed in terms of conditional PDF. f X . X 2 . x > 10 ⎩ i) ii) Find the constant K Evaluate P ( X ≤ 5 ) and P ( 5 < X ≤ 7 ) 2.27) Hence.23 Indian Institute of Technology Madras .30b) Similarly.3. 2. y . z ) = f X ( x ) fY ( y ) fz ( z ) (2. Example 2.30a) (2. y ) = f X ( x ) fY ( y ) Making use of Eq. Z are independent. . 2. f X . we have f X |Y ( x | y ) fY ( y ) = f X ( x ) fY ( y ) or f X |Y ( x | y ) = f X ( x ) (2.30(a) and 2.. V. We call k random variables X1 . Then.

⎩ x < 0 0 ≤ x ≤ 10 x > 10 Note: Specification of f X ( x ) or FX ( x ) is complete only when the algebraic expression as well as the range of X is given. Venkata Rao iii) What is f X ( x ) ? i) ii) FX ( ∞ ) = 100 K = 1 ⇒ K = 1 .4 Consider the random variable X defined by the PDF fX ( x ) = a e −bx . As can be seen f X ( x ) ≥ 0 for − ∞ < x < ∞ . 0 ∞ ∞ 1 .24 Indian Institute of Technology Madras . ⎩ x < 0 x ≥ 0. ⎧0 ⎪ = ⎨0. 100 ⎛ 1 ⎞ P ( x ≤ 5 ) = FX ( 5 ) = ⎜ ⎟ × 25 = 0. we require That is.24 fX ( x ) = d FX ( x ) dx . ⎪ fX ( x ) = ⎨ − b x ⎪a e . Determine the corresponding FX ( x ) Find P [1 < X ≤ 2] . In order for f X ( x ) to ∞ i) represent a legitimate PDF.Principles of Communication Prof. 2 ii) the given PDF can be written as ⎧a e b x . i) ii) iii) Determine the relation between a and b so that f X ( x ) is a PDF. hence b = 2 a . − ∞ < x < ∞ where a and b are positive constants.25 ⎝ 100 ⎠ P ( 5 < X ≤ 7 ) = FX ( 7 ) − FX ( 5 ) = 0.02 x . Example 2. ∫ a e − b x dx = 0 −∞ ∫ ae −bx dx = 2 ∫ a e − b x dx = 1 . V. ⎪0 . 2.

2 x < 0 x ≥ 0 1 −b e − e− 2 b 2 ( ) Example 2.25 Indian Institute of Technology Madras . 2 1 − bx e . y ) fX ( x ) 2. y ) = ⎨ 2 ⎪0 . ∫ f X ( x ) d x = 2 ∞ −2 −∞ ∫ f (x) d x X Therefore. x > 0 2 We can now write the complete expression for the CDF as ⎧ ⎪ ⎪ FX ( x ) = ⎨ ⎪1 − ⎪ ⎩ iii) FX ( 2 ) − FX (1) = 1 bx e . 2 2 −∞ ∫ 2 x Consider x > 0 . Venkata Rao For x < 0 .Y ( x . V. FX ( x ) = 1 − 1 − bx e . Take a specific value of x = 2 FX ( 2 ) = −∞ ∫ f (x) d x X ∞ = 1 − ∫ fX ( x ) d x 2 But for the problem on hand.Principles of Communication Prof.5 ) Are X and Y independent? (a) fY | X ( y | x ) = f X .5 ⎧1 . 0 ≤ y ≤ 2 ⎪ Let f X .Y ( x . 0 ≤ x ≤ y. otherwise ⎩ Find (a) (b) i) fY | X ( y | 1) and ii) fY | X ( y | 1. we have FX ( x ) = b bα 1 e d α = eb x .

The maximum value taken by y is 2 . then fY | X ( y | x ) = fY ( y ) . 1.5 ) = 2 = ⎨ 1 ⎩0 . (i) ∫2 dy x 2 1 = 1− x . otherwise 4 the dependence between the random variables X and Y is evident from the statement of the problem because given a value of X = x1 . 1 ≤ y ≤ 2 ⎨ 1 ⎩0 . Y should be greater than or equal to x1 for the joint PDF to be non zero. fX ( x ) = Hence. 0 ≤ x ≤ 2 2 fY | X ( y | 1) = 1 2 = ⎧1 .Principles of Communication Prof. in addition.26 Indian Institute of Technology Madras . y ≥ x .5 ≤ y ≤ 2 fY | X ( y | 1. Venkata Rao f X ( x ) can be obtained from f X .Y by integrating out the unwanted variable y over the appropriate range. Also we see that fY | X ( y | x ) depends on x whereas if X and Y were to be independent. 2. V. Hence. otherwise 2 (ii) b) 1 ⎧2 . for any given x .

the following density functions have been given. 0 ≤ y ≤ 10 ⎪100 ⎪ fY ( y ) = ⎨ ⎪ 1 − y . detection.6) Fig. (Fig. Venkata Rao Exercise 2. filtering etc. We will now develop the necessary theory for the statistical characterization of the output random variables. y ) Show that ⎧ y .4 Transformation of Variables The process of communication involves various operations such as modulation.Y ( x . 2.27 Indian Institute of Technology Madras . given the transformation and the input random variables. V. 2. we typically generate new random variables from the given ones.3 For the two random variables X and Y .Principles of Communication Prof. 10 < y ≤ 20 ⎪ 5 100 ⎩ 2.6: PDFs for the exercise 2.3 Find a) b) f X . 2. In performing these operations.

. We see that the equation y 1 = g ( x ) has three roots namely. obtained from X by the transformation Y = g ( X ) . (2.7. Theorem 2. x2 and x3 . f X ( x ) . + g ' ( xn ) f X ( xn ) + . 2.. x1 ....... V.1 Functions of one random variable Assume that X is the given random variable of the continuous type with the PDF...... . Denoting its real roots by xn . 2. Let Y be the new random variable. for the specific y .31) where g ' ( x ) is the derivative of g ( x ) .Principles of Communication Prof..28 Indian Institute of Technology Madras .4. This can be obtained with the help of the following theorem (Thm... Proof: Consider the transformation shown in Fig.. y = g ( x1 ) = .1 Let Y = g ( X ) .. By this we mean the number Y ( s1 ) associated with any sample point s1 is Y ( s1 ) = g ( X ( s1 ) ) Our interest is to obtain fY ( y ) . solve the equation y = g ( x ) ......1). 2.. we will show that fY ( y ) = g ' ( x1 ) f X ( x1 ) + ... Venkata Rao 2.... To find fY ( y ) ... = g ( xn ) = . .

1 We know that fY ( y ) d y = P [ y < Y ≤ y + d y ] . 2. this set consists of the following intervals: x1 < x ≤ x1 + d x1 . we need to find the set of values x such that y1 < g ( x ) ≤ y1 + d y and the probability that X is in this set. it follows that P [ y1 < Y ≤ y1 + d y ] = P [ x1 < X ≤ x1 + d x1 ] + P [ x2 + dx2 < X ≤ x2 ] + P [ x3 < X ≤ x 3 + d x3 ] This probability is the shaded area in Fig. V. As we see from the figure.7: X − Y transformation used in the proof of theorem 2. but d x2 < 0 . From the above. x2 + d x2 < x ≤ x2 . d x 3 = dy g ' ( x2 ) dy g ' ( x3 ) 2. P [ x1 < X ≤ x1 + d x1 ] = f X ( x1 ) d x1 .7. 2.Principles of Communication Prof. for any given y 1 . x3 < x ≤ x3 + d x3 where d x1 > 0 . d x3 > 0 .29 Indian Institute of Technology Madras . d x1 = dy g ' ( x1 ) P [ x2 + d x2 < x ≤ x2 ] = f X ( x2 ) d x2 . d x2 = P [ x3 < X ≤ x3 + d x 3 ] = f X ( x 3 ) d x3 . Venkata Rao Fig. Therefore.

We have g ' ( x ) = 1 and x = y − a . elsewhere ⎩ and a = − 1 Then.Principles of Communication Prof. fY ( y ) = 1 − y + 1 As y = x − 1. where a is a constant. V. and x ranges from the interval ( − 1. x ≤ 1 ⎪ Let f X ( x ) = ⎨ ⎪ 0 . 0 ) . by canceling d y from the Eq. 2. fY ( y ) d y = g '(x) fX ( x ) d y and as g ' ( x ) = 1. then we have P [Y = y1 ] = P ( x0 < X ≤ x1 ) = FX ( x1 ) − FX ( x0 ) .32) and Eq. we have fY ( y ) = f X ( y − a ) ⎧1 − x . Note that if g ( x ) = y1 = constant for every x in the interval ( x0 . We shall take up a few examples.30 Indian Institute of Technology Madras .31 follows. 1) .6 Y = g ( X ) = X + a . that is FY ( y ) is discontinuous at y = y1 . For a given y . 2. x1 ) . Hence fY ( y ) contains an impulse. Venkata Rao We conclude that fY ( y ) d y = g ' ( x1 ) f X ( x1 ) dy + g ' ( x2 ) f X ( x2 ) dy + g ' ( x3 ) f X ( x3 ) dy (2. 2. δ ( y − y1 ) of area FX ( x1 ) − FX ( x0 ) . Let us find fY ( y ) . Example 2. there is a unique x satisfying the above transformation. we have the range of y as ( − 2.32. Hence.

otherwise ⎩ and b = − 2 . 2. we have X = 1 Y .8.6 As can be seen. Again for a given y . 0 ≤ x ≤ 2 Let f X ( x ) = ⎨ 2 ⎪ 0 . there is a unique b x . Let us find fY ( y ) . we have fY ( y ) = x ⎧ ⎪1 − . Fig. Example 2. the transformation of adding a constant to the given variable simply results in the translation of its PDF. V. where b is a constant. b ⎝b⎠ 2.8: FX ( x ) and FY ( y ) of example 2. Venkata Rao ⎧1 − y + 1 . Solving for X .Principles of Communication Prof. − 2 ≤ y ≤ 0 ⎪ Hence fY ( y ) = ⎨ 0 .7 Let Y = b X . then 1 ⎛y⎞ fX ⎜ ⎟ . 2.31 Indian Institute of Technology Madras . As g ' ( x ) = b . elsewhere ⎪ ⎩ FX ( x ) and FY ( y ) are shown in Fig.

2. compute and sketch fY ( y ) for a = − 2 . − 4 ≤ y ≤ 0 ⎣ ⎦ 0 .2. where a and b are constants.8 Y = a X 2 . V. otherwise f X ( x ) and fY ( y ) are sketched in Fig. Example 2.4 Let Y = a X + b . and b = 1. 2.Principles of Communication Prof. Venkata Rao ⎧1 ⎪ fY ( y ) = ⎨ 2 ⎪ ⎩ y⎤ ⎡ ⎢1 + 4 ⎥ . a ⎝ a ⎠ If f X ( x ) is as shown in Fig.9: f X ( x ) and fY ( y ) of example 2. Let us find fY ( y ) . a > 0 . Fig. 2.7 Exercise 2.8.32 Indian Institute of Technology Madras .9. Show that fY ( y ) = 1 ⎛y − b⎞ fX ⎜ ⎟.

If y < 0 . otherwise ⎩ Sketching of f X ( x ) and fY ( y ) is left as an exercise.31 yields a y and a fY ( y ) ⎧ 1 ⎡ ⎛ y⎞ ⎛ ⎪ ⎢f X ⎜ ⎟ + fX ⎜ − ⎜ a⎟ ⎜ ⎪ ⎠ ⎝ ⎣ = ⎨ 2a y ⎢ ⎝ a ⎪ ⎪ 0 ⎩ y ⎞⎤ ⎟⎥ . If y ≥ 0 . x1 = x2 = − y .Principles of Communication Prof. −∞ < x < ∞ ⎜ 2 ⎟ ⎟ 2π ⎝ ⎠ 1 (Note that exp ( α ) is the same as e α ) Then fY ( y ) = 1 2 y ⎡ ⎛ y⎞ ⎛ y ⎞⎤ ⎢ exp ⎜ − 2 ⎟ + exp ⎜ − 2 ⎟ ⎥ 2π ⎣ ⎝ ⎠ ⎝ ⎠⎦ ⎧ 1 ⎛ y⎞ exp ⎜ − ⎟ . otherwise ⎩ We shall compute and sketch fY ( y ) . then the equation y = a x 2 has no real solution. Example 2. y ≥ 0 a ⎟⎥ ⎠⎦ . − < x < Let f X ( x ) = ⎨ 2 2 2 ⎪ 0 . X ≤ 0 Y =⎨ ⎩X . otherwise Let a = 1 . Hence fY ( y ) = 0 for y < 0 . b) 2. X > 0 a) Let us find the general expression for fY ( y ) 1 3 ⎧1 ⎪ . Venkata Rao g ' ( x ) = 2 a x . 2. V. and Eq. and f X ( x ) = ⎛ x2 ⎞ exp ⎜ − . then it has two solutions.33 Indian Institute of Technology Madras .9 Consider the half wave rectifier transformation given by ⎧0 . y ≥ 0 ⎪ = ⎨ 2π y ⎝ 2⎠ ⎪ 0 .

As Y is nonnegative. Fig. let f X ( x ) = ⎨ 2 2 2 ⎪0 . fY ( y ) f X ( x ) and fY ( y ) are sketched in Fig. otherwise ⎪0 ⎪ ⎩ Then. there is an impulse in fY ( y ) at y = 0 whose area is equal to FX ( 0 ) . As Y = X for x > 0 . V.10. elsewhere ⎩ ⎧1 ⎪4 δ(y ) . otherwise ⎩ ⎧1 . − <x< Specifically. Hence ⎧f ( y ) w ( y ) + FX ( 0 ) δ ( y ) ⎪ fY ( y ) = ⎨ X ⎪0 . Hence. 2. 0 ) . y ≥ 0 where w ( y ) = ⎨ ⎩0 .34 Indian Institute of Technology Madras .9 2. we have fY ( y ) = f X ( y ) for y > 0 . fY ( y ) = 0 for y < 0 . 0 < y ≤ 2 ⎪2 . 2. otherwise b) 1 3 ⎧1 ⎪ . Venkata Rao a) Note that g ( X ) is a constant (equal to zero) for X in the range of ( − ∞ . y = 0 ⎪ 3 ⎪1 = ⎨ .10: f X ( x ) and fY ( y ) for the example 2.Principles of Communication Prof.

Venkata Rao Note that X . 2. namely ±1. X ≥ 0 a) b) Let us find the general expression for fY ( y ) . X < 0 Let Y = ⎨ ⎩+ 1. 3 1 and P− 1 = . Hence the PDF of Y has only two impulses. 2. Y . we have P1 = has the sketches f X ( x ) and fY ( y ) . V. Y assumes only two values. We shall compute and sketch fY ( x ) assuming that f X ( x ) is the same as that of Example 2.Principles of Communication Prof.11 4 4 Fig. 2. a continuous RV is transformed into a mixed RV. In this case. Fig.10 Note that this transformation has converted a continuous random variable X into a discrete random variable Y . Let us write fY ( y ) as fY ( y ) = P1 δ ( y − 1) + P−1 δ ( y + 1) where a) P− 1 = P [ X < 0 ] and P1 [ X ≥ 0 ] b) Taking f X ( x ) of example 2.9.35 Indian Institute of Technology Madras .9.10 ⎧− 1.11: f X ( x ) and fY ( y ) for the example 2. Example 2.

Principles of Communication Prof. 2. 2. Venkata Rao Exercise 2. Compute and sketch fY ( y ) . Compute and sketch fY ( y ) .12(a) be the input to a device with the input-output characteristic shown in Fig.5 (b) Input-output transformation Exercise 2.6 The random variable X of exercise 2.12(b). V.5 is applied as input to the X − Y transformation shown in Fig. 2.5 Let a random variable X with the PDF shown in Fig.36 Indian Institute of Technology Madras .13. Fig. 2. 2.12: (a) Input PDF for the transformation of exercise 2.

the RV. then Y takes the 6 a) If X takes the values (1.. then 6 Y takes the values 0. 62 with probability 1 1 . 2. find fY ( y ) .. 6 ) with probability of values 12 .2 Functions of two random variables Given two random variables X and Y (assumed to be continuous type).. 9 with probabilities That is. Y ) and W = h ( X . find fY ( y ) .. .. fY ( y ) = 6 6 i = 1 ∑ δ(x − i ) ...11 Let Y = X 2 .. y k = g ( x ) for x = xk and x = xm . . then P [Y = y k ] = Pk + Pm . however. 1 . 2. .. then P [Y = y k ] = P [ X = xk ] = Pk .37 Indian Institute of Technology Madras . Example 2. X takes the values − 2. 1. 6 2 b) If.4. Y ) 2. 1. V.. − 1. fY ( y ) = 1 1 1 1 .. That is. ∑ 3 6 b) 1 6 i = −2 δ ( x − i ) ... Z and W are defined using the transformation Z = g ( X . assuming the value Yk = g ( x k ) with probability Pk . 6 3 3 6 1 1 ⎡δ ( y ) + δ ( y − 9 ) ⎤ + ⎡δ ( y − 1) + δ ( y − 4 ) ⎤ ⎣ ⎦ 3⎣ ⎦ 6 2. 0. respectively. 4. a) 1 If f X ( x ) = 6 If f X ( x ) = i = 1 ∑ δ ( x − i ) . If however. 22 . 3 with probability 1 . two new random variables. Venkata Rao We now assume that the random variable X is of discrete type taking on the value x k with probability Pk . . If y k = g ( x ) for only one x = xk .Principles of Communication Prof. Y = g ( X ) is also discrete.. In this case.

38 Indian Institute of Technology Madras . Theorem 2. y ) = w1 . w ⎞ J⎜ ⎟ = ∂w ⎝ x.W ( z . y1 ⎠ (2.W ( z . y1 ) ⎛ z. h ( x . Let ( x1 . We shall now state the theorem which relates fZ .Y ( x1 . Venkata Rao Given the above transformation and f X . (2. the Jacobian is the determinant of the appropriate partial derivatives. 2. y ⎠ ∂x = ∂z ∂y ∂w ∂y ∂ z ∂w ∂ z ∂w − ∂x ∂y ∂y ∂x That is.W ( z . ( x1 . y ⎠ ∂z ∂x ⎛ z. y ) = z1 . w ) and f X . y ) = w1 . w ) = f X . w ⎞ J⎜ ⎟ where ⎝ x. V. refer [1]. y ) .34) Proof of this theorem is given in appendix A2. we require the Jacobian of the transformation. For a more general version of this theorem.Y ( x . That is.2: To obtain fZ . given g ( x . we would like to obtain fZ .1. solve the system of equations g ( x . for x and y . y ) .Principles of Communication Prof. fZ . y1 ) satisfying Eq. w ⎞ J⎜ ⎟ ⎝ x1 . y1 ) be the result of the solution. w ) . We shall assume that the transformation is one-to-one.33b) then there is a unique set of numbers. Then. h ( x . 2.33.Y ( x . For this.33a) (2. w ) . denoted ⎛ z.W ( z . y ) = z1 .

39 Indian Institute of Technology Madras .Principles of Communication Prof. we obtain x = 1 (z + w ) 2 and y = 1 ( z − w ) .W ( z .Y ( x . V. 2. Fig.14(a) 2 depicts the (product) space A on which f X . let us find (a) fZ .W is non-zero 2. 2. y ≤ 1 2 and (b) fZ ( z ) . If Z = X + Y and W = X − Y . x ≤ 1 2 1 .12 X and Y are two independent RVs with the PDFs .14: (a) The space where f X .Y is non-zero (b) The space where fZ . y ) is non-zero. Fig. Venkata Rao Example 2. fX ( x ) = fY ( y ) = 1 . We see that the mapping is one-to-one. w ) a) From the given transformations.

14(b). V. otherwise ⎩ ) = 2. −2 ≤ z ≤ 0 4 2.14(b). b) fZ ( z ) = ∞ −∞ ∫ f (z.Principles of Communication Prof. we have fZ ( z ) = −z ∫ 8 dw z = − 1 z. Venkata Rao We can obtain the space B (on which fZ . w ) is non-zero) as follows: A space The line x = 1 B space 1 (z + w ) = 1 ⇒ w = − z + 2 2 1 ( z + w ) = − 1 ⇒ w = − ( z + 2) 2 1 (z − w ) = 1 ⇒ w = z − 2 2 1 (z − w ) = − 1 ⇒ w = z + 2 2 The line x = − 1 The line y = 1 The line y = − 1 The space is B is shown in Fig. we can see that. w ) = ⎨ 2 ⎪0 . 0 ≤ z ≤ 2 4 For z negative.40 Indian Institute of Technology Madras . w can take values only in the − z to z . w ∈ B (z.W ( z . Hence fZ ( z ) = + z −z ∫ 8 dw 1 1 = 1 z.W From Fig. z. The Jacobian of the transformation 1 1 ⎛ z. w ) d w Z . 2.W ⎧1 1 4 = ⎪8 . 2. y ⎠ Hence fZ . for a given z ( z ≥ 0 ). w ⎞ J⎜ = − 2 and J ( ⎟ = 1 −1 ⎝ x.

Y ) is specified and what is required is fZ ( z ) . fR . Theorem 2. and the transformation is one -to -one. Φ ( r .Y ( x .Principles of Communication Prof. y ) = exp ⎢ − ⎜ ⎟⎥ . ϕ ) . − ∞ < x.13 Let the random variables R and Φ be given by. R = X 2 + Y 2 and ⎛Y ⎞ Φ = arc tan ⎜ ⎟ where we assume R ≥ 0 and − π < Φ < π .2 can also be used when only one function Z = g ( X . z ≤ 2 4 Example 2. V. fΦ ( ϕ ) and to show that R and Φ are independent variables. Φ ( r . we can write x = r cos φ and y = r sin φ . To apply the theorem. a conveniently chosen auxiliary or dummy variable W is introduced. Typically W = X or W = Y . w ) is found from which fZ ( z ) can be obtained. 2. It is given that ⎝X⎠ ⎡ ⎛ x 2 + y 2 ⎞⎤ 1 f X . otherwise ⎩0 It is left as an exercise to find fR ( r ) . 2π 2 ⎠⎦ ⎣ ⎝ Let us find fR .41 Indian Institute of Technology Madras . using the theorem fZ . ϕ ) = ⎨ 2 π ⎝ 2⎠ ⎪ . − π < ϕ < π ⎪ Hence. Venkata Rao Hence fZ ( z ) = 1 z . ∂r ∂x ∂ϕ ∂x ∂r ∂y cos ϕ = − sin ϕ ∂ϕ r ∂y sin ϕ 1 cos ϕ = r r J = ⎧ r ⎛ r2 ⎞ exp ⎜ − ⎟ . y < ∞ . 0 ≤ r ≤ ∞ .W ( z . As the given transformation is from cartesian-to-polar coordinates.

35) If X and Y are independent. w ) = f X . with ⎧1 ⎪ .Y ∞ (2.36a) (2.14 Let X and Y be two independent random variables. −1 ≤ x ≤ 1 fX ( x ) = ⎨ 2 ⎪0 . Let us introduce a dummy variable W = Y .15.Principles of Communication Prof. Carrying out the convolution. X = Z − W . let us find P [ Z ≤ − 2] . fZ ( z ) is the convolution of f X ( z ) and fY ( z ) . otherwise ⎩ If Z = X + Y . then Eq. Then.36(b).W ( z . we obtain fZ ( z ) as shown in Fig. 2. V. and Y = W . From Eq.36b) That is.35 becomes fZ ( z ) = −∞ ∫ f ( z − w ) f (w ) d w X Y (2.42 Indian Institute of Technology Madras . otherwise ⎩ ⎧1 ⎪ . w ) and fZ ( z ) = ∞ −∞ ∫ f (z − w .Y ( z − w . As J = 1 . w ) d w X . 2. −2 ≤ y ≤ 1 fY ( y ) = ⎨ 3 ⎪0 . 2. fZ . 2. Venkata Rao Let Z = X + Y and we require fZ ( z ) . fZ = f X ∗ fY Example 2.

V. Venkata Rao Fig. 12 Example 2. we have X = ZW Y = W As J = 1 .43 Indian Institute of Technology Madras .Y ( zw . let us find an expression for fZ ( z ) .15: PDF of Z = X + Y of example 2.Y ( x . elsewhere ⎩ Then fZ ( z ) = ∞ −∞ ∫ w 1 + zw 2 dw 4 2. x ≤ 1.14 P [ Z ≤ − 2] is the shaded area = 1 . we have w ∞ fZ ( z ) = −∞ ∫ w f X .15 Let Z = X . y ) = ⎨ 4 ⎪0 . Y Introducing the dummy variable W = Y . y ≤ 1 ⎪ Let f X . 2.Principles of Communication Prof. w ) dw ⎧1 + x y .

16(b). w ) be non-zero if ( z . Under x ≤ 1 and y ≤ 1 (Fig.Y ( x . where A is the product space B .W ( z . w ) . y ) ∈ A .16a). B will be as shown in Fig.44 Indian Institute of Technology Madras . Venkata Rao f X .Principles of Communication Prof. 2. we have fZ ( z ) = 2 iii) 1 z 1 + zw 2 1⎛ 1 1 ⎞ ∫ 4 w dw = 4 ⎜ z2 + 2 z3 ⎟ ⎝ ⎠ 0 For z < − 1. Let fZ .2. y ) is non-zero if (x.W ( z .16: (a) The space where f X . then 1 + zw 2 1 + zw 2 fZ ( z ) = ∫ w dw = 2 ∫ w dw 4 4 0 −1 1 1 = z⎞ 1⎛ ⎜1 + 2 ⎟ 4⎝ ⎠ ii) For z > 1 . 2. w ) ∈ the given transformation. we have fZ ( z ) = 2 − 1 z ∫ 0 1 + zw 2 1⎛ 1 1 ⎞ w dw = ⎜ 2 + ⎟ 4 4 ⎝z 2 z3 ⎠ 2. Fig. V.Y is non-zero (b) The space where fZ . we have to integrate out w over the appropriate ranges. i) Let z < 1.W is non-zero To obtain fZ ( z ) from fZ .

we may encounter a situation where one of the variables is continuous and the other is discrete. Venkata Rao ⎧1 ⎪4 ⎪ Hence fZ ( z ) = ⎨ ⎪1 ⎪4 ⎩ z⎞ ⎛ ⎜1 + 2 ⎟ ⎝ ⎠ . as illustrated below.45 Indian Institute of Technology Madras .Y ⎜ . z ≤ 1 ⎛ 1 1 ⎞ ⎜ 2 + ⎟. z > 1 2 z3 ⎠ ⎝z Exercise 2. x ≤ 1 and y ⎧ ⎪y e− fY ( y ) = ⎨ ⎪0 ⎩ 2 . V.7 Let Z = X Y and W = Y . y ≥ 0 . a) b) Show that fZ ( z ) = ∞ −∞ ∫ 1 ⎛z ⎞ f X . by making use of the distribution function approach.16 The input to a noisy channel is a binary random variable with P [ X = 0] = P [ X = 1] = 1 . − ∞ < z < ∞. The output of the channel is given by Z = X + Y 2 2. Example 2. such cases are handled better. In transformations involving two random variables. otherwise Show that fZ ( z ) = 1 2π e 2 −z 2 . w ⎟ d w w ⎝w ⎠ X and Y be independent with fX ( x ) = 1 π 1 + x2 2 .Principles of Communication Prof.

Obtain fZ ( z ) . The method of obtaining PDFs based on theorem 2.Y ( x .17 Let Z = X + Y . V. if Y = g ( X ) . Find Let us first compute the distribution function of Z from which the density function can be derived. 1 2π e − y2 2 . Similarly.1 or 2. we have FY ( z ) + FY ( z − 1) . we have P [ Z ≤ z | X = 0 ] = FY ( z ) Similarly P [ Z ≤ z | X = 1] = P ⎡Y ≤ ( z − 1) ⎤ = FY ( z − 1) ⎣ ⎦ Hence FZ ( z ) = 1 1 d FZ ( z ) .2 is called change of variable method. Example 2. for transformations involving more than one random variable. y ) . Venkata Rao where Y is the channel noise with fY ( y ) = fZ ( z ) . (In this method. 2. as illustrated by means of example 2. P ( Z ≤ z ) = P [ Z ≤ z | X = 0] P [ X = 0 ] + P [ Z ≤ z | X = 1] P [ X = 1] As Z = X + Y . we compute FY ( y ) and then obtain fY ( y ) = d FY ( y ) dy .Principles of Communication Prof.46 Indian Institute of Technology Madras . As fZ ( z ) = 2 2 dz ⎛ z − 1 2 ⎞⎤ ( ) ⎟⎥ exp ⎜ − ⎜ ⎟⎥ 2 2π ⎝ ⎠⎦ 1 ⎡ ⎛ z2 ⎞ 1⎢ 1 ⎟+ exp ⎜ − fZ ( z ) = ⎜ 2 ⎟ 2 ⎢ 2π ⎝ ⎠ ⎣ The distribution function method.16. − ∞ < y < ∞ . is a very basic method and can be used in all situations. given f X . Of course computing the CDF may prove to be quite difficult in certain situations.) We shall illustrate the distribution function method with another example.

17: Shaded area is the P [ Z ≤ z ] That is. 2. z − x) d x X . y ) d y ⎥ ∫∞ ⎢ − ∞ ⎥ − ⎣ ⎦ ∞ ∞ ⎡z − x ⎤⎤ ∂ ⎡ fZ ( z ) = ⎢ ∫ d x ⎢ ∫ f X . y ) d y ⎥ ⎥ ⎦ (2. Venkata Rao P [Z ≤ z ] = P [ X + Y ≤ z ] = P [Y ≤ z − x ] This probability is the probability of ( X . Fig.37a) = −∞ ∫ f (x.Principles of Communication Prof. 2.Y ( x . y ) d y ⎥ ⎥ ∂ z ⎢− ∞ ⎢−∞ ⎥⎥ ⎣ ⎦⎦ ⎣ = ⎡ ∂ ∫∞ d x ⎢ ∂ z ⎢ − ⎣ ∞ ∞ z− x −∞ ∫ ⎤ f X . Y ) lying in the shaded area shown in Fig.47 Indian Institute of Technology Madras .Y ( x .Y ( x .Y It is not too difficult to see the alternative form for fZ ( z ) . namely. V. 2. FZ ( z ) = ⎡z − x ⎤ d x ⎢ ∫ f X .17.

A player is to be 2. However. 2. the expected value of a function of X .37b) If X and Y are independent.38) where E denotes the expectation operator. This can be illustrated as follows: Three small similar discs. Details can be found in [1.39) Remarks: The terminology expected value or expectation has its origin in games of chance. 2]. We note that Eq. y ) d y X . We now define a few of these averages and explore their significance. Some of these averages do provide a simple and fairly adequate (though incomplete) description of the random variable.Y (2.48 Indian Institute of Technology Madras . is defined by E ⎡g ( X ) ⎤ = g ( X ) = ⎣ ⎦ ∞ −∞ ∫ g (x) f (x) d x X (2. Similarly.35. g ( X ) .5 Statistical Averages The PDF of a random variable provides a complete statistical characterization of the variable.Principles of Communication Prof. Note that m X is a constant. Venkata Rao fZ ( z ) = ∞ −∞ ∫ f (z − y . 2. we have fZ ( z ) = f X ( z ) ∗ fY ( z ) .37(b) is the same as Eq. 2. mathematical expectation or simply expectation) of random variable X is defined as mX = E [ X ] = X = ∞ −∞ ∫ x fX ( x ) d x (2. This can be generalized to the case of functions of n variables. So far we have considered the transformations involving one or two variables. numbered 1. V. we might be in a situation where the PDF is not available but are able to estimate (with reasonable accuracy) certain (statistical) averages of the random variable.2 and 2 respectively are placed in bowl and are mixed. The mean value (also called the expected value.

42) We can extend the definition of expectation to the case of functions of k ( k ≥ 2 ) random variables. the mathematical expectation of g ( X ) is precisely the player's claim or expectation [3]. he will receive nine dollars. that is. E ⎡Xn ⎤ = ⎣ ⎦ ∞ −∞ ∫ x n fX ( x ) d x (2. which results in the mean value of Eq. If we take X to be (discrete) random variable with the PDF fX ( x ) = 1 2 δ ( x − 1) + δ ( x − 2 ) and g ( X ) = 15 − 6 X . Consider a function of two random variables. he will receive 3 dollars.40) The most widely used moments are the first moment ( n = 1. Then. if he draws either disc numbered 2. It seems reasonable to assume that the player has a '1/3 claim' on the 9 dollars and '2/3 claim' on three dollars. Y ) .49 Indian Institute of Technology Madras . X . Note that g ( x ) is such that g (1) = 9 and g ( 2 ) = 3 . resulting in the mean square value of X ). For the special case of g ( X ) = X n . we obtain the n-th moment of the probability distribution of the RV.38) and the second moment ( n = 2 . 2. If he draws the disc numbered 1. Venkata Rao blindfolded and is to draw a disc from the bowl. then 3 3 ∞ ⎡ ⎤ E ⎣g ( X ) ⎦ = −∞ ∫ (15 − 6 x ) f ( x ) d x X = 5 That is. then E ⎡ g ( X ) ⎤ gives n-th central moment. 2. that is. ⎣ ⎦ n E ⎡( X − m X ) ⎤ = ⎣ ⎦ ∞ −∞ ∫ (x − m ) X n fX ( x ) d x (2. His total claim is 9(1/3) + 3(2/3). V. or five dollars.Principles of Communication Prof. E ⎡X2⎤ = X2 = ⎣ ⎦ n ∞ −∞ ∫ x 2 fX ( x ) d x (2.41) If g ( X ) = ( X − m X ) . g ( X .

y ) d x dy X . y ) d x d y X . This result can be established as follows. Y ) = α X + βY where α and β are constants. y ) f (x.43) An important property of the expectation operator is linearity. From Eq. we have the first central moment being always zero because.Principles of Communication Prof.Y ∞ (2. we have E [Z ] = ∞ −∞ ∫ α x f (x) d x + ∫ β y f (y ) d y X Y −∞ ∞ = α X + βY 2.Y ∞ Integrating out the variable y in the first term and the variable x in the second term. V.50 Indian Institute of Technology Madras . that is.5. 2 2 E ⎡ X 2 − 2 mX X + mX ⎤ = E ⎡ X 2 ⎤ − 2 mX E [ X ] + mX ⎣ ⎦ ⎣ ⎦ 2 2 = E ⎡ X 2 ⎤ − 2 mX + mX ⎣ ⎦ 2 = X 2 − mX = X 2 − ⎡ X ⎤ ⎣ ⎦ 2 2. Y ) ⎤ = ⎣ ⎦ ∞ − ∞− ∞ ∫ ∫ g (x. y ) d x dy + ∞ − ∞− ∞ ∫ ∫ (β y ) f ( x .Y ( x . Venkata Rao E ⎡g ( X . y ) d x dy X .1 Variance Coming back to the central moments. 2. then Z = α X + βY . we have E [Z ] = = ∞ − ∞− ∞ ∞ ∞ ∫ ∫ ( α x + β y ) f ( x . if Z = g ( X .43. E ⎡( X − m X ) ⎤ = ⎣ ⎦ ∞ −∞ ∫ (x − m ) f (x) d x X X = mX − mX = 0 Consider the second central moment 2 2 E ⎡( X − m X ) ⎤ = E ⎡ X 2 − 2 m X X + m X ⎤ ⎣ ⎦ ⎣ ⎦ From the linearity property of expectation.Y ∞ − ∞− ∞ ∫ ∫ ( α x ) fX .

3: Let g ( X ) be a non-negative function of the random variable X . which is the desired result. we use a subscript on σ 2 ) The variance provides a measure of the variable's spread or randomness. This can be made more precise with the help of the Chebyshev Inequality which follows as a special case of the following theorem. Specifying the variance essentially constrains the effective width of the density function. If E ⎡ g ( X ) ⎤ exists.3 is useful in establishing similar inequalities involving random variables. E ⎡ g ( X ) ⎤ ≥ c P ⎡ g ( X ) ≥ c ⎤ . (If necessary. 2. V. then g ( x ) ≥ c . ⎣ ⎦ ⎣ ⎦ Note: The kind of manipulations used in proving the theorem 2. E ⎡g ( X ) ⎤ = ⎣ ⎦ = ∞ −∞ ∫ g (x) f (x) d x X X X B ∫g (x) f (x) d x + ∫ g (x) f (x) d x A Since each integral on the RHS above is non-negative. ⎣ ⎦ P ⎡g ( X ) ≥ c ⎤ ≤ ⎣ ⎦ E ⎡g ( X ) ⎤ ⎣ ⎦ c (2. The symbol σ 2 is generally used to denote the variance.51 Indian Institute of Technology Madras . then for every positive constant c . hence E ⎡g ( X )⎤ ≥ c ∫ f X ( x ) d x ⎣ ⎦ A But ∫ f X ( x ) d x = P [ x ∈ A] = P ⎡g ( X ) ≥ c ⎤ ⎣ ⎦ A That is. Theorem 2.Principles of Communication Prof. we can write E ⎡g ( X )⎤ ≥ ⎣ ⎦ ∫g (x) f (x) d x X A If x ∈ A .44) Proof: Let A = { x : g ( x ) ≥ c} and B denote the complement of A . Venkata Rao The second central moment of a random variable is called the variance and its (positive) square root is called the standard deviation.

let g ( X ) = ( X − m X ) and c = k 2 σ2 in theorem X 2 2. Chebyshev 2. the k2 probability of X − m X ≥ 2 σ X does not exceed 1/4 or 25%.44 states that the probability of choosing a person over 16 m tall is at most 1 ! (In a population of 1 billion. smaller the standard deviation.45b) To establish 2. By the same token. Naturally. at most 100 million would be as tall as a 10 full grown Palmyra tree!) Chebyshev inequality can be stated in two equivalent forms: i) ii) 1 P ⎡ X − mX ≥ k σ X ⎤ ≤ 2 . Venkata Rao To see how innocuous (or weak. smaller is the width of the interval around m X . perhaps) the inequality 2.3. We then have. V. let g ( X ) represent the height of a randomly chosen human being with E ⎡ g ( X ) ⎤ = 1.45a) (2. we would take the positive number k to be greater than one to have a meaningful result. k > 0 ⎣ ⎦ k 1 P ⎡ X − mX < k σ X ⎤ > 1 − 2 ⎣ ⎦ k where σ X is the standard deviation of X .Principles of Communication Prof. 1 P ⎡ X − mX ≥ k σ X ⎤ ≤ 2 ⎣ ⎦ k which is the desired result.52 Indian Institute of Technology Madras . (2.6m . we expect X to occur within the range ( m X ± 2 σ X ) for more than 75% of the observations. ⎣ ⎦ Then Eq. 2.45(a). 1 2 P ⎡( X − m X ) ≥ k 2 σ 2 ⎤ ≤ 2 X ⎣ ⎦ k In other words. where the required probability is concentrated. That is. Chebyshev inequality can be interpreted as: the probability of observing any RV outside ± k standard deviations off its mean value is no larger than 1 . With k = 2 for example.44 is.

Then. For example.2 Covariance An important joint expectation is the quantity called covariance which is obtained by letting g ( X .Y ] = E ⎡( X − m X ) (Y − mY ) ⎤ ⎣ ⎦ (2. Suppose X and Y are independent. That is. 2. λ X Y = cov [ X .46a) Using the linearity property of expectation. We use the symbol λ to denote the covariance. Venkata Rao inequality thus enables us to give a quantitative interpretation to the statement 'variance is indicative of the spread of the random variable'. then X = 0 but X 2 is not finite.46b) The cov [ X .5. ρX Y = E [ X Y ] − mX mY σ X σY (2. That is. Y ) = ( X − m X ) (Y − mY ) in Eq. V. Note that it is not necessary that variance exists for every PDF.Principles of Communication Prof. we have λ X Y = E [ X Y ] − m X mY (2. α +x (This is called Cauchy’s PDF) 2.53 Indian Institute of Technology Madras .43.Y ] . y ) d x d y x y f X ( x ) fY ( y ) d x d y ∞ − ∞− ∞ ∞ ∫ ∫ ∫ = x fX ( x ) d x −∞ −∞ ∫ y f (y ) d y Y = m X mY 2. if α f X ( x ) = 2 π 2 . E[XY] = = ∞ ∞ − ∞− ∞ ∞ ∞ ∫ ∫ x y f X . normalized with respect to the product σ X σY is termed as the correlation coefficient and we denote it by ρ .Y ( x .47) The correlation coefficient is a measure of dependency between the variables. − ∞ < x < ∞ and α > 0 .

Similarly if the probability of ( x − m X ) and ( y − mY ) being of the opposite sign is quite large. That is. When the joint experiment is performed many times. Suppose for example. the outcome y is conditioned on the outcome x in such a manner that there is a greater possibility of (y − mY ) being of the same sign as ( x − mX ) .Principles of Communication Prof. let 2. the sum of the numbers x1 ( y − mY ) would be very small and the quantity. Assume X and Y to be independent. When the random variables are independent. α being constant. That is. On the other hand. Then ρ X Y = ± 1 . let X and Y be so conditioned that. However.54 Indian Institute of Technology Madras . and given X = x1 . the random variables X and Y are said to be uncorrelated. this result is appealing. In the course of many trials of the experiments and with the outcome X = x1 . number of trials tends to zero as the number of trials keep increasing. then we expect ρ X Y to be negative. Two random variables X and Y are said to be orthogonal if E [ X Y ] = 0 . for X and Y be independent. If the variables are neither independent nor totally dependent. Then we expect ρ X Y to be positive. and sometimes negative with respect to mY . then X Y = X Y . As an example. Taking the extreme case of this dependency. sum . we have ρ X Y = 0 and for the totally dependent (y = ± α x ) case. ρ X Y = 1. Venkata Rao Thus. given X = x1 . then ρ will have a magnitude between 0 and 1. they are uncorrelated. we have λ X Y (and ρ X Y ) being zero. If ρ X Y ( or λ X Y ) = 0 . then Y = ± α x1 . Intuitively. let X and Y be dependent. if X and Y are uncorrelated. V. the fact that they are uncorrelated does not ensure that they are independent. then Y would occur sometimes positive with respect to mY .

⎪0 . σ2 i = σ2 . 2 2 Then σ2 = σW = σ1 + σ2 .48a) If X1 and X 2 are uncorrelated. 2 . Let ρ12 be the correlation coefficient between X1 and X 2 . That is.Principles of Communication Prof. i = 1. Then. then Y = x1 ! Let Y be the linear combination of the two random variables X1 and X 2 . we can show that 2 2 σY = k1 σ1 + k 2 σ2 + 2 ρ12 k1 k 2 σ1 σ 2 2 (2.48b) Note: Let X1 and X 2 be uncorrelated. Let E [ X i ] = mi . But X and Y are 2 not independent because. V. Venkata Rao α α ⎧1 < x < ⎪ .55 Indian Institute of Technology Madras . − fX ( x ) = ⎨ α 2 2. That is Y = k1 X1 + k 2 X 2 where k1 and k2 are constants. X Y = 0 which means λ X Y = X Y − X Y = 0 . then 2 2 σY = k1 σ1 + k2 σ2 2 (2. 2 σY = E ⎡Y 2 ⎤ − ⎡E [Y ]⎤ ⎦ ⎣ ⎦ ⎣ 2 E [Y ] = k1 m1 + k 2 m2 2 E ⎡Y 2 ⎤ = E ⎡ k1 X12 + k 2 X 2 + 2 k1 k 2 X1 X 2 ⎤ ⎣ ⎦ ⎣ ⎦ With a little manipulation. XY = X3 1 1 = ∫ x3 d x = α α −∞ ∞ α −α ∫ 2 x3 d x = 0 2 As X = 0 . X i 2 We will now relate σY to the known quantities. if 2. if X = x1 . otherwise ⎩ and Y = X 2 . the sum as well as the difference random Z 2 2 variables have the same variance which is larger than σ1 or σ 2 ! 2 The above result can be generalized to the case of linear combination of n variables. That is. and let Z = X1 + X 2 and W = X1 − X 2 .

Venkata Rao Y = 2 σY = i = 1 ∑k n n i X i . Example 2.49) where the meaning of various symbols on the RHS of Eq. 2 ≤ x ≤ 4 4 ≤ x Therefore. . We shall now give a few examples based on the theory covered so far in this section. 0 ≤ x ≤ 2 . x < 0 . 2. 2 ≤ x ≤ 4 4 ≤ x We shall find a) X and b) σ2 X a) The given CDF implies the PDF ⎧0 ⎪1 ⎪ ⎪8 fX ( x ) = ⎨ ⎪1 x ⎪8 ⎪0 ⎩ . .Principles of Communication Prof. E[X] = 1 1 2 ∫8x dx + ∫8x dx 0 2 2 4 2.18 Let a random variable X have the CDF ⎧0 ⎪x ⎪ ⎪8 FX ( x ) = ⎨ 2 ⎪x ⎪16 ⎪ ⎩1 .49 is quite obvious.56 Indian Institute of Technology Madras . then 2 i i = 1 ∑k σ2 + 2 ∑∑ k i k j ρi j σi σ j i i j i < j (2. V. x < 0 . 0 ≤ x ≤ 2 .

where 1 1 ⎧ ⎪1.38 we have.0636 π 1 = 0. 0 < x < 1. otherwise ⎩ 2 Let us find E [Y ] and σY . elsewhere ⎩0 2.57 Indian Institute of Technology Madras .Principles of Communication Prof. 0 < y < 1 f X . y ) = ⎨ .5 2 E ⎡Y ⎤ = ⎣ ⎦ 2 2 Hence σY = − 1 2 ∫ cos2 ( π x ) d x = 1 4 − 2 = 0. E [Y ] = 1 2 − 1 2 ∫ cos ( π x ) d x = 1 2 2 = 0. − < x < fX ( x ) = ⎨ 2 2 ⎪0.Y ( x .20 Let X and Y have the joint PDF ⎧ x + y .96 2 π Example 2. Venkata Rao = 1 7 31 + = 4 3 12 2 b) σ 2 = E ⎡ X 2 ⎤ − ⎡ E [ X ]⎤ X ⎦ ⎣ ⎦ ⎣ E ⎡X2⎤ = ⎣ ⎦ = σ 2 X 1 2 1 3 ∫8x dx + ∫8x dx 0 2 1 15 47 + = 3 2 6 2 2 4 47 ⎛ 31 ⎞ 167 = −⎜ ⎟ = 6 144 ⎝ 12 ⎠ Example 2.19 Let Y = cos π X . V. From Eq. 2.

We can easily show that E [ X ] = E [Y ] = 7 11 48 2 and E [ X Y ] = . given Y = y . If g ( X ) = X . We shall illustrate the calculation of conditional mean with the help of an example. σ X and σY . E[X | Y = y] = E[X |Y ] = ∞ −∞ ∫ x f X |Y ( x | y ) dx (2. y ) dx dy = ∫ ∫ x y ( x + y ) dx dy 2 0 0 1 1 = 17 72 b) To find ρ X Y . E [ X ]. E [ X | Y ] .51) Similarly. E [Y ] . we have E ⎡XY 2⎤ = ⎣ ⎦ ∞ − ∞− ∞ ∫ ∫ ∞ x y 2 f X .21 Let the joint PDF of the random variables X and Y be 2. E ⎡g ( X ) | Y = y ⎤ = ⎣ ⎦ ∞ −∞ ∫ g ( x ) f ( x | y ) dx X |Y (2. The quantity. Example 2.58 Indian Institute of Technology Madras . namely. V. 2. we can define the conditional variance etc.Principles of Communication Prof. we require E [ X Y ]. Venkata Rao Let us find a) E ⎡ X Y 2 ⎤ and ⎣ ⎦ b) ρ X Y a) From Eq.50) is called the conditional expectation of g ( X ) . σ2 = σY = X 144 12 144 1 11 Hence ρ X Y = − Another statistical average that will be found useful in the study of communication theory is the conditional expectation.43. then we have the conditional mean.Y ( x .

That is. y ) ≡ ⎨ x . Venkata Rao ⎧1 ⎪ . − ln y x ln y y < x < 1 Hence E ( X | Y ) = = ∫ x ⎢ − x ln y ⎥ d x ⎣ ⎦ y 1 ⎡ 1 ⎤ y −1 ln y Note that E [ X | Y = y ] is a function of y .Principles of Communication Prof. say. 0 < y < 1 1 1 x f X |Y ( x | y ) = = − . Discrete random variables i) Binomial: Consider a random experiment in which our interest is in the occurrence or nonoccurrence of an event A .1. we require the conditional PDF.6 Some Useful Probability Models In the concluding section of this chapter. we shall discuss certain probability distributions which are encountered quite often in the study of communication theory.6. We will begin our discussion with discrete random variables. 0 < x < 1. B . Let the experiment be repeated n times and let p be the 2.Y ( x . f X |Y ( x | y ) = fY ( y ) = f X .Y ( x .59 Indian Institute of Technology Madras . V. we are interested in the event A or its complement. ⎪0 . 2. y ) fY ( y ) ∫x dx y 1 1 = − ln y . outside ⎩ Let us compute E [ X | Y ] . 0 < y < x f X . To find E [ X | Y ] . 2.

will map into real number 3 as shown in the Fig.. let n = 5 and k = 3 . 2. Fig. n . s32 .18: Binomial RV for the special case n = 5 P ( A B A A B ) = P ( A ) P ( B ) P ( A ) P ( A ) P ( B ) . The sample point s1 could represent the sequence ABBBB . n .. 1.. The sample space (representing the outcomes of these five repetitions) has 32 sample points. (Each sample point is actually an element of the five dimensional Cartesian product space).. ‘number of occurrences of A in n trials'.... k = 0.... ⎝3⎠ In other words.. we have the binomial density... 2. . Taking a special case. The sample points such as ABAAB . as trails are independent..18. 1.Principles of Communication Prof.... say...60 Indian Institute of Technology Madras . for n = 5 ... . s1 . then we can write f X ( x ) . given by 2. V. Venkata Rao probability of occurrence of A on each trial and the trials are independent.. ⎛ 5⎞ 2 P [ X = 3] = ⎜ ⎟ p3 (1 − p ) ⎝ 3⎠ Generalizing this to arbitrary n and k . . A A A B B etc.... = p (1 − p ) p 2 (1 − p ) = p 3 (1 − p ) 2 ⎛5⎞ There are ⎜ ⎟ = 10 sample points for which X ( s ) = 3 . Let X denote random variable. k = 3 .. If we can compute P [ X = k ].. 2. X can be equal to 0...

22 A digital communication system transmits binary digits over a noisy channel in blocks of 16 digits.52) ⎛n⎞ n−i where Pi = ⎜ ⎟ p i (1 − p ) ⎝i ⎠ As can be seen.61 Indian Institute of Technology Madras . Let X be the random variable representing the number of errors per block. (Though X the formulae for the mean and the variance of a binomial PDF are simple.01 = 0. 0. V.Principles of Communication Prof. p ) to indicate X has a binomial PDF with parameters n and p defined above. Then X is b (16. We write X is b ( n . 2. The following example illustrates the use of binomial PDF in a communication problem. the algebra to derive them is laborious).01 and that errors in various digit positions within a block are statistically independent. Assume that the probability of a binary digit being in error is 0. i) ii) Find the expected number of errors per block Find the probability that the number of errors per block is greater than or equal to 3. Example 2. f X ( x ) ≥ 0 and ∞ −∞ ∫ fX ( x ) d x = i = 0 ∑ n Pi = i = ∑ ⎜i n ⎛n⎞ i n −i ⎟ p (1 − p ) 0⎝ ⎠ n = ⎡(1 − p ) + p ⎤ ⎣ ⎦ = 1 It is left as an exercise to show that E [ X ] = n p and σ2 = n p (1 − p ) .16. Venkata Rao fX ( x ) = i = 0 ∑ P δ(x − i ) i n (2.01) . i) E [ X ] = n p = 16 × 0.

Principles of Communication Prof. m = ∞ Since. m = 0 m! λ ∞ we have d eλ dλ ( ) = eλ = ∞ m λm − 1 1 ∑0 m ! = λ m = ∞ m = 1 ∑ ∞ m λm m! E[X] = m λm e− λ ∑ 1 m ! = λ eλ e − λ = λ m = Differentiating the series again.002 Exercise 2. Evidently f X ( x ) ≥ 0 and We will now show that E [ X ] = λ = σ2 X ∞ −∞ ∫ f X ( x ) d x = 1 because λm ∑0 m ! = eλ . 2. Chebyshev inequality results in P ⎡ X ≥ 3 ⎤ ≤ 0. we obtain. Venkata Rao ii) P ( X ≥ 3 ) = 1 − P [ X ≤ 2] = 1− i =0 ⎛ ⎞ ∑ ⎜ i ⎟ ( 0.62 Indian Institute of Technology Madras . λm e = ∑ . Note that Chebyshev inequality is not very A random variable X which takes on only integer values is Poisson distributed.53) where λ is a positive constant. ii) Poisson: 0.02 .22.1) (1 − p ) 2 16 i 16 − i ⎝ ⎠ = 0. if ∞ fX ( x ) = m = 0 ∑ δ ( x − m) λm e− λ m! (2.0196 ⎣ ⎦ tight. V.8 Show that for the example 2.

2. namely 1 .19: Uniform PDF It is easy show that (b − a) a+b E[X] = and σ2 = X 2 12 2 Note that the variance of the uniform PDF depends only on the width of the interval ( b − a ) . Hence σ2 = λ .Principles of Communication Prof. Therefore.2 Continuous random variables i) Uniform: A random variable X is said to be uniformly distributed in the interval a ≤ x ≤ b if.2. V. it has the same variance.2.63 Indian Institute of Technology Madras .54) Fig. ⎧ 1 .6. elsewhere ⎩ A plot of f X ( x ) is shown in Fig. 3 ii) Rayleigh: An RV X is said to be Rayleigh distributed if. (2. whether X is uniform in ( − 1.19. 1) or ( 2. Venkata Rao E ⎡ X 2 ⎤ = λ 2 + λ . X ⎣ ⎦ 2. 4 ) . a ≤ x ≤ b ⎪ fX ( x ) = ⎨ b − a ⎪0 .

55) A typical sketch of the Rayleigh PDF is given in Fig.2. (2. elsewhere ⎩0 where b is a positive constant.Principles of Communication Prof.2. We will encounter it later in the study of narrow-band noise processes.20: Rayleigh PDF Rayleigh PDF frequently arises in radar and communication problems. Venkata Rao ⎧x ⎛ x2 ⎞ exp ⎜ − ⎪ ⎟.20. 2. x ≥ 0 fX ( x ) = ⎨ b ⎝ 2 b⎠ ⎪ .64 Indian Institute of Technology Madras .) Fig.12 is Rayleigh PDF. V. ( fR ( r ) of example 2.

2. 1) .56) where m X is the mean value and σ2 the variance. N ( 0. then Y = 2 ( ) ⎛ X − mX ⎞ ⎜ σ ⎟ is N ( 0. In appendix A2. σ X . 2. As can be seen from the Fig. ⎝ ⎠ X 2. That is.56 is a valid PDF. 1 TP PT In this notation. The Gaussian PDF is symmetrical with respect to m X .65 Indian Institute of Technology Madras . −∞ < x < ∞ exp ⎢− 2 σ2 ⎢ ⎥ X ⎣ ⎦ (2. then E [ X ] = and iii) Gaussian By far the most widely used PDF. Note that if X is N m X . x d x = b) Show that if X E ⎡X2⎤ = 2 b ⎣ ⎦ dz .3. we show that X PT f X ( x ) as given by Eq. Venkata Rao Exercise 2. σ2 ) to denote the Gaussian density1. m X and σ2 . 1) denotes the Gaussian PDF with zero mean and unit variance. Hint: Make 0 ∞ the change of variable x 2 = z . Then. in the context of communication theory is the Gaussian (also called normal) density.55. 2.9 a) Let f X ( x ) be as given in Eq. V. specified by fX ( x ) = 1 2 π σX ⎡ ( x − m X )2 ⎤ ⎥. Show that ∫ f X ( x ) d x = 1. We use the symbol X N ( m X .21. 2 πb 2 is Rayleigh distributed. the Gaussian PDF is X completely specified by the two parameters.Principles of Communication Prof.

P [ X ≥ a] = ∞ ∫ a 1 2 π σX ⎡ ( x − m X )2 ⎤ exp ⎢− ⎥dx 2 σ2 ⎢ ⎥ X ⎣ ⎦ This integral cannot be evaluated in closed form. Venkata Rao Fig.2 at the end of the chapter.57 is N ( 0. we have ⎝ σX ⎠ P [ X ≥ a] = ∞ a − mX σX ∫ 1 2π e − z2 2 dz ⎛ a − mX ⎞ = Q⎜ ⎟ ⎝ σX ⎠ where Q ( y ) = ∞ ∫ y ⎛ x2 ⎞ exp ⎜ − ⎟dx 2π ⎝ 2 ⎠ 1 (2.21: Gaussian PDF Hence FX ( m X ) = mX −∞ ∫ f (x) d x X = 0. We have. 2. A small list is given in appendix A2. 1) . By making a change of variable ⎛ x − mX ⎞ z = ⎜ ⎟ .57) Note that the integrand on the RHS of Eq.Principles of Communication Prof. 2.66 Indian Institute of Technology Madras . V.5 Consider P [ X ≥ a ] . 2. Q( ) function table is available in most of the text books on communication theory as well as in standard mathematical tables.

then FX ( x ) approaches Gaussian as N becomes large. ⎧ ⎡ ( ln y − α )2 ⎤ 1 ⎪ ⎥. y ≥ 0 exp ⎢− ⎪ fY ( y ) = ⎨ 2 π y β 2 β2 ⎢ ⎥ ⎣ ⎦ ⎪ 0 . regardless of the distribution of the individual components. to the total.67 Indian Institute of Technology Madras . a) b) c) Show that Y is log-normal Find E (Y ) If m is such that FY ( m ) = 0. We shall develop Gaussian random processes in detail in Chapter 3 and. Example 2. V.23 A random variable Y is said to have a log-normal PDF if X = ln Y has a Gaussian (normal) PDF.Principles of Communication Prof. The electrical noise in a communication system is often due to the cumulative effects of a large number of randomly moving charged particles. Let Y have the PDF. find m . each particle making an independent contribution of the same amount. where each component makes only a small contribution to the sum. we shall make use of this theory in our studies on the noise performance of various modulation schemes. Hence the instantaneous value of the noise can be fairly adequately modeled as a Gaussian variable. fY ( y ) given by. Let X = ln Y or x = ln y (Note that the transformation is one-to-one) a) 2. you may refer [1-3]. Venkata Rao The importance of Gaussian density in communication theory is due to a theorem called central limit theorem which essentially states that: If the RV X is the weighted sum of N independent random components. in Chapter 7. otherwise ⎪ ⎩ where α and β are given constants.5 . For a more precise statement and a thorough discussion of this theorem.

Principles of Communication Prof. ∞ ) is 1. k1 = 2 π σ X σY 1 − ρ2 2. ⎧ 1 ⎡ ( x − m X )2 ( y − mY )2 ( x − mX ) ( y − mY ) ⎤⎫ exp ⎨− fX . y ) = + −2ρ ⎢ ⎥⎬ 2 2 k1 σX σY σ X σY ⎩ k2 ⎣ ⎦⎭ 1 (2.5 . x → ∞ Hence f X ( x ) = ⎡ ( x − α )2 ⎤ ⎥. − ∞ < x . then P [ X ≤ ln m ] = 0. Y ( x . Venkata Rao dx 1 1 = → J = dy y y Also as y → 0 . x → − ∞ and as y → ∞ .68 Indian Institute of Technology Madras . y ) . ln m = α or m = e α iv) Bivariate Gaussian As an example of a two dimensional density. f X .5 That is. β2 ) b) Y = E ⎡e X ⎤ = ⎣ ⎦ ⎡ − ( x − α) 2 1 x ⎢ e e 2β ∫ 2π β − ∞ ⎢ ⎣ ∞ 2 2 ⎤ ⎥dx ⎥ ⎦ = e β2 α+ 2 ⎡ x − ( α + β2 ) ⎤ ⎡∞ ⎤ ⎦ − ⎣ 1 ⎢ ⎥ 2 β2 e d x⎥ ⎢∫ ⎢− ∞ 2 π β ⎥ ⎣ ⎦ As the bracketed quantity being the integral of a Gaussian PDF between the limits ( − ∞ . V.Y ( x . y < ∞ given by.58) where. exp ⎢− 2 β2 ⎥ 2π β ⎢ ⎣ ⎦ 1 −∞ < x < ∞ Note that X is N ( α . we will consider the bivariate Gaussian PDF. we have Y = e α + β2 2 c) P [Y ≤ m ] = P [ X ≤ ln m ] Hence if P [Y ≤ m ] = 0.

unless X and Y independent. Y and let fX are and fY be Gaussian. σ2 ) and Y is N ( mY . Y is jointly Gaussian.2). This does not imply fX .Principles of Communication Prof. then the marginal density of X or Y is 2 Gaussian. We can construct examples of a joint PDF fX . then Z is Gaussian. with respect to non-Gaussian variables (we have already seen an example of this in Sec. Z Figure 2. Let fX and fY be obtained from fX . then they are independent. if the Gaussian variables are uncorrelated. σY ) 1 X TP PT P2) f X Y ( x .5.22 gives the plot of a bivariate Gaussian PDF for the case of ρ = 0 and σ X = σY . X is N ( m X . which is not Gaussian but results in fX and fY that are Gaussian. in general. that is. V. Y . 2. P3) If Z = α X + βY where α and β are constants and X and Y are jointly Gaussian. That is not true. 1 TP PT Note that the converse is not necessarily true. y ) = f X ( x ) fY ( y ) iff ρ = 0 That is. 2.5. Venkata Rao k 2 = 2 (1 − ρ2 ) ρ = Correlation coefficient between X and Y The following properties of the bivariate Gaussian density can be verified: P1) If X and Y are jointly Gaussian. Therefore fZ ( z ) can be written after computing mZ and σ2 with the help of the formulae given in section 2.69 Indian Institute of Technology Madras .

the striker missing! For ρ ≠ 0 . Let us find the probability of 2 (X. σ2 = σY = 1 and X ρX Y = − 1 .70 Indian Institute of Technology Madras . If ρ > 0 . we have two cases (i) ρ . of course. 2. positive and (ii) ρ . imagine the bell being compressed along the X = − Y axis so that it elongates along the X = Y axis.Y) lying in the shaded region D shown in Fig. with.Y resembles a (temple) bell.22: Bivariate Gaussian PDF ( σ X = σY and ρ = 0 ) For ρ = 0 and σ X = σY . Similarly for ρ < 0.Principles of Communication Prof.2.23. negative. Example 2.24 2 Let X and Y be jointly Gaussian with X = − Y = 1 . Venkata Rao Fig. V. 2. f X .

24(a) and B be the shaded region in Fig.24: (a) Region A and (b) Region B used to obtain region D A ⎤ − P ⎡( x . Fig. we have y ≥ − x + 1 and for the region B . 2. Hence the required probability is. 2 2.24 Let A be the shaded region shown in Fig. 2. V.71 Indian Institute of Technology Madras .24(b). we have 1 x + 2 .Principles of Communication Prof. y ) ∈ B ⎤ ⎦ ⎣ ⎦ The required probability = P ⎡( x . Venkata Rao Fig. 2.23: The region D of example 2. y ) ∈ ⎣ For the region y ≥ − 1 2 A . 2.

Principles of Communication

Prof. V. Venkata Rao

X X ⎡ ⎤ ⎡ ⎤ P ⎢Y + ≥ 1⎥ − P ⎢Y + ≥ 2⎥ 2 2 ⎣ ⎦ ⎣ ⎦

Let Z = Y +

X 2

Then Z is Gaussian with the parameters, Z =Y + σ2 = Z = 1 1 X = − 2 2

1 2 1 2 σ X + σY + 2 . ρ X Y 4 2 1 1 1 3 + 1− 2. . = 4 2 2 4
Z+ 3 4 1 2 is N ( 0, 1)

⎛ 1 3⎞ That is, Z is N ⎜ − , ⎟ . Then W = ⎝ 2 4⎠

P [ Z ≥ 1] = P ⎡W ≥ ⎣

3⎤ ⎦

5 ⎤ ⎡ P [ Z ≥ 2] = P ⎢W ≥ ⎥ 3⎦ ⎣
Hence the required probability = Q

( 3) − Q⎛ ⎜ ⎝

5 ⎞ ⎟ 3⎠

( 0.04 − 0.001) =

0.039

2.72
Indian Institute of Technology Madras

Principles of Communication

Prof. V. Venkata Rao

Exercise 2.10

X and Y are independent, identically distributed (iid) random variables, each being N ( 0, 1) . Find the probability of X , Y lying in the region Fig. 2.25.

A shown in

Fig. 2.25: Region

A of Exercise 2.10

Note: It would be easier to calculate this kind of probability, if the space is a product space. From example 2.12, we feel that if we transform

(X,Y)
B.

into

(Z , W )

such that Z = X + Y , W = X − Y , then the transformed space

B

would be square. Find fZ ,W ( z , w ) and compute the probability ( Z , W ) ∈

2.73
Indian Institute of Technology Madras

Principles of Communication

Prof. V. Venkata Rao

Exercise 2.11

Two random variables X and Y are obtained by means of the transformation given below. X = ( − 2 loge U1 ) 2 cos ( 2 π U2 ) Y = ( − 2 loge U1 ) 2 sin ( 2 π U2 )
1 1

(2.59a) (2.59b)

U1 and U2 are independent random variables, uniformly distributed in the range 0 < u1 , u2 < 1 . Show that X and Y are independent and each is
N ( 0, 1) .

Hint: Let X1 = − 2 loge U1 and Y1 = Show that Y1

X1

is Rayleigh. Find f X ,Y ( x , y ) using X = Y1 cos Θ and

Y = Y1 sin Θ , where Θ = 2 π U2 . Note: The transformation given by Eq. 2.59 is called the Box-Muller transformation and can be used to generate two Gaussian random number sequences from two independent uniformly distributed (in the range 0 to 1) sequences.

2.74
Indian Institute of Technology Madras

Principles of Communication

Prof. V. Venkata Rao

Appendix A2.1 Proof of Eq. 2.34
The proof of Eq. 2.34 depends on establishing a relationship between the differential area d z d w in the z − w plane and the differential area d x d y in the x − y plane. We know that
fZ ,W ( z , w ) d z d w = P [ z < Z ≤ z + d z , w < W ≤ w + d w ]

If we can find d x d y such that
fZ ,W ( z , w ) d z d w = f X ,Y ( x , y ) d x d y , then fZ ,W can be found. (Note that

the variables x and y can be replaced by their inverse transformation quantities, namely, x = g − 1 ( z , w ) and y = h − 1 ( z , w ) ) Let the transformation be one-to-one. (This can be generalized to the case of one-to-many.) Consider the mapping shown in Fig. A2.1.

Fig. A2.1: A typical transformation between the ( x − y ) plane and ( z − w ) plane in the z − w

Infinitesimal rectangle

ABC D

plane is mapped into the

parallelogram in the x − y plane. (We may assume that the vertex A transforms to A' , B to B' etc.) We shall now find the relation between the differential area of the rectangle and the differential area of the parallelogram.

2.75
Indian Institute of Technology Madras

76 Indian Institute of Technology Madras . Fig. the area A of the parallelogram is. y + dw⎟ ∂w ∂w ⎝ ⎠ Consider the vectors V1 and V2 shown in the Fig. Then. V. Venkata Rao Consider the parallelogram shown in Fig. Then the P2 and P3 are given by ⎛ ⎞ ∂ g− 1 ∂ h− 1 P2 = ⎜ x + dz. y + d z⎟ ⎜ ⎟ ∂z ∂z ⎝ ⎠ ⎛ ⎞ ∂x ∂y dz. That is. P3 and P4 . with vertices P1 . A2. P2 . A2. y ) be the co-ordinates of P1 . V1 = V2 = ∂x ∂y dz i + dz j ∂z ∂z ∂x ∂y dw i + dw j ∂w ∂w where i and j are the unit vectors in the appropriate directions. y + d z⎟ = ⎜x + ∂z ∂z ⎝ ⎠ ⎛ ⎞ ∂x ∂y P3 = ⎜ x + dw .2: Typical parallelogram Let ( x .2 where V1 = ( P2 − P1 ) and V2 = ( P3 − P1 ) . A = V1 × V2 2. A2.2.Principles of Communication Prof.

we have V1 × V2 = ∂x ∂y ∂y ∂x d z dw − ∂ z ∂w ∂ z ∂w ⎛ x. w ⎠ That is. Y ( x . V.77 Indian Institute of Technology Madras .W ( z . ⎛ x. y ⎠ . y ) ⎛ z. Venkata Rao As i × i = 0 . 2. w ) = f X . w ⎞ J⎜ ⎟ ⎝ x. w ⎠ = fX .Principles of Communication Prof.Y ( x . y ⎞ fZ. j × j = 0 . y ) J ⎜ ⎟ ⎝ z. and i × j = − ( j × i ) = k where k is the unit vector perpendicular to both i and j . y ⎞ A = J⎜ ⎟ d z dw ⎝ z.

4013 1.0179 0.0156 2.27 4.2743 1.Principles of Communication Prof.0808 2.30 0.35 0.0019 0.20 0.4405 1.55 0.40 0.50 0.0287 3.1469 2.30 0.2 Q( ) Function Table Q (α) = ∞ ∫ α 1 2π e 2 − x 2 dx It is sufficient if we know Q ( α ) for α ≥ 0 .60 0.15 0.05 0.00023 0.5 .0139 0.80 0. Note that Q ( 0 ) = 0.0668 3.45 0.00 0.55 0.70 0. V.20 0.2578 1.10 0.70 3.25 0.35 0.2420 1.80 0.0548 3.0359 3.0495 3.2119 1.10 0.00 0.3821 1. y Q (y ) y Q (y ) y Q (y ) y Q (y ) 0.95 0.50 0.00 0.95 0.3085 1.90 0.15 0.1841 1.00016 0.3632 1.85 0.0968 2.0035 0.10 3.65 0.0047 0.20 0.0322 3.50 0.90 0.3264 1.3446 1.00034 0.0885 2.75 0.05 0.78 0.0228 4.90 4.4602 1.40 0.40 0.70 0.0446 3.1251 2.1711 1.10 0.0401 3.00007 0.2912 1.0256 3.28 3.0107 0.4207 1.30 0.1977 1.1151 2.1587 2.60 0.00048 0.80 0.4801 1.50 0.25 0.00010 0.60 0.75 0.85 0.10 0.90 0.60 0.80 0.00003 2.0026 0.0010 10− 3 10− 3 2 10− 4 10− 4 2 10− 5 10− 6 3.78 Indian Institute of Technology Madras .70 0.70 0.0013 0.0082 0.20 0.90 0.0606 3. Venkata Rao Appendix A2.40 0.00069 0.2266 1.00 0.0735 2.0062 0.00005 1. because Q ( − α ) = 1 − Q ( α ) .65 0.30 0.1357 2.45 0.

V. 2 ⎝ 2⎠ 2. Venkata Rao Note that some authors use erfc ( given by ) .79 Indian Institute of Technology Madras .Principles of Communication Prof. the complementary error function which is ∞ erfc ( α ) = 1 − erf ( α ) = and the error function. erf ( α ) = Hence Q ( α ) = 2 π 2 ∫e α 0 − β2 dβ π ∫e α − β2 dβ 1 ⎛ α ⎞ erfc ⎜ ⎟.

3) Then.1) x − mX σx dx σx (A2.3 Proof that N m X . r = v 2 + y 2 and θ = tan−1 ⎜ ⎟ .2) (A2.3. Venkata Rao Appendix A2.3. Then.3.3 in Eq. σ2 is a valid PDF X We will show that f X ( x ) as given by Eq. V.3.3.Principles of Communication Prof. d v = Using Eq. Then. Let v = 1 2π ∞ −∞ ∫e − v2 2 dv = 1 (A2. 2π ∞ I 2 = ∫ ∫e 0 0 − r2 2 r dr dθ 2π = 2 π or I = That is.3. 2. ⎝v ⎠ and d x d y = r d r d θ . 2. A2. we have the required result. 2 −y Let I = −∞ ∫e dv = −∞ ∫e 2 dy. A2. (Note that f X ( x ) ≥ 0 for − ∞ < x < ∞ ). is a valid PDF by ∞ ( ) establishing ∞ −∞ 2 −v ∫ f (x) d x X ∞ 2 = 1 . A2.2 and Eq.80 Indian Institute of Technology Madras . I 2 ⎡ ∞ − v2 ⎤ ⎡ ∞ − y2 ⎤ 2 dv⎥ ⎢ ∫ e 2 d y⎥ = ⎢∫e ⎢− ∞ ⎥ ⎢− ∞ ⎥ ⎣ ⎦⎣ ⎦ ∞ = − ∞− ∞ ∫ ∫e ∞ − v2 + y2 2 dv d y ⎛y⎞ Let v = r cos θ and y = r sin θ . (Cartesian to Polar coordinate transformation).1.56.

. V. 2nd edition.. V. J. 1991. I. ‘Probability. Random Variables and Stochastic Processes’. P P 2. M. The Macmillan Co. 3) Hogg. and Jacobs. McGraw Hill (3rd edition). A.. A. 1965. P P 2) Wozencraft. ‘Principles of Communication Engineering’. R. 1965.81 Indian Institute of Technology Madras . J. Venkata Rao References 1) Papoulis..Principles of Communication Prof.. ‘Introduction to Mathematical Statistics”. and Craig. T. John Wiley.

Sign up to vote on this title
UsefulNot useful