You are on page 1of 81

Principles of Communication Prof. V.

Venkata Rao

2 CHAPTER 2
U

Probability and Random Variables

2.1 Introduction
At the start of Sec. 1.1.2, we had indicated that one of the possible ways
of classifying the signals is: deterministic or random. By random we mean
unpredictable; that is, in the case of a random signal, we cannot with certainty
predict its future value, even if the entire past history of the signal is known. If the
signal is of the deterministic type, no such uncertainty exists.

Consider the signal x ( t ) = A cos ( 2 π f1 t + θ ) . If A , θ and f1 are known,

then (we are assuming them to be constants) we know the value of x ( t ) for all t .

( A , θ and f1 can be calculated by observing the signal over a short period of

time).

Now, assume that x ( t ) is the output of an oscillator with very poor

frequency stability and calibration. Though, it was set to produce a sinusoid of

frequency f = f1 , frequency actually put out maybe f1' where f1' ∈ ( f1 ± ∆ f1 ) .

Even this value may not remain constant and could vary with time. Then,
observing the output of such a source over a long period of time would not be of
much use in predicting the future values. We say that the source output varies in
a random manner.

Another example of a random signal is the voltage at the terminals of a
receiving antenna of a radio communication scheme. Even if the transmitted

2.1

Indian Institute of Technology Madras

Principles of Communication Prof. V. Venkata Rao

(radio) signal is from a highly stable source, the voltage at the terminals of a
receiving antenna varies in an unpredictable fashion. This is because the
conditions of propagation of the radio waves are not under our control.

But randomness is the essence of communication. Communication
theory involves the assumption that the transmitter is connected to a source,
whose output, the receiver is not able to predict with certainty. If the students
know ahead of time what is the teacher (source + transmitter) is going to say
(and what jokes he is going to crack), then there is no need for the students (the
receivers) to attend the class!

Although less obvious, it is also true that there is no communication
problem unless the transmitted signal is disturbed during propagation or
reception by unwanted (random) signals, usually termed as noise and
interference. (We shall take up the statistical characterization of noise in
Chapter 3.)

However, quite a few random signals, though their exact behavior is
unpredictable, do exhibit statistical regularity. Consider again the reception of
radio signals propagating through the atmosphere. Though it would be difficult to
know the exact value of the voltage at the terminals of the receiving antenna at
any given instant, we do find that the average values of the antenna output over
two successive one minute intervals do not differ significantly. If the conditions of
propagation do not change very much, it would be true of any two averages (over
one minute) even if they are well spaced out in time. Consider even a simpler
experiment, namely, that of tossing an unbiased coin (by a person without any
magical powers). It is true that we do not know in advance whether the outcome
on a particular toss would be a head or tail (otherwise, we stop tossing the coin
at the start of a cricket match!). But, we know for sure that in a long sequence of
tosses, about half of the outcomes would be heads (If this does not happen, we
suspect either the coin or tosser (or both!)).

2.2

Indian Institute of Technology Madras

Principles of Communication Prof. V. Venkata Rao

Statistical regularity of averages is an experimentally verifiable
phenomenon in many cases involving random quantities. Hence, we are tempted
to develop mathematical tools for the analysis and quantitative characterization
of random signals. To be able to analyze random signals, we need to understand
random variables. The resulting mathematical topics are: probability theory,
random variables and random (stochastic) processes. In this chapter, we shall
develop the probabilistic characterization of random variables. In chapter 3, we
shall extend these concepts to the characterization of random processes.

2.2 Basics of Probability
We shall introduce some of the basic concepts of probability theory by
defining some terminology relating to random experiments (i.e., experiments
whose outcomes are not predictable).

2.2.1. Terminology
Def. 2.1: Outcome
The end result of an experiment. For example, if the experiment consists
of throwing a die, the outcome would be anyone of the six faces, F1 ,........, F6

Def. 2.2: Random experiment
An experiment whose outcomes are not known in advance. (e.g. tossing a
coin, throwing a die, measuring the noise voltage at the terminals of a resistor
etc.)

Def. 2.3: Random event
A random event is an outcome or set of outcomes of a random experiment
that share a common attribute. For example, considering the experiment of
throwing a die, an event could be the 'face F1 ' or 'even indexed faces'

( F2 , F4 , F6 ). We denote the events by upper case letters such as A , B or

A1 , A2 ⋅ ⋅ ⋅ ⋅


2.3

Indian Institute of Technology Madras

Principles of Communication Prof. V. Venkata Rao

Def. 2.4: Sample space
The sample space of a random experiment is a mathematical abstraction
used to represent all possible outcomes of the experiment. We denote the
sample space by S.


Each outcome of the experiment is represented by a point in S and is
called a sample point. We use s (with or without a subscript), to denote a sample
point. An event on the sample space is represented by an appropriate collection
of sample point(s).
Def. 2.5: Mutually exclusive (disjoint) events
Two events A and B are said to be mutually exclusive if they have no
common elements (or outcomes).Hence if A and B are mutually exclusive, they
cannot occur together.

Def. 2.6: Union of events
The union of two events A and B , denoted A ∪ B , {also written as

(A + B ) or ( A or B )} is the set of all outcomes which belong to A or B or both.

This concept can be generalized to the union of more than two events.

Def. 2.7: Intersection of events
The intersection of two events, A and B , is the set of all outcomes which
belong to A as well as B . The intersection of A and B is denoted by ( A ∩ B )

or simply ( A B ) . The intersection of A and B is also referred to as a joint event

A and B . This concept can be generalized to the case of intersection of three or
more events.

Def. 2.8: Occurrence of an event
An event A of a random experiment is said to have occurred if the
experiment terminates in an outcome that belongs to A .


2.4

Indian Institute of Technology Madras

Two of the most popular definitions are: i) the relative frequency definition. denoted by A is the event containing all points in S but not in A . it is likely that ⎜ A ⎟ will fluctuate quite badly. let the experiment be that of tossing a coin and A the event ⎛n ⎞ 'outcome of a toss is Head'.1) n → ∞ ⎝ n ⎠ ⎛ nA ⎞ ⎜ n ⎟ represents the fraction of occurrence of A in n trials.2. 2. Venkata Rao Def. denoted φ . we expect.9: Complement of an event The complement of an event A .  2. Def. 2.2 Probability of an Event The probability of an event has been defined in several ways.Principles of Communication Prof. ⎝ ⎠  ⎛n ⎞ For small values of n .5 Indian Institute of Technology Madras .  Def.10: Null event The null event. then the probability of A .11: The relative frequency definition: Suppose that a random experiment is repeated n times. then A B = φ and vice versa). 2. If the event A occurs nA times. V. Thus φ = S (note that if A and B are disjoint events. ⎜ A ⎟ may not deviate from ⎝ n ⎠ 2. is an event with no sample points. But ⎝ n ⎠ ⎛n ⎞ as n becomes larger and larger. is defined as ⎛n ⎞ P ( A ) = lim ⎜ A ⎟ (2. For example. If n is the order of 100. ⎜ A ⎟ to tend to a definite limiting ⎝ n ⎠ value. denoted by P ( A ) . and ii) the classical definition.

. then P ( A + B ) = P ( A ) + P ( B ) (2. Venkata Rao 1 by more than.6 Indian Institute of Technology Madras .4) ii) Let A1 ..3. we require the probability measure (to the various events on the sample space) to obey the following postulates or axioms: P1) P ( A ) ≥ 0 (2. to denote the union of A and B and to denote the addition of two real numbers).3a) P2) P ( S ) = 1 (2. 2. This is done by counting the total number N of the possible outcomes of the experiment. A2 .. then NA P ( A) = (2.3(c). for i ≠ j and (2.2) N where it is assumed that all outcomes are equally likely!  Whatever may the definition of probability. namely. then P ( A + B ) = P ( A ) + P ( B ) − P ( A B ) (2.Principles of Communication Prof. we 2 ⎛n ⎞ 1 expect ⎜ A ⎟ to converge to .3c) (Note that in Eq. say ten percent and as n becomes larger and larger. In the classical approach.. 2. V.3b) P3) ( AB ) = φ .5a) 2. it is possible for us to derive some additional relationships: i) If A B ≠ φ .. If N A of those outcomes are favorable to the occurrence of the event A . the probability of the event A is found without experimentation. 2.12: The classical definition: The relative frequency definition given above has empirical flavor. ⎝ n ⎠ 2 Def.. Using Eq. the symbol + is used to mean two different things.. An be random events such that: a) Ai A j = φ .

.7 is left as an exercise.7) The derivation of Eq. In a real world random experiment. 2. Such dependencies between the events is brought out using the notion of conditional probability . let a bowl contain 3 resistors and 1 capacitor. To give a simple example.4. denoted P ( B | A ) . ⋅ ⋅ ⋅.5b).. (2.. The conditional probability P ( B | A ) can be written in terms of the joint probability P ( A B ) and the probability of the event P ( A ) .5a) and exhaustive (Eq. V. 2. it represents the probability of B occurring. we have ⎛ nAB ⎞ P ( AB ) = lim ⎜ ⎟ n → ∞ ⎝ n ⎠ ⎛n ⎞ P ( A ) = lim ⎜ A ⎟ n → ∞ ⎝ n ⎠ 2. 2.. iii) P ( A ) = 1 − P ( A ) (2..5b) Then.. The occurrence of the event 'the capacitor on the second draw' is very much dependent on what has been drawn at the first instant. 2. given that A has occurred. + An = S. A very useful concept in probability theory is that of conditional probability. it is quite likely that the occurrence of the event B is very much influenced by the occurrence of the event A . P ( A ) = P ( A A1 ) + P ( A A2 ) + . Venkata Rao b) A1 + A2 + . Note: A1 . Using the former. A2 . This relation can be arrived at by using either the relative frequency definition of probability or the classical definition. + P ( A An ) (2. An are said to be mutually exclusive (Eq...6) where A is any event on the sample space..7 Indian Institute of Technology Madras .6 and 2..Principles of Communication Prof.

2. As P ( B | A ) refers to the probability of B occurring. say P ( B | A ) and the (unconditional) probability P ( A ) .9 expresses the probability of joint event AB in terms of conditional probability.8(a) as P (B ) P ( A | B ) P (B | A) = . Venkata Rao where nAB is the number of times AB occurs in n repetitions of the experiment.9) In view of Eq.9. we can also write Eq. we have P ( AB ) P ( A | B) = .10(a) or 2.8 Indian Institute of Technology Madras .8(b) can be written as P ( A B ) = P (B | A) P ( A) = P (B ) P ( A | B ) (2. P ( A) ≠ 0 (2.10a) P ( A) Similarly P ( A) P (B | A) P ( A | B) = . For example P ( A BC ) can be written as 2. Similar relation can be derived for the joint probability of a joint event involving the intersection of three or more events. 2.8b) P (B ) Eq. 2. we have Def 2. 2.8a) n → ∞ ⎜ nA ⎟ P ( A)  ⎜ n ⎟ ⎝ ⎠ or P ( A B ) = P (B | A) P ( A) Interchanging the role of A and B .10b) P (B ) Eq. P (B ) ≠ 0 (2. 2.Principles of Communication Prof.10(b) is one form of Bayes’ rule or Bayes’ theorem. Eq.8(a) and 2.13: Conditional Probability nAB P ( B | A ) = lim n → ∞ nA ⎛ nAB ⎞ ⎜ ⎟ P ( AB ) = lim ⎜ n ⎟ = . P (B ) ≠ 0 (2. V. P ( A) ≠ 0 (2. given that A has occurred.

Another useful probabilistic concept is that of statistical independence.. 2.. A2 .13) That is.Principles of Communication Prof.12 represents another form of Bayes’ theorem. Show that ( ) ( ) P B | Aj P Aj ( P Aj | B =) n (2.1 Let A1 ... A2 . Ak are said to be statistically independent if and only if (iff) the probability of every intersection of k or fewer events equal the product of the probabilities of its constituents. Suppose the events A and B are such that P (B | A) = P (B ) (2.13.. Alternatively. Note that if A and B are independent. C are independent when 2. the events A and B are said to be statistically independent.... whereas if they are disjoint. An be n mutually exclusive and exhaustive events and B is another event defined on the same space.. knowledge of occurrence of A tells no more about the probability of occurrence B than we knew without that knowledge.14 can be used to define the statistical independence of two events. B . The notion of statistical independence can be generalized to the case of more than two events. 2.. 2.. Then. then P ( A B ) = P ( A ) P ( B ) . A set of k events A1 . Venkata Rao P ( A BC ) = P ( A B ) P (C | AB ) = P ( A ) P ( B | A ) P (C | AB ) (2. Thus three events A ... V.12) ∑ P (B | Aj ) P ( Aj ) i = 1 Eq. then P ( A B ) = P ( A) P (B ) (2.9 Indian Institute of Technology Madras .13 or 2.14) Either Eq.11) Exercise 2.. if A and B satisfy the Eq. then P ( A B ) = 0 .

after seeing each other for some time (and after a few tiffs) decide to get married. 2. Mark this event on the sample space. a) The sample space is the rectangle.1 Priya (P1) and Prasanna (P2). a) Picture the sample space b) Let the event A stand for “P1 and P2 meet”. 2. Unfortunately. both are somewhat lacking in punctuality and their arrival times are equally likely to be anywhere in the interval 11 to 12 hrs on that day. Venkata Rao P ( A B ) = P ( A) P (B ) P ( AC ) = P ( A ) P (C ) P ( BC ) = P ( B ) P (C ) and P ( A BC ) = P ( A ) P ( B ) P (C ) We shall illustrate some of the concepts introduced above with the help of two examples. on the ensuing Friday (looks like they are not aware of Rahu Kalam or they don’t care about it). V. c) Find the probability that the lovebirds will get married and (hopefully) will live happily ever after. much against the wishes of the parents on both the sides.Principles of Communication Prof. before leaving in a huff never to meet again. They agree to meet at the office of registrar of marriages at 11:30 a.m. Example 2. Also arrival of one person is independent of the other.10 Indian Institute of Technology Madras .1(a). shown in Fig. However. both are also very short tempered and will wait only 10 min.

Assuming that P1 arrives at 11: x . 2. indicating the possibility of P1 and P2 meeting.1(b).1(b): The event A of Example 2.11 Indian Institute of Technology Madras . The event A .Principles of Communication Prof.1 b) The diagonal OP represents the simultaneous arrival of Priya and Prasanna. Fig. 2. is shown in Fig. as shown in the figure.1(a): S of Example 2. 2. meeting between P1 and P2 would take place if P2 arrives within the interval a to b . V. Venkata Rao Fig.1 2.

2: Let two honest coins.12 Indian Institute of Technology Madras . H1 H2 . T1 H2 . (Match comprises the two outcomes T1 T2 . 1 Hence P ( A B ) = . H1 H2 ).e. ( T1 indicates toss of coin 1 resulting in tails. therefore. The four possible outcomes are T1 T2 . V.. The event A comprises of the outcomes ‘ T1 T2 . marked 1 and 2. Venkata Rao Shaded area 11 c) Probability of marriage = = Total area 36  Example 2. T1 H2 and 4 H1 T2 ’. given 'not H1 H2 ’ the 3 toss would have resulted in anyone of the three other outcomes which can be 2. that is the probability of occurrence of any of these four outcomes 1 is . i. Let the event A be 'not H1 H2 ' and B be the event 'match'. similarly T2 etc. Are A and B independent? P ( AB ) We know that P ( B | A ) = . P ( A) A B is the event 'not H1 H2 ' and 'match'. 3 P ( A) = 4 1 1 P (B | A) = 4 = 3 3 4 1 Intuitively. Find P ( B | A ) . be tossed together. H1 T2 . (Treating each of these outcomes as an event. we find that these events 4 are mutually exclusive and exhaustive). it represents the outcome T1 T2 . the result P ( B | A ) = is satisfying because.Principles of Communication Prof.) We shall treat that all these outcomes are equally likely.

This implies that the outcome T1 T2 given 3 1 'not H1 H2 '. Venkata Rao 1 treated to be equally likely. that is. Fig.2: A mapping X ( ) from S to the real line. A and B are dependent events. 2.1 Distribution function: Taking a specific case. 3 1 1 As P ( B ) = and P ( B | A ) = . say X . 2 3  2.Principles of Communication Prof. to each si ∈ S .2.3 Random Variables Let us introduce a transformation or function. The six faces of the die can be treated as the six sample points in S . has a probability of .13 Indian Institute of Technology Madras . namely . 2. as shown in Fig. that is. let the random experiment be that of throwing a die. X assigns a real number. a2 ] . The figure shows the transformation of a few individual sample points as well as the transformation of the event A . V.3. whose domain is the sample space (of a random experiment) and whose range is in the real line. which falls on the real line segment [a1 . X ( si ) . 2.2.

4 . 2. then 8 8 2 the distribution function FX ( x ) . i = 1. provided we know the Distribution Function of X. 2. i = 1. Then we can enquire into the probabilities such as P ⎡⎣{s : X ( s ) < a1}⎤⎦ P ⎡⎣{s : b1 < X ( s ) ≤ b2 }⎤⎦ or P ⎡⎣{s : X ( s ) = c}⎤⎦ These and other probabilities can be arrived at. P ( s3 ) = and P ( s4 ) = . (Note that. 2. s1 to s4 ...3. Venkata Rao Fi = si . But. Once the transformation is induced.. comprising all those sample points which are transformed by X into real numbers less than or equal to x . then the events on the sample space will become transformed into appropriate segments of the real line.14 Indian Institute of Technology Madras .5. let S consist of four sample points.15) That is. with 1 each with sample point representing an event with the probabilities P ( s1 ) = . . 2. 1] ). for convenience. 6 . FX ( a1 ) etc. it could be any other symbol and we may use FX ( α ) . denoted by FX ( ) which is given by FX ( x ) = P ⎡⎣{s : X ( s ) ≤ x}⎤⎦ (2. FX ( x ) is the probability of the event.) Evidently. As an example of a distribution function (also called Cumulative Distribution Function CDF). 4 1 1 1 P ( s2 ) = . will be as shown in Fig. V. If X ( si ) = i − 1. Let X ( si ) = i . .. we use x as the argument of FX ( ) . 3. FX ( ) is a function whose domain is the real line and whose range is the interval [0.Principles of Communication Prof..

there is a discontinuity in FX of magnitude Pa at a point x = a .Principles of Communication Prof.5 .5 whereas 1 1 FX ( − 0. In general. note that FX ( x ) = 0 for x < − 0. 2. 2. then ⎡⎣FX ( a ) − FX ( b ) ⎤⎦ = P ⎡⎣{s : b < X ( s ) ≤ a}⎤⎦ v) If a > b . − ∞ < x < ∞ ii) FX ( − ∞ ) = 0 iii) FX ( ∞ ) = 1 iv) If a > b . there is a discontinuity of at the point 4 4 x = − 0.3: An example of a distribution function FX ( ) satisfies the following properties: i) FX ( x ) ≥ 0.16) 2. V. if and only if P ⎡⎣{s : X ( s ) = a}⎤⎦ = Pa (2. Properties iv) and v) follow from the fact {s : X ( s ) ≤ b} ∪ {s : b < X ( s ) ≤ a} = {s : X ( s ) ≤ a} Referring to the Fig. In other words.15 Indian Institute of Technology Madras . Venkata Rao Fig. then FX ( a ) ≥ FX ( b ) The first three properties follow from the fact that FX ( ) represents the probability and P ( S ) = 1.5 ) = .3.

for any real x . is continuous to the right at each point x 1. Hence. every transformation from S into the real line need not be a random variable. Venkata Rao The properties of the distribution function are summarized by saying that Fx ( ) is monotonically non-decreasing. F X ( x ) is continuous to the right. P ( B ) = and P (C ) = .16 is satisfied. Let the transformation X be X ( si ) = i . s2 }. TP PT Functions such as X ( ) for which distribution functions exist are called Random Variables (RV). s1 to s6 . the limit of the set s : a < X ( s ) ≤ a + ∆ is the null set and can be proved to be so. 2. We see that the probabilities for the 6 2 6 various unions and intersections of A . F X ( a+ ) − FX ( a ) = 0 . 1 TP PT Let x = a . lim P ⎡⎣a < X ( s ) ≤ a + ∆ ⎤⎦ = lim ⎡⎣FX ( a + ∆ ) − FX ( a ) ⎤⎦ ∆ → 0 ∆ → 0 { } We intuitively feel that as ∆ → 0 . V.5 ] = P ( s4 ) is not known as s4 is not an event on the sample space. s4 . In other words. The only events that have been identified on the sample space are: A = {s1 .16 Indian Institute of Technology Madras . B and C can be obtained.Principles of Communication Prof. B = {s3 . and has a step of size Pa at point a if and if Eq. 2. let S consist of six sample points. where a + = lim ∆ → 0 (a + ∆ ) That is. (The term “random variable” is somewhat misleading because an RV is a well defined function from S into the real line. Consider. with ∆ > 0 . For example.) However. s5 } and C = {s6 } and their probabilities 2 1 1 are P ( A ) = .5 < x ≤ 4. {s : X ( s ) ≤ x} should be an event in the sample space with some assigned probability. Then the distribution function fails to exist because P [s : 3.

2 Let S be a sample space with six sample points. 4. 1 1 1 B = {s3 . Sketch FY ( y ) . in practice we find it more convenient to deal with Probability Density Function (PDF). 5 ⎪3 .17a) dx or x FX ( x ) = ∫ f (α) d α X (2.17b) −∞ The distribution function may fail to have a continuous derivative at a point x = a for one of the two reasons: i) the slope of the Fx ( x ) is discontinuous at x = a ii) Fx ( x ) has a step discontinuity at x = a The situation is illustrated in Fig. s5 } and C = {s6 } with P ( A ) = .17 Indian Institute of Technology Madras . A = {s1 . s2 } . f X ( x ) is defined as the derivative of the CDF. 2. The events identified on S are the same as above. 3 2 6 Let Y ( ) be the transformation. 2 ⎪ Y ( si ) = ⎨2 . ⎧1. namely.4. 2. Venkata Rao Exercise 2. 2. i = 1. s4 . i = 6 ⎩ Show that Y ( ) is a random variable by finding FY ( y ) . The PDF. i = 3. that is d FX ( x ) fX ( x ) = (2. P ( B ) = and P (C ) = .2 Probability density function Though the CDF is a very fundamental concept.3.Principles of Communication Prof. s1 to s6 . V.

f X ( x ) has an impulse of weight at x = as 8 2 ⎡ 1⎤ 1 P ⎢ X = ⎥ = . 2. we resolve the ambiguity by taking f X to be a derivative on the right.4: A CDF without a continuous derivative As can be seen from the figure. 2. if there is a discontinuity in FX at x = a of magnitude Pa .3. the PDF will be.) The second case is taken care of by introducing the impulse in the probability domain. For example. In the first case. 1 ⎛ 1⎞ 1 ⎛ 1⎞ 1 ⎛ 3⎞ 1 ⎛ 5⎞ fX ( x ) = δ⎜ x + ⎟ + δ⎜ x − ⎟ + δ⎜ x − ⎟ + δ⎜ x − ⎟ (2. Venkata Rao Fig. for the CDF shown in Fig. This impulse function cannot be taken as the limiting case of ⎣ 2⎦ 8 1 ⎛x⎞ an even function (such as ga ⎜ ⎟ ) because. we include an impulse Pa δ ( x − a ) in the PDF.18) 4 ⎝ 2⎠ 8 ⎝ 2⎠ 8 ⎝ 2⎠ 2 ⎝ 2⎠ 1 1 In Eq.18.Principles of Communication Prof. That is. 2. (Note that FX ( x ) is continuous to the right. FX ( x ) has a discontinuous slope at x = 1 and a step discontinuity at x = 2 . ε ⎝ε⎠ 1 1 2 2 1 ⎛ 1⎞ 1 lim ∫ f X ( x ) dx = lim ∫ δ ⎜ x − ⎟ dx ≠ ε → 0 1 −ε ε → 0 1 −ε 8 ⎝ 2⎠ 16 2 2 2. V.18 Indian Institute of Technology Madras .

FX ( x ) . ε → 0 8 1 −ε 2 ⎧2 1 1 ⎪⎪ 8 . Later on in this lesson. We can induce more than one transformation on a given sample space. given by FX . Y ( s ) ≤ y }⎤⎦ (2. 2. then X 1 is a continuous random variable. y ) = P ⎡⎣{s : X ( s ) ≤ x . If we induce k such transformations. say X and Y . As FX is non-decreasing and FX ( ∞ ) = 1. lim ∫ f X ( x ) dx = . then we have a set of k co-existing random variables.19a) ∞ ii) ∫ f ( x ) dx −∞ X = 1 (2. Venkata Rao 1 2 1 However. We say that X is a mixed random variable if FX ( x ) is discontinuous but not a staircase.20) 1 TP PT As the domain of the random variable X ( ) is known. 2. a random variable can be classified as: i) continuous (ii) discrete and (iii) mixed. − 2 ≤ x < 2 FX ( x ) = ⎨ ⎪3 . we have i) fX ( x ) ≥ 0 (2. is a continuous function of x for all x .Y ( x . it is convenient to denote the variable simply by X . This ensures. V. If the CDF.3 Joint distribution and density functions Consider the case of two random variables. we shall discuss some examples of these three types of variables.3.19 Indian Institute of Technology Madras . then X corresponds to a discrete variable. They can be characterized by the (two-dimensional) joint distribution function. If FX ( x ) is a TP PT staircase.19b) Based on the behavior of CDF.Principles of Communication Prof. 1 ≤ x < 3 ⎪⎩ 8 2 2 Such an impulse is referred to as the left-sided delta function.

Principles of Communication Prof.Y ( x . FX .Y ( − ∞ . In other words.Y ( x1 .Y ( ∞ . if B is comprised of all those sample points s ∈ S such that Y ( s ) ≤ y1 . ∞ ) = FX ( x ) 2. then F ( x1 . y1 ) is the probability associated with the set of all sample points whose transformation does not fall outside the shaded region in the two dimensional (Euclidean) space shown in Fig. V. Venkata Rao That is. let A be the set of all those sample points s ∈ S such that X ( s ) ≤ x1 . y ) ≥ 0 . Properties of the two dimensional distribution function are: i) FX . −∞ < x < ∞. 2. their transformed values will be less than or equal to x and at the same time.5. y1 ) is the probability associated with the event AB . − ∞ ) = 0 iii) FX . under Y .Y ( x . y ) = FY ( y ) v) FX . −∞ < y < ∞ ii) FX .Y ( x1 .5: Space of { X ( s ) . y1 ) Looking at the sample space S. FX . 2.Y ( ∞ . Y ( s )} corresponding to FX . Fig.Y ( x . ∞ ) = 1 iv) FX .20 Indian Institute of Technology Madras . y ) = FX .Y ( x . y ) is the probability associated with the set of all those sample points such that under X . Similarly. the transformed values will be less than or equal to y .

FX . ∞ ) = FX ( x1 ) . Given the joint PDF of random variables X and Y . where k ≥ 3 .Y ( x2 .Y − ∞− ∞ d ⎧⎪ 1 ⎡ ⎤ ⎫⎪ x ∞ f X ( x1 ) = ⎨ ∫ ⎢ ∫ f X . y1 ) We define the two dimensional joint PDF as ∂2 f X .21a) ∂x ∂y y x or FX . 2.Principles of Communication Prof.Y (2.Y ( x1 . f X ( x ) and fY ( y ) .21b) − ∞− ∞ The notion of joint CDF and joint PDF can be extended to the case of k random variables. FX ( x1 ) = ∫ ∫ f ( α . Hence it is common to refer FX ( x ) as the marginal 2. x1 ∞ That is. β ) dβ X .22 involves the derivative of an integral.Y 1 dx x = x1 −∞ ∞ or fX ( x ) = ∫ f ( x .Y (2.Y ( x . y ) = ∫ ∫ f ( α .Y ( x2 .23b) −∞ (In the study of several random variables.Y ( x . V. fY ( y ) = ∫ f ( x . the statistics of any individual variable is called the marginal. y ) dx X . β ) d β⎥ d α ⎬ (2.Y ( x . β) d β d α X . We know that.Y (2. then FX . y ) dy X .22) d x1 ⎩⎪− ∞ ⎣⎢ − ∞ ⎦⎥ ⎭⎪ Eq.Y ( x1 . Hence. β ) d α dβ X . d FX ( x ) ∞ f X ( x1 ) = = ∫ f ( x . y ) (2. it is possible to obtain the one dimensional PDFs.Y ( α .23a) −∞ ∞ Similarly. y1 ) ≥ FX . y ) = FX . Venkata Rao vi) If x2 > x1 and y 2 > y1 .21 Indian Institute of Technology Madras . y 2 ) ≥ FX .

y ) = f X |Y ( x | y ) fY ( y ) (2. we might as well drop the subscript on y and denote it by f X |Y ( x | y ) .25d) The function f X |Y ( x | y ) may be thought of as a function of the variable x with variable y arbitrary.4 Conditional density Given f X . Accordingly.26a) ∞ and ∫ f ( x | y ) dx −∞ X |Y = 1 (2.Y ( x . f X |Y ( x | y ) ≥ 0 (2. but fixed. y ) f X |Y ( x | y ) = (2. y ) and fY | X ( y | x ) = (2.Y ( x . An analogous relation can be defined for fY | X ( y | x ) . given y = y1 .24) fY ( y1 ) where it is assumed that fY ( y1 ) ≠ 0 .Y ( x .Y ( x .Y ( x .) 2.3.26b) 2.25c) = fY | X ( y | x ) f X ( x ) (2. That is.22 Indian Institute of Technology Madras . that is.25a) fY ( y ) f X . We might also be interested in the PDF of X given a specific value of y = y1 . V. This is called the conditional PDF of X . we know how to obtain f X ( x ) or fY ( y ) . Venkata Rao distribution function of X and f X ( x ) as the marginal density function.25b) fX ( x ) or f X . we have the pair of equations. y ) . f X . Once we understand the meaning of conditional PDF. y1 ) f X |Y ( x | y1 ) = (2. denoted f X |Y ( x | y1 ) and defined as f X . it satisfies all the requirements of an ordinary PDF.Principles of Communication Prof. FY ( y ) and fY ( y ) are similarly referred to.

.29) Statistical independence can also be expressed in terms of conditional PDF.3 A random variable X has ⎧ 0 . We call k random variables X1 .25(c).. we have f X |Y ( x | y ) fY ( y ) = f X ( x ) fY ( y ) or f X |Y ( x | y ) = f X ( x ) (2.Y .30b) Eq. Z ( x . 2... V. the definition of statistical independence is somewhat simpler than in the case of events.. Venkata Rao 2. Y . 2. y ) = f X ( x ) fY ( y ) Making use of Eq...3. fY | X ( y | x ) = fY ( y ) (2. X 2 . y . Example 2. iff.28) and three random variables X . f X .Y ( x . X k statistically independent iff. Let X and Y be independent. We shall now give a few examples to illustrate the concepts discussed in sec.30(a) and 2.30a) Similarly..30(b) are alternate expressions for the statistical independence between X and Y . two random variables X and Y are statistically independent. z ) = f X ( x ) fY ( y ) fz ( z ) (2. Then.Y ( x . .27) i = 1 Hence. the k -dimensional joint PDF factors into the product k ∏ f X i ( xi ) (2. 0 ≤ x ≤ 10 ⎪100 K . iff f X .Principles of Communication Prof. y ) = f X ( x ) fY ( y ) (2.3. f X . 2. Z are independent. x < 0 ⎪ FX ( x ) = ⎨ K x 2 . x > 10 ⎩ i) Find the constant K ii) Evaluate P ( X ≤ 5 ) and P ( 5 < X ≤ 7 ) 2..23 Indian Institute of Technology Madras .5 Statistical independence In the case of random variables.

2. x ≥ 0. − ∞ < x < ∞ where a and b are positive constants. In order for f X ( x ) to ∞ ∞ ∫ dx = 2 ∫ a e − b x dx = 1 .24 Indian Institute of Technology Madras .  Example 2. ii) Determine the corresponding FX ( x ) iii) Find P [1 < X ≤ 2] . Venkata Rao iii) What is f X ( x ) ? 1 i) FX ( ∞ ) = 100 K = 1 ⇒ K = . ∫ a e − b x dx = .25 ⎝ 100 ⎠ P ( 5 < X ≤ 7 ) = FX ( 7 ) − FX ( 5 ) = 0.4 Consider the random variable X defined by the PDF fX ( x ) = a e −bx . i) Determine the relation between a and b so that f X ( x ) is a PDF. V. we require ae −∞ 0 ∞ 1 That is. −bx represent a legitimate PDF. 100 ⎛ 1 ⎞ ii) P ( x ≤ 5 ) = FX ( 5 ) = ⎜ ⎟ × 25 = 0. 0 ≤ x ≤ 10 dx ⎪0 ⎩ . x < 0 fX ( x ) = ⎨ − b x ⎪⎩a e .Principles of Communication Prof. hence b = 2 a . x > 10 Note: Specification of f X ( x ) or FX ( x ) is complete only when the algebraic expression as well as the range of X is given. x < 0 d FX ( x ) ⎪ fX ( x ) = = ⎨0. 0 2 ii) the given PDF can be written as ⎧⎪a e b x .02 x .24 ⎧0 . i) As can be seen f X ( x ) ≥ 0 for − ∞ < x < ∞ .

FX ( x ) = 1 − e .5 ⎧1 ⎪ .5 ) (b) Are X and Y independent? f X . y ) = ⎨ 2 ⎪⎩0 .Principles of Communication Prof.25 Indian Institute of Technology Madras . −∞ 2 2 Consider x > 0 . x ≥ 0 ⎪⎩ 2 iii) FX ( 2 ) − FX (1) = 1 −b 2 ( e − e− 2 b )  Example 2. x < 0 FX ( x ) = ⎨ 2 ⎪1 − 1 − bx e . otherwise Find (a) i) fY | X ( y | 1) and ii) fY | X ( y | 1. Take a specific value of x = 2 2 FX ( 2 ) = ∫ f (x) d x X −∞ ∞ = 1 − ∫ fX ( x ) d x 2 ∞ −2 But for the problem on hand. y ) (a) fY | X ( y | x ) = fX ( x ) 2. we have x b bα 1 FX ( x ) = ∫ e d α = eb x . Venkata Rao For x < 0 . 0 ≤ y ≤ 2 Let f X . ∫ f X ( x ) d x = ∫ f (x) d x X 2 −∞ 1 − bx Therefore. x > 0 2 We can now write the complete expression for the CDF as ⎧ 1 bx ⎪⎪ e . V.Y ( x .Y ( x . 0 ≤ x ≤ y.

y ≥ x . Venkata Rao f X ( x ) can be obtained from f X . V. The maximum value taken by y is 2 .Y by integrating out the unwanted variable y over the appropriate range.26 Indian Institute of Technology Madras . otherwise 4 b) the dependence between the random variables X and Y is evident from the statement of the problem because given a value of X = x1 . 2 1 x fX ( x ) = ∫2 dy = 1− . Also we see that fY | X ( y | x ) depends on x whereas if X and Y were to be independent.  2. 1 ≤ y ≤ 2 (i) 1 ⎨ 2 ⎩0 .5 ) = 2 = ⎨ 1 ⎩0 . 1 fY | X ( y | 1) = 2 = ⎧1 . Hence. Y should be greater than or equal to x1 for the joint PDF to be non zero.5 ≤ y ≤ 2 (ii) fY | X ( y | 1.Principles of Communication Prof. in addition. 0 ≤ x ≤ 2 x 2 Hence. 1. otherwise 1 ⎧2 . for any given x . then fY | X ( y | x ) = fY ( y ) .

(Fig. the following density functions have been given.4 Transformation of Variables The process of communication involves various operations such as modulation.3 For the two random variables X and Y .6) Fig.27 Indian Institute of Technology Madras . 2. In performing these operations. V. 0 ≤ y ≤ 10 fY ( y ) = ⎨ ⎪ 1 − y .Y ( x . y ) b) Show that ⎧ y ⎪⎪100 . 2. given the transformation and the input random variables. detection. 2. we typically generate new random variables from the given ones. Venkata Rao Exercise 2.3 Find a) f X . We will now develop the necessary theory for the statistical characterization of the output random variables. 10 < y ≤ 20 ⎪⎩ 5 100 2.6: PDFs for the exercise 2. filtering etc.Principles of Communication Prof.

To find fY ( y ) . x1 ......31) g ' ( x1 ) g ' ( xn ) where g ' ( x ) is the derivative of g ( x ) . We see that the equation y 1 = g ( x ) has three roots namely. .1). Venkata Rao 2.Principles of Communication Prof. 2... V.. x2 and x3 . obtained from X by the transformation Y = g ( X ) . 2. Denoting its real roots by xn .... we will show that f X ( x1 ) f X ( xn ) fY ( y ) = + ..4....7.. solve the equation y = g ( x ) . y = g ( x1 ) = . Let Y be the new random variable.. This can be obtained with the help of the following theorem (Thm.1 Let Y = g ( X ) . Theorem 2....... 2.1 Functions of one random variable Assume that X is the given random variable of the continuous type with the PDF.... By this we mean the number Y ( s1 ) associated with any sample point s1 is Y ( s1 ) = g ( X ( s1 ) ) Our interest is to obtain fY ( y ) ... .. = g ( xn ) = ... for the specific y . + + . f X ( x ) .28 Indian Institute of Technology Madras . (2. Proof: Consider the transformation shown in Fig.

we need to find the set of values x such that y1 < g ( x ) ≤ y1 + d y and the probability that X is in this set.7. x2 + d x2 < x ≤ x2 . d x2 = g ' ( x2 ) dy P [ x3 < X ≤ x3 + d x 3 ] = f X ( x 3 ) d x3 .29 Indian Institute of Technology Madras . dy P [ x1 < X ≤ x1 + d x1 ] = f X ( x1 ) d x1 . this set consists of the following intervals: x1 < x ≤ x1 + d x1 . for any given y 1 . Venkata Rao Fig. d x3 > 0 . As we see from the figure. Therefore. V. x3 < x ≤ x3 + d x3 where d x1 > 0 . 2. it follows that P [ y1 < Y ≤ y1 + d y ] = P [ x1 < X ≤ x1 + d x1 ] + P [ x2 + dx2 < X ≤ x2 ] + P [ x3 < X ≤ x 3 + d x3 ] This probability is the shaded area in Fig.7: X − Y transformation used in the proof of theorem 2.1 We know that fY ( y ) d y = P [ y < Y ≤ y + d y ] . but d x2 < 0 .Principles of Communication Prof. d x 3 = g ' ( x3 ) 2. d x1 = g ' ( x1 ) dy P [ x2 + d x2 < x ≤ x2 ] = f X ( x2 ) d x2 . 2. From the above.

0 ) .32.31 follows. and x ranges from the interval ( − 1. then we have P [Y = y1 ] = P ( x0 < X ≤ x1 ) = FX ( x1 ) − FX ( x0 ) . where a is a constant. V. we have g '(x) fY ( y ) = f X ( y − a ) ⎧⎪1 − x . Let us find fY ( y ) . Note that if g ( x ) = y1 = constant for every x in the interval ( x0 . Hence. fY ( y ) = 1 − y + 1 As y = x − 1.Principles of Communication Prof. by canceling d y from the Eq. there is a unique x satisfying the above transformation. Example 2. 2. that is FY ( y ) is discontinuous at y = y1 . elsewhere and a = − 1 Then. For a given y .32) g ' ( x1 ) g ' ( x2 ) g ' ( x3 ) and Eq. Venkata Rao We conclude that f X ( x1 ) f X ( x2 ) f X ( x3 ) fY ( y ) d y = dy + dy + dy (2. fX ( x ) fY ( y ) d y = d y and as g ' ( x ) = 1. x ≤ 1 Let f X ( x ) = ⎨ ⎪⎩ 0 . x1 ) . 2. We have g ' ( x ) = 1 and x = y − a . δ ( y − y1 ) of area FX ( x1 ) − FX ( x0 ) .  We shall take up a few examples. Hence fY ( y ) contains an impulse.6 Y = g ( X ) = X + a .30 Indian Institute of Technology Madras . 2. 1) . we have the range of y as ( − 2.

the transformation of adding a constant to the given variable simply results in the translation of its PDF.  Example 2. − 2 ≤ y ≤ 0 Hence fY ( y ) = ⎨ ⎪⎩ 0 . where b is a constant.8. Let us find fY ( y ) . 2. elsewhere FX ( x ) and FY ( y ) are shown in Fig. we have fY ( y ) = fX ⎜ ⎟ . Fig. then 2.31 Indian Institute of Technology Madras . otherwise and b = − 2 .7 Let Y = b X . Again for a given y . there is a unique b 1 ⎛y⎞ x .Principles of Communication Prof. V. we have X = Y . 2. 0 ≤ x ≤ 2 Let f X ( x ) = ⎨ 2 ⎪⎩ 0 . 1 Solving for X .8: FX ( x ) and FY ( y ) of example 2.6 As can be seen. b ⎝b⎠ ⎧ x ⎪1 − . Venkata Rao ⎧⎪1 − y + 1 . As g ' ( x ) = b .

32 Indian Institute of Technology Madras . − 4 ≤ y ≤ 0 ⎣ ⎦ ⎪ ⎩ 0 .7  Exercise 2.Principles of Communication Prof. 2. compute and sketch fY ( y ) for a = − 2 . V.2. otherwise f X ( x ) and fY ( y ) are sketched in Fig. 2. a ⎝ a ⎠ If f X ( x ) is as shown in Fig. Fig.9. and b = 1. 2.9: f X ( x ) and fY ( y ) of example 2. Venkata Rao ⎧1 ⎡ y⎤ ⎪ fY ( y ) = ⎨ 2 ⎢1 + 4 ⎥ . where a and b are constants.8.8 Y = a X 2 . Show that 1 ⎛y − b⎞ fY ( y ) = fX ⎜ ⎟. Example 2. Let us find fY ( y ) . a > 0 .4 Let Y = a X + b .

V.  Example 2. Venkata Rao g ' ( x ) = 2 a x . X > 0 a) Let us find the general expression for fY ( y ) ⎧1 1 3 ⎪ . otherwise We shall compute and sketch fY ( y ) . y ≥ 0 = ⎨ 2π y ⎝ 2⎠ ⎪ ⎩ 0 . − < x < b) Let f X ( x ) = ⎨ 2 2 2 ⎪⎩ 0 . −∞ < x < ∞ 2π ⎜ 2 ⎟⎟ ⎝ ⎠ (Note that exp ( α ) is the same as e α ) 1 ⎡ ⎛ y⎞ ⎛ y ⎞⎤ Then fY ( y ) = ⎢ exp ⎜ − 2 ⎟ + exp ⎜ − 2 ⎟ ⎥ 2 y 2π ⎣ ⎝ ⎠ ⎝ ⎠⎦ ⎧ 1 ⎛ y⎞ ⎪ exp ⎜ − ⎟ . 2. otherwise Sketching of f X ( x ) and fY ( y ) is left as an exercise.31 yields a ⎧ 1 ⎡ ⎛ y⎞ ⎛ y ⎞⎤ ⎪ ⎢f X ⎜⎜ ⎟⎟ + f X ⎜⎜ − ⎟⎥ . y Hence fY ( y ) = 0 for y < 0 .33 Indian Institute of Technology Madras . If y ≥ 0 .Principles of Communication Prof. then it has two solutions.9 Consider the half wave rectifier transformation given by ⎧0 . 2. x1 = and a y x2 = − . X ≤ 0 Y =⎨ ⎩X . otherwise 1 ⎛ x2 ⎞ Let a = 1 . If y < 0 . y ≥ 0 ⎪ a ⎟⎠ ⎦⎥ fY ( y ) = ⎨ 2a y ⎣⎢ ⎝ a⎠ ⎝ ⎪ a ⎪⎩ 0 . and Eq. then the equation y = a x 2 has no real solution. and f X ( x ) = exp ⎜ − .

y ≥ 0 where w ( y ) = ⎨ ⎩0 . otherwise ⎧1 . As Y is nonnegative. otherwise ⎧1 1 3 ⎪ . As Y = X for x > 0 . fY ( y ) = ⎨ . otherwise ⎪ ⎩ f X ( x ) and fY ( y ) are sketched in Fig. 2. Hence ⎪⎧f ( y ) w ( y ) + FX ( 0 ) δ ( y ) fY ( y ) = ⎨ X ⎪⎩0 . 0 < y ≤ ⎪2 2 ⎪0 .10.34 Indian Institute of Technology Madras . Fig. elsewhere ⎧1 ⎪4 δ(y ) . 0 ) . 2. − <x< b) Specifically. fY ( y ) = 0 for y < 0 .9 2. V.Principles of Communication Prof.10: f X ( x ) and fY ( y ) for the example 2. there is an impulse in fY ( y ) at y = 0 whose area is equal to FX ( 0 ) . Venkata Rao a) Note that g ( X ) is a constant (equal to zero) for X in the range of ( − ∞ . we have fY ( y ) = f X ( y ) for y > 0 . y = 0 ⎪ ⎪1 3 Then. let f X ( x ) = ⎨ 2 2 2 ⎪⎩0 . Hence.

35 Indian Institute of Technology Madras .10 Note that this transformation has converted a continuous random variable X into a discrete random variable Y . we have P1 = and P− 1 = . Hence the PDF of Y has only two impulses. V. Fig. a) In this case. X ≥ 0 a) Let us find the general expression for fY ( y ) . 2. namely ±1.  Example 2. a continuous RV is transformed into a mixed RV.10 ⎧− 1. Let us write fY ( y ) as fY ( y ) = P1 δ ( y − 1) + P−1 δ ( y + 1) where P− 1 = P [ X < 0 ] and P1 [ X ≥ 0 ] 3 1 b) Taking f X ( x ) of example 2. Y assumes only two values. Fig.9. X < 0 Let Y = ⎨ ⎩+ 1.9. Venkata Rao Note that X . b) We shall compute and sketch fY ( x ) assuming that f X ( x ) is the same as that of Example 2.11 4 4 has the sketches f X ( x ) and fY ( y ) . Y . 2.11: f X ( x ) and fY ( y ) for the example 2.Principles of Communication Prof.  2.

2.12(a) be the input to a device with the input-output characteristic shown in Fig. Compute and sketch fY ( y ) .12(b). 2.13. 2. Fig.5 Let a random variable X with the PDF shown in Fig.5 (b) Input-output transformation Exercise 2.5 is applied as input to the X − Y transformation shown in Fig. 2.36 Indian Institute of Technology Madras . V. Venkata Rao Exercise 2.12: (a) Input PDF for the transformation of exercise 2. Compute and sketch fY ( y ) .Principles of Communication Prof. 2.6 The random variable X of exercise 2.

1. If y k = g ( x ) for only one x = xk . Example 2. 6 3 3 6 That is.. . 22 . 2.2 Functions of two random variables Given two random variables X and Y (assumed to be continuous type).. Z and W are defined using the transformation Z = g ( X . two new random variables..4. .11 Let Y = X 2 . 3 with probability . Y ) and W = h ( X . fY ( y ) = 2 6 6 i = 1 1 b) If. In this case. ..Principles of Communication Prof.. 1. however. 4. Y = g ( X ) is also discrete.... Y ) 2. . 1 1 fY ( y ) = ⎡⎣δ ( y ) + δ ( y − 9 ) ⎤⎦ + ⎡⎣δ ( y − 1) + δ ( y − 4 ) ⎤⎦ 6 3  2... Venkata Rao We now assume that the random variable X is of discrete type taking on the value x k with probability Pk . find fY ( y ) . 6 1 1 values 12 . V.. respectively.. i = 1 3 1 b) If f X ( x ) = 6 ∑ δ ( x − i ) . the RV.. then P [Y = y k ] = P [ X = xk ] = Pk . then 6 1 1 1 1 Y takes the values 0. If however. 6 1 a) If f X ( x ) = 6 ∑ δ ( x − i ) . 6 ) with probability of . find fY ( y ) . then P [Y = y k ] = Pk + Pm . y k = g ( x ) for x = xk and x = xm . 0. 62 with probability . − 1.37 Indian Institute of Technology Madras . i = −2 1 a) If X takes the values (1.. then Y takes the 6 ∑ δ(x − i ) . X takes the values − 2. 2. 9 with probabilities . assuming the value Yk = g ( x k ) with probability Pk . That is.

solve the system of equations g ( x .W ( z . y1 ) satisfying Eq. y ⎠ ∂w ∂w ∂x ∂y ∂ z ∂w ∂ z ∂w = − ∂x ∂y ∂y ∂x That is. h ( x .33a) h ( x . y ) = z1 . For a more general version of this theorem. Venkata Rao Given the above transformation and f X .W ( z . we require the Jacobian of the transformation.Principles of Communication Prof.34) ⎛ z. w ⎞ J⎜ ⎟ where ⎝ x. y1 ⎠ Proof of this theorem is given in appendix A2. w ) . y1 ) fZ .Y ( x1 . We shall assume that the transformation is one-to-one. 2. given g ( x . Let ( x1 . For this. w ) and f X .W ( z .W ( z . ( x1 . w ⎞ ∂x ∂y J⎜ ⎟ = ⎝ x. y ) = w1 . Theorem 2. refer [1]. Then. 2.38 Indian Institute of Technology Madras . the Jacobian is the determinant of the appropriate partial derivatives. for x and y . y ) = z1 .Y ( x . y ) . f X .2: To obtain fZ .33b) then there is a unique set of numbers. w ) . V.Y ( x . (2. w ⎞ J⎜ ⎟ ⎝ x1 .1. y ) = w1 . y ⎠ ∂z ∂z ⎛ z. y ) . That is. denoted ⎛ z. w ) = (2. we would like to obtain fZ . We shall now state the theorem which relates fZ . (2.33. y1 ) be the result of the solution.

y ) is non-zero.14: (a) The space where f X . we obtain x = (z + w ) and 2 1 y = ( z − w ) . Fig. Fig. let us find (a) fZ . 2.Principles of Communication Prof. Venkata Rao Example 2. y ≤ 1 2 If Z = X + Y and W = X − Y . V.Y is non-zero (b) The space where fZ . 2.12 X and Y are two independent RVs with the PDFs .W ( z .39 Indian Institute of Technology Madras . We see that the mapping is one-to-one. 1 fX ( x ) = .W is non-zero 2. 1 a) From the given transformations.14(a) 2 depicts the (product) space A on which f X . w ) and (b) fZ ( z ) .Y ( x . x ≤ 1 2 1 fY ( y ) = .

The Jacobian of the transformation is ⎛ z. w ) = 4 = ⎨8 2 ⎪⎩0 . for a given z ( z ≥ 0 ). 2. we have −z 1 1 fZ ( z ) = ∫ 8 dw = − z. otherwise ∞ b) fZ ( z ) = ∫ f (z. w ⎞ 1 1 J⎜ ⎟ = = − 2 and J ( ) = 2. V. y ⎠ 1 −1 1 ⎧1 ⎪ .14(b). w ) d w Z . −2 ≤ z ≤ 0 z 4 2. 0 ≤ z ≤ 2 −z 4 For z negative.W (z. w ∈ B Hence fZ .40 Indian Institute of Technology Madras . 2. w can take values only in the − z to z .14(b).W ( z .W −∞ From Fig. we can see that. Hence + z 1 1 fZ ( z ) = ∫ 8 dw = z.Principles of Communication Prof. w ) is non-zero) as follows: A space B space 1 The line x = 1 (z + w ) = 1 ⇒ w = − z + 2 2 1 The line x = − 1 ( z + w ) = − 1 ⇒ w = − ( z + 2) 2 1 The line y = 1 (z − w ) = 1 ⇒ w = z − 2 2 1 The line y = − 1 (z − w ) = − 1 ⇒ w = z + 2 2 The space B is shown in Fig. Venkata Rao We can obtain the space B (on which fZ . ⎝ x. z.

To apply the theorem. Typically W = X or W = Y . − ∞ < x. y < ∞ . Φ ( r . − π < ϕ < π Hence. fR . w ) is found from which fZ ( z ) can be obtained. using the theorem fZ . a conveniently chosen auxiliary or dummy variable W is introduced.Principles of Communication Prof. ϕ ) = ⎨ 2 π ⎝ 2⎠ ⎪ ⎩0 . we can write x = r cos φ and y = r sin φ . y ) = exp ⎢ − ⎜ ⎟⎥ .2 can also be used when only one function Z = g ( X .13 Let the random variables R and Φ be given by. Y ) is specified and what is required is fZ ( z ) .41 Indian Institute of Technology Madras . 2π ⎣ ⎝ 2 ⎠⎦ Let us find fR . It is given that ⎝X⎠ 1 ⎡ ⎛ x 2 + y 2 ⎞⎤ f X . fΦ ( ϕ ) and to show that R and Φ are independent variables. As the given transformation is from cartesian-to-polar coordinates. ϕ ) .  Theorem 2. 0 ≤ r ≤ ∞ . and the transformation is one -to -one. z ≤ 2 4  Example 2. R = X 2 + Y 2 and ⎛Y ⎞ Φ = arc tan ⎜ ⎟ where we assume R ≥ 0 and − π < Φ < π . Φ ( r . otherwise It is left as an exercise to find fR ( r ) . ∂r ∂r cos ϕ sin ϕ ∂x ∂y 1 J = = − sin ϕ cos ϕ = ∂ϕ ∂ϕ r r r ∂x ∂y ⎧ r ⎛ r2 ⎞ ⎪ exp ⎜ − ⎟ . V. 2.W ( z .Y ( x . Venkata Rao 1 Hence fZ ( z ) = z .

2. w ) ∞ and fZ ( z ) = ∫ f (z − w . fZ ( z ) is the convolution of f X ( z ) and fY ( z ) . with ⎧1 ⎪ . we obtain fZ ( z ) as shown in Fig. 2.36b) Example 2. otherwise If Z = X + Y .Y (2. 2. −1 ≤ x ≤ 1 fX ( x ) = ⎨ 2 ⎪⎩0 .W ( z . w ) d w X .Y ( z − w .Principles of Communication Prof. and Y = W . −2 ≤ y ≤ 1 fY ( y ) = ⎨ 3 ⎪⎩0 . w ) = f X .14 Let X and Y be two independent random variables. From Eq.35) −∞ If X and Y are independent.35 becomes ∞ fZ ( z ) = ∫ f ( z − w ) f (w ) d w X Y (2.36a) −∞ That is. As J = 1 .42 Indian Institute of Technology Madras . Venkata Rao Let Z = X + Y and we require fZ ( z ) . 2. fZ .36(b).15. X = Z − W . fZ = f X ∗ fY (2. otherwise ⎧1 ⎪ . let us find P [ Z ≤ − 2] . V. then Eq. Then. Let us introduce a dummy variable W = Y . Carrying out the convolution.

12  Example 2. let us find an expression for fZ ( z ) .15 X Let Z = . x ≤ 1. w ) dw −∞ ⎧1 + x y ⎪ .Principles of Communication Prof.Y ( x . elsewhere ∞ 1 + zw 2 Then fZ ( z ) = ∫ w dw −∞ 4 2.14 1 P [ Z ≤ − 2] is the shaded area = . we have X = ZW Y = W 1 As J = . y ≤ 1 Let f X . y ) = ⎨ 4 ⎪⎩0 . we have w ∞ fZ ( z ) = ∫ w f X .Y ( zw .15: PDF of Z = X + Y of example 2.43 Indian Institute of Technology Madras . 2. Venkata Rao Fig. V. Y Introducing the dummy variable W = Y .

w ) ∈ B .Y is non-zero (b) The space where fZ . 2.Principles of Communication Prof. y ) ∈ A .16a). Let fZ .2. B will be as shown in Fig. i) Let z < 1. we have − 1 z 1 + zw 2 1⎛ 1 1 ⎞ fZ ( z ) = 2 ∫ w dw = ⎜ 2 + ⎟ 0 4 4 ⎝z 2 z3 ⎠ 2.W ( z . we have to integrate out w over the appropriate ranges. then 1 + zw 2 1 + zw 2 1 1 fZ ( z ) = ∫ w dw = 2 ∫ w dw −1 4 0 4 1⎛ z⎞ = ⎜ 1+ ⎟ 4⎝ 2⎠ ii) For z > 1 . 2. Fig. where A is the product space x ≤ 1 and y ≤ 1 (Fig. w ) . Under the given transformation.W ( z .W is non-zero To obtain fZ ( z ) from fZ .16(b). y ) is non-zero if (x.Y ( x .16: (a) The space where f X . w ) be non-zero if ( z . V.44 Indian Institute of Technology Madras . Venkata Rao f X . we have 1 z 1 + zw 2 1⎛ 1 1 ⎞ fZ ( z ) = 2 ∫0 4 w dw = 4 ⎜⎝ z2 + 2 z3 ⎟⎠ iii) For z < − 1.

Venkata Rao ⎧1 ⎛ z⎞ ⎪4 ⎜1 + 2 ⎟ . ∞ 1 ⎛z ⎞ a) Show that fZ ( z ) = ∫ f X . The output of the channel is given by Z = X + Y 2 2. 2π In transformations involving two random variables. w ⎟ d w −∞ w ⎝w ⎠ b) X and Y be independent with 1 fX ( x ) = .7 Let Z = X Y and W = Y .Y ⎜ . otherwise 1 2 Show that fZ ( z ) = −z e 2 .Principles of Communication Prof. such cases are handled better. z ≤ 1 ⎪ ⎝ ⎠ Hence fZ ( z ) = ⎨ ⎪1 ⎛ 1 1 ⎞ ⎪⎩ 4 ⎜ 2 + ⎟.45 Indian Institute of Technology Madras . we may encounter a situation where one of the variables is continuous and the other is discrete. Example 2. z > 1 ⎝z 2 z3 ⎠  Exercise 2.16 The input to a noisy channel is a binary random variable with 1 P [ X = 0] = P [ X = 1] = . x ≤ 1 π 1 + x2 ⎧⎪ 2 −y and fY ( y ) = ⎨ y e 2 . y ≥ 0 ⎪⎩0 . by making use of the distribution function approach. as illustrated below. − ∞ < z < ∞. V.

Y ( x .1 or 2. V.) We shall illustrate the distribution function method with another example. we have P [ Z ≤ z | X = 0 ] = FY ( z ) Similarly P [ Z ≤ z | X = 1] = P ⎡⎣Y ≤ ( z − 1) ⎤⎦ = FY ( z − 1) 1 1 d Hence FZ ( z ) = FY ( z ) + FY ( z − 1) . Obtain fZ ( z ) . Similarly. Of course computing the CDF may prove to be quite difficult in certain situations. is a very basic method and can be used in all situations.46 Indian Institute of Technology Madras . 2. if d FY ( y ) Y = g ( X ) . we have 2 2 dz ⎡ ⎛ z2 ⎞ ⎛ z − 1 2 ⎞⎤ fZ ( z ) = 1⎢ 1 exp ⎜ − ⎟+ 1 exp ⎜ − ( ) ⎟⎥  2 ⎢ 2π ⎜ 2 ⎟ 2π ⎜ 2 ⎟⎥ ⎣ ⎝ ⎠ ⎝ ⎠⎦ The distribution function method. we compute FY ( y ) and then obtain fY ( y ) = .2 is called change of variable method. Find 2π fZ ( z ) . Venkata Rao y2 1 − where Y is the channel noise with fY ( y ) = e 2 .Principles of Communication Prof. as illustrated by means of example 2. y ) . The method of obtaining PDFs based on theorem 2. Example 2. As fZ ( z ) = FZ ( z ) . (In this method. − ∞ < y < ∞ . given f X . P ( Z ≤ z ) = P [ Z ≤ z | X = 0] P [ X = 0 ] + P [ Z ≤ z | X = 1] P [ X = 1] As Z = X + Y .16.17 Let Z = X + Y . Let us first compute the distribution function of Z from which the density function can be derived. for dy transformations involving more than one random variable.

47 Indian Institute of Technology Madras . Venkata Rao P [Z ≤ z ] = P [ X + Y ≤ z ] = P [Y ≤ z − x ] This probability is the probability of ( X .Y ( x .37a) It is not too difficult to see the alternative form for fZ ( z ) .Principles of Communication Prof. Y ) lying in the shaded area shown in Fig. y ) d y ⎥⎥ d x ⎣ ⎦ ∂ ⎡ ⎡z − x ⎤⎤ ∞ fZ ( z ) = ⎢ ∫ d x ⎢ ∫ f X .Y (2. namely.Y ( x . 2. 2. Fig. z − x) d x −∞ X .17. 2. V. y ) d y ⎥ ⎥ ∂ z ⎢⎣ − ∞ ⎣⎢ − ∞ ⎦⎥ ⎥⎦ ∞ ⎡ ∂ z− x ⎤ = ∫− ∞ d x ⎢⎢ ∂ z ∫ f X .Y ( x .17: Shaded area is the P [ Z ≤ z ] That is. ∞ ⎡z − x ⎤ FZ ( z ) = ∫− ∞ ⎢⎢ −∫∞ fX . y ) d y ⎥ ⎣ −∞ ⎥⎦ ∞ = ∫ f (x.

the expected value of a function of X . This can be illustrated as follows: Three small similar discs.  So far we have considered the transformations involving one or two variables.2 and 2 respectively are placed in bowl and are mixed.37(b) is the same as Eq.Principles of Communication Prof.48 Indian Institute of Technology Madras . 2.35. Venkata Rao ∞ fZ ( z ) = ∫ f (z − y . numbered 1. mathematical expectation or simply expectation) of random variable X is defined as ∞ mX = E [ X ] = X = ∫ x fX ( x ) d x (2. we might be in a situation where the PDF is not available but are able to estimate (with reasonable accuracy) certain (statistical) averages of the random variable.39) −∞ Remarks: The terminology expected value or expectation has its origin in games of chance. V. y ) d y X . 2.37b) −∞ If X and Y are independent. Note that m X is a constant.5 Statistical Averages The PDF of a random variable provides a complete statistical characterization of the variable. Some of these averages do provide a simple and fairly adequate (though incomplete) description of the random variable. We now define a few of these averages and explore their significance.Y (2. Similarly. However.38) −∞ where E denotes the expectation operator. The mean value (also called the expected value. is defined by ∞ E ⎡⎣g ( X ) ⎤⎦ = g ( X ) = ∫ g (x) f (x) d x X (2. we have fZ ( z ) = f X ( z ) ∗ fY ( z ) . This can be generalized to the case of functions of n variables. 2. g ( X ) . We note that Eq. Details can be found in [1. 2]. A player is to be 2.

41) −∞ If g ( X ) = ( X − m X ) . Consider a function of two random variables.Principles of Communication Prof. For the special case of g ( X ) = X n . that is. if he draws either disc numbered 2. that is. g ( X . V. If he draws the disc numbered 1. Venkata Rao blindfolded and is to draw a disc from the bowl. Y ) . X . resulting in the mean square value of X ). Note that g ( x ) is such that g (1) = 9 and g ( 2 ) = 3 . 2. ∞ E ⎡⎣ X n ⎤⎦ = ∫ x n fX ( x ) d x (2. the mathematical expectation of g ( X ) is precisely the player's claim or expectation [3]. If we take X to be (discrete) random variable with the PDF 1 2 fX ( x ) = δ ( x − 1) + δ ( x − 2 ) and g ( X ) = 15 − 6 X . or five dollars. It seems reasonable to assume that the player has a '1/3 claim' on the 9 dollars and '2/3 claim' on three dollars.42) ⎣ ⎦ X −∞ We can extend the definition of expectation to the case of functions of k ( k ≥ 2 ) random variables. Then. n ∞ E ⎡( X − m X ) ⎤ = ∫ (x − m ) fX ( x ) d x n n (2.38) and the second moment ( n = 2 .49 Indian Institute of Technology Madras . he will receive nine dollars. then 3 3 ∞ E ⎣⎡g ( X ) ⎦⎤ = ∫ (15 − 6 x ) f ( x ) d x X = 5 −∞ That is. then E ⎡⎣ g ( X ) ⎤⎦ gives n-th central moment. His total claim is 9(1/3) + 3(2/3).40) −∞ The most widely used moments are the first moment ( n = 1. he will receive 3 dollars. 2. which results in the mean value of Eq. we obtain the n-th moment of the probability distribution of the RV. ∞ E ⎡⎣ X 2 ⎤⎦ = X 2 = ∫ x 2 fX ( x ) d x (2.

V. From Eq. Venkata Rao ∞ ∞ E ⎡⎣g ( X . E ⎡⎣ X 2 − 2 m X X + m X2 ⎤⎦ = E ⎡⎣ X 2 ⎤⎦ − 2 m X E [ X ] + m X2 = E ⎡⎣ X 2 ⎤⎦ − 2 m X2 + m X2 2 = X 2 − m X2 = X 2 − ⎡⎣ X ⎤⎦ 2. Y ) ⎤⎦ = ∫ ∫ g (x. we have ∞ ∞ E [Z ] = ∫ α x f (x) d x + ∫ β y f (y ) d y X Y −∞ −∞ = α X + βY 2. y ) f (x.1 Variance Coming back to the central moments.50 Indian Institute of Technology Madras . Y ) = α X + βY where α and β are constants.Y − ∞− ∞ ∞ ∞ ∞ ∞ = ∫ ∫ ( α x ) fX .Y (2.43. if Z = g ( X .43) − ∞− ∞ An important property of the expectation operator is linearity. we have the first central moment being always zero because.5. y ) d x dy X .Principles of Communication Prof.Y − ∞− ∞ − ∞− ∞ Integrating out the variable y in the first term and the variable x in the second term. then Z = α X + βY . ∞ E ⎡⎣( X − m X ) ⎤⎦ = ∫ (x − m ) f (x) d x X X −∞ = mX − mX = 0 Consider the second central moment E ⎡( X − m X ) ⎤ = E ⎡⎣ X 2 − 2 m X X + m X2 ⎤⎦ 2 ⎣ ⎦ From the linearity property of expectation. This result can be established as follows. y ) d x d y X . 2. that is. y ) d x dy + ∫ ∫ (β y ) f ( x . y ) d x dy X .Y ( x . we have ∞ ∞ E [Z ] = ∫ ∫ ( α x + β y ) f ( x .

Principles of Communication Prof. hence E ⎡⎣g ( X ) ⎤⎦ ≥ c ∫ f X ( x ) d x A But ∫ f X ( x ) d x = P [ x ∈ A] = P ⎡⎣g ( X ) ≥ c ⎤⎦ A That is. E ⎡⎣ g ( X ) ⎤⎦ ≥ c P ⎡⎣ g ( X ) ≥ c ⎤⎦ .3 is useful in establishing similar inequalities involving random variables. ∞ E ⎡⎣g ( X ) ⎤⎦ = ∫ g (x) f (x) d x X −∞ = ∫g (x) f (x) d x + ∫ g (x) f (x) d x A X B X Since each integral on the RHS above is non-negative. which is the desired result.3: Let g ( X ) be a non-negative function of the random variable X . then for every positive constant c . If E ⎡⎣ g ( X ) ⎤⎦ exists. we use a subscript on σ 2 ) The variance provides a measure of the variable's spread or randomness. Venkata Rao The second central moment of a random variable is called the variance and its (positive) square root is called the standard deviation.44) c Proof: Let A = { x : g ( x ) ≥ c} and B denote the complement of A . 2. Theorem 2. we can write E ⎡⎣ g ( X ) ⎤⎦ ≥ ∫g (x) f (x) d x X A If x ∈ A . E ⎡⎣g ( X ) ⎤⎦ P ⎡⎣g ( X ) ≥ c ⎤⎦ ≤ (2. (If necessary. The symbol σ 2 is generally used to denote the variance. This can be made more precise with the help of the Chebyshev Inequality which follows as a special case of the following theorem. V. then g ( x ) ≥ c .  Note: The kind of manipulations used in proving the theorem 2.51 Indian Institute of Technology Madras . Specifying the variance essentially constrains the effective width of the density function.

45(a). we expect X to occur within the range ( m X ± 2 σ X ) for more than 75% of the observations. Naturally. where the required probability is concentrated.Principles of Communication Prof. the k2 probability of X − m X ≥ 2 σ X does not exceed 1/4 or 25%. That is. k > 0 (2. 2. perhaps) the inequality 2. Chebyshev 2. smaller is the width of the interval around m X . smaller the standard deviation. 1 P ⎡( X − m X ) ≥ k 2 σ2X ⎤ ≤ 2 2 ⎣ ⎦ k In other words. By the same token.45a) k 1 ii) P ⎡⎣ X − m X < k σ X ⎤⎦ > 1 − 2 (2. Chebyshev inequality can be interpreted as: the probability of observing any RV outside ± k standard 1 deviations off its mean value is no larger than . Venkata Rao To see how innocuous (or weak.45b) k where σ X is the standard deviation of X . We then have. let g ( X ) represent the height of a randomly chosen human being with E ⎡⎣ g ( X ) ⎤⎦ = 1. Then Eq.52 Indian Institute of Technology Madras .44 states that the probability of choosing a person over 16 m tall is at 1 most ! (In a population of 1 billion. To establish 2.6m . V. With k = 2 for example. we would take the positive number k to be greater than one to have a meaningful result.44 is. 1 P ⎡⎣ X − m X ≥ k σ X ⎤⎦ ≤ 2 k which is the desired result. at most 100 million would be as tall as a 10 full grown Palmyra tree!) Chebyshev inequality can be stated in two equivalent forms: 1 i) P ⎡⎣ X − m X ≥ k σ X ⎤⎦ ≤ 2 .3. let g ( X ) = ( X − m X ) and c = k 2 σ2X in theorem 2 2.

Y ) = ( X − m X ) (Y − mY ) in Eq. V. E [ X Y ] − mX mY ρX Y = (2. if α f X ( x ) = 2 π 2 .46b) The cov [ X . y ) d x d y − ∞− ∞ ∞ ∞ = ∫ ∫ x y f X ( x ) fY ( y ) d x d y − ∞− ∞ ∞ ∞ = ∫ x fX ( x ) d x ∫ y f (y ) d y Y = m X mY −∞ −∞ 2. α +x (This is called Cauchy’s PDF) 2. For example. Then.Y ( x .Y ] . 2. Note that it is not necessary that variance exists for every PDF. That is. then X = 0 but X 2 is not finite. ∞ ∞ E[XY] = ∫ ∫ x y f X .53 Indian Institute of Technology Madras .46a) Using the linearity property of expectation. λ X Y = cov [ X . we have λ X Y = E [ X Y ] − m X mY (2. − ∞ < x < ∞ and α > 0 .Y ] = E ⎡⎣( X − m X ) (Y − mY ) ⎤⎦ (2. normalized with respect to the product σ X σY is termed as the correlation coefficient and we denote it by ρ .43. Venkata Rao inequality thus enables us to give a quantitative interpretation to the statement 'variance is indicative of the spread of the random variable'. Suppose X and Y are independent.5.47) σ X σY The correlation coefficient is a measure of dependency between the variables.Principles of Communication Prof.2 Covariance An important joint expectation is the quantity called covariance which is obtained by letting g ( X . We use the symbol λ to denote the covariance. That is.

However. Intuitively. we have ρ X Y = 0 and for the totally dependent (y = ± α x ) case. Taking the extreme case of this dependency. That is. the outcome y is conditioned on the outcome x in such a manner that there is a greater possibility of (y − mY ) being of the same sign as ( x − mX ) . V. On the other hand. As an example. then we expect ρ X Y to be negative. Then we expect ρ X Y to be positive. number of trials tends to zero as the number of trials keep increasing. ρ X Y = 1. then Y = ± α x1 . That is. α being constant.Principles of Communication Prof. . let 2. Then ρ X Y = ± 1 . When the joint experiment is performed many times. and given X = x1 . Venkata Rao Thus. Similarly if the probability of ( x − m X ) and ( y − mY ) being of the opposite sign is quite large. let X and Y be dependent. If ρ X Y ( or λ X Y ) = 0 . In the course of many trials of the experiments and with the outcome X = x1 . Assume X and Y to be independent. then ρ will have a magnitude between 0 and 1. for X and Y be independent. then Y would occur sometimes positive with respect to mY . When the random variables are independent. let X and Y be so conditioned that. they are uncorrelated. Two random variables X and Y are said to be orthogonal if E [ X Y ] = 0 . Suppose for example. If the variables are neither independent nor totally dependent. the sum of the sum numbers x1 ( y − mY ) would be very small and the quantity. and sometimes negative with respect to mY . the random variables X and Y are said to be uncorrelated. the fact that they are uncorrelated does not ensure that they are independent. this result is appealing. given X = x1 . if X and Y are uncorrelated. we have λ X Y (and ρ X Y ) being zero. then X Y = X Y .54 Indian Institute of Technology Madras .

i = 1. X Y = 0 which means λ X Y = X Y − X Y = 0 . we can show that σY2 = k1 σ12 + k 2 σ22 + 2 ρ12 k1 k 2 σ1 σ 2 (2. if X = x1 . Let ρ12 be the correlation coefficient between X1 and X 2 . But X and Y are not independent because.48a) If X1 and X 2 are uncorrelated. α ∞ 2 1 1 XY = X3 = ∫ x3 d x = ∫ x3 d x = 0 −∞ α α −α 2 As X = 0 . σY2 = E ⎡⎣Y 2 ⎤⎦ − ⎡⎣E [Y ]⎤⎦ 2 E [Y ] = k1 m1 + k 2 m2 E ⎡⎣Y 2 ⎤⎦ = E ⎡⎣ k1 X12 + k 2 X 22 + 2 k1 k 2 X1 X 2 ⎤⎦ With a little manipulation. otherwise and Y = X 2 . Venkata Rao ⎧1 α α ⎪ . the sum as well as the difference random variables have the same variance which is larger than σ12 or σ 22 ! The above result can be generalized to the case of linear combination of n variables.Principles of Communication Prof. ⎪⎩0 . V. and let Z = X1 + X 2 and W = X1 − X 2 . then σY2 = k1 σ12 + k2 σ22 (2. σ2X i = σ2i . 2 . That is.55 Indian Institute of Technology Madras . Then. if 2. then Y = x12 ! Let Y be the linear combination of the two random variables X1 and X 2 . Let E [ X i ] = mi . Then σ2Z = σW2 = σ12 + σ22 . − < x < fX ( x ) = ⎨ α 2 2. That is Y = k1 X1 + k 2 X 2 where k1 and k2 are constants.48b) Note: Let X1 and X 2 be uncorrelated. We will now relate σY2 to the known quantities. That is.

18 Let a random variable X have the CDF ⎧0 . Therefore. We shall now give a few examples based on the theory covered so far in this section. Venkata Rao n Y = ∑k i = 1 i X i .49) where the meaning of various symbols on the RHS of Eq. 0 ≤ x ≤ 2 ⎪8 fX ( x ) = ⎨ ⎪1 x . V. 4 ≤ x We shall find a) X and b) σ2X a) The given CDF implies the PDF ⎧0 . 2 ≤ x ≤ 4 ⎪8 ⎪0 4 ≤ x ⎩ . 0 ≤ x ≤ 2 ⎪8 FX ( x ) = ⎨ 2 ⎪x . then n σY2 = ∑k i = 1 i 2 σ2i + 2 ∑∑ k i k j ρi j σi σ j i j i < j (2.56 Indian Institute of Technology Madras . 2 ≤ x ≤ 4 ⎪16 ⎪ ⎩1 . 2. 2 4 1 1 2 E[X] = ∫0 8 x d x + ∫2 8 x d x 2. Example 2. x < 0 ⎪1 ⎪ . x < 0 ⎪x ⎪ .49 is quite obvious.Principles of Communication Prof.

elsewhere 2.96 2 π  Example 2.57 Indian Institute of Technology Madras . otherwise Let us find E [Y ] and σY2 .0636 − 1 π 2 1 2 1 E ⎡⎣Y ⎤⎦ = 2 ∫ cos2 ( π x ) d x = = 0. 0 < x < 1. Venkata Rao 1 7 31 = + = 4 3 12 σ2X = E ⎡⎣ X 2 ⎤⎦ − ⎡⎣E [ X ]⎤⎦ 2 b) 2 4 1 2 1 3 E ⎡⎣ X 2 ⎤⎦ = ∫0 8 x d x + ∫2 8 x d x 1 15 47 = + = 3 2 6 2 47 ⎛ 31 ⎞ 167 σ 2 = −⎜ ⎟ = X 6 ⎝ 12 ⎠ 144  Example 2.38 we have. 1 2 2 E [Y ] = ∫ cos ( π x ) d x = = 0. 2.Principles of Communication Prof.Y ( x . V. 0 < y < 1 f X .20 Let X and Y have the joint PDF ⎧ x + y .19 Let Y = cos π X . where ⎧ 1 1 ⎪1. From Eq.5 − 1 2 2 1 4 Hence σY2 = − 2 = 0. y ) = ⎨ ⎩0 . − < x < fX ( x ) = ⎨ 2 2 ⎪⎩0.

namely. V.21 Let the joint PDF of the random variables X and Y be 2. ∞ E ⎡⎣g ( X ) | Y = y ⎤⎦ = ∫ g ( x ) f ( x | y ) dx X |Y (2. y ) dx dy − ∞− ∞ 1 1 17 = ∫ ∫ x y ( x + y ) dx dy = 2 0 0 72 b) To find ρ X Y . 2. E [ X | Y ] . The quantity. σ X and σY .Y ( x .58 Indian Institute of Technology Madras . We can easily show that 7 11 48 E [ X ] = E [Y ] = .43. we have ∞ ∞ E ⎡⎣ X Y 2 ⎤⎦ = ∫ ∫ x y 2 f X .51) −∞ Similarly. ∞ E[X | Y = y] = E[X |Y ] = ∫ x f X |Y ( x | y ) dx (2.Principles of Communication Prof. then we have the conditional mean. If g ( X ) = X . E [ X ]. given Y = y .50) −∞ is called the conditional expectation of g ( X ) . E [Y ] . σ2X = σY2 = and E [ X Y ] = 12 144 144 1 Hence ρ X Y = − 11  Another statistical average that will be found useful in the study of communication theory is the conditional expectation. Example 2. we require E [ X Y ]. We shall illustrate the calculation of conditional mean with the help of an example. we can define the conditional variance etc. Venkata Rao Let us find a) E ⎡⎣ X Y 2 ⎤⎦ and b) ρ X Y a) From Eq.

we require the conditional PDF. B . say. 0 < x < 1. V.Y ( x . we shall discuss certain probability distributions which are encountered quite often in the study of communication theory.6.Y ( x .59 Indian Institute of Technology Madras . f X . 2. We will begin our discussion with discrete random variables. y < x < 1 − ln y x ln y 1 ⎡ 1 ⎤ Hence E ( X | Y ) = ∫ x ⎢⎣ − x ln y ⎥⎦ d x y y −1 = ln y  Note that E [ X | Y = y ] is a function of y .Principles of Communication Prof. Let the experiment be repeated n times and let p be the 2. 0 < y < 1 y 1 1 f X |Y ( x | y ) = x = − . 2. y ) To find E [ X | Y ] . Discrete random variables i) Binomial: Consider a random experiment in which our interest is in the occurrence or non- occurrence of an event A . 0 < y < x f X . Venkata Rao ⎧1 ⎪ . ⎪⎩0 . y ) ≡ ⎨ x . f X |Y ( x | y ) = fY ( y ) 1 1 fY ( y ) = ∫x dx = − ln y . outside Let us compute E [ X | Y ] .1. we are interested in the event A or its complement. That is.6 Some Useful Probability Models In the concluding section of this chapter.

.. The sample space (representing the outcomes of these five repetitions) has 32 sample points. say.18. ..18: Binomial RV for the special case n = 5 P ( A B A A B ) = P ( A ) P ( B ) P ( A ) P ( A ) P ( B ) . The sample point s1 could represent the sequence ABBBB .. 2.. 2.... .. as trails are independent..Principles of Communication Prof. k = 0. k = 3 .. . 1.. n . ⎛ 5⎞ P [ X = 3] = ⎜ ⎟ p3 (1 − p ) 2 ⎝ 3⎠ Generalizing this to arbitrary n and k . let n = 5 and k = 3 . = p (1 − p ) p 2 (1 − p ) = p 3 (1 − p ) 2 ⎛5⎞ There are ⎜ ⎟ = 10 sample points for which X ( s ) = 3 . n .. s32 . Let X denote random variable.. Taking a special case. given by 2.. The sample points such as ABAAB . A A A B B etc. (Each sample point is actually an element of the five dimensional Cartesian product space)... for n = 5 .. then we can write f X ( x ) .. s1 . 2..60 Indian Institute of Technology Madras .. Venkata Rao probability of occurrence of A on each trial and the trials are independent.. If we can compute P [ X = k ]. ‘number of occurrences of A in n trials'.. ⎝3⎠ In other words. we have the binomial density.... Fig. X can be equal to 0.. V. will map into real number 3 as shown in the Fig... 1.

i) E [ X ] = n p = 16 × 0. i) Find the expected number of errors per block ii) Find the probability that the number of errors per block is greater than or equal to 3. (Though the formulae for the mean and the variance of a binomial PDF are simple. 2. f X ( x ) ≥ 0 and ∞ n n ⎛n⎞ i fX ( x ) d x = ∑ ∑ ⎜i ⎟ p (1 − p ) n −i ∫ −∞ i = 0 Pi = i = 0⎝ ⎠ = ⎡⎣(1 − p ) + p ⎤⎦ n = 1 It is left as an exercise to show that E [ X ] = n p and σ2X = n p (1 − p ) .Principles of Communication Prof. Let X be the random variable representing the number of errors per block.52) ⎛n⎞ where Pi = ⎜ ⎟ p i (1 − p ) n−i ⎝i ⎠ As can be seen. Venkata Rao n fX ( x ) = ∑ P δ(x − i ) i = 0 i (2. the algebra to derive them is laborious). p ) to indicate X has a binomial PDF with parameters n and p defined above.22 A digital communication system transmits binary digits over a noisy channel in blocks of 16 digits. Then X is b (16.01) . V. 0. Assume that the probability of a binary digit being in error is 0. We write X is b ( n .01 = 0. The following example illustrates the use of binomial PDF in a communication problem.01 and that errors in various digit positions within a block are statistically independent. Example 2.16.61 Indian Institute of Technology Madras .

Chebyshev inequality results in P ⎡⎣ X ≥ 3 ⎤⎦ ≤ 0.002  Exercise 2.02 .22. ii) Poisson: A random variable X which takes on only integer values is Poisson distributed. 2. ∞ ∞ λm Evidently f X ( x ) ≥ 0 and ∫ f X ( x ) d x = 1 because ∑ m = 0 m! = eλ . we obtain. ∞ λm e = ∑ λ .8 Show that for the example 2.1) (1 − p ) i 16 − i = 1− i =0 ⎝ ⎠ = 0.Principles of Communication Prof. if ∞ λm e− λ fX ( x ) = ∑ m = 0 δ ( x − m) m! (2. Venkata Rao ii) P ( X ≥ 3 ) = 1 − P [ X ≤ 2] 2 ⎛ ⎞ 16 ∑ ⎜ i ⎟ ( 0.53) where λ is a positive constant.0196  0. we have m = 0 m! ( ) d eλ ∞ m λm − 1 1 ∞ λm dλ = eλ = ∑ m = 0 m! = λ ∑ m = 1 m m! ∞ m λm e− λ E[X] = ∑ = λ eλ e − λ = λ m = 1 m! Differentiating the series again. −∞ We will now show that E [ X ] = λ = σ2X Since.62 Indian Institute of Technology Madras . V. Note that Chebyshev inequality is not very tight.

54) ⎪0 ⎩ .19.2 Continuous random variables i) Uniform: A random variable X is said to be uniformly distributed in the interval a ≤ x ≤ b if. 3 ii) Rayleigh: An RV X is said to be Rayleigh distributed if.2. a ≤ x ≤ b fX ( x ) = ⎨ b − a (2. ⎧ 1 ⎪ .19: Uniform PDF It is easy show that (b − a) 2 a+b E[X] = and σ2X = 2 12 Note that the variance of the uniform PDF depends only on the width of the interval ( b − a ) . namely . whether X is uniform in ( − 1. 4 ) . 1) or ( 2. 2.6. Venkata Rao E ⎡⎣ X 2 ⎤⎦ = λ 2 + λ .Principles of Communication Prof. Therefore. 2.2. Hence σ2X = λ .63 Indian Institute of Technology Madras . V. it has the 1 same variance. elsewhere A plot of f X ( x ) is shown in Fig. Fig.

64 Indian Institute of Technology Madras .55) ⎪ ⎩0 .20.2. ( fR ( r ) of example 2. elsewhere where b is a positive constant. x ≥ 0 fX ( x ) = ⎨ b ⎝ 2 b⎠ (2.Principles of Communication Prof.20: Rayleigh PDF Rayleigh PDF frequently arises in radar and communication problems.2. A typical sketch of the Rayleigh PDF is given in Fig. Venkata Rao ⎧x ⎛ x2 ⎞ ⎪ exp ⎜− ⎟. 2.) Fig. V.12 is Rayleigh PDF. We will encounter it later in the study of narrow-band noise processes.

Hint: Make 0 dz the change of variable x 2 = z . ⎝ X ⎠ 2. As can be seen from the Fig. 2. −∞ < x < ∞ (2.56) 2 π σX ⎢⎣ 2 σ2X ⎥ ⎦ where m X is the mean value and σ2X the variance.3. 2.65 Indian Institute of Technology Madras . In appendix A2.Principles of Communication Prof.9 ∞ a) Let f X ( x ) be as given in Eq. we show that PT f X ( x ) as given by Eq. That is. 2 πb b) Show that if X is Rayleigh distributed. Venkata Rao Exercise 2. Then. We use the symbol N ( m X . N ( 0. 1 TP PT In this notation. 1) . 2. Note that ⎛ X − mX ⎞ ( ) if X is N m X . in the context of communication theory is the Gaussian (also called normal) density. m X and σ2X . the Gaussian PDF is completely specified by the two parameters. σ X .21. Show that ∫ f X ( x ) d x = 1.56 is a valid PDF. σ2X ) to denote the Gaussian density1.55. V. 1) denotes the Gaussian PDF with zero mean and unit variance. then E [ X ] = and 2 E ⎡⎣ X 2 ⎤⎦ = 2 b iii) Gaussian By far the most widely used PDF. The Gaussian PDF is symmetrical with respect to m X . x d x = . specified by 1 ⎡ ( x − m X )2 ⎤ fX ( x ) = exp ⎢− ⎥. then Y = 2 ⎜ σ ⎟ is N ( 0.

21: Gaussian PDF mX Hence FX ( m X ) = ∫ f (x) d x X = 0.2 at the end of the chapter.5 −∞ Consider P [ X ≥ a ] . we have ⎝ σX ⎠ ∞ z2 1 − P [ X ≥ a] = ∫ e 2 dz a − mX 2π σX ⎛ a − mX ⎞ = Q⎜ ⎟ ⎝ σX ⎠ ∞ 1 ⎛ x2 ⎞ where Q ( y ) = ∫ exp ⎜ − ⎟dx (2.Principles of Communication Prof. Q( ) function table is available in most of the text books on communication theory as well as in standard mathematical tables. 2. A small list is given in appendix A2. We have. ∞ 1 ⎡ ( x − m X )2 ⎤ P [ X ≥ a] = ∫ exp ⎢− ⎥dx 2 π σX ⎢⎣ 2 σ2X ⎥ a ⎦ This integral cannot be evaluated in closed form. 2. 1) .66 Indian Institute of Technology Madras . V.57 is N ( 0. 2.57) y 2π ⎝ 2 ⎠ Note that the integrand on the RHS of Eq. Venkata Rao Fig. By making a change of variable ⎛ x − mX ⎞ z = ⎜ ⎟ .

Principles of Communication Prof. Venkata Rao The importance of Gaussian density in communication theory is due to a theorem called central limit theorem which essentially states that: If the RV X is the weighted sum of N independent random components. Hence the instantaneous value of the noise can be fairly adequately modeled as a Gaussian variable. fY ( y ) given by. Example 2. then FX ( x ) approaches Gaussian as N becomes large. each particle making an independent contribution of the same amount. a) Show that Y is log-normal b) Find E (Y ) c) If m is such that FY ( m ) = 0. regardless of the distribution of the individual components. ⎧ 1 ⎡ ( ln y − α )2 ⎤ ⎪⎪ exp ⎢− ⎥. to the total.5 . you may refer [1-3].67 Indian Institute of Technology Madras . Let Y have the PDF. a) Let X = ln Y or x = ln y (Note that the transformation is one-to-one) 2. The electrical noise in a communication system is often due to the cumulative effects of a large number of randomly moving charged particles. For a more precise statement and a thorough discussion of this theorem. y ≥ 0 fY ( y ) = ⎨ 2 π y β ⎢⎣ 2 β2 ⎥⎦ ⎪ ⎪⎩ 0 . where each component makes only a small contribution to the sum. in Chapter 7. find m . otherwise where α and β are given constants. V. we shall make use of this theory in our studies on the noise performance of various modulation schemes. We shall develop Gaussian random processes in detail in Chapter 3 and.23 A random variable Y is said to have a log-normal PDF if X = ln Y has a Gaussian (normal) PDF.

then P [ X ≤ ln m ] = 0. y < ∞ given by. ln m = α or m = e α  iv) Bivariate Gaussian As an example of a two dimensional density. Venkata Rao dx 1 1 = → J = dy y y Also as y → 0 . y ) . f X . we will consider the bivariate Gaussian PDF. β2 ) ⎡ − ( x − α) ⎤ 2 ∞ 1 x ⎢ ⎥dx Y = E ⎡⎣e X ⎤⎦ = ∫ 2 β2 b) e e 2π β − ∞ ⎢ ⎥ ⎣ ⎦ ⎡∞ ⎡ x − ( α + β2 ) ⎤ ⎤ 2 β2 − ⎣ ⎦ α+ ⎢ 1 ⎥ ⎢∫ 2 β2 = e 2 e d x⎥ ⎢− ∞ 2 π β ⎥ ⎣ ⎦ As the bracketed quantity being the integral of a Gaussian PDF between the limits ( − ∞ .Y ( x . − ∞ < x . 1 ⎧ 1 ⎡ ( x − m X )2 ( y − mY )2 ( x − mX ) ( y − mY ) ⎤⎫ fX . x → − ∞ and as y → ∞ . x → ∞ 1 ⎡ ( x − α )2 ⎤ Hence f X ( x ) = exp ⎢− ⎥.58) σX σY σ X σY 2 2 k1 ⎩ k2 ⎣ ⎦⎭ where.5 .68 Indian Institute of Technology Madras . y ) = exp ⎨− ⎢ + −2ρ ⎥⎬ (2. k1 = 2 π σ X σY 1 − ρ2 2.5 That is.Principles of Communication Prof. we have β2 α + Y = e 2 c) P [Y ≤ m ] = P [ X ≤ ln m ] Hence if P [Y ≤ m ] = 0. ∞ ) is 1. Y ( x . V. −∞ < x < ∞ 2π β ⎢⎣ 2 β2 ⎥ ⎦ Note that X is N ( α .

We can construct examples of a joint PDF fX . This does not imply fX . then they are independent. y ) = f X ( x ) fY ( y ) iff ρ = 0 That is. σY2 ) 1 TP PT P2) f X Y ( x . in general. P3) If Z = α X + βY where α and β are constants and X and Y are jointly Gaussian.22 gives the plot of a bivariate Gaussian PDF for the case of ρ = 0 and σ X = σY . then Z is Gaussian. V. unless X and Y are independent.5. with respect to non-Gaussian variables (we have already seen an example of this in Sec. That is not true. then the marginal density of X or Y is Gaussian. Figure 2. Y .2). 2.Principles of Communication Prof. which is not Gaussian but results in fX and fY that are Gaussian. Venkata Rao k 2 = 2 (1 − ρ2 ) ρ = Correlation coefficient between X and Y The following properties of the bivariate Gaussian density can be verified: P1) If X and Y are jointly Gaussian. Y is jointly Gaussian. X is N ( m X .69 Indian Institute of Technology Madras . 2. if the Gaussian variables are uncorrelated. Let fX and fY be obtained from fX . σ2X ) and Y is N ( mY . Therefore fZ ( z ) can be written after computing mZ and σ2Z with the help of the formulae given in section 2. that is. 1 TP PT Note that the converse is not necessarily true. Y and let fX and fY be Gaussian.5.

2. If ρ > 0 .23. negative.24 Let X and Y be jointly Gaussian with X = − Y = 1 . 2. Venkata Rao Fig. Let us find the probability of (X. Example 2. with. the striker missing! For ρ ≠ 0 . of course.Y resembles a (temple) bell.Y) lying in the shaded region D 2 shown in Fig. imagine the bell being compressed along the X = − Y axis so that it elongates along the X = Y axis. positive and (ii) ρ . V. Similarly for ρ < 0. σ2X = σY2 = 1 and 1 ρX Y = − . f X .Principles of Communication Prof.22: Bivariate Gaussian PDF ( σ X = σY and ρ = 0 ) For ρ = 0 and σ X = σY .70 Indian Institute of Technology Madras . we have two cases (i) ρ . 2.

2. we have 2 1 y ≥ − x + 2 .71 Indian Institute of Technology Madras . Venkata Rao Fig.24 Let A be the shaded region shown in Fig. y ) ∈ A ⎤⎦ − P ⎡⎣( x . 2.24(b).Principles of Communication Prof. Hence the required probability is. Fig. 2 2. y ) ∈ B ⎤⎦ 1 For the region A . we have y ≥ − x + 1 and for the region B .23: The region D of example 2. 2.24(a) and B be the shaded region in Fig.24: (a) Region A and (b) Region B used to obtain region D The required probability = P ⎡⎣( x . 2. V.

Principles of Communication Prof. V. Venkata Rao

⎡ X ⎤ ⎡ X ⎤
P ⎢Y + ≥ 1⎥ − P ⎢Y + ≥ 2⎥
⎣ 2 ⎦ ⎣ 2 ⎦
X
Let Z = Y +
2
Then Z is Gaussian with the parameters,
1 1
Z =Y + X = −
2 2
1 2 1
σ2Z = σ X + σY2 + 2 . ρ X Y
4 2
1 1 1 3
= + 1− 2. . =
4 2 2 4
1
Z+
⎛ 1 3⎞ 2 is N ( 0, 1)
That is, Z is N ⎜ − , ⎟ . Then W =
⎝ 2 4⎠ 3
4

P [ Z ≥ 1] = P ⎡⎣W ≥ 3 ⎤⎦

⎡ 5 ⎤
P [ Z ≥ 2] = P ⎢W ≥ ⎥
⎣ 3⎦

Hence the required probability = Q ( 3 ) − Q ⎛⎜⎝ 5 ⎞
⎟  ( 0.04 − 0.001) = 0.039
3⎠ 

2.72

Indian Institute of Technology Madras

Principles of Communication Prof. V. Venkata Rao

Exercise 2.10
X and Y are independent, identically distributed (iid) random variables,
each being N ( 0, 1) . Find the probability of X , Y lying in the region A shown in
Fig. 2.25.

Fig. 2.25: Region A of Exercise 2.10

Note: It would be easier to calculate this kind of probability, if the space is a
product space. From example 2.12, we feel that if we transform (X,Y) into

(Z , W ) such that Z = X + Y , W = X − Y , then the transformed space B
would be square. Find fZ ,W ( z , w ) and compute the probability ( Z , W ) ∈ B.

2.73

Indian Institute of Technology Madras

Principles of Communication Prof. V. Venkata Rao

Exercise 2.11
Two random variables X and Y are obtained by means of the
transformation given below.
1
X = ( − 2 loge U1 ) 2 cos ( 2 π U2 ) (2.59a)
1
Y = ( − 2 loge U1 ) 2 sin ( 2 π U2 ) (2.59b)

U1 and U2 are independent random variables, uniformly distributed in the

range 0 < u1 , u2 < 1 . Show that X and Y are independent and each is

N ( 0, 1) .

Hint: Let X1 = − 2 loge U1 and Y1 = X1

Show that Y1 is Rayleigh. Find f X ,Y ( x , y ) using X = Y1 cos Θ and

Y = Y1 sin Θ , where Θ = 2 π U2 .
Note: The transformation given by Eq. 2.59 is called the Box-Muller
transformation and can be used to generate two Gaussian random number
sequences from two independent uniformly distributed (in the range 0 to 1)
sequences.

2.74

Indian Institute of Technology Madras

Principles of Communication Prof. V. Venkata Rao

Appendix A2.1
Proof of Eq. 2.34
The proof of Eq. 2.34 depends on establishing a relationship between the
differential area d z d w in the z − w plane and the differential area d x d y in
the x − y plane. We know that

fZ ,W ( z , w ) d z d w = P [ z < Z ≤ z + d z , w < W ≤ w + d w ]

If we can find d x d y such that

fZ ,W ( z , w ) d z d w = f X ,Y ( x , y ) d x d y , then fZ ,W can be found. (Note that

the variables x and y can be replaced by their inverse transformation

quantities, namely, x = g − 1 ( z , w ) and y = h − 1 ( z , w ) )

Let the transformation be one-to-one. (This can be generalized to the case of
one-to-many.) Consider the mapping shown in Fig. A2.1.

Fig. A2.1: A typical transformation between the ( x − y ) plane and ( z − w ) plane

Infinitesimal rectangle ABC D in the z − w plane is mapped into the
parallelogram in the x − y plane. (We may assume that the vertex A transforms

to A' , B to B' etc.) We shall now find the relation between the differential area of
the rectangle and the differential area of the parallelogram.

2.75

Indian Institute of Technology Madras

y + d z⎟ ⎝ ∂z ∂z ⎠ ⎛ ∂x ∂y ⎞ P3 = ⎜ x + dw . Fig.2 where V1 = ( P2 − P1 ) and V2 = ( P3 − P1 ) . A2. Venkata Rao Consider the parallelogram shown in Fig.Principles of Communication Prof. y + dw⎟ ⎝ ∂w ∂w ⎠ Consider the vectors V1 and V2 shown in the Fig. V. That is. P2 . Then. A2. A2. A = V1 × V2 2.2: Typical parallelogram Let ( x . the area A of the parallelogram is.76 Indian Institute of Technology Madras . y ) be the co-ordinates of P1 . ∂x ∂y V1 = dz i + dz j ∂z ∂z ∂x ∂y V2 = dw i + dw j ∂w ∂w where i and j are the unit vectors in the appropriate directions. y + d z⎟ ⎜ ∂z ∂z ⎟ ⎝ ⎠ ⎛ ∂x ∂y ⎞ = ⎜x + dz. Then the P2 and P3 are given by ⎛ ∂ g− 1 ∂ h− 1 ⎞ P2 = ⎜ x + dz. P3 and P4 . with vertices P1 .2.

w ⎠ That is. w ) = f X . y ) = . Y ( x . we have ∂x ∂y ∂y ∂x V1 × V2 = − d z dw ∂ z ∂w ∂ z ∂w ⎛ x.Y ( x . and i × j = − ( j × i ) = k where k is the unit vector perpendicular to both i and j .W ( z . ⎛ x.Principles of Communication Prof. y ) J ⎜ ⎟ ⎝ z. y ⎠ 2.77 Indian Institute of Technology Madras . y ⎞ A = J⎜ ⎟ d z dw ⎝ z. y ⎞ fZ. Venkata Rao As i × i = 0 . w ⎞ J⎜ ⎟ ⎝ x. w ⎠ fX . V. ⎛ z. j × j = 0 .

3264 1.80 0.45 0.0062 10− 6 4.55 0.60 0.75 0.00 0.4602 1.3446 1.95 0.1587 2.0179 3.50 0.0735 2.70 0.0082 2 10− 5 4.95 0.55 0.0107 10− 4 3.5 .90 0.10 0.00003 2.10 10− 3 0. Note that Q ( 0 ) = 0.35 0.50 0.4207 1.78 Indian Institute of Technology Madras .90 0.20 0.0808 2. Venkata Rao Appendix A2.60 0.0256 3.0010 0.2119 1.0968 2.15 0.30 0.1251 2.3821 1.75 0.05 0.00048 0.50 0.60 0.90 0.2578 1.0035 0.1841 1.1711 1.00007 0.40 0.20 0.0156 2.70 0.1977 1.4013 1.25 0.4405 1.0019 0.10 0.27 0.0359 3.25 0.30 0.10 0.00010 0.2743 1.70 0.00 0.78 0.2420 1.00016 0.1151 2.40 0.90 0.60 0.1357 2.0495 3.20 0.90 0.20 0.0885 2.0287 3.0139 10− 4 3.85 0.0026 0.4801 1.50 0.85 0.2266 1. y Q (y ) y Q (y ) y Q (y ) y Q (y ) 10− 3 3.00034 0. V.Principles of Communication Prof.80 0.40 0.45 0.0446 3.0047 0.35 0.00 0.80 0.2 Q( ) Function Table ∞ 1 2 Q (α) = − x ∫ α 2π e 2 dx It is sufficient if we know Q ( α ) for α ≥ 0 .65 0. because Q ( − α ) = 1 − Q ( α ) .00 0.0322 3.3085 1.00005 1.15 0.30 0.1469 2.0668 3.0606 3.65 0.0401 3.0228 4.0548 3.2912 1.00069 0.40 0.0013 0.30 0.00023 0.3632 1.28 2 0.70 0.05 0.80 0.10 0.70 0.

Principles of Communication Prof. erf ( α ) = ∫e − β2 dβ π 0 1 ⎛ α ⎞ Hence Q ( α ) = erfc ⎜ ⎟.79 Indian Institute of Technology Madras . V. the complementary error function which is given by ∞ 2 erfc ( α ) = 1 − erf ( α ) = ∫e − β2 dβ π α α 2 and the error function. Venkata Rao Note that some authors use erfc ( ) . 2 ⎝ 2⎠ 2.

80 Indian Institute of Technology Madras . (Cartesian to Polar coordinate transformation). σ2X is a valid PDF ) We will show that f X ( x ) as given by Eq. ⎡ ∞ − v2 ⎤ ⎡ ∞ − y2 ⎤ Then. ⎝v ⎠ and d x d y = r d r d θ . ∞ 2 ∞ 2 −v −y Let I = ∫e −∞ 2 dv = ∫e −∞ 2 dy. we have the required result. 2.Principles of Communication Prof.3. 2π −∞ ∫e 2 dv = 1 (A2. is a valid PDF by ∞ establishing ∫ f (x) d x −∞ X = 1 .3. d v = (A2. Then. A2. (Note that f X ( x ) ≥ 0 for − ∞ < x < ∞ ). A2.2) σx dx Then.3.3.2 and Eq. V. Venkata Rao Appendix A2.3.3 in Eq. r = v 2 + y 2 and θ = tan−1 ⎜ ⎟ .3.3 ( Proof that N m X .1) x − mX Let v = (A2.1. 2. A2.56.3) σx Using Eq. I 2 = ⎢∫e 2 dv⎥ ⎢ ∫ e 2 d y⎥ ⎣⎢ − ∞ ⎥⎦ ⎣⎢ − ∞ ⎥⎦ ∞ ∞ v2 + y2 − = ∫ ∫e − ∞− ∞ 2 dv d y ⎛y⎞ Let v = r cos θ and y = r sin θ . 2π ∞ r2 − = ∫ ∫e r dr dθ 2 2 I 0 0 = 2 π or I = 2π ∞ v2 1 − That is.

A. P P 2. Random Variables and Stochastic Processes’. M. ‘Principles of Communication Engineering’.Principles of Communication Prof. and Jacobs. I.81 Indian Institute of Technology Madras . R. J. T. McGraw Hill (3rd edition). and Craig. John Wiley. 2nd edition. ‘Introduction to Mathematical Statistics”. V. 1991.. Venkata Rao References 1) Papoulis. ‘Probability. P P 2) Wozencraft... V. 3) Hogg.. A. 1965. The Macmillan Co. 1965. J..