You are on page 1of 164

PROBABILITY THEORY AND

STOCHASTIC PROCESSES(PTSP-8C304)

Unit II Random Variables

Faculty: V.RAJENDRA CHARY(VRC),


Assistant Professor,ECE,SNIST.
Class: II B.Tech,I Semester, ECE
UNIT I PROBABILITY THEORY

• Set Definitions, Sample Points and Sample Spaces,


Probability of Random events, Laws of Probability. Joint,
Marginal and Conditional Probabilities. Total Probability.
Bayes Theorem. Statistical Independence.

• Applications: Bayes theorem in calculation of channel


capacity, Information Theory i.e., Entropy, Mutual
information (rate at which source generates information)
.
15-Nov-21 RAJENDRA CHARY 2
UNIT II RANDOM VARIABLES
• Probability Distribution Functions,Discrete Random Variables and
Probability Mass Function,Expected values.Continuous Random
Variables.Probability Density Functions.Complex Random Variables.
Moments and Characteristic Functions. Distributions and Density
Functions and their Properties. Expected Values. Moments and
CharacteristicFunctions,Binomial,Poisson,Uniform,Gaussian,Exponen
tial,Rayleigh.Transformations of Random Variables.
• Applications : Design of queue for Tele communications using
Binomial, Poisson distributions.

15-Nov-21 RAJENDRA CHARY 3


UNIT III RANDOM VECTORS
• Joint Probability Distribution Functions. Joint Probability
Densities. Conditional Probability Distributions Functions.
Marginal Distributions and Density Functions. Conditional
Probability Densities. Expected Value of a Function of
Random Variables. Joint Moments. Joint Characteristic
Functions. Sum of Two Random Variables. Sum of Several
Random Variables. Central limit theorem (proof not
expected)Jointly Gaussian Random Variables. Independent
Random Variables. Transformations (Functions) of Multiple
Random Variables.
• Applications :Design of optimum filter
15-Nov-21 RAJENDRA CHARY 4
Outline of the unit
• Random variable-definition,conditions and types(1)
• Probability distribution and density functions with their
properties,probability mass function,problems(2)
• Standard Random variables with their PDF,pdf(2)
• Expected value and their properties,problems and Complex
random variable(2)
• Moments,Characteristic function,MGF,problems(3)
• Transformations on random variables(1)
15-Nov-21 RAJENDRA CHARY 5
15-Nov-21 RAJENDRA CHARY 6
Random Variable(RV)
• A Random variable is a real function of the events of the
given sample space. It is denoted with any capital letter like
X(S).
• A random variable X can be considered as a function that
maps all events of the sample space into points on the real
axis.
Eg. To be discussed using chalk and board taking example
of tossing 2 coins at a time.

15-Nov-21 RAJENDRA CHARY 7


Conditions for a function to be RV
• Every point in the sample space must correspond to only one
value of the random variable.
• The set{X≤x} shall be an event for any real number x.The
probability of this event is equal to the sum of the
probabilities of all the events corresponding to set{X≤x} .This
is denoted as P{X≤x} .
• The probabilities of events {X= ∞}and {X=- ∞} are zero.

15-Nov-21 RAJENDRA CHARY 8


Types of Random variable
• Continuous Random Variable
• Discrete Random Variable
• Mixed Random Variable

15-Nov-21 RAJENDRA CHARY 9


Continuous Random Variable
• A Continuous random variable is one having continuous range
of values.
• The sample space for continuous random variable is
continuous.
eg. Temperature calculation

15-Nov-21 RAJENDRA CHARY 10


Discrete Random Variable
• A Discrete random variable is one having only discrete
values.
• The sample space for a discrete random variable can be
discrete, continuous or mixture of continuous and discrete.
eg. No. of telephone calls received in an office in a finite
interval of time.

15-Nov-21 RAJENDRA CHARY 11


Mixed Random Variable
• A mixed random variable is one for which some of its values
are discrete and some are continuous.
• The mixed case is usually the least important but it occurs in
some problems of practical significance.

15-Nov-21 RAJENDRA CHARY 12


15-Nov-21 RAJENDRA CHARY 13
Probability Distribution Function(PDF)
• The Probability Distribution Function describes probabilistic
behaviour of a random variable.
• The Probability Distribution Function of a Random variable X
is given by FX(x).
FX(x)= P{X ≤ x}
P{X ≤ x} is the probability of all the events {X ≤ x}
x is a real number in the range - ∞ ≤ x ≤ ∞

15-Nov-21 RAJENDRA CHARY 14


PDF-other names & example
• PDF is also called as cummulative distribution function(CDF)
or general distribution function(GDF).
eg.-To be discussed using chalk and board taking example of
tossing 3 coins at a time.

15-Nov-21 RAJENDRA CHARY 15


PDF for Continuous RV
• If X is continuous random variable,then the distribution
function is integration of all continuous probabilities of X
upto the value x. It is given by the expression:

15-Nov-21 RAJENDRA CHARY 16


PDF for Discrete RV
• If X is discrete random variable, then the distribution function
is the cummulative sum of all probabilities X upto the value x.
It is given by the expression:

15-Nov-21 RAJENDRA CHARY 17


PDF for Discrete RV
• u(x) is the unit step function,which is given by:

15-Nov-21 RAJENDRA CHARY 18


Properties of PDF
1) FX(- ∞)=0
FX(x)= P{X ≤ x}
FX(- ∞)= P{X ≤ - ∞}
=minimum value of probability
=0

15-Nov-21 RAJENDRA CHARY 19


Properties of PDF
2) FX(∞)=1
FX(x)= P{X ≤ x}
FX(∞)= P{X ≤ ∞}
= maximum value of probability
=1

15-Nov-21 RAJENDRA CHARY 20


Properties of PDF
3) 0 ≤ FX(x) ≤ 1
Since the distribution function takes the values of all the
probabilities, the range of distribution function is also same as
the range of probability.
4) FX(x1) ≤ FX(x2) , if x1 < x2
FX(x) is a non decreasing function of x, i.e the distribution
function is monotonically increasing function between 0 and 1.

15-Nov-21 RAJENDRA CHARY 21


Properties of PDF
5)P{x1 < X ≤ x2} = FX(x2) - FX(x1)
It states that the probability that X will have values larger than
some number x1 but not exceeding another number x2 is equal
to the difference in FX(x) evaluated at the two points
6) FX(x+) = FX(x)
It states that the probability distribution function is continuous
from x to x+, where x+ is an infinitesimal increment in x

15-Nov-21 RAJENDRA CHARY 22


Properties of PDF
7) P{X > x}=1- FX(x)
• Properties 1,2,4 and 6 may be used to test whether a given
function is valid distribution function or not.

• Property 5,6 explained taking rolling 3 dice example using


chalk and board

15-Nov-21 RAJENDRA CHARY 23


15-Nov-21 RAJENDRA CHARY 24
Probability Density Function(pdf)
• The probability density function of a random variable X is
defined as the derivative of PDF.
• It is given by the expression:

fX(x)= d/dx(FX(x) )

15-Nov-21 RAJENDRA CHARY 25


Pdf for Discrete RV
• For discrete random variable,pdf is given by:

15-Nov-21 RAJENDRA CHARY 26


Pdf for Discrete RV

15-Nov-21 RAJENDRA CHARY 27


Pdf for Discrete RV

15-Nov-21 RAJENDRA CHARY 28


Properties of pdf

• This property states that the probability density function is


non-negative.

15-Nov-21 RAJENDRA CHARY 29


Properties of pdf

• This property states that the area under probability density


function is unity.

15-Nov-21 RAJENDRA CHARY 30


Properties of pdf

• This property states that the distribution function is equal to


the integral of density function upto the value of x or is equal
to the area under the density curve from - ∞ to x as shown in
figure.

15-Nov-21 RAJENDRA CHARY 31


Properties of pdf

15-Nov-21 RAJENDRA CHARY 32


Properties of pdf

• This property states that the probability of X from the values


x1 to x2 is equal to the area under the density curve from x1
to x2 as shown in figure.
• Note:P{x1 < X ≤ x2} = FX(x2) - FX(x1)

15-Nov-21 RAJENDRA CHARY 33


Properties of pdf

15-Nov-21 RAJENDRA CHARY 34


Properties of pdf
• Properties 1 and 2 may be used to test whether a given
function is valid density function or not.

15-Nov-21 RAJENDRA CHARY 35


Probability Mass Function
• Consider a discrete random variable with infinite number of
possible outcomes i.e X={x1,x2,....xn….. ∞}
• If the probability of X,P(xi),i=1,2……∞ satisfies the
following conditions:
(i) P(xi) ≥ 0,for all I
(ii) Σ P(xi) =1[Σ limits are from1 to ∞]
Then the function P(xi) is called the probability mass function

15-Nov-21 RAJENDRA CHARY 36


Problems
• A random variable X has probabilities as shown in
table,(a)find the value of K,(b) find FX(x) ,fX(x) and draw their
plots.(eg.2.2)

X -3 -2 -1 0 1 2
P(X) 0.2 0.5K K 0.1 0.3K K

15-Nov-21 RAJENDRA CHARY 37


15-Nov-21 RAJENDRA CHARY 38
Tutorial Problems
• If the pdf of a random variable is given by fX(x) =K(1- x2 ),
0 < x < 1,find the value K and FX(x) (eg2.5)
• Find the value of b so that the function
f (x) = e3x; 0<x<b
= 0; otherwise
is a valid density function.

15-Nov-21 RAJENDRA CHARY 39


Tutorial Problems

15-Nov-21 RAJENDRA CHARY 40


Few PDFs and pdfs
For Continuous random variables,
• Gaussian/Normal
• Uniform
• Exponential
• Rayleigh
For Discrete random variables,
• Binomial
• Poisson
15-Nov-21 RAJENDRA CHARY 41
Few more functions
For Continuous random variables,
• Gamma
• Beta
• Chi-square
• Nakagami-m
• Cauchy
• Fishers-F etc….

15-Nov-21 RAJENDRA CHARY 42


Few more functions
For discrete random variables,
• Bernoulli
• Negative Binominal
• Hypergeometric
• Geometric etc…

15-Nov-21 RAJENDRA CHARY 43


Gaussian/Normal Random Variable
• A random variable which satisfies the Gaussian density
function is called a Gaussian random variable. It can be
denoted by N(μ x , σx ).
• The Gaussian density function of a random variable X is
given by:

15-Nov-21 RAJENDRA CHARY 44


Gaussian/Normal Random Variable
• σx > 0 and - ∞ ≤ μ x ≤ ∞ are constants
• σx =Standard deviation of X, μ x = mean of X

15-Nov-21 RAJENDRA CHARY 45


Gaussian/Normal Random Variable
• The Gaussian distribution function of a random variable X is
given by:

15-Nov-21 RAJENDRA CHARY 46


Normalized Gaussian Random Variable
• Substitute μ x =0, σx2 =1 in Gaussian density and distribution
functions, we get the normalized Gaussian density and
distribution equations.

fX(x)= exp(- x2/2)

FX(x)= ∫ exp(- x2/2)

15-Nov-21 RAJENDRA CHARY 47


Gaussian Random Variable
a)Density function plot, b)Distribution function plot

15-Nov-21 RAJENDRA CHARY 48


Gaussian Random Variable
• The plot of Gaussian density function is bell shaped and
symmetric about its mean value.
• The total area under the density function is one.
• The maximum value of fX(x) is

• It occurs at x= μ x and the function decreases to 0.607 times


of its maximum value at x= μ x + σx and x= μ x – σx.

15-Nov-21 RAJENDRA CHARY 49


15-Nov-21 RAJENDRA CHARY 50
Gaussian Random Variable
• The Distribution function FX(x) is 0.5 at μ x ,0.159 at = μ x - σx
and 0.841 at μ x + σx.

15-Nov-21 RAJENDRA CHARY 51


Gaussian Random Variable
• If FX(x) is Gaussian distribution function and F(x) is a
normalized Gaussian distribution function, then the
relationship between these two functions is given by:

15-Nov-21 RAJENDRA CHARY 52


Gaussian RV-Applications
• The Gaussian random variable is most important among all
the random variables in the fields of engineering.
• It gives accurate description of many practical random
quantities.
• The distribution of noise signal exactly matches the Gaussian
probability function.
.

15-Nov-21 RAJENDRA CHARY 53


Gaussian RV-Applications
• It is possible to eliminate noise by
knowing its behaviour using the Gaussian Probability density
function

15-Nov-21 RAJENDRA CHARY 54


Uniform Random Variable
• The uniform probability density function is given by:

15-Nov-21 RAJENDRA CHARY 55


Uniform Random Variable
• The uniform distribution function –

15-Nov-21 RAJENDRA CHARY 56


Uniform Random Variable

15-Nov-21 RAJENDRA CHARY 57


Uniform Random Variable

15-Nov-21 RAJENDRA CHARY 58


Uniform Random Variable
• The uniform probability density function has a constant
amplitude in the given range. The total area of the function is
always one.

15-Nov-21 RAJENDRA CHARY 59


Uniform RV-Applications
• The random distribution of errors introduced in round-off
process are uniformly distributed.
• Quantization of signal samples prior to encoding in digital
communication systems.Quantization refers to rounding off
the actual sample to the nearest of a large number of discrete
quantum levels.

15-Nov-21 RAJENDRA CHARY 60


Exponential Random Variable
• The exponential probability density function for a random
variable X is given by:

15-Nov-21 RAJENDRA CHARY 61


Exponential Random Variable
• The exponential distribution function –

15-Nov-21 RAJENDRA CHARY 62


Exponential Random Variable

15-Nov-21 RAJENDRA CHARY 63


Exponential Random Variable
• The exponential probability density function has a maximum
value of 1/b at x=a. The function starts at x=a and
exponentially decreases to zero at x=∞.
• The distribution function starts at x=a and exponentially
increases to one at x=∞.

15-Nov-21 RAJENDRA CHARY 64


Exponential RV-Applications
• The fluctuations in signal strength received by radar receivers
from certain types of targets are exponentially distributed.
• Raindrop sizes, when a large number of rainstorm
measurements are made are also exponentially distributed.

15-Nov-21 RAJENDRA CHARY 65


Rayleigh Random Variable
• The Rayleigh density function is given by:

15-Nov-21 RAJENDRA CHARY 66


Rayleigh Random Variable
• The Rayleigh distribution function –

15-Nov-21 RAJENDRA CHARY 67


Rayleigh Random Variable

15-Nov-21 RAJENDRA CHARY 68


Rayleigh Random Variable

15-Nov-21 RAJENDRA CHARY 69


Rayleigh RV-Applications
• The Rayleigh density describes the envelope of white noise,
when the noise is passed a bandpass filter.
• It is used in analysis of errors in various measurement
systems.
• Some type of fluctuations received by the receiver are
modelled as Rayleigh distribution.

15-Nov-21 RAJENDRA CHARY 70


Binomial Random Variable
• Consider an experiment having only two possible outcomes
such as yes or no, tails or heads, one or zero etc..
• If such an experiment in repeated for N trials, then the
binomial probability density function of a discrete random
variable X is given by:

15-Nov-21 RAJENDRA CHARY 71


Binomial Random Variable
• The Binomial distribution function is given by:

15-Nov-21 RAJENDRA CHARY 72


Binomial Random Variable

15-Nov-21 RAJENDRA CHARY 73


Binomial RV-Applications
• Binomial distribution can be applied to many game of chance,
detection problems in radar and sonar.
• It is used in many experiments having only two possible
outcomes in any given trial.
• It can be applied to Bernoulli trial experiment.

15-Nov-21 RAJENDRA CHARY 74


Poisson’s Random Variable
• The poisson’s pdf and PDF are given by:

here, b > 0 is a real constant


15-Nov-21 RAJENDRA CHARY 75
Poisson’s Random Variable
• The poisson’s distribution is the approximated function of
binomial distribution when N→∞ and P →∞.Here the
constant b=NP.
• The poisson’s distribution and density function graphs are
similar to that of graphs of binomial function.

15-Nov-21 RAJENDRA CHARY 76


Poisson’s RV-Applications
• The Poisson RV applies to a wide variety of counting type
applications. It describes:
• The no. of telephone calls made during a period of time.
• The no. of defective elements in a given sample.
• The no. of electrons emitted from a cathode in a given time
interval.
• The no. of items waiting in a queue etc…

15-Nov-21 RAJENDRA CHARY 77


15-Nov-21 RAJENDRA CHARY 78
Problems
• The random variable X has the discrete variables in the set
{-1,-0.5,0.7,1.5,3}.The corresponding probabilities are assumed
to be {0.1,0.2,0.1,0.4,0.2}.Plot its distribution function.
(eg 2.14)

15-Nov-21 RAJENDRA CHARY 79


Problems
• Let X be a continuous random variable with density function

(i)Find the value of K,(ii)find P(2 ≤X ≤ 5) (eg.2.32)

15-Nov-21 RAJENDRA CHARY 80


Problems
• If the probability density of a random variable is given by

Find the value that C must have and evaluate FX(0.5) (eg.2.27)

15-Nov-21 RAJENDRA CHARY 81


Problems
• Assume that the height of clouds above the ground at some
location is Gaussian random variable X with μ x =1830m and
σx =460m.Find the probability that clouds will be higher than
2750m.(eg2.4.2),[F(2)=0.9722]
• Find the probability of event{X ≤ 5.5} for a Gaussian random
variable with μ x =3 and σx =2.[F(1.25)=0.8944]

15-Nov-21 RAJENDRA CHARY 82


Expected Value
• The average value of a probability distribution function of a
random variable X is called expected value of X or
mathematical expectation of X or statistical average of X or
mean value of X.
• It is denoted by E[X] or X̅.

15-Nov-21 RAJENDRA CHARY 83


Expected value for Continuous RV
• If X is a continuous random variable with valid probability
density function fX(x) ,then the expected value of X is given
by the expression:

15-Nov-21 RAJENDRA CHARY 84


Expected value for Discrete RV
• If X is a discrete random variable with a set of elements
{x1,x2,....xN} and a set of corresponding probabilities
{P(x1),P(x2)…P(xN)},then the expected value of X is given by
the expression:

Summation limits are from:i=1 to N


15-Nov-21 RAJENDRA CHARY 85
Expected value for Discrete RV
• For equiprobable elements, where P(x1)= P(x2)= P(x3)=
……… P(xN-1)= P(xN)= 1/N, the expected value is given by:

15-Nov-21 RAJENDRA CHARY 86


Expected value of a function of a RV
• Consider a random variable X with pdf fX(x) .If g(x) is a real
function of x, then the expected value of g(x) for a continuous
and discrete random variables respectively is given by:

15-Nov-21 RAJENDRA CHARY 87


Properties of Expected Value
1)If a random variable X is constant, i.e X=a then E[a]=a,
where a is a constant.

2)If a is any constant, then E[aX]=aE[X]

3)If a and b are any two constants, then


E[aX+b]=aE[X]+b
15-Nov-21 RAJENDRA CHARY 88
15-Nov-21 RAJENDRA CHARY 89
15-Nov-21 RAJENDRA CHARY 90
Properties of Expected Value
4)If X ≥ 0,then E[X] ≥ 0
5)If X is any random variable, then the inequality,
|E[X]| ≤ E|X| exists.
6)If g1(x ) and g2(x) are two functions of a random variable X,
then
E[g1(x ) + g2(x) ]=E[g1(x)]+E[ g2(x)]
Proofs-To be derived using chalk and board
15-Nov-21 RAJENDRA CHARY 91
Properties of Expected Value
• Similarly for n functions,
E[g1(x ) + g2(x) +………+ gn(x)]=E[g1(x )]+E[ g2(x)]+……..+
E[ g2(x)]+…+ E[gn(x )]

15-Nov-21 RAJENDRA CHARY 92


Complex Random Variables
• A complex random variable Z can be defined in terms of real
random variables X and Y by:
Z=X +jY, j= √−1
• Its expectation is given by:
E[Z]=E[X] +jE[Y]

15-Nov-21 RAJENDRA CHARY 93


Problems
• Find the expected value of a uniformly distributed random
variable.(eg 3.1), (ii)find E[X2].
• If X is a discrete random variable with probabilities as shown
in table, find(i)E[X],(ii)E[2X+3],(iii)E[X2],(iv)E[(2X+1) 2].
(eg 3.2)
X -2 -1 0 1 2
P(X) 1/5 2/5 1/10 1/10 1/5

15-Nov-21 RAJENDRA CHARY 94


Problems
• When a die is thrown, what is the expected value of
probability of getting a particular number.(eg.3.11)
• When two unbiased dice are thrown, find the expected value
of the sum of the numbers shown on dice(eg 3.29)

15-Nov-21 RAJENDRA CHARY 95


Problems
• Consider a pdf of a random variable X :

and another random variable Y=2X,then find a)value of K,


b)E[X], c)E[Y], d)E[XY] (eg3.32)

15-Nov-21 RAJENDRA CHARY 96


15-Nov-21 RAJENDRA CHARY 97
Moments
• Moments about Origin
• Moments about mean/central moments

15-Nov-21 RAJENDRA CHARY 98


Moments about Origin

• Let g(x) be a real function of random variable X, such that


g(x)=Xn for n=0,1,2…..then the expected value of the function
g(x) is called the moments about origin of a random variable
X. It is denoted by mn,n=order of moments.

15-Nov-21 RAJENDRA CHARY 99


Moments about Origin

• Mathematically, for a continuous random variable,

• For a discrete random variable,


E[Xn]=Σ xinP(xi)
15-Nov-21 RAJENDRA CHARY 100
Moments about Origin

• n=0,

• n=1,

15-Nov-21 RAJENDRA CHARY 101


Moments about Origin

• n=2, we get m2=E[X2]


• The zeroth moment, m0=1.It gives the area of pdf
• The first moment, m1=E[X]
• The second moment, m2=E[X2].It gives mean square value of
X and average power of a signal.Square root of m2 gives the
rms value of the signal

15-Nov-21 RAJENDRA CHARY 102


Moments about mean/central moments

• Let g(x) be a real function of random variable X, such that


g(x)= for n=0,1,2…..then the expected value of the
function g(x) is called the moments about mean of a random
variable X. It is denoted by μ n,n=order of moments.

15-Nov-21 RAJENDRA CHARY 103


Moments about mean/central moments

• Mathematically, for a continuous random variable,

• For a discrete random variable,


μ n=Σ (xi- x̄)nP(xi)

15-Nov-21 RAJENDRA CHARY 104


Moments about Mean

• n=0,

• n=1,

15-Nov-21 RAJENDRA CHARY 105


Moments about Mean

• n=2,

15-Nov-21 RAJENDRA CHARY 106


Moments about Mean

• μ 0= m0=1=area of pdf
• μ 1=0.i.e the first central moment is always equal to zero.
• μ 2=variance

15-Nov-21 RAJENDRA CHARY 107


Variance
• The variance of the pdf fX(x) for a random variable X is
defined as the second central moments μ 2 of X.
• It is denoted by σx2 or var(X) and is given by:

15-Nov-21 RAJENDRA CHARY 108


Variance
• For a discrete random variable,

σx2 = Σ (xi- x̄)2P(xi)

15-Nov-21 RAJENDRA CHARY 109


Standard deviation
• The standard deviation of a random variable is defined as the
square root of the variance .
• It is expressed as:

15-Nov-21 RAJENDRA CHARY 110


Skew
• The skew of the pdf fX(x) for a random variable X is defined
as the third central moments μ 3 of X.
• It is given by:

15-Nov-21 RAJENDRA CHARY 111


Skew
• For a discrete random variable,

• The physical significance of skew is that is a measure of the


asymmetry of fX(x) about its mean.It is the amount of
deviation of symmetry from the mean value.

15-Nov-21 RAJENDRA CHARY 112


15-Nov-21 RAJENDRA CHARY 113
coefficient of skewness
• The normalised third central moment or the ratio of the third
central moment to the cube of standard deviation is called
skewness of the density function or the coefficient of
skewness.
• It is given by:

15-Nov-21 RAJENDRA CHARY 114


Properties Of Variance
1)The variance of a constant is zero.i.e var(k)=0

2)If k is any constant, then for a random variable X,


var(kX)= k2 var(X)

3)For any given random variable X, the relationship between


the variance and moments is given by σx2= m2 - m12
15-Nov-21 RAJENDRA CHARY 115
Properties Of Variance
4)If X is a random variable, a and b are real constants,then
var(aX+b)= a2 var(X)
5)If two random variables X1, X2 are independent,then
var(X1 +X2) = var(X1) + var(X2)
var(X1 -X2) = var(X1) + var(X2)

15-Nov-21 RAJENDRA CHARY 116


Problems
• Consider a random variable X with E[X]=5 and σx2
=3.Another random variable is given as Y=-8X+10,find
(a)E[X2],(b)E[XY],(c)E[Y2],(d) σY2 (eg 3.31)

• Find the variance of uniform probability density function(eg


3.3)
• Find the skew and skewness of Uniform density function(eg
3.4)

15-Nov-21 RAJENDRA CHARY 117


15-Nov-21 RAJENDRA CHARY 118
Transformations of RVs
• Transformation is used to convert a given random variable X
into another random variable Y.
• It is denoted as: Y=T(X), T represents transformation.
• The transformation of X to Y is :

15-Nov-21 RAJENDRA CHARY 119


Types of Transformations
• Monotonically increasing transformation
• Monotonically decreasing transformation
• Non monotonic transformation

15-Nov-21 RAJENDRA CHARY 120


Monotonic Transformations of continuous RV

• Consider a random variable X.


• If the transformation is T(x1) < T(x2) for any x1 < x2,then it is
called a monotonically increasing transformation.
• If the transformation is T(x1) > T(x2) for any x1 < x2,then it is
called a monotonically decreasing transformation.

15-Nov-21 RAJENDRA CHARY 121


Monotonically Increasing Function
• Assume that the transformation T is continuous and
differentiable for all values of x with fX(x) ≠ 0. Let another
random variable Y have a value y0 corresponding to x0 of X as
shown in figure

15-Nov-21 RAJENDRA CHARY 122


Monotonically Increasing Function
• The relationship between random variables X and Y in terms
of density function is given by:

15-Nov-21 RAJENDRA CHARY 123


Monotonically Increasing Function
The transformation is given as:
Y=T(X)
y0 =T(x0)
or x0= T-1(y0)
where T-1 represents the inverse of the transformation T.

15-Nov-21 RAJENDRA CHARY 124


Monotonically Increasing Function
• Since transformation provides a one-to-one correspondence
between X and Y, the probability of the event {Y ≤ y0 } must
be equal to the probability of the event {X ≤ x0}.
Thus,
P{Y ≤ y0 }= P{X ≤ x0}
FY(y0)= FX(x0)

15-Nov-21 RAJENDRA CHARY 125


Monotonically Increasing Function

15-Nov-21 RAJENDRA CHARY 126


Monotonically Decreasing Function

15-Nov-21 RAJENDRA CHARY 127


Monotonically Decreasing Function
• The relationship between random variables X and Y in terms
of density function is given by:

15-Nov-21 RAJENDRA CHARY 128


Monotonically Decreasing Function

15-Nov-21 RAJENDRA CHARY 129


Monotonically Decreasing Function
• Using Leibniz’s rule and differentiating w.r.t y0

15-Nov-21 RAJENDRA CHARY 130


Monotonically Increasing/Decreasing
Function
• Therefore for a monotonic transformation, either increasing or
decreasing, the density function of y is:

15-Nov-21 RAJENDRA CHARY 131


Non-Monotonic Transformations of
continuous RV
• Consider a random variable Y which is non-monotonic
transformation of a random variable X as shown in figure.

15-Nov-21 RAJENDRA CHARY 132


Non-Monotonic Transformations of
continuous RV
• The relationship between random variables X and Y in terms
of density function is given by:

15-Nov-21 RAJENDRA CHARY 133


Non-Monotonic Transformations of
continuous RV
• For any given event {Y ≤ y0 },there is more than one value of
From figure, it is found that the event {Y ≤ y0 }corresponds to
events {X ≤ x1 and x2 ≤ X ≤ x3}.
• Thus the probability of the event {Y ≤ y0 } is equal to the
probability of the event{X|Y ≤ y0 }.

15-Nov-21 RAJENDRA CHARY 134


Non-Monotonic Transformations of
continuous RV

• By differentiating, the density function is given by:

15-Nov-21 RAJENDRA CHARY 135


Non-Monotonic Transformations of
continuous RV
• This can be simplified by using Leibniz’s rule as:

15-Nov-21 RAJENDRA CHARY 136


Transformation of a discrete RV
• If X is a discrete random variable and the transformation is
monotonic, then the set{yn} of a random variable Y has one-
to-one correspondence with the set {xn},through the equation
yn= T(xn) and P(yn)=P(xn)
Therefore,

15-Nov-21 RAJENDRA CHARY 137


Transformation of a discrete RV
• If the transformation is non-monotonic, there may exist more
than one value xn which corresponds to yn.P(yn) will be equal
to sum of all the probability of values for which yn= T(xn) .

15-Nov-21 RAJENDRA CHARY 138


Transformations derivations
• Slides no.124,125,126-monotonically increasing function
• Slides no.129,130-monotonically decreasing function
• Slides no.134,135,136-non monotonic function
• All images in these slides are from Mallikarjun textbook
given only for reference.

15-Nov-21 RAJENDRA CHARY 139


Problem
• A discrete random variable X with pdf is given in the table,
find the density function Y for the transformation:
Y= 3X3-3X2+2

X 0 1 2 3 4

P(X) 0.2 0.15 0.3 0.15 0.2

15-Nov-21 RAJENDRA CHARY 140


15-Nov-21 RAJENDRA CHARY 141
Functions to generate moments
• Moment Generating Function(MGF)-not in syllabus
• Characteristic Function

15-Nov-21 RAJENDRA CHARY 142


Moment Generating Function(MGF)
• Consider a random variable X with pdf fX(x) ,then the MGF
of X is defined as the expected value of the function e vX.It is
given by:
MX(v)=E[e vX]
v is a real variable , - ∞ ≤ v ≤ ∞

15-Nov-21 RAJENDRA CHARY 143


Moment Generating Function(MGF)
• For a continuous random variable,
MX(v)= ∫ e vXfX(x) dx

• For a discrete random variable,


MX(v)= Σ e vxiP(xi)

15-Nov-21 RAJENDRA CHARY 144


Calculation of moments using MGF
• If MX(v) is a moment generating function of a random
variable X, then the nth moment of X is given by:

15-Nov-21 RAJENDRA CHARY 145


Disadvantage of MGF
• The main disadvantage of MGF is that it may not exist for all
random variables and values of v. To overcome this
disadvantage, we prefer another method for generating
moments which is characteristic function, which exists for all
values of X and ω.

15-Nov-21 RAJENDRA CHARY 146


Properties of MGF
1)The MGF at v=0 is unity. It is given as:
MX(v)|v=0=MX(0)=1

2)Let X be a random variable with MGF MX(v),then the MGF


for Y=aX+b is given by:
MY(v)=e bv MX(av)

15-Nov-21 RAJENDRA CHARY 147


Properties of MGF
3)If MX(v) is a MGF of a random variable X,then
MX (cv)=Mcx(v),where c=real constant

4)If X1 and X2 are two independent random variables with MGF


Mx1(v) and Mx2(v) then
MX1+X2(v)=MX1(v) MX2(v)

15-Nov-21 RAJENDRA CHARY 148


Fourier Transform and Inverse
Fourier transform
• Fourier Transform-

• Inverse Fourier Transform-

15-Nov-21 RAJENDRA CHARY 149


Characteristic Function
• Consider a random variable X with a pdf fX(x) ,then the
expected value of the function e jωX is called the characteristic
function.
• It is given by:
(ω) =E[e jωX ]
x
• It is a function of real variable, - ∞ ≤ ω ≤ ∞,j is an imaginary
operator.

15-Nov-21 RAJENDRA CHARY 150


Characteristic Function
• For a continuous random variable,

(ω)= ∫ e jωX f (x) dx


x X

• For a discrete random variable,

(ω)= Σ e jωxiP(x )
x i

15-Nov-21 RAJENDRA CHARY 151


Characteristic Function
• The characteristic function transforms the random variable X
into another real variable ω.It can be expressed as Fourier
transform of fX(x) with the sign of ω reversed.
• Therefore the inverse fourier transform of x(ω) gives the pdf
with the sign of x reversed.
• So the functions x(ω) and fX(x) are fourier transform pairs
with the sign of the variable reversed.

15-Nov-21 RAJENDRA CHARY 152


Characteristic Function

15-Nov-21 RAJENDRA CHARY 153


Calculation of moments using
Characteristic Function
• If x(ω) is the characteristic function of a random variable X,
then the nth moment of X is given by:

15-Nov-21 RAJENDRA CHARY 154


15-Nov-21 RAJENDRA CHARY 155
Properties Of Characteristic Function
1)The characteristic function is unity at ω =0 and given by:

x(ω)| ω=0 = x(0) =1

2)The maximum amplitude of the characteristic function is


unity at ω =0 i.e
| x(ω)| ≤ x(0) or | x(ω)| ≤ 1

15-Nov-21 RAJENDRA CHARY 156


Properties Of Characteristic Function
3) x(ω) is continuous function of ω in the range - ∞ ≤ ω ≤ ∞
4) x(-ω) and x(ω) are conjugate functions i.e
(-ω)= *(ω) and *(-ω)=
x x x x(ω)
5)If x(ω) is a characteristic function of a random variable
X,then the characteristic function of Y=aX+b is given by:
(ω)= e jωb (aω) ,where a,b are constants
Y x

15-Nov-21 RAJENDRA CHARY 157


Properties Of Characteristic Function
6)If x(ω)
is a characteristic function of a random variable X, then
x(cω) = x(ω),where c is a real constant.

7)If X1and X2 are two independent random variables, then:

x1+x2(ω)= x1(ω) x2(ω)

15-Nov-21 RAJENDRA CHARY 158


Problems
• Find the characteristic function of a uniformly distributed
random variable X in the range [0,1] ( eg 3.5)
• Find the characteristic function of a random variable with
density function:
fX(x) =x, 0 ≤ x ≤ 1 (eg.3.36)

15-Nov-21 RAJENDRA CHARY 159


Problems
• The density function of a random variable is given as:
fX(x) = ae-bx, 0 ≤ x
Find the characteristic function and the first two
moments(eg.3.37)

15-Nov-21 RAJENDRA CHARY 160


Integration of uv

15-Nov-21 RAJENDRA CHARY 161


Applications
• Design of Telecommunications using Binomial, Poisson
distributions

15-Nov-21 RAJENDRA CHARY 162


Conclusion of the Unit
In this unit we studied about:
• Random variable-definition, conditions & types
• PDF, pdf with properties
• Expected value, moments
• PMF, complex random variable
• Characteristic function
• Standard random variables with their PDF,pdf and plots
• Transformations on random variables
• Problems
15-Nov-21 RAJENDRA CHARY 163
15-Nov-21 RAJENDRA CHARY 164

You might also like