Module 1 - Part 1 Probability and Stochastic Process

Module 1 (18MAB203T)
Dr. Anuradha Yadav

Assistant Professor
Department of Mathematics
College of Engenerring & Technology
SRM Institute of Science and Technology
Academic Year 2020 − 2021 (Even Semester)

18MAB203T - Probability & Stochastic Processes
Dr. Anuradha Yadav (SRMIST, Chennai) Module 1 (18MAB203T) 8 February 2021

RANDOM VARIABLE
Random Variable in One Dimensional
A random variable (R.V.) is a real valued function defined over the sample
space of an experiment.
A random variable is a function X (s) = x, which assigns a real number (x) to

every element (s) of the sample space (S) corresponding to random
experiment (E), i.e.
X : S → R.
X (s) = x ∈ R.

Example
Example
Consider an experiment of tossing 2 coins simultaneously with a random
variable X denoting the number of heads. Then we define the following:
Let 2 coins be tossed simultaneously. Then
E : An experiment of tossing 2 unbiased coins.
S : Outcomes of the experiment E.
i.e. Sample space = {HH, HT , TH, TT }.
X : a random variable which denotes the number of heads.
X (HH) = 2 = x1 (say)
X (HT ) = 1 = x2
X (TH) = 1 = x2
X (TT ) = 0 = x3
Random variable X takes set values {0, 1, 2}, which is range space of X
denoted as RX .
Note: Here X is a discrete random variable.

Types of Random Variable
Types of random variables

There are two types of random variables according to their range space. They
are
1 Discrete random variable (D.R.V.)
2 Continuous random variable (C.R.V.)
Discrete Random Variable

If X is a random variable (R.V.) which can take a finite number or countably
infinite number of values, X is called a Discrete Random Variable. When the
R.V. is discrete, the possible values of X may be assumed as x1 , x2 , ..., xn , ....
In the finite case, the list of values terminates and in the countably infinite
case, the list goes upto infinity.
Continuous Random Variable

If X is an R.V. which can take all values (i.e. infinite number of values) in an
interval, then X is called Continuous Random Variable.

Discrete Random Variable
Example
1 The number of telephone calls received by the telephone operator.

2 The number of printing mistakes in a book.
Probability Mass Function

Let X be a discrete random variable which takes values x1 , x2 , · · · . Each value
is associated with probability pi = P (X = xi ), then pi is called the Probability
Mass Function of the random variable X , provided pi (i = 1, 2, · · · ) satisfies
the following two conditions:
1. pi ≥ 0, ∀i
X
2. pi = 1
i
Note: The pair {xi , pi } is called probability distribution of the discrete

random variable X .

Cumulative Distribution Function of D.R.V.
Cumulative Distribution Function (C.D.F.)
The function F (x) or FX (x) is called the Cumulative Distribution function of

the discrete random variable X and is defined as:
X
F (x) = P (X ≤ x) = p (xi )
xi ≤x
Properties of C.D.F.
1 0 ≤ F (x) ≤ 1
2 F (x) is a non- decreasing function of x,
i.e. F (x1 ) ≤ F (x2 ) if x1 < x2
3 F (−∞) = lim F (x) = 0 = P(X ≤ −∞)
x→−∞
F (∞) = lim F (x) = 1 = P(X ≤ ∞)
x→∞
4 If X is a discrete R.V. taking values x1 , x2 , ...., where x1 < x2 < ...., Then
P(X = xi ) = F (xi ) − F (xi−1 ).

Mathematical Expectation of Discrete Random
Variable
Mean value of X
The value E(X ) is called the Expectation of X (or) Expected value of X (or)
Mean value of X and is defined as:
P
Mean of X = E(X ) = X = xi p (xi ), if X is a discrete random variable.
i
Properties of Mean value of X

1 E(aX ) = aE(X ), where a is a constant.
2 E(aX + b) = aE(X ) + b, where a and b are constants.
3 E (k1 X1 + k2 X2 + · · · + kn Xn ) = k1 E (X1 ) + k2 E (X2 ) + · · · + kn E (Xn ),
where k1 , k2 , · · · , kn are constants.
4 If X1 , X2 , · · · , Xn , are independent random variables, then
E (X1 X2 · · · Xn ) = E (X1 ) E (X2 ) · · · E (Xn ) , Multiplication theorem on expectation

5 E X − X = E[X − E(X )] = E(X ) − E(X ) = 0. X = E(X ) = Mean of X

Variance and Properties of Discrete Random
Variable
Variance
Variance of X :
2
Var (X ) = V(X ) i.e. σX2 = E X − X , where X = E (X )
P 2
= x − X p (x) , if X is a D.R.V. and p(x) is P.M.F. ofX .
x
(or)σX = E X 2 − [E(X )]2

2

" #2
X X
= x 2 p(x) − xp(x) , if X is a D.R.V. and p(x) is P.M.F. of X .
x x
Properties of Variance
1 Var(X ) ≥ 0
2 Var(a) = 0, where a is a constant.
3 Var (a ± bX ) = Var (a) + b2 Var (X ) = b2 Var (X ), where a and b are
constants.
Standard Deviation and Moments of Discrete
Random Variable
Standard Deviation
Standard Deviation of p
X = Var (X ) = σX .
Moments
The expected value of an integral power of a random variable is called

its moments.
Moments are classified as two types.
1 Moments about mean (µ).
2 Moments about any point (a).

Examples of Discrete R.V.
Example 1
From a lot containing 25 items, 5 of which are defective, 4 items are
chosen at random. If X is the number of defective found, obtain the
probability distribution of X, when the items are chosen
(i) without replacement and
(ii) with replacement.
Solution: Since only 4 items are chosen, X can take the values 0, 1, 2, 3
and 4. The lot contains 20 non- defective and 5 defective items.
Case (i): When the items are chosen without replacement, we can assume
that all the 4 items are chosen simultaneously.
P(X = r ) = P (choosing exactly r defective items),

= P (choosing r defective and (4 − r ) good items),
5 20
Cr × C4−r
= 20 C
(r = 0, 1, .., 4)
4
Examples of Discrete R.V. Continued...
Case (ii): When the items are chosen with replacement, we note that the
probability of an item being defective remains the same in each draw.
5 1 4
p= = , q = and n = 4.
25 5 5
The problem is one of performing 4 Bernoulli’s trials and finding the probability
of exactly r successes.
4 4−r
1 44
P(X = r ) = Cr , (r = 0, 1, .., 4).
5 5
Example 2
A shipment of 6 television sets contains 2 defective sets. A hotel makes
a random purchase of 3 of the sets. If X is the number of defective sets
purchased by the hotel, find the probability distribution of X . Solution:
All the 3 sets are purchased simultaneously. Since there are only 2 defective
sets in the lot, X can take the values 0, 1 and 2.
P(X = r ) = P (choosing exactly r defective items),

= P (choosing r defective and (3 − r ) good items),
2
Cr × 4 C3−r
= 6C
, (r = 0, 1, 2)
3
The required probability distribution is represented in the form of the following

table.
X =4 pr
1
0
5
3
1
5
1
2
5
Total 1

Example 3
Let X denotes the number of heads in an experiment of tossing two
coins. Find
(a) probability distribution, (b) Cumulative distribution
(c) Mean of X (d) Variance of X
(e) P (X ≤ 1) (f) P (|X | ≤ 1)
(g) P (X ≥ 1) (h) find minimum value of c such that P (X ≤ c) > 1/2.
Solution
We know that, by tossing two coins, the sample space is
S = {HH, HT , TH, TT }
n(S) = |S| = 4
Given that X is a random variable which denotes the number of heads.

X (HH) = 2, X (HT ) = 1, X (TH) = 1, X (TT ) = 0
The range space of X is RX = 0, 1, 2. ⇒ X is a discrete random variable.

X (HH) = 2, X (HT ) = 1, X (TH) = 1, X (TT ) = 0 (1)

The range space of X is RX = {0, 1, 2}.
⇒ X is a discrete random variable.
(a) Probability distribution of X is {x, p(x)}:
1
P(X = 0) = [number of times X = 0 is 1 by (1)]
4
2
4
1
4
Probability distribution of X is
R.V. X 0 1 2
1 2 1 1
P.M.F. P(X = x) 4 = 2 4
4

(b) Cumulative distribution F (X = x) = P(X ≤ x) :
R.V. X 0 1 2
C.D.F. F (X = x) F (x = 0) = P(X ≤ 0) = 1
4
F (x = 1) = P(X ≤ 1) = 3
4
F (x = 2) = P(X ≤ 2) = 4 = 1
4
(c) Mean of X :
X
E(X ) = xP(X = x)
x
x=2
X
= x · P(X = x)
x=0
= 0 · P(X = 0) + 1 · P(X = 1) + 2 · P(X = 2)
1 2 1
= 0· +1· +2· [refer (2)]
4 4 4
1 1
= 0+ +
2 2
E(X ) = 1

2
(d) Variance of X : V (X ) = E X 2 − [E(X )] then

X 2
E X2 = x · P(X = x)
x
x=2
X
= x 2 · P(X = x)
x=0
= 0 · P(X = 0) + 12 · P(X = 1) + 22 · P(X = 2)

2
2 1
=0+1· +4· [refer (2)]
4 4
1 3
= +1=
2 2
3 1
V (X ) = − 1 = [E(X ) = 1]
2 2

(e) P (X ≤ 1) = P(X = 0) + P(X = 1)

1 2 3
= + = [refer (2)]
4 4 4
(f) P (|X | ≤ 1) = P(−1 ≤ X ≤ 1) by the definition of |X | ≤ 1
= P(X = −1) + P(X = 0) + P(X = 1)

1 2 3
= 0+ + = [refer (2)]
4 4 4
(g)) P (X ≥ 1) = 1 − P (X < 1)
1
P(X = 0) = [refer (2)]
4

(h) Minimum value of c such that P (X ≤ c) > 1/2.
X P(X ≤ c) > 1 Remark
2
0 P(X ≤ 0) = P(X = 0) = 1 ≯ 1 c 6= 0
4 2
1 P(X ≤ 1) = P(X = 0) + P(X = 1) = 1 + 2 = 3 > 1 c = 1
4 4 4 2
2 P(X ≤ 2) = P(X = 0) + P(X = 1) + P(X = 2) = 1 + 2 + 1 = 1 > 1 c = 2
4 4 4 2
P(X ≤ c) > 12 satisfies for c = 1, 2.

Minimum value of c is 1.
Example 4
Let X takes values 1, 2, 3, 4 such that
2P (X = 1) = 3P (X = 2) = P (X = 3) = 5P (X = 4) .
Find the distributions of X .
Solution
Here X is a discrete random variable.
Let 2P (X = 1) = 3P (X = 2) = P (X = 3) = 5P (X = 4) = k
k
P(X = 1) =
2
k
P(X = 2) =
3
P(X = 3) = k
k
P(X = 4) =
5
We know that, by the property of probability
X
P(X = x) = 1
x
x=4
X
P(X = x) = 1
x=1
P(X = 1) + P(X = 2) + P(X = 3) + P(X = 4) = 1
k k k
+ +k + = 1
2 3 5
15k + 10k + 30k + 6k

=1
30
30
61k = 30 ⇒ k =
61
The probability and cumulative distributions table of X is
R.V. x 1 2 3 4
P.M.F. P(X = x) 15/61 10/61 30/61 6/61
When x < 1, F (x) = 0

When 1 ≤ x < 2, F (x) = P(X = 1) = 15
61
When 2 ≤ x < 3, F (x) = P(X = 1) + P(X = 2) = 25
61
When 3 ≤ x < 4, F (x) = P(X = 1) + P(X = 2) + P(X = 3) = 55
61
When x ≥ 4, F (x) = P(x = 1) + P(x = 2) + P(x = 3) + P(x = 4) = 1.

Example 5
A discrete random variable X has the probability function given below:
X 0 1 2 3 4 5 6 7
P(X ) 0 a 2a 2a 3a a2 2a2 7a2 + a
Find
(a) a (e)P (3 ≤ X < 6)
(b) cumulative distribution function (f) P (2X + 3 < 7)
(c) P (X < 6) (g) P (X > 1 / X < 3)
(d) find maximum value of c such that
P (X ≤ c) < 34 .
Solution
Given X is discrete R.V. with P.M.F. P(X ) with
P an unknown a.
(a) We know that, by the definition P.M.F. is P(X = x) = 1
x

x=7
X
P(X = x) = 1
x=0
2 2 2
0 + a + 2a + 3a + a + 2a + 7a + a = 1
2
10a + 9a = 1
2
10a + 9a − 1 = 0
2
10a + 10a − a − 1 = 0
10a(a + 1) − (a + 1) = 0
(10a − 1)(a + 1) = 0
a = 1 or − 1
10
But a 6= −1[ Probability ∈ [0, 1]]
a = 1
10
Probability distribution table becomes

R.V. x 0 1 2 3 4 5 6 7
P.M.F. p(x) 0 1/10 2/10 2/10 3/10 1/100 2/100 17/100
(b) Cumulative distribution function F (X = x) = P(X ≤ x) :
R.V. x: 0 1 2 3 4 5 6 7
C.D.F. F (x) : 0 1/10 3/10 5/10 8/10 81/100 83/100 1

(c) P(X < 6) = P(X = 0) + P(X = 1) + P(X = 2) + P(X = 3)

+ P(X = 4) + P(X = 5)
= 1 − [P(X ≥ 6)] (P(A) = 1 − P(A))
= 1 − [P(X = 6) + P(X = 7)] = 1 − [2/100 + 17/100]
100 − 19 81
= 1 − [19/100] = = = 0.81
100 100
(d) To find the maximum value of c such that P (X ≤ c) < 34 .
X P(X ≤ c) > 34 = 0.75 Remarks

0 P(X ≤ 0) = F (0) = 0 < 0.75 c=0
1
1 P(X ≤ 1) = F (1) = 10 = 0.1 < 0.75 c=1
3
2 P(X ≤ 2) = F (2) = 10 = 0.3 < 0.75 c=2
5
3 P(X ≤ 3) = F (3) = 10 = 0.5 < 0.75 c=3
8
4 P(X ≤ 4) = F (4) = 10 = 0.8 ≯ 0.75 c 6= 4
81
5 P(X ≤ 5) = F (5) = 100 = 0.81 ≯ 0.75 c 6= 5
X P(X ≤ c) > 34 = 0.75 Remarks

83
6 P(X ≤ 6) = F (6) = 100 = 0.83 ≯ 0.75 c 6= 6
100
7 P(X ≤ 7) = F (7) = 100 = 1 ≯ 0.75 c 6= 7
P(X ≤ c) < 34 satisfies for c = 0, 1, 2, 3.

Maximum value of c is 3.
(e) P(3 ≤ X < 6) = P(X = 3) + P(X = 4) + P(X = 5)

51
= 2/10 + 3/10 + 1/100 = = 0.51
100
(f) P(2X + 3 < 7) = P(2X + < 7 − 3)

= P(2X < 4)
= P(X < 2)
= P(X = 0) + P(X = 1)
1
= 0 + 1/10 = = 0.1
10

P[(X > 1) ∩ (X < 3)] P(A ∩ B)
(g) P(X > 7/X < 3) = P(A/B) =
P(X < 3) P(B)
P[(X = 2, 3, 4, 5, 6, 7) ∩ (X = 0, 1, 2)]
=
P(X = 0, 1, 2)
P[(X = 2)]
=
P(X = 0) + P(X = 1) + P(X = 2)
2 2
10 10
= 1 2
= 3
0+ 10 + 10 10
2
= = 0.666666
3
∼
= 0.667

Continuous Random Variable
Example
1 The duration of telephonic conversation.
2 The path of the aeroplane from Chennai to Hydarabad.
Probability Density Function

Let X be a continuous random variable such that

1 1
P x − dx ≤ X ≤ x + dx = f (x)dx
2 2
then f (x) is called the probability density function of X , provided f (x)

satisfies the following conditions:
1. f (x) ≥ 0, for all x ∈ Rx , and

R∞
2. f (x) dx = 1.
−∞

Cumulative Distribution Function of C.R.V.
Cumulative Distribution Function
The function F (x) or FX (x) is called the Cumulative Distribution function of
the continuous random variable X and is defined as:
Zx
F (x) = P(−∞ < X < x) = f (x) dx
−∞
Properties of C.D.F.
1 0 ≤ F (x) ≤ 1.
2 F (x) is a non-decreasing function of X i.e. F (x1 ) ≤ F (x2 ) ( if x1 < x2 )
3 F (−∞) = lim F (x) = 0 = P(X ≤ −∞), F (∞) = lim F (x) = 1 = P(X ≤ ∞)
x→−∞ x→∞
4 Moreover, P(a ≤ X ≤ b) or P(a < X < b) or etc. of a C.R.V. X (for curve f (x),
the probability curve of the R.V. X ) is defined as: Z b
P(a ≤ X ≤ b) = P(a < X < b) = P(a ≤ X < b) = P(a < X ≤ b) = f (x)dx
a
5 The relation between C.D.F. F (x) and P.D.F. f (x):

d
[F (x)] = f (x)
dx
Mathematical Expectation of Continuous R.V.
Mean value of X
The value E(X ) is called the Expectation of X (or) Expected value of X (or)
and is defined as:
Mean of X ,
Z∞
E(X ) = X = x f (x) dx,
−∞
if X is a continuous random variable.

Note: 1. E(X ) is the first moment of X .
2. In general, E [X r ] is the r th moment of the random variable X .

1 E(a) = a, where a is a constant.
2 E(aX ) = aE(X ), where a is a constant.
3 E(aX + b) = aE(X ) + b, where a and b are constants.

1 E (k1 X1 + k2 X2 + · · · + kn Xn ) = k1 E (X1 ) + k2 E (X2 ) + · · · + kn E (Xn ),
where k1 , k2 , · · · , kn are constants.
2 If X1 , X2 , · · · , Xn , are independent random variables, then
E (X1 X2 · · · Xn ) = E (X1 ) E (X2 ) · · · E (Xn )

(Multiplication theorem on expectation)

3 E X − X = E[X − E(X )] = E(X ) − E(X ) = 0.

∵ X = E(X ) = Mean of X
Variance
Variance of X : Var(X )=V (X ) i.e.
2
σX2 = E X − X , where X = E (X )
R 2
= R x − X f (x) dx, if X is a C.R.V. and f (x) is P.D.F. of X .
x
(or)
σX2 = E X 2 − [E(X )]2

" #2
2 R∞ 2 R∞
σX = x f (x) − xf (x) , if X is a C.R.V. and f (x) is P.D.F. of X .
−∞ −∞
Standard Deviation of X
p
X = Var (X ) = σX
Properties of Variance
1 Var(X ) ≥ 0
2 Var(a) = 0, where a is a constant.
3 Var(a ± bX ) =Var(a) + b2 Var(X ) = b2 Var(X ), where a and b are
constants.
4 Var(aX ± bY ) = a2 Var(X ) + b2 Var(Y ). If X and Y are independent.

Examples of Continuous R.V.
Example 1
Check whether the following functions are probability density function
(P.D.F.):
(a) f (x) = 6x(1 − x), 0 ≤ X ≤ 1 (b) f (x) = π1 1+x
1
2 , −∞ < X < ∞
100
2 , X > 100
(c) f (x) = x (d) f (x) = sin x, 0 < x < π
 0, X < 100
 0, x < 2
1
(e) f (x) = (3 + 2x) , 2≤x ≤4
 18
0, x > 4
Solution
Here f (x) is defined in the interval (a, b) which contains uncountably infinite
values.
X is a continuous random variable. If X is a continuous R.V. and f (x) is a
function defined in an interval of the form (a, b), then
Zb Zb
f (x)dx = 1, given f (x) is pdf in (a, b), f (x)dx 6= 1, given f (x) is not pdf.
a a
Examples of Continuous R.V. Continued...
(a) Given f (x) = 6x(1 − x) = 6 x − x 2 , 0 ≤ X ≤ 1.

R1
We have to prove f (x) dx = 1 (1)
0 Z1 Z1
LHS of (1) = f (x)dx = 6 x − x 2 dx

0 0
2 3 1

6x 6x 1
= 3x 2 − 2x 3 0

= −
2 3 0
= [(3 − 2) − (0 − 0)] = 1
= RHS of (1)
The given f (x) is P.D.F.
1
(b) Given f (x) = π1 1+x 2 , −∞ < X < ∞.
∞
R
We have to prove f (x)dx = 1 (2)
−∞

Z∞ Z∞
1 1
LHS of (2) = f (x) dx = dx
π 1 + x2
−∞ −∞
1 −1 ∞ 1 −1
tan (∞) − tan−1 (−∞)

= tan x −∞ =
π π
1 π −π
= −
π 2 2
1
= [π] = 1 = RHS of (2)
π
The given f (x) is
P.D.F.
100
x2
, x > 100
(c) Given f (x) =
0, x < 100
Z∞
−∞

Z100 Z∞ Z∞
100
LHS of (3) = f (x)dx + f (x)dx = 0 + dx
x2
−∞ 100 100
∞
−1 −1
= 100 = 100 0 −
x 100 100
= 1 = RHS of (3)

(d) Given f (x) = sin x, 0 < x < π
Zπ
We have to prove f (x) dx = 1 (4)
Z0π Zπ
π π
LHS of (4) = f (x)dx = sin xdx = [− cos x]0 = −[cos x]0
0 0
= −[(cos 0) − (cos π)] = −[1 − (−1)] = 2 6= 1 6= RHS of (4)
The given f (x) is not P.D.F.


 0, x < 2
1
(e) Given f (x) = 18 (3 + 2x) , 2 ≤ x ≤ 4
0, x > 4

R∞
−∞
Z 2 Z 4 Z ∞
LHS of (5) = f (x) dx + f (x) dx + f (x) dx
−∞ 2 4
1 4
=0+ 3x + x 2 2 + 0
18
1
= [(12 + 16) − (6 + 4)] = 1 = RHS of (5)
18

Example 2
A continuous random variable X has a p.d.f. f (x) = 3x 2 , 0 ≤ X ≤ 1. Find
k & α such that
(a) P (X ≤ k ) = P (X > k ) [(b)] P (X > α) = 0.1
(c) P (|X | ≤ 1) [(d)] P(X > β) = 0.05
Solution : Here X is a continuous random variable.

Given, pdf f (x) = 3x 2 , 0 ≤ x ≤ 1. (1)
(a). Find k from the given equation P (X ≤ k ) = P (X > k ) (2)
We know that, P (X ≤ k ) + P (X > k ) = 1

2P (X ≤ k ) = 1 [P (X ≤ k ) = P (X > k )]
1
P (X ≤ k ) =
2
1
(2) ⇒ P (X ≤ k ) = P (X > k ) = (3)
2

1
From (3), we have P(X ≤ k ) = 2 or P(X > k ) = 12 .
Method 1 Method 2
P(X ≤ k ) = 21 P(X > k ) = 21
Rk 1
R∞ 1
i.e. f (x)dx = 2 i.e. f (x)dx = 2
−∞ k
Rk 1
R1 1
3x 2 dx = 2 3x 2 dx = 2
0 k
3 1
[x 3 ]k0 = 21 x k = 21
k 3 = 21 1 − k 3 = 12
1 1
k = 12 3 k = 12 3
(b). Find α from given equation P(X > α) = 0.1
Z∞ Z1
i.e. f (x)dx = 0.1 ⇒ 3x 2 dx = 0.1 [ by (1)]
α α

3 1 1
x α = 0.1 =
10
1
1 − α3 =
10
3 1
−α = −1
10
9
α3 =
10
13
9
α=
10
(c). P(|X | ≤ 1) = P(−1 ≤ X ≤ 1)

Z1 Z0 Z1
= f (x)dx = (0)dx + 3x 2 dx [ by (1)]
−1 −1 0
1
= x3 0 = 1
(d). Find β from given equation P (X > β) = 0.05 :
i.e., P (X > β) = 0.05

Z1
f (x) dx = 0.05
β
Z1
3x 2 dx = 0.05
β
1
x3

3 = 0.05
3 β
3
1 − β = 0.05
β 3 = 1 − 0.05
β 3 = 0.95
β = 0.9830476

Example 3
For thetriangular distribution
 x, X ∈ [0, 1]
f (x) = 2 − x, X ∈ [1, 2] .
0, otherwise

Find mean and variance.
Solution : Here X is a continuous random variable with pdf

 x, X ∈ [0, 1]
f (x) = 2 − x, X ∈ [1, 2] (1)
0, otherwise

Mean of X :
Z∞
E(X ) = xf (x)dx
−∞
Z0 Z1 Z2 Z∞
= (0)dx + x · xdx + x · (2 − x)dx + (0)dx [ by (1)]
−∞ 0 1 0
Z1 Z2
2
= x dx + (2x − x 2 )dx
0 1
3 1
2
2x 2 x3

x
= + −
0 3 2 3 1

1 8 1
= −0 + 4− − 1−
3 3 3

1 7
= + 3−
3 3
1 2
= +
3 3
∴ E[X ] = 1 (2)

2
Variance of X : Var(X ) = E X 2 − [E(X )]

Now,
Z∞ Z0 Z1 Z2
2 2
E X = x f (x)dx = x (0) dx + x · xdx + x 2 · (2 − x)dx
2 2
−∞ −∞ 0 1
Z∞
+ x 2 (0) dx
0
Z1 Z2 4 1 3 2
3 x 2x x4
2x 2 − x 3 dx =

= x dx + + −
4 0 3 4 1
0 1

1 16 16 2 1
= [1 − 0] + − − −
4 3 4 3 4

1 14 15 1 56 − 45 1 11 14
= + − = + = + =
4 3 4 4 12 4 12 12
7 2 7 1
E X 2 = ⇒ Var(X ) = E X 2 − [E(X )] = − 1 =

6 6 6
Central Moments
1. Moments about Mean (Central Moments)

The r th moment about mean (µ) for a random variable X is defined as
r
µr = E (X − µ)
The first four moments about the mean are

1
µ1 = E (X − µ) = E(X ) − E(µ) = µ − µ = 0
h i2
2
µ2 = E (X − µ) = Variance of X = E X 2 − E(X )
3
µ3 = E (X − µ)
4
µ4 = E (X − µ)

Central Moment Continued...
2. Moments about Any Point

The r th moment about any point(a) for a random variable X is defined as
r
µ0r = E (X − a)
Moments about Origin (Raw Moments)

The r th moment about origin for a random variable X is defined as
µ0r = E (X r )
The first four moments about the origin are
µ01 = E X 1 , µ02 = E X 2 , µ03 = E X 3 , µ04 = E X 4


Central Moment Continued...
Central moments in terms of Raw Moments
µ1 = 0
2
µ2 = Var (X ) = µ02 − (µ01 )
3
µ3 = µ03 − 3µ02 µ01 + 2 (µ01 )
2 4
µ4 = µ04 − 4µ03 µ01 + 6µ02 (µ01 ) − 3 (µ01 )
2 3
In general, µr = µ0r − r C1 µ0r −1 (µ01 ) + r C2 µ0r −2 (µ01 ) − r C3 µ0r −3 (µ01 )
r
+ · · · + (−1)r (µ01 )
Note: µ00 = 1, µ01 = Mean of X = E(X ), µ02 = E X 2


Example of Continuous R.V. for Central Moments
Example 4
The density function of a random variable X is given by
f (x) = Kx(2 − x), 0 < x < 2.
(i) Find K .
(ii) r th moment
(i). To find K : Solution : Here X is continuous random variable with pdf has
unknown K . To find K :
Z2 Z2
2x − x 2 dx = 1

Kx (2 − x) dx = 1, K
0 0
2 2
x3

x 8
K 2 − = 1, K 4− − (0 − 0) = 1
2 3 0 3

4
K =1
3
3
K =
4
(ii). r th moment:
Z2 Z2
r r 3
E [X ] = x f (x) dx = x r x (2 − x) dx
4
0 0
Z2 2
3 2x r +2 x r +3

3 r +1 r +2

= 2x −x dx = −
4 4 r +2 r +3 0
0
2r +2 2r +3 3 2r +3 2r +3

3
= 2· − − (0 − 0) = −
4 r +2 r +3 4 r +2 r +3
3 2r +3

1 1
= −
4 r +2 r +3
3 2r +3

r +3−r −2
=
22 (r + 2) (r + 3)
r +1

3 2
=
(r + 2) (r + 3)
Characteristic function
Characteristic function
Although higher order moments of a R.V. X may be obtained directly by using

the definition of E(X n ), it will be easier in many problems to compute them
through the characteristic function or equivalently through the moment
generating function of the R.V. X . While the characteristic function always
exists, the moment generating function need not.
Characteristic function of a R.V. X (discrete or continuous) is defined as

E(eiX ω ) and denoted as f (ω). If X is a discrete R.V. that can take the values
x1 , x2 , ... such that P(X = xr ) = pr , then
X
φ(ω) = eixr ω p(xr ).
r
If X is a continuous R.V. with density function f (x), then

Z ∞
φ(ω) = eixω f (x)dx.
−∞

Properties of Characteristic function
Properties
i n ωn
1 µ0n = E(X n )= the coefficient of n in the expansion of φ(ω) in series of
ascending powers of iω.
h n i
2 µ0n = i1n dω
d
n φ(ω) .
ω=0
3 If the characteristic function of a R.V. X is φx (ω) and if Y = aX + b, then
φY (ω) = eibω φY (aω).
4 If X and Y are independent R.Vs., then
φX +Y (ω) = φX (ω)φ̇Y (ω)

5 If the characteristic function of a continuous R.V. X with density function
f (x) is φ(ω),
R ∞then the−ixω
Fourier inversion formula
1
f (x) = 2π −∞
φ(ω)e dω Z ∞
φ(ω) = f (x)eixω dx
−∞
6 If the density function of X is known, the density function of Y = g(X ) can

be found from the C.F. of Y , provided Y = g(X ) is one-to-one.
Binomial Distribution
Bernoulli trial
A Bernoulli trial (or binomial trial) is a random experiment in which there are
only two possible outcomes namely success (s) and failure (f ).
The sample space of a Bernoulli trial is S = {s, f }.
Bernoulli experiment
The experiment consists of ‘n’ independent repeated Bernoulli trials.
Bernoulli distribution
Let us consider an experiment consists of 0 n0 independent trials which results
successes(S) and failures (F ) of the random form
S S F S ··· F F S
Let X be a random variable which denotes the number of success, specifically

x ∈ X be the number of successes and hence we have 0 n − x 0 number of
failures.
Bernoulli distribution Continued...
Let p be the corresponding probability to get a success and q be the corre-
sponding probability to get a failure.
The probability to get this form S S F S · · · F F S is
P(S S F S · · · F F S) = P(S) · P(S) · P(F ) · P(S) · · · P(F ) · P(F ) · P(S)

[∵ independent trials]
= p · p · q · p···q · q · p
h i h i
= p · p···p q · q···q
(x times) (n−x times)
x n−x
=p q
This probability value is for ‘x’ successes in sequence above form SSFS · · · FFS
only.
But 0 x 0 successes in 0 n0 trials can occur in n Cx ways.
∴ Probability of 0 x 0 successes in 0 n0 trials is n Cx px q n−x , i.e.
n
P(X = x successes) = Cx px q n−x
where x = 0, 1, 2, · · · , n with p + q = 1.
Bernoulli distribution Continued...
Note
1 P(X = x) = n Cx px q n−x is the (x + 1)th term in the binomial expansion of
n
(p + q)
.
(p + q)n = n C0 p0 q n + n C1 p1 q n−1 + n C2 p2 q n−2 + · · · + n Cn pn q 0

n
X n
X
n
2 P(x) = Cx px q n−x = (p + q)n = 1n = 1.
x=0 x=0
Definition
The random variable X that counts the number of successes, in the ‘n’
Bernoulli trials is said to follow a Binomial distribution with parameters n
and p, written as B(n, p). Symbolically, X ∼ B(n, p), n ∈ N, p ∈ [0, 1].
The Probability mass function of the Binomial distributed discrete random
variable X is
n
P (X = x) = P(x) = Cx px q n−x , x = 0, 1, 2, 3, ...n, with p + q = 1.

Assumptions of the Binomial distribution
(1) The random experiments corresponds to two possible outcomes

(success or failure).
(2) Number of trials is finite.
(3) The trials are independent.
(4) The probability of success is constant in any trial.
Formula
(1) Probability mass function of X ∼ B(n, p) is
n
P(X = x) = Cx px q n−x
where x = 0, 1, 2, 3, ..., n with p + q = 1.

Continued...
where
n = number of trials.
X = Random variable (Discrete), represents number of successes,
which follows Binomial distribution.
x = value of random variable X .
p = probability of success in single trial.
q = probability of failure in single trial = 1 − p.
N = Number of times ‘n’ trials are repeated (or)
Total number of sets

Mathematical Expectation of Binomial distribution
The Probability mass function of the Binomial distributed discrete random vari-
able X is
n
P(X = x) = Cx px q n−x , x = 0, 1, · · · , n where p + q = 1
Characteristic function of X
φ(ω) = (1 − p + peiω )n = (q + peiω )n
Mean of X in terms of Characteristic Function

As we know that
n
P(X = x) = Cx px q n−x , x = 0, 1, 2, ...., n
Characteristic Function φ(ω) = E(eiωx )
Xn
= eiωx · n Cx px q n−x
x=0

Continued...
n
X
n
φ(ω) = Cx (peiω )x q n−x
x=0
= q n + n C1 (peiω )q n−1 + ...... + (peiω )n

= (q + peiω )n
From the properties of characteristic function, we know that
1 dn

µ0n = n φ(ω)
i dω n ω=0
Variance of X in terms of Characteristic Function

2
We have Var (X ) = µ02 − µ01 , we find
dn
φ(ω) = n(q + peiω )n−1 · pieiω
dω n
Continued...
dn
φ(ω) = n(q + peiω )n−1 · pieiω
dω n
dn
φ(ω)|w=0 = n(q + p)n−1 pi
dω n
1 d 1
µ01 = φ(ω) = n npi = np
i dω ω=0 i
Differentiating φ(ω) w.r.to ω,
d
npi (q + peiω )n−1 eiω

φ(ω) =
dω
d2
npi (n − 1)(q + peiω )n−2 eiω · peiω + (q + peiω )n−1 ieiω

φ(ω) =
dω 2
2
d
φ(ω) = npi[(n − 1)pi + i]
dω 2 ω=0
= npi 2 [(n − 1)p + 1]
Continued...
d2

φ(ω) = npi[(n − 1)pi + i]
dω 2 ω=0
= npi 2 [(n − 1)p + 1]
Now Variance of X is
2
Var (X ) = µ2 = µ02 − µ01
= np[(n − 1)p + 1] − n2 p2
= np[np − p + 1 − np]
= npq.

Examples of Binomial Distribution
Example 1
Consider an example of tossing 2 coins. Then the way of finding the
same values of probability using,
(1) the ideas probability,
(2) random variable probability,
(3) and Binomial distribution probability.
Solution:
(1) (2) (3)
S.S. = {HH, HT , TH, TT } S.S. = {HH, HT , TH, TT } 1
1 1 P(X = 2) =
P(2 heads) = P(X = 2) = 4
4 4 2
2 2 P(X = 1) =
P(1 head) = P(X = 1) = 4
4 4 1
1 1 P(X = 0) =
P(0 head) = P(X = 0) = 4
4 4

Examples of Binomial Distribution Continued...
Importance of the table is to find probability of ‘x’ heads

We can write the sample space for 2 coins in limited time. But we cannot
write the sample space for 10 or more coins in limited time. That’s why, we
use Binomial distribution for any number of finite coins.
Example 2
Let X denotes the number of heads in an experiment of tossing two
coins. Find probability distribution by using Binomial distribution.
Solution:
X = a discrete R.V. which denotes the number of heads.
1
p = Probability of getting a head from a single coin =
2
1 1
q =1−p =1− =
2 2
n = number of coins = 2.
X ∼ B.D. (n, p).

We know that, the P.M.F. of the Binomial Distribution is

P(X = x) = n Cx px q n−x
0 2−0
1 2 1 1 1
When X = 0, P(X = 0) = C0 =1·1· =
2 2 4 4
1 2−1
1 1 1 1 2
When X = 1, P(X = 1) = 2 C1 =2· · =
2 2 2 2 4
2 2−2
1 1 1 1
When X = 2, P(X = 2) = 2 C2 =1· ·1=
2 2 4 4
X: 0 1 2
The probability distribution is
P(x): 1/4 2/4 1/4

Example 3
Find Binomial distribution and P(X = 4), for
(a) mean 4, variance = 3 (b) mean 4, variance = 5.
Solution : Given distribution is Binomial. We know that the P.M.F. of the
Binomial distribution is
n
P(X = x) = Cx px q n−x , x = 0, 1, 2, · · · , n (1)
Mean of B.D. = np (2)
Variance of B.D. = npq (3)
Relation between Mean & Variance is of B.D is Mean > Variance. (4)
(a) Mean = np = 4 (5)

Variance = npq = 3 (6)
4q = 3 [ by (5)]
3
q=
4

1 1
⇒ np = 4 ⇒ n =4 p= ⇒ n = 16
4 4
x 16−x
16 1 3
⇒ P(X = x) = Cx
4 4
4 12
1 3
i.e. P(X = 4) = 16 C4 = 0.22
4 4
(b) Here Mean < Variance, which is not possible for Binomial distribution.
Explanation:
Mean = np = 4
5
Variance = npq = 5 ⇒ 4q = 5 ⇒ q = >1
4
which is not possible. [0 ≤ Probability (p (or) q) ≤ 1.]
⇒ Given data in (b) is not applicable for Binomial distribution.

Example 4
An irregular 6-faced dice is such that the probability that it gives 3 even
numbers in 5 throws is twice the probability that it gives 2 even numbers
in 5 throws. How many sets of exactly 5 trials can be expected to give no
even number out of 2500 sets?
Solution: Let the probability of getting an even number with the unfair dice be
p. Let X denote the number of even numbers obtained in 5 trials (throws).
P(X = 3) = 2 × P(X = 2)
5
C3 p 3 q 2 = 2 × 5 C2 p 2 q 3
p = 2q = 2(1 − p)
2 1
3p = 2 or p = and q =
3 3
Now P (getting no even number)
1 1
= P(X = 0) = 5 C0 p0 q 5 = ( )5 =
5 243
Number of sets having no success (even number) out of N sets

= N × P(X = 0)
1
Required number of sets = 2500 ×
243
= 10, nearly

Module 1 - Part 1 Probability and Stochastic Process

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Module 1 - Part 1 Probability and Stochastic Process

Uploaded by

Copyright:

Available Formats

Module 1 (18MAB203T)

Dr. Anuradha Yadav

Academic Year 2020 − 2021 (Even Semester)

Dr. Anuradha Yadav (SRMIST, Chennai) Module 1 (18MAB203T) 8 February 2021

Random Variable in One Dimensional

A random variable is a function X (s) = x, which assigns a real number (x) to

Dr. Anuradha Yadav (SRMIST, Chennai) Module 1 (18MAB203T) 8 February 2021

Note: Here X is a discrete random variable.

Types of random variables

Discrete Random Variable

Continuous Random Variable

Dr. Anuradha Yadav (SRMIST, Chennai) Module 1 (18MAB203T) 8 February 2021

1 The number of telephone calls received by the telephone operator.

Probability Mass Function

Note: The pair {xi , pi } is called probability distribution of the discrete

Dr. Anuradha Yadav (SRMIST, Chennai) Module 1 (18MAB203T) 8 February 2021

The function F (x) or FX (x) is called the Cumulative Distribution function of

Dr. Anuradha Yadav (SRMIST, Chennai) Module 1 (18MAB203T) 8 February 2021

Properties of Mean value of X

Dr. Anuradha Yadav (SRMIST, Chennai) Module 1 (18MAB203T) 8 February 2021

(or)σX = E X 2 − [E(X )]2

The expected value of an integral power of a random variable is called

Dr. Anuradha Yadav (SRMIST, Chennai) Module 1 (18MAB203T) 8 February 2021

P(X = r ) = P (choosing exactly r defective items),

P(X = r ) = P (choosing exactly r defective items),

The required probability distribution is represented in the form of the following

Dr. Anuradha Yadav (SRMIST, Chennai) Module 1 (18MAB203T) 8 February 2021

Given that X is a random variable which denotes the number of heads.

Dr. Anuradha Yadav (SRMIST, Chennai) Module 1 (18MAB203T) 8 February 2021

X (HH) = 2, X (HT ) = 1, X (TH) = 1, X (TT ) = 0 (1)

Dr. Anuradha Yadav (SRMIST, Chennai) Module 1 (18MAB203T) 8 February 2021

(b) Cumulative distribution F (X = x) = P(X ≤ x) :

Dr. Anuradha Yadav (SRMIST, Chennai) Module 1 (18MAB203T) 8 February 2021

= 0 · P(X = 0) + 12 · P(X = 1) + 22 · P(X = 2)

Dr. Anuradha Yadav (SRMIST, Chennai) Module 1 (18MAB203T) 8 February 2021

(e) P (X ≤ 1) = P(X = 0) + P(X = 1)

= P(X = −1) + P(X = 0) + P(X = 1)

Dr. Anuradha Yadav (SRMIST, Chennai) Module 1 (18MAB203T) 8 February 2021

P(X ≤ c) > 12 satisfies for c = 1, 2.

Find the distributions of X .

15k + 10k + 30k + 6k

When x < 1, F (x) = 0

Dr. Anuradha Yadav (SRMIST, Chennai) Module 1 (18MAB203T) 8 February 2021

Dr. Anuradha Yadav (SRMIST, Chennai) Module 1 (18MAB203T) 8 February 2021

Probability distribution table becomes

Dr. Anuradha Yadav (SRMIST, Chennai) Module 1 (18MAB203T) 8 February 2021

(c) P(X < 6) = P(X = 0) + P(X = 1) + P(X = 2) + P(X = 3)

(d) To find the maximum value of c such that P (X ≤ c) < 34 .

X P(X ≤ c) > 34 = 0.75 Remarks

X P(X ≤ c) > 34 = 0.75 Remarks

P(X ≤ c) < 34 satisfies for c = 0, 1, 2, 3.

(e) P(3 ≤ X < 6) = P(X = 3) + P(X = 4) + P(X = 5)

(f) P(2X + 3 < 7) = P(2X + < 7 − 3)

Dr. Anuradha Yadav (SRMIST, Chennai) Module 1 (18MAB203T) 8 February 2021

Probability Density Function

then f (x) is called the probability density function of X , provided f (x)

1. f (x) ≥ 0, for all x ∈ Rx , and

Dr. Anuradha Yadav (SRMIST, Chennai) Module 1 (18MAB203T) 8 February 2021

5 The relation between C.D.F. F (x) and P.D.F. f (x):

if X is a continuous random variable.

Properties of Mean value of X

Dr. Anuradha Yadav (SRMIST, Chennai) Module 1 (18MAB203T) 8 February 2021