Outline

 Aim: To review

Probability  Definitions and some basic p

properties
p of probability
p y
 Random variables and distributions
 Expected values and variances
 Some common continuous and discrete prob. distributions

1. Basic concepts  Uniform

 Normal
2. Random variables and distribution  Binomial
3. Common PDF  Poisson
 Exponential, etc..

Definitions Definitions
 An experiment (trial) is any process that generates well-  Finite and infinite sample space
defined outcomes. Example  A sample
p space
p is called ffinite if the number of ppossible
- Tossing a coin once or several times outcomes in the sample space is finite, otherwise it is called
- Rolling a six-sided die (sides numbered 1, 2, …, 6 ) infinite
 e.g rolling a die S=(1,2,3,4,5,6) or selection once card from
 The sample space for an experiment is the set of all
a standard deck s= (1,2,3,…52)
outcomes
 Discrete and continuous sample spaces
 An event is a subset of the sample space
 A sample space is called discrete if it contains only finite but
 Probability is a numerical measure of the likelihood that an countable number of possible outcomes, otherwise it is
event will occur. continuous

3 4

1
 Definitions Some Properties of Probability
 Let S be a sample space and A be an event. The probability  Let S be a sample space and A ,B  S.
of occurrence of A, Pr(A), is determined by one of the following:
g 0 ≤ Pr(A) ≤ 1; Pr(Ø) =0;
0; Pr(S) = 1.
 1) If the number of elements of S, n(S) , and that of A, n(A), are
known: A probability near 0 indicates an event is very unlikely to occur.
n( A)
Pr( A ) 
n(S ) A probability near 1 indicates an event is almost certain to occur.
 2) If the number of elements of S and/or that of A are unknown: A probability of 0.5 indicates the occurrence of the event is just
as likely as it is unlikely.
n( A)
Pr( A ) 
N 1. Complementary events
 where N is a sufficiently large number of trials and n(A) is the
number of times that event A occurred in the N trials. Pr(A' ) = 1 – Pr(A) where A' is complement of A

5 6

Some Properties of Probability Some Properties of Probability

2. Addition law of Prob. 3. Independent events : Event A and B occur simultaneously
Pr(A or B) = Pr(A) + Pr(B) – Pr( A and B) or P (A andd B) = Pr(A)
Pr(A P (A) * Pr(B)
P (B) or
Pr(A  B) = Pr(A) * Pr(B)
Pr(A U B) = Pr(A) + Pr(B) – Pr( A  B)
Example:
Mutually exclusive events : Two events A and B are said
A box contains 5 white and 8 black balls. Another box contains 6
to be mutually exclusive if they can not occur together; i.e.,
white and 10 black balls. One ball is taken out of each of the box.
A B=Ø
A Ø. If so, PPr(AUB)
(AUB) = PPr(A)
(A) + PPr(B)
(B)
What is the probability that the balls taken out are both white?
Example: Calculate the probability of drawing either an ace or a
spade in a pack of cards; (not mutually exclusive) = (5/13)*(6/16) = 15/104

= (4/52) + (13/52) – (1/52) = 4/13

7 8

2
 Law of conditional Probability Conditional probabilty
 Given that event A has occurred, find the probability that B  Determine the prob. that the outcome is a 6 in a die game,
also occurs. The event B is dependent on A and occurs only if being told that the roll has turned up an even number.
A has
h occurred. d (Conditional
d l Probability)
b bl  A = {2,4,6} and B = {6}
 P(B/A) = P(A  B)/P(A)  P(B/A) = P(A  B)/P(A) = P(B)/P(A)
 If A and B are independent P(A  B) =P(A)*P(B)  = (1/6)/1/2 = 1/3
 If B is the subset of A  P(A  B) = P(B)  Bayes’ theorem (Rule)
 Examples:  Reading g Assignment
g
 Given a red card drawn, the prob. that it is a king heart
 Randomly selected person has a history hypertension, the
probability that this person has hypertension

9 10

Random Variables and Distributions Random Variables and Distributions

• A random variable X is a function that associates numerical  A random variable X is called discrete if its range, RX , is finite or
countable infinity. In this case we denote RX by
values to every outcome of a random experiment.
 RX = { x1 , x2 , x3 , . . . }
• The distribution of a random variable X is the collection of  Given a discrete random variable X, we may define a function
all possible outcomes (range of X) along with their  f : RX [ 0, 1 ]
probabilities. Example: X = number of heads in three  given by f(xi) = Pr(X= xi ), for every xi  RX.
tosses of a coin • Such f is called the probability distribution function (pdf) of the discrete
Outcomes X Distribution: Range of X, RX = { 0, 1, 2 ,3} random variable X
HHH 3
P (X 0) = 1/8
Pr(X=0)
HHT 2  Note:
Pr(X=1) = 3/8
• f(xi)  0 , for all i
HTH 2
HTT 1
Pr(X=2) = 3/8
THH 2 • ∑ f(xi) = 1 , summation is over all i .
THT 1 Pr(X=3) = 1/8
TTH 1
11 TTT 0 Note: ∑ Pr (X= x ) = 1 x RX 12

3
 Random Variables and Distributions Exercise-1
 The cumulative distribution function (cdf) of a discrete  A pair of fair dice is rolled once. Let x be the random
random variable X with a pdf f(xi) is denoted by F(x) and variable whose value for any outcome is the sum of the two
defined by F(x) := Pr( xi ≤ x ) = ∑ f (xi ) numbers on the dice.
 (i) Find the probability function x . Construct the probability
f(x) F(x)
table and a probability chart
2p 1 F(1) = 0.5p  (ii) Find the probability that x is an odd number (ans. ½)
1.5p F(2) = 2p
p
 (iii) Find P(3  xi  9) and P(0  xi  4) (ans. 29/36 and 1/6)
F(3) = 3p
0 5p
0.5p  (iv) Find the CDF and represent it graphically
F(4) = 5p
1 2 3 4 5 1 2 3 4 5
F(5) = 6p
(a) (b)

(a) Example of a probability function of a discrete random variable. We may require p

= 1/6 . (b) is the corresponding cumulative distribution function
13 14

Continuous random Variables Continuous random Variables

 A random variable X is called continuous if its range, RX , takes a range
of values in an interval (often –  to  ).
 Example: X = the time between successive failures of an electronic
equipment 0< x < 
 The cumulative distribution function (cdf) of a
continuous random variable X with pprobabilityy densityy
function f(x) is denoted by F(x) and defined by
x

 Analogue to the pdf for discrete case, we can define a probability

F ( x)  Pr( X  x)   f (t )dt

distribution function (pdf) f(x) on RX , also called density function,
for a continuous random variable X so that  In this case
 Pr( x < X < x
x+ dx ) = f(x)dx i.e , f(x)dx is the probability that 1. F ( x) is defined for all x  R and 0  F ( x)  1
X lies in the interval (x, x+dx ) . Such f should satisfies the 2. F ( x) is non decreasing function of x.
following:
1. f ( x)  0 for all x  R X and f ( x)  0 for all x  R X . 3. lim x F ( x)  0 and lim x F ( x)  1.
dF ( x)
2. 
RX
f ( x )dx  1 4.  f ( x)
b
dx
15 3. Pr(a  X  b)   f ( x )dx 16 5. Pr( a  X  b)  F (b)  F (a )
a

4
 Expected values and variances Example
 Let X be a random variable and let f(x) be its pdf.  Suppose a biased die has probabilities p/2, p, p, p, p, 2p, of showing 1, 2, 3, 4,
5, 6, respectively. Find the mean and variance of this distribution.
 The expected value (or mean ) of X is defined by
 We require p = 2/13 (so that the sum of the distribution is 1 )
i x i f ( xi ) , for a discrete distributi on

E[ X ]    E[X] = ∑i xif(xi) = 1p/2 + 2p + 3p + 4p + 5p + 6 2p = 53p/2 =
 R X xf ( x ) dx , for a continuous distributi on
53/13 .  = 53/13,
 The variance of a distribution, Var(X), also written as 2
 Var(X) = ∑i ( xi –  )2 f(xi)

i ( xi   ) 2 f ( xi ) , for a discrete distribution  = (1-  )2p/2 + (2-  )2p + (3- )2p + (4-  )2p + (5-  )2p +

Var(X)  E[( X   ) 2 ]   (6  )22p
(6- 2 = 3120
3120p/169
/169 = 480/169
RX ( x   ) f ( x)dx , for a continuous distribution
2

 HW: Calculate the mean, var, std. of the random variable

 where  represents the mean (expected value) of X. in exercise1 (Ans 7, 5.8, 2.4)
 The standard deviation of the distribution, denoted by  measures
the spread of X about x =  )   Var ( X )
17 18

Probability Distributions Probability Distributions

 Some common Probability distributions: theoretical defn.  The distribution is called uniform because its density
1. Uniform distribution: A continuous random variable X function is uniform in the interval (a,b). It has application in
is called a uniform distribution over [a, b] if its pdf is given statistical problems
by  The density and distribution functions are shown below
1
f ( x)  , a xb
ba
f(x) F(x)
 The expected value and variance are given by 1.0

ab (b  a) 2
E[ X ]  , and Var[ X ] 
2 12
 The cdf of the uniform distribution is 1/(b-a)

x x 1 xa a b a b
F ( x)   f (t )dt   dt  .
a a ba ba
19 20

5
 Probability Distributions Probability Distributions
2. Normal (Gaussian) distribution: A continuous random  The cdf F(x) of the normal distribution
variable X is called a normal distribution (Gaussian distribution)
with mean, E[x] =  and variance, Var[x] = 2 if its pdf is given f(x)
F ( x)  Pr( X  x)  
x
f (t )dt

1  x 
2
  
2  
f ( x)   1
2
e ,   x   F(x)

x
f(x)  Note that:
b
Pr(a  X  b)   f (t )dt  F (b)  F (a )
a
f(x)
Pr( a≤ X ≤ b )

x=  -  x=  x=  + 

 The graph of f is symmetric about x =  and the point x =  –

a b
21  and x=  +  are points of inflexions. 22

Probability Distributions Probability Distributions

 The transformation z = ( x- )/  transforms normal • The cdf of the standard normal distribution, denoted by (z), is
distribution to that of  =0 and  2=1 . given by z
 ( z )  Pr( Z  z )    (t )dt


 A normal distribution with mean  =0 and standard

  Pr( a ≤ Z ≤ b) = (b) – (a)
deviation  =1 is called standard normal distribution
and is denoted by Z. The pdf of Z, denoted by (z), is then • Tables provide (z) value in many books
given 1 • If X is a normally distributed random variable with mean =  and
 z2
(z))
(  ( z)  1
e 2
,   z   variance =  2, then we can use (z) to determine Pr( a≤ X≤ b)
2

Pr( a  X  b)  Pr( a   X 

  b
 )
 Pr( a   Z  b
 )
z=-1 =0 z= 1
23 24
  ( b  )   ( a  )

6
 Example Probability Distributions
 Let X be a normally distributed random variable with 3. Poisson distribution: - a discrete distribution with indefinite
mean  = –1.2 and variance 2 = 4.6. Find Pr( 2 ≤ X ≤ 4 ) or undetermined number of outcomes

 If we wish to use a table of normal distribution:  Poisson distribution is used to describe events that occur in a
“totally random” fashion, meaning there is no way to predict
 2  (1.2) X   4  (1.2)  when the event will occur. Such events include, for instance,
Pr(2  X  4)  Pr   
 4.6  4.6  • number of customers arrive at a service facility during a specific
 Pr1.492  Z  2.145
 (2.145)  (1.492) time interval
 0.9838  0.9320
9320 ( from table )
 0.5818 • number of accidents in a city on a day
 The pdf for describing the number of occurrence of such events
during a specified time period follows the Poisson distribution –
defined below.
25 26

Probability Distributions Example

 Customers arrive at a facility according to a Poisson distribution at the
 Let X = number of events (e.g., arrivals) and Let λ =
rate of four per minute, on the average. What is the probability that at
expected (or average) number of arrivals per the specified unit l t one person will
least ill arrive
i iin any given
i 30 30-seconds
d iinterval?
t l?
time. Then, the Poisson pdf is given by  Let X = number of customers arrive within 30 seconds.
 k
• The average number of arrivals is min = 4 per minute.
f (k )  Pr( X  k )  e  , k  0, 1, 2, 3, ...
k!
 The mean and variance of the Poisson distribution is: • To compute the desired probability in the required time interval, we
E[ X ]   need to compute the average arrival rate  per 30-seconds - namely,
Var [ X ]    = 0.5  4 = 2 per 30 seconds
 Note: The probability of at least m arrivals (or at least m
 Pr(X  1) = 1 – Pr(X = 0)
events) is
= 1 – (0) e– / 0! ,  = 2.
 Pr( X  m) = 1 – Pr( X< m)
= 1– e–2 = 0.8647
27
 = 1 – [ Pr( X= 0) + Pr( X= 1) + … + Pr( X= m – 1 ) ] 28

7
 Probability Distributions Example
 4. Exponential distribution :  Cars arrive at a gas station randomly every 2 minutes, on
 Note: If  events occur per unit time
time, the average time interval the average. Determine the probability that the inter-arrival
between successive events is 1/  time of cars does not exceed 1 minute.
• If  is the average number of occurrence (arrival) of Poisson events,  Let X= inter-arrival time of cars
then the distribution of time between successive arrivals, X, is given
by the exponential pdf  x
• The desired probability is Pr( X ≤ 1) = 1 – e–(1)
f ( x)  e , x  0.
 The mean and variance of the exponential distribution are : • Given: average inter-arrival of cars 1/  = 2 minutes  =
1 1 0 5 arrivals
0.5 i l per minute
i
E[ X ]  , Var [ X ] 
 2
 Note that the cdf of the exponential distribution is given by
• Therefore,
 12 (1)
Pr( X  1)  1  e  0.3935
x x
Pr( X  x)   f (t )dt   e t dt 1  e x
0 0
29 30

Probability Distributions Beta –distribution (PERT Analysis)

5. Beta Distribution
 A random variable x is said to have a beta distribution if its density Most Likely
y
function is given by;  x p 1 (1 - x) q 1
f ( x)   if 0  x  1, p, q  0,
  ( p, q )
or 0 if x  0 and x  1

p
mean 
pq Optimistic Pessimistic
p ( p  1)
var iance 
 p  q  p  q  1
Time
0 a m b
31 32

8
 Probability Distributions
6. Gamma Distribution
 A random variable x is said to have a ggamma distribution if it
assumes only non-negative values . The pdf is given by;
 ( x) n 1 e x
f ( x)  , if 0  x  
(n  1)!
 f(x) = 0 otherwise f(x)
Pdf if n=1 and m =0.5
 m and n are non-negative
g 0.5

 If n=1 pdf will be exponential

33 0 2 4 6 8 10