Lecture 3

probability and Random Processes

By Tafari Lemma

由NordriDesign提供
www.nordridesign.com
Lecture Outline
• Random Variables
• Type of random variable
• Discrete random variable
• Probability Distributions
• Mean and Variance
• Independence of Random Variables
• Specific Discrete Distributions
• Continuous Random Variable
• Specific Continuous Distributions

2
 Random Variables
Often it is useful to consider random variables. These map the
outcomes of a random experiment to a number.
A random variable X is a function from the sample space to the
real numbers.

Example:
•X is the random variable corresponding to the temperature of
the room at time t.
• x is the measured temperature of the room at time t.

3
Type of Random Variables

4
Discrete Random Variables

Random variable X is a discrete random variable, if its

range is countable.
The range of a random variable X, shown by Range(X) or
Rx, is the set of possible values of X
Many situations can be adequately modeled using discrete
random variables:

• Number of failed hard-drives in a system

• Number of customers attempting to access a webserver

5
Probability Distributions
The probability distribution of a random variable X is a
description of the probabilities associated with each possible
outcome of X.

For a discrete random variable X this is actually quite simple, as

X can only take a finite/countable number of possible values.

Definition: Probability Mass Function (p.m.f.)

6
 Example
You toss a fair coin twice, and let X be defined as the number of heads
you observe. Find the range of X, RX, as well as its probability mass
function .
Here, our sample space is given by
S={HH,HT,TH,TT}

The number of heads will be 0, 1 or 2. Thus RX={0,1,2}.

PX(k)=P(X=k) for k=0,1,2.

We have
PX(0)=P(X=0)=P(TT)=1/4,
PX(1)=P(X=1)=P({HT,TH})=1/4+1/4=1/2,
PX(2)=P(X=2)=P(HH)=1/4.
7
 Example
Consider a situation in which a user can make one of three possible
requests to a GUI. We can view this as a random experiment with
the following sample space:

We can identify each request with a number. For instance identify Print,
Save and Cancel with 0,1, and 2, respectively. Let X be the random variable
corresponding to the random experiment above.

It’s very easy to check this is a valid probability mass function.

8
A valid probability mass function

9
Cumulative Distribution Function
Properties:

IMPORTANT: note that the c.d.f. is defined for any real

number, and not only for the values X can take !!!

10
 C.D.F. Example

Let’s plot the p.m.f. and c.d.f.:

1.0 1.0
0.9 0.9
0.8 0.8
0.7 0.7
0.6 0.6
0.5 0.5
0.4 0.4
0.3 0.3
0.2 0.2
0.1 0.1

-1 0 1 2 3 4 5 -1 0 1 2 3 4 5 11

11
C.D.F. Example
1.0
0.9
0.8
0.7
0.6
0.5

0.4
0.3
0.2
0.1

-1 0 1 2 3 4 5

12
 Example of CDF
The CDF of the random variable X is given by

13
 Cont’d
a. The graph of the CDF is as shown in below

14
 Mean and Variance
There are certain “summaries” of the distribution of a random
variable that can give a lot of information about it. These have
also easy intuitive interpretations.
Definition: Mean and Variance (discrete r.v.)

15
 Mean and Variance
The mean is a weighted average of the possible values of X,
where the weights are their corresponding probabilities.
Therefore the mean describes the “center” of the distribution.
In other words X takes values “around” its mean…

The variance measures the dispersion of X around the mean. If

this value is large means X varies a lot.

Definition: Standard deviation

14

16
 Example

The number of surface defects on the CD-RW of a certain

manufacturer is well modeled by a random variable X with the
following distribution:

17
Example

18
Properties of the Mean and Variance

Properties:

WARNING!!!

19
 Properties of Independent R.V.s

Definition:

This result will be very useful to us, since many common

distributions can be described as the sum of independent
random variables!!

20
Specific Discrete Distributions

In what follows we are going to study and learn about some

specific discrete distributions, namely:

• Discrete Uniform Distribution

• Bernoulli Distribution

• Binomial Distribution

• Poisson Distribution

All these are very important in a number of scenarios, fitting

many practical situations one might encounter.

21
Discrete Uniform Distribution
Definition: Discrete Uniform Distribution

Example:

German tanks had serial numbers ranging from 1 to N. Every

time a tank was seen/captured a note was made on its serial
number. This information, together with the assumption that
the serial number from the tank captured was an independent
sample from a discrete uniform distribution was sufficient to
accurately estimate N.

22
Discrete Uniform Distribution

23
 Dices

A regular dice has six sides, with respectively 1,

2, 3, 4, 5 and 6 dots. If the dice is fair then
outcome of rolling the dice is given by a uniform
random variable:

24
Bernoulli Random Variables
Definition: Bernoulli Random Variable

Examples:

• Flipping a “fair” coin: X ~ Ber(1/2)

• Success starting Windows: X ~ Ber(0.95)

25
 Binomial Distribution
Definition: Binomial Random Variable

26
 Binomial Distribution

The computation of the mean and variance of X can also be

done directly from the p.m.f., but it is rather tedious and
complicated… But there is a much easier way.

27
Binomial Distribution

28
Binomial Distribution Example
Samples from a certain water supply have a 10% chance of
containing an organic pollutant. Suppose you collect 20 samples
over time, and these are taking in such a way that you can
assume these are independent. Let X be the number of
contaminated samples.

29
 Poisson Process
Assumptions:

• The customers behave independently from one another

•The expected number of customers per unit time is a constant
(e.g. =12 customers per day, on average)
Let’s break the day into 4 hour intervals…

If the intervals were REALLY small the number of customers

there are either 0 or 1 with very high probability.

30
 Poisson Distribution
Definition: Poisson random variable

We can easily check that this is a valid p.m.f…

31
 Properties

The mean and variance are easily calculated from the p.m.f:

32
 Example
There are many situations where the Poisson distribution
seems to be a good fit.

Example: When manufacturing silicon wafers for IC there is

always the risk of contamination by small particles in the
atmosphere. There are on average two contaminating particles
in an area of 100 cm2, and this contamination process is well
described by a Poisson process.

Let X denote the number of particles in a circular wafer with

radius of 5cm. Calculate the probability that there are more
than 4 contaminating particles in the wafer…

33
Example

34
Continuous Random Variable

A continuous random variable can assume the infinitely many

values corresponding to points on a line interval.

Examples:
• Heights
• Birth weights
• Blood pressure
• Length of life of a medicine

35
Probability Density Function
Probability distribution describes how the probabilities are
distributed over all possible values. A probability distribution for a
continuous random variable x is specified by a mathematical
function denoted by f(x) which is called the density function.

Technical remark: note that the cumulative distribution

function might not be differentiable at all points.

36
Probability Density Functions

Definition: Probability Density Function (p.d.f.)

A valid p.d.f. must always satisfy (i) and (ii) !

37
Probability Density Functions
Probability Density Function Probability Mass Function
(continuous random variables) (discrete random variables)

There is a strong parallel between discrete and

continuous random variables
38
Example
Let the random variable X denote the time it takes (in days) for a
hard drive to fail. We have reason to believe that this random
variable has the following density

39
Cumulative Distribution Function
We already saw that we can consider also different descriptions
of a random variable, namely we defined in the first lecture the

Definition: Cumulative Distribution Function (c.d.f.)

This seemingly simple object is extremely powerful, as all the

information contained in the probability density function is also
contained in the c.d.f.

This is EXACTLY the same definition we had for discrete

random variables!!!

40
Cumulative Distribution Function

All the information in the density is also contained in the

cumulative distribution function (also referred to simply as the11
distribution function)
41
C.D.F. Example

42
C.D.F. Example

43
 Mean and Variance
As we did for discrete random variables, we can consider also
certain summaries of the distributions

Definition: Mean and Variance (continuous r.v.)

44
 Mean and Variance
The mean is a weighted average of the possible values of X,
where the weights are their corresponding probabilities.
Therefore the mean describes the “center” of the distribution.
In other words X takes values “around” its mean.

The variance measures the dispersion of X around the mean. If

this value is large means X varies a lot.

Definition: Standard deviation

16

45
Example
Suppose you want to model the clock frequency (in GHz) of a
certain mobile device processor. This is a random quantity, and
in some cases it can be well modeled by a continuous random
variable with density

Let’s study this random variable in more detail…

46
 Example
Let’s first check this is a valid probability density function:

47
Example

48
Example
Suppose you replace the old processor with a better one, with
the same average speed, but much more steady.

49
Example

50
Properties of the Mean and Variance

WARNING!!!

51
Specific Continuous Distributions

In what follows we are going to study and learn about some

specific continuous distributions, namely:

• Continuous Uniform Distribution

• Normal Distribution
• Exponential Distribution
• Erlang Distribution
• Weibull Distribution

All these are very important in a number of scenarios, fitting

many practical situations one might encounter.

52
Continuous Uniform Distribution
Definition: Continuous Uniform Distribution

This is exactly the type of distribution we saw on our last

example. It is extremely simple, which makes it attractive
in many situations.

53
Countinuous Uniform Distribution

54
 Normal Distribution
Remarkably the sum of many independent random variables
almost always looks similar to a bell curve !!! No matter what
these random variables look like themselves.

This bell curve is known as the normal distribution.

Definition: Normal Density Function

30

55
 Why is the Normal Distribution Important?

The result relating the sum of many independent random variables and
the normal distribution is called the Central Limit Theorem. It is a very
important result that we will state later on. For now it is important to
note that normal distributions can be used to model many different
things:

•Total time it takes to download a file (the time is the sum of the time
is takes to receive each packet)

•Average grade of a test (this is the sum of all the grades, divided
by the number of tests)

•Noise level in a phone line (the noise is the superposition of many

interferences).

56
Example

57
Basic Properties

58
 Standard Normal Distribution
Unfortunately it is not possible to calculate in closed form
the areas under the normal density :(. One must resort to
either numerical evaluation of the integral, or the use of
tables – however, we need only one table for a very specific
normal distribution.
Definition: Standard Normal Density

36

59
Standardizing a Normal R. Variable

So, if we have access to the cumulative distribution function of

the standard normal random variable, we can do computations37
with any other normal random variable!
60
Standardizing a Normal R. Variable
We essentially just proved the following:

We can now use this to compute probabilities associated with X

using the standard normal table at the end of the book, or in
the statistical compendium…

61
 A Practical Example
In a communication system each bit to be transmitted is
converted into an electrical signal, that is send through a
copper wire. This transmitted signal is then converted into a
potential (in volt). Let X be a random variable denoting this
value:

62
Normal Distribution table

63
A Practical Example

64
A Practical Example

65
 A Practical Example

Modification: Actually, the system only decodes a bit if the

signal has a magnitude greater than 0.08, regardless of the
sign.

66
 Exponential Distribution
When we discussed discrete random variables we encountered
the Poisson or geometric random variables, which had an
interesting memoryless property. In the continuous setting
we also have a similar random variable.

Definition: Exponential Random Variable

67
 Properties

The facts above are easily checked, and left as an exercise. The
lack of memory property is rather interesting. If X denotes the
time until a certain event happens:

•Given that we already waited t1 for the event implies that the
probability that we must wait t2 more time does not depend on
the value of t1.

68
 Applicability of the Exponential Dist.

Because of it’s simplicity and important properties, the

exponential distribution is used to model many important
situations:

•The time until a device fails (if the device doesn’t wear out,
as it is the case in some electronic components)

• The time in between consecutive calls to a call center

• The time between the arrival of packets at a router

69
 Example
The duration of IC555 integrated circuits under normal
operation conditions is well modeled by an exponential
distribution with mean 2000 days.

What is the probability such an IC lasts more than 10 years?

70
 Example
The duration of IC555 integrated circuits under normal
operation conditions is well modeled by an exponential
distribution with mean 2000 days.

What is the probability such an IC lasts more than 10 years?

You have a specific IC555 that has already lasted 3 years.

What is the probability that it will last another 10 years?

71
 Erlang Distribution

Definition: Erlang Distribution

72
 Gamma Distribution
The Erlang distribution can be generalized for the case r is not an integer
(although our interpretation of this random variable is no longer based on
the Poisson process). For this we need to generalize the factorial function

We can use this to generalize the Erlang distribution in a very

natural way.

73
 Gamma Distribution
Definition: Gamma Distribution

74
 Example
For record keeping purposes, each electronic transaction made
in a ATM machine much be stored recorded in a paper roll. The
transactions are well modeled by a Poisson process, with an
average of 25 transactions per hour.

Suppose you install a new roll. Let X denote the time (in
minutes) until three transactions are registered. What is the
distribution of X ?

You installed a new roll at 12:00. What is the probability that

the third transaction happens before 12:10?

75
 Weibull Distribution
If we want to model the time until failure of a part that is
subject to wear over time the exponential distribution is no
longer a suitable candidate, due to the lack-of-memory
property. In that case the Weibull distribution is much more
reasonable:

Definition: Weibull Distribution

63

76
Properties

77
 Example
The lifetime of bottom bracket (dichte trapas) from a certain bicycle
manufacturer is well modeled by a Weibull distribution with parameters
=2 and =5000.

• What’s the mean time until failure?

•What’s the probability that bottom bracket lasts at least 6000 hours?
• What’s the probability it lasts 3000 hours
•What’s the probability that is lasts 3000 hours more given that it
already lasted 3000 hours?

78
 Example

•What’s the probability that bottom bracket lasts at least

6000 hours?
• What’s the probability it lasts more than 3000 hours
•What’s the probability that is lasts 6000 hours given that it
already lasted 3000 hours?

79
Thank You !!!