Hawassa University

Institute of Technology
Electrical and Computer Engineering
Department
Course Title: Probability and Random Processes
Course Code: ECEg-3201

Lecture No.1
Chapter 1: Basic Concepts of Probability Theory

Oct. 15/2013
 1.1. Introduction
 Is everything on this planet determined by randomness?

 This question is open to philosophical debate. What is certain

is that every day thousands and thousands of engineers,
scientists, business persons, manufacturers, and others are
using tools from probability and statistics.
 What is probability?
 Probability is the study of randomness and uncertainty.
Probability allows us to quantify the chance that a certain
event will occur.
Example “There is a 30% chance of rain tomorrow?”, “Our
chance of winning the lottery is1 in 41,416,353”.
 Probability is based on the idea of a random experiment that is
an experiment where the outcome cannot be predicted with
absolute certainty before it is run.
 Probability models in electrical engineering
 Need to build systems that work in a chaotic environment
o Communication systems that must provide continuous
and error free communication.
o Computer system that must satisfy the diverse
processing demands.
 Models as tools in analysis and design
o Help the marking of choices from various feasible
alternatives.
o Actual implementation and experimental evaluation of
the alternatives are too costly.
o Models can be used to make decisions.
o Probability models enable the designers to successfully
build systems that are efficient, reliable, and cost
effective.
 Model
o An approximate representation of a physical system.
o Attempts to explain observed behavior using a set of simple
and understandable rules.
 Mathematical models
o Used when the observational phenomenon has measurable
properties.
o Consists of a set of assumption about how a system works.
o Stated in the form of mathematical relations involving
parameters and variable of the system.
 Computer simulation model
o Consists of computer program that simulates the system.
o A means of checking the predictions made by a
mathematical model.
  Types of Models

 Deterministic model
o The conditions under which an experiment is carried out
determine the exact outcome of the experiment (the outcome
can be predicted exactly in advance)
o The solution of a set of mathematical equations specifies the
exact Outcome of the experiment e.g. Force = mass x
acceleration. If we are given values for mass and acceleration,
we exactly know the value of force

 Probability model
o For systems involving unpredictable variation and randomness.
An experiment in which the outcome varies in an Unpredictable
fashion when the experiment is repeated under the same
condition
 1.2. Sample Space and Events

A. Random Experiments
 In the study of probability, any process of observation is referred
to as an experiment
 The results of an observation are called the outcomes of the
experiment. An experiment is called a random experiment if its
outcome cannot be predicted.
 Typical examples of a random experiment are the roll of a die,
the toss of a coin, drawing a card from a deck, or selecting a
message signal for transmission from several messages.
B. The Sample Space
 The set of all possible outcomes of a random experiment is called
the sample space (or universal set), and it is denoted by S.
 The sample space S can be specified compactly by using set
notation. It can be visualized by drawing tables, diagrams,
intervals of the real line, or regions of the plane.
 Two basic ways to specify a set:

o List all the elements, separated by commas, inside a pair of

braces. For example:

o Give a property that specifies the elements of the set

 Pick a number X at random between zero and one, and then

pick a number Y at random between zero and X.
EXAMPLE
1. A manager supervises the operation of three power plants, at any
given time; each of the three plants can be classified as either
generating electricity (1) or being idle (0).

2. If two dice are rolled (or, equivalently, if one die is rolled twice),
derive the sample space
 Probabilities

 In some situations, notably games of chance, the experiments are

conducted in such a way that all of the possible outcomes can be
considered to be equally likely, so that they must be assigned
identical probability values.
 n outcomes in the sample space that are equally likely => each
probability value be 1/n. 9
 C. Events and Complements
 An event A is a subset of the sample space S. It collects
outcomes of particular interest. The probability of an event is
obtained by summing the probabilities of the outcomes contained
within the event A.
 An event is said to occur if one of the outcomes contained within
the event occurs
The complement of an event is the opposite event.
both coins come out heads E1 = { HH }

both coins do not come out heads E1c = { HT, TH, TT }

Ec S
E
 Intersections of Events

P( A  B)  P( A  B)  P( A)
P( A  B)  P( A  B)  P( B)

The intersection of events happens when all events happen.

 Mutually Exclusive Events
A B  
 Two events A and B are said to be mutually exclusive if
so that they have no outcomes in common.
 A B  A
A B  B  A
A A  A
A S  A
A  
A  A  
Fig.1
A  ( B  C )  ( A  B)  C

 Unions of Events

The event A  B is the union of events A and B and consists of the

outcomes that are contained within at least one of the events A and B. The
probability of this event, P( A  B), is the probability that at least one of the
events A and B occurs.

 The union of events happens when at least one of the

events happens.
P( A  B)  P( A  B)  P( A  B)  P( A  B)
 Simple results concerning the unions of events (Refer Fig.1)

Example :Tossing of two coins

(a) first coin comes out heads E2 = { HH, HT }
(b) both coins come out same E3 = { HH, TT }
 E2 E3 = { HH }
both (a) and (b) happen

S
E1 E2

Example :Tossing of two coins

(a) first coin comes out heads E2 = { HH, HT }

(b) both coins come out same E3 = { HH, TT }
at least one happens E2 ∪ E3 = { HH, HT, TT }

S
E2 E3
 Probability for finite spaces

The probability of an event is the sum of the probabilities of its

elements
S = { HH, HT, TH, TT }
Example ¼ ¼ ¼ ¼
both coins come out heads E1 = { HH } P(E1) = ¼
both coins come out same E3 = { HH, TT } P(E ) = ½
3
first coin comes out heads E2 = { HH, HT } P(E2) = ½
Example
A company that manufactures television sets performs a final quality check on
each appliance before packing and shipping it. The quality check has an
evaluation of the quality of the picture and the appearance. Each of the two
evaluations is graded as Perfect (P), Good (G), Satisfactory (S), or Fail (F).

Determine the probability of an appliance that fails on either of the two evaluations
and that score an evaluation of Satisfactory on both accounts will not be shipped.
An appliance that fails on either of the two evaluations and that score an
evaluation of Satisfactory on both accounts will not be shipped.
A = { an appliance cannot be shipped }
= { (F,P), (F,G), (F,S), (F,F), (P,F), (G,F), (S,F), (S,S) }

P(A) = 0.074

About 7.4% of the television

sets will fail the quality check.
B = { picture satisfactory or fail }

P(B) = 0.178 P(B) = 0.178

A B = { Not shipped and the picture satisfactory or fail }

P( A  B)  0.055
 A B = { the appliance was either not shipped or the picture
was evaluated as being either Satisfactory or Fail }

P( A  B)  0.197

Exercise
= { Television sets that have a picture evaluation of either
P( A  B ') Perfect or Good but that cannot be shipped }
• Union of Three Events
The probability of the union of three events A, B, and C is the sum of the
probability values of the simple outcomes that are contained within at
least one of the three events. It can also be calculated from the expression

P( A  B  C )   P( A)  P ( B )  P (C ) 
  P( A  B)  P( A  C )  P( B  C ) 
 P( A  B  C )
1.3. Axioms and Properties of Probability

 A finite probability space is specified by:

A finite sample space S.

 Every event E over S is given a number P(E) called the
probability of E such that

1. for every e, 0 ≤ p(e) ≤ 1 e s

2. P(s) = 1 (sum of probabilities of all events in set s) s

3. If EF = ∅ (disjoint events) then S

E F
P(E∪F) = P(E) + P(F)
 Rules for calculating probability

Complement rule: Ec S
E
P(Ec) = 1 – P(E)
Difference rule: If E ⊆ F S
E F
P(F and not E) = P(F) – P(E)
in particular, P(E) ≤ P(F)

Inclusion-exclusion: S
P(E∪F) = P(E) + P(F) – P(EF) E F
 Exercise
In some town 10% of the people are rich, 5% are famous, and 3%
are rich and famous. For a random resident of the town what is the
chance that:
(a) The person is not rich?
(b) The person is rich but not famous?
(c) The person is either rich or famous (but not both)?
Solution