You are on page 1of 41

# Topic: Probability

KNF2053
Numerical Methods and Statistics

## Ade Syaheda Wani Binti Marzuki

Faculty of Engineering
Semester 2 (2010/2011)

Probability

## The ratio of the number of occurrences of an

event to the number of all possible occurrences
of equally likely and mutually exclusive events.

## An idealization of the proportion of times that

an event would be expected to occur in
repeated trials.

Definition of Probability

## If an experiment can result in any one of N

different equally likely outcomes, and if
exactly n of these outcomes correspond to
event A, then the probability of event A is:
n
P(A)
N

## Example: Getting a head when a coin is tossed

once.
1
2

Random Experiment

## An experiment that satisfies the following

conditions
a) The outcome of experiment cannot be
predicted with certainty.
b) The experiment can be repeated under
the same condition.

E.

Sample space, S

## The set of all possible outcomes of a

statistical experiment.

## S S, the sample space S itself is an

event, called the certain event.

## S , is called the null event.

Sample space, S
Example 1
Experiment: E :Tossing a die
Sample Space: S = {1,2,3,4,5,6}
Event:

## A = {getting an even number}

P (A) = 3/6 = 1/2

Sample space, S
Simple exercise
A dice is tossed once.
a) What is the sample space, S?
b) Give the set of the following events
i.

ii.

## the number shows on top is more than 4

Sample space, S
Simple exercise
A coin is tossed twice
a) What is the sample space, S?
b) Give the set of the following events:
i.

ii.

Points

## All sample point (event ) probabilities

must lie between 0 and 1.

## The probabilities of all the sample points

within a sample space must sum to 1.

1 1
2 2

## Relations between event A and

B
S

Include
A

B include A is denoted as B A.
All the elements of A is also
belonging to B.

Equal
A equal to B is denoted A = B. It
means A B and B A. A
happen if and only if B happen.

S
A=B

## Relations between event A and

B
S

Union
Denoted by the symbol A B,
is the event containing all the
elements that belong to A and
B or both.

Intersection
Denoted by the symbol A B,
is the event containing all
elements that common to A and
B

## Relations between event A and

B
S

Mutually exclusive or
disjoint
A and B are mutually exclusive,
or disjoint if A and B cannot
happen at the same time
(A B = )

Complement
The complement of an event A
with respect to S is the subset
of all elements of S that are not

A
B

S
A'
A

Axioms of Probability

P () = 0

P (A) = 1 P(A)

## If A and B are mutually exclusive, then:

P ( A B) P ( A) P( B )

## In many cases, events will not be mutually

exclusive
P ( A B) P( A) P( B ) P ( A B)

Example 1
The probability that John passes Mathematics is 2/3,
and the probability he passes English is 3/9. If the
probability of passing both subjects is 1/4, what is
the probability that John will pass at least one of
these subjects?
Let M : the event passing Math
Let E : the event passing English

Example 1-Solution
The probability that John will pass at least one of
these subjects.
Let M : the event passing Math
Let E : the event passing English

P( M E ) P( M ) P( E ) P( M E )
2 3 1

3 9 4
3

## According to multiplication principle,

If the first sampling can be performed in m ways.
Second sampling can be performed in n ways

ways

mn

## Lets look at an examples of counting sample with

replacement and without replacement

## Sampling with replacement

Consider this case!
How many ways are to choose 3 balls with
replacement from a box, which contains 5 distinct
balls?
First attempt

## Second attempt :5 ways to choose second ball

Third attempt :5 ways to choose third ball
Total ways to choose the ball = 5 5 5 = 125

## Sampling without replacement

Consider this case!
How many ways are to choose 3 balls without
replacement from a box, which contains 5 distinct
balls?
First attempt

## Second attempt :4 ways to choose second ball

Third attempt :3 ways to choose third ball
Total ways to choose the ball = 5 4 3 = 60

Permutation

## Given n distinct objects, we wish to choose r of

these objects 0 < r < n and arrange the chosen r
objects.

## The number of permutations of size r that can be

constructed from n objects denotes by nPr
n

n!
Pr n: no. of object
(n r )!
r: no. of ways

Permutation

n

n!
Pr
n: no.
n ofr!object
r: no. of ways

n

Pr n

Example 2

## In a coded telegram the alphabet letters are

arranged in groups of five letters. What is the
arrangement of such words.
n

Pr 265
11881376

n

n!
26!
Pr

7893600
(n r )! (26 5)!

Combination

Denotes by nCr

## For example, taking a combination of letter a, b, c

and two letter taken at a time.

Without repetition:
3 combinations ab, ac and bc

With repetitions:
6 combinations aa, ab , ac, bb, bc and cc

Combination

n

## Combination with repetitions

n

n
n!
Cr
r r!(n r )!

n r 1
(n r 1)!

Cr
r r!((n r 1) r )!

## Every combination has r! of ways to rearrange

them (with regard to order)
n

Pr nCr r!

Example 3

## Consider a case of taking letters a, b, c from a

box. a) Find the combination of taking 2 letter from
a box
i) Without repetitions
ii) With repetitions:
b)How many ways to arrange the combinations in
both cases

Example 3 - Solution
a)

n

n
3!
Cr
3
r 2!(3 2)!

## ii) With repetitions:

n

n r 1
4!

Cr
6
r 2!(4 2)!

Example 3 - Solution
b)

n

Pr nCr r!
3 2!
6

## ii) With repetitions:

n

Pr nCr r!
6 2!
12

Conditional Probability

## One occurrence event can increase the chances

of occurrence of another event

Example:
A : rain next Sunday
B : rain next Saturday

## The occurrence of B increase the chance of

occurrence A.

Conditional Probability

## For any two events, A and B with P(B) > 0. The

conditional probability of A given B has occurred
is defined by:

P( A B)
P( A | B)
P( B)

Example 4

## Consider a case of tossing a dice.

Let event A = {1,2,3} and event B = {1,3,5}.
Find P (A|B)

Example 4 - Solution

event A = {1,2,3}

event B = {1,3,5}
P( A B)
P( B)
A B {1,3}
2 1
P( A B)
6 3
3 1
P( B)
6 2
P( A | B)

1
P( A B)
2
3
P( A | B)

1
P( B)
3
2

Multiplicative Rule

## The probability of an intersection of two events

can be calculated using multiplicative rule, which
employs the conditional probabilities we defined
in the previous section.

## The multiplicative rule of probability is

P ( A B ) P ( A) P ( B | A)
or equivalently

P( A B) P( B) P( A | B)

Independent Events

## Two events A and B are independent if and only

if:

P( A B) P( A) P( B)

Remarks:
When P(B) > 0, A and B are independent
P(A|B) = P(A)
When P(A) = 0 or P(B) = 0, A and B are
independent

Example 5
Lets consider when a coin is tossed twice. Let A
= head appears in the first time and B = Tail
appears in the second time. Show that A and B
are independent events.

Example 5 - Solution
S = {(H,H), (H,T), (T,H), (T,T)}
A = head appears in the first time = {(H,H),
(H,T)}
B = Tail appears in the second time = {(H,T),
(T,T)}
P(B) > 0, so:
P( A | B)

P( A B)
1
4

2
P( B)
2
4

Bayes Rule

## Let A and B be two events,

S
A

We may express A as
A B'

A B

A ( A B) ( A B ' )

## As AB and AB are clearly mutually exclusive,

then

P ( A) P( A B) P( A B' )
P( A | B) P( B) P( A | B' ) P( B' )

Bayes Rule
Definition:

## A sequence of event A1, A2,, An represents a

partition of the sample space S if
a) Ai Aj = , for every 1 < i j < n
b) Total union of Ai = S
c) P(Ai ) > 0 for all i = 1,2,, n

S
A3
A1

A2

A4

A5

Probability

## Let A1, A2, An be a partition of the sample space

S. Then for any other event B associate with S
n

P( B) P( B Ai )
i 1
n

P( B | Ai )P( Ai )

S
B

A3

i 1

A2
A1

A4

A5

Bayes Rule

## Let A1, A2, An be a partition of the sample space

S. Then for any other event B associate with S for
which P(B)>0

P( Ai B )
P( B | Ai ) P( Ai )
P( Ai | B )
n
P( B)
P( B | Ai ) P( Ai )
i 1

For i = 1, 2, , n

Example 6
In a factory, production line A,B and C are all
producing boxes with the same size. On their
production, production line A, B and C produce 1%,
2% and 3% defective boxes, respectively. Of the
total production, line A produces 25%, line B
produces 35% and line C produces 40%. A box is
selected randomly from the total production of a day
a) what is the probability that it is defective?
b) if the box is defective, what is the conditional
probability that it was produced by line C?

Example 6 - Solution
Let D = defective box
A = box from line A
B = box from line B
C = box from line C
a)

P ( D) P( D A) P ( D B) P ( D C )
P( D | A) P( A) P( D | B ) P( B ) P ( D | C ) P (C )

1 25 2 35 3 40

## 100 100 100 100 100 100

215
43

10000 2000

Example 6 - Solution
b) By Bayes Theorem

P(C ) P ( D | C )
P (C | D)
P( D)
40
3
100 100

43
2000
24

43