Mathematical Preliminaries

Mathematical Preliminaries
Topics
• Set Theory

• Combinatorics

• Mathematical Functions

• Summations

• Probability Theory
Set Theory
 Set Theory
Definitions
ƒ A Set is a collection of objects called members. If a, b, c, d are members of a set S we write :
S ={a, b, c, d}

where a, b, c, d ∈ S ( read a, b, c, d belong to S )

ƒ The set builder notation can be used to define a set. For example, the set
S={1, 3, 5, 7, 11, 13}
can be defined as
S = {x: where x is prime number and x<15}

ƒ In set notation the order of members does not matter

S = {a, b, c, d} is identical to set {d, b, c, a}

ƒ In a set all members are distinct objects , i. e, duplicate members are not allowed

ƒ A set that contains no elements is called empty set. It is symbolically denoted by φ

 Set Theory
Subsets
ƒA set A is called a subset of set B if every element of A is also a member B. Symbolically:

A ⊆ B, if x ∈A then x ∈B
ƒ The set A is proper subset of B, if A contains only some of the elements of B. Symbolically
the relationship is represented as:

A ⊂ B
ƒ The relationship among sets is usually shown by a picture, which is called Venn diagram.
Figures (a), (b) show Venn diagrams for a subset and proper subset

B
A=B
A

A ⊂ B A ⊆ B
(a) A is proper subset of B (b) A is subset of B
 Set Theory
Universal Set
ƒ The largest set for a given collection of objects is called Universal Set. All other sets are
subsets of the Universal set U. Symbolically:
φ ⊆ U, A ⊆ U, B⊆ U, C ⊆ U

Example: The Universal set, U, of all small English alphabets is

U = {a, b, c,…..,x, y, z}
The subsets of the Universal set are, for example, A = {a, f, g}, B = {p, q, r, s , t},
C = {x ,y, w}, as shown in the diagram

A B
C

Subsets of Universal set of alphabets (a, b, c,…x, y ,z}

ƒ Given a Universal set, we define complement Ac of set A as

Ac = U – A
The set Ac contains all of elements of Universal set, excluding those in the set A.
Example: If A={a, b, c}, then Ac = { d, e, f,……, x, y, z}
 Set Operations
Union
ƒ The union of set A and set B is set of all elements x such that x is in A or x is in B.
Symbolically:
A U B, if x ∈ A or x ∈ B
ƒ The Union operation is pictorially represented by Venn diagram, as shown below:

A B A U B
Union set
Example (1): A={ 1, 3, 4, 5}, B={ 3, 4, 7, 8, 9}. The Union of sets A and B is the
set { 1, 3, 4, 5, 7,8, 9}.
Example(2): Union operation can be used to add elements to a set. Consider sets {a, b}, {c, d}
(i) Setting S to empty set φ
S=φ
(ii) To insert elements of first set into S, we perform union of S with {a, b}
S = S U {a, b} = {a, b}
(iii) To insert elements of second set {b, c} into S, we perform union of S with {c, d}
S = S U {c, d} = {a, b, c, d}
 Set Operations
Intersection
ƒ The intersection of set A and set B is set of all elements x, such that x is in A and
x is in B. Symbolically:

A I B, if x ∈A and x ∈B
ƒ The intersection operation is pictorially represented by Venn Diagram, as shown below.

A B A I B
Set intersection

Example: Let A={1,2, 3, 4, 5} and B={2, 4, 7, 8} then

A I B = {2, 4}

ƒTwo sets are called disjoint, if their intersection is empty set. Thus, if A and B are disjoint,

A I B=φ
 Set Operations
Difference Set
ƒ The difference of set A and set B is set of all elements x, such that x is not in B if x is in A.
Symbolically:
B – A , if x ∈A then x ∉B
ƒ The difference operation is illustrated by Venn Diagram, as shown below.

A B B-A
Difference set

Example(1): If A={1,2,3,4,5} and B={4, 6, 8, 10} then A –B = {1, 2, 3, 5}

Example(2): We can use difference operation to delete elements from a set. Consider
S = {a, b, c, d, e, f }
(i) To delete element a we perform difference operation on S and set {a}.
S = S – {a} = { b, c, d, e, f }

(ii) To delete element d we perform difference operation on set S and set {d}.
S = S-{d} = { b, c, e, f }
 Set Theory
Cardinality
The cardinality of a set is the number of elements in the set. It is also called size. The
cardinality of set S is denoted by |S| .

Example: If S = {a, b, c, d, e}, then | S | = 5

¾ For any sets A and B, | A UB | = | A | + | B | - | A ∩ B |

Consider, for example, the sets A, B
Let A = {a, b, c, d, e, f}, |A|=6

Let B ={d, e, f, g, h, i, j ,k}, |B|=8

A U B = {a, b, c, d, e, f, g, h, i, j, k}, | A U B | = 11

A ∩ B = {d, e, f}, |A∩B|=3

It follows that | A | + | B | - | A ∩ B | = 6 + 8 – 3 = 11
 Set Theory
Cartesian Product
The Cartesian product of two sets A and B, denoted by A X B, is a set of all
ordered pairs such that the first element of the pair is in set A and the second element
is in set B. It is also called cross product. Mathematically,

A X B = { (a , b) : a ∈A and b∈ B }

Example: Let A = {a, b, c} and B = {g, h}, then

A X B = {(a, g), (a, h), (b, g), (b, h), (c, g), (c, h) }

¾ The cardinality of Cartesian product is the product of cardinalities of sets

| A X B | = | A | .| B |
 Set Theory
Binary Relation
ƒ A binary relation R on sets A and B is a subset of the Cartesian product A X B of the sets .
In set notation, the relation is represented as:

R ⊆ AXB

ƒ Since a cross product set can have one or more subsets there can be many binary relations

Example: Consider the sets A = {a, b, c}, B = {x, y}

Their cross product is set A X B = {(a, x), (a, y), (b, x), (b, y), (c, x), (c, y) }

(1) A binary relation R1 is

R1 = { (a, y), (b, y), (c, y) }

(2) Another binary relation R2 is

R2 = {(a, x), (a, y) }

 Binary Relation
Example
A graph G is a pair (V, E ), where V is a finite set, called vertex set, and E is a binary
relation on V, called edge set

Example: Consider the graph G = (V, E) , and V = { a, b, c }, vertex set for the graph
A binary relation is

E = {(a, a), (a, b), (a, c), (b, c), (c, b) }

The set E represents edges of graph G. The graph is shown pictorially in the diagram below.
Combinatorics
 Permutations
Definition
ƒ A permutation is an arrangement of n objects in some order, such that each object appears
exactly once.

ƒ Elements of a set of n objects can be arranged in the following ways

n.(n-1).(n-2)…..3.2.1
because, the first place can be filled in n ways, and the second place in n-1 ways. Thus, first
and second places together can be filled in n(n-1) ways. The first, second and third places can
be filled in n.(n-1).(n-2) ways, and so on

ƒ The product n(n-1)(n-2)….3.2.1 is called n-factorial, which is denoted as n!

Example: The elements of Set S={ a, b, c } can be permuted in six ways because
3! = 3.2.1 = 6
The six arrangements are: abc, acb, bac, bca, cab, cba

¾ It is assumed that 0! = 1 (factorial of zero is one ). The factorial of a negative integer is

not defined
 Factorial Function
Stirling‘s Approximation
ƒ The n! factorial increases rapidly with the increase of n. The table below shows the
growth.

n n!
2 2

4 24

8 40,320

16 20,922,789,888,000

32 2.6 x 10 35

ƒ It can be shown that 1000! is an integer consisting of 2,500 digits.

¾ The analysis of algorithms sometimes involves large inputs. For large values of n, the
n! is approximated by the following formula,:

n! ≈ √ ( 2 π n ) ( n / e) n , where e = 2.718

¾The above formula is referred to as Stirling’s approximation

 Combinations
Definition
ƒThe arrangement of n objects taken k ( k ≤ n ) at a time, without regard to the order,
is called k-combination.

Example: The objects a, b, c, d have following 2-combinations:

ab, ac, ad, bc, bd, cd

Observe that the combination ab ≡ ba i.e, order does not matter

ƒ The number of k-combinations of n objects is denoted by the symbol

Another notation is C(n ,k).

¾ It can be shown that :

C(n,k) =

¾ The k-combinations are called binomial coefficients because they occur in

binomial theorem for the expansion of the expression:
Mathematical Functions
 Mathematical Functions
Use in Analysis
ƒ Several mathematical functions are used to analyze algorithms and study their behavior.
We briefly examine the properties of some these functions From analysis perspective, the
following functions are particularly useful:
i. y=└x ┘ (Floor function)

ii. y=┌x ┐ (Ceil function )

iii. y = log a x (Logarithm function)

iv. y = x (Linear function )

v. y = x2 (Quadratic function)

vi. y = x3 (Cubic function)

vii. y = 2x (Exponential function with base 2)

viii. y =nn (Exponential)

ƒ When a mathematical function is used to represent the running time of an algorithm, the
function is referred to as growth function. The behavior (growth rate) of a function can be
represented pictorially by plotting a graph of (x,y) points given by the functional relation
 Floor Function
Definition
ƒ The floor function y(x) is defined as:

y= └x ┘
where x is a real number and y is the largest integer which is smaller than or equal to
number x.

Example: └ 4.9┘= 4,

└ 4.0 ┘ = 4
ƒ It follows from the definition : x -1 <└ x ┘ ≤ x

¾ The floor function is used for mapping of a real valued function into integer function
 Ceil Function
Definition

ƒ The ceil function y(x) is defined as:

y= ┌x ┐

where x is real number and y is the smallest integer larger than or equal to x.

Example: ┌ 4.9 ┐= 5

┌ 4.0 ┐ = 4

ƒ It follows from the definition : x ≤ ┌ x ┐ < x +1

¾ The ceil function is used for mapping a real valued functions into integer function
 Logarithm Function
Definition
ƒ The logarithm function y= log a x is the inverse of exponential function ay, where a is some
positive constant.

ƒ Thus, if y = log a x ,
then
x=ay
Example: Consider, y= log 5 25 . Since 25= 52, it follows y=2 .

ƒ The figure shows a plot of logarithm function to the base 2

¾ The logarithm function will frequently arise in the analysis of algorithms

 Logarithm Function
Bases
ƒ The choice of base depends on the nature of application and data type.

¾ The base 10 is normally used in computations involving decimal numbers. Here are some
examples
log 10 1 =0 because 1= 10 0 by definition
log 10 10 =1 10= 10 1
log 10 100 = 2 100=10 2
log 10 1000000 = 6 1000000=10 6
¾ The base 2 is often used in the analysis of algorithms. Some examples of base 2 are
log 2 1=0 because 1= 20 by definition
log 2 2= 1 2=21
log 2 16= 4 16= 24
log 2 64= 6 64= 26
The logarithm to base 2 are referred to as binary logarithm. A special notation lg is used
to represent a binary algorithm, as follows
lg (n) ≡ log 2 (n)
¾ The base e ≈ 2.718 is used in calculus for purposes of differentiation and integration It is
called natural logarithm. A special notation ln is used to represent natural logarithms.
ln (n) ≡ log e (n)
 Logarithm Function
Properties
The following properties of logarithms are often used in several applications
• log b x . y = logb x + logb y (Sum rule)

• logb x / y = logb x – logb y (Difference rule)

• logb x n = n logb x (Exponentiation rule)

• x logb y = y logb x ( Symmetry rule )

¾ The last property is particularly useful for interchanging logarithmic exponential and
base of exponentiation. Here are some examples.

2log 2 n = n log 2 2 = n (Interchanging 2 and n)

8log 2 n = n log 2 8 = n3 (Interchanging 8 and n)

3log 2 n = n log 2 3 = n1.585 ( Interchanging 3 and n and using lg 3 = 1.585)

 Logarithm Function
Change of Base
ƒ Sometimes it is necessary to change the base of a logarithm. The formula for the
transformation from base a to b is as follows:

loga x = logb x / logb a

Example : lg (binary logarithm) can be converted to ln (natural logarithm):

lg x = log2 x

= log e x / loge2

= ln x / ln 2 ( ln 2 = 0.6931)
 Mathematical Functions
Growth Rates
The table shows the growth rate of common mathematical functions, which are useful in the
analysis of algorithms

n lg n √n n lg n n2 n3 2n n! nn
2 1 1.4 2 4 8 4 2 2

4 2 2 8 16 64 16 24 256

8 3 2.8 24 64 512 256 40,320 16,777,216

16 4 4 64 256 4,096 65,536 20,922,789,888,000 1.845 x 1019

32 5 5.7 160 1,024 32,768 4,294,967,296 2.631 x 10 35 1.461 x 10 48

64 6 8 384 4,096 262,144 1.8 x 10 19 1.269 x 10 89 3.940 x 10 115

128 7 11 896 16,384 2,097,152 3.4 x 10 38 3.856 x 10 215

5.283 x 10 269

¾ The logarithm lg n has the lowest growth rate , and the exponential function n n
the highest
¾ The relationship of functions in terms of their growth rates is symbolically represented as
lg n < √n < n < n lgn < n2 < n3 < 2n < n! < n n
 Growth Functions
Graph
The plot of mathematical functions lg n, n, n lg n n2, n3, 2n is shown below.

f(n)

¾ Because of extremely rapid growth of n! and nn , these functions are not shown in the
ranges shown in the graph scale
Summations
 Summation
Arithmetic
ƒ Thenarithmetic summation consist of sum natural numbers (1, 2, 3….n)
∑k
k =1
= 1 + 2 + 3+………..+ n = n ( n+1 ) / 2

¾ The sum can be simply evaluated, as under.

Let S be sum of first n terms of the series.

Therefore,
S = 1+ 2 + 3 +……………… + (n-2)…+(n-1)+ n [ n terms] ………(1)

Writing the summation in reverse order

S = n + (n-1)+(n-2)+……………+3 + 2 + 1 [n terms] ….… .(2)

Adding (1) and (2)

2S = (n+1)+(n+1)+(n+1)+…….(n+1) + (n+1) + (n+1) [ n terms]…… …...(3)
= n(n+1)

Thus, S = n ( n + 1) / 2

ƒ Another summation sometimes used in analysis is sum of squares of natural numbers. It has
the sum:
S= 12 + 22 + 32+ ……+n2 = n(n+1)(2n+1)/6
 Summation
Geometric
ƒ The geometric summation consists of sum of powers of some fixed number.
The fixed number is referred to as geometric ratio. Hers is a common form of exponential
summation:
n

∑r
k
= r 0 + r 1+ r 2 + ……..+ r n-1 +r n = ( rn+1 - 1) / (r - 1)
k =0
where r is the geometric ratio.
¾ The summation is evaluated as follows. Let S be the sum of first n terms of the series. Then,
S = r 0 + r 1 + r 2+r3 ……..+ r n-1 + r n ………….(1)
Multiplying both sides of (1) with r
r .S = r 1 + r 2+r 3+ … +rn-1 + rn +r n+1 …………..(2)
Subtracting (1) from (2)
(r-1).S = rn+1-r0
= rn+1 -1
Or S = (rn+1-1) / ( r -1) …………(3)

Example(1) : Let r = 2, the sum of geometric series is as under

20 + 21+ 22+…..+2n =2n+1 - 1

Example (2): Let r = 2/3, the sum is as under

(2/3)0 + (2/3)1 + ……+(2/3)n = ( (2/3)n+1 -1)/( 2/3 - 1 ) = 3 – 3(2/3)n+1
 Summation
Arithogeometric
ƒ Another useful series, referred to as arithogeometric . It consists of sum of the product of
natural numbers with exponentials of a constant. :
k =n

∑ km
k
= 1.m1 + 2.m2 + 3.m3+ …….+ n.mn
k =0

where m is some constant

ƒ The following summation of arithogeomatric series, with base 2, arises in the analysis of
some algorithms.
n k
∑ k.2 = 1.21 + 2.22 + 3.23+ …….+ n.2n = (n-1) 2n+1 + 2
k =0
 Summation
Harmonic
ƒ The harmonic summation consists of reciprocals of natural numbers (1,2,…..n).
n

∑1 / k
k =1
= 1/1 + 1/2 + 1/3 +…….+1/n ≈ ln n + γ + 1 / 2n - 1 / 12n

γ =0.57721 is called Euler’s constant

¾ The Harmonic series does not have an exact formula for the sum . The approximate
sum is obtained by converting the summation into an integral

¾ By using mathematical analysis it can be shown that upper and lower bounds of
harmonic series are given by the relation
n

ln(n+1) ≤ ∑ 1 / k ≤ 1 + ln(n)
k =1
 Summation
Logarithmic
ƒ The logarithmic summation includes logarithms of natural numbers (1,2,…,n ). Assuming
base 2 for the logarithm, the series is expressed as follows:
n

∑ lg(k )
k =1
= lg(1) + lg(2) + lg(3)+……..+lg(n) . = lg(1.2.3..n) =lg( n! )

ƒ An estimate for the above summation is obtained, by using the Stirling ‘s approximation
for large factorial. Since n! ≈ √ ( 2 π n ) ( n / e) n where e = 2.718, it
follows that
lg n! ≈ n lg( n) ( ignoring lower order terms and constants)

ƒ Therefore, the approximate summation for logarithm series for large n is given by :

lg(1) + lg(2) + lg(3)+……..+lg(n) ≈ n lg( n )

Probability Theory
 Sample Space
Definition
ƒ The Probability Theory is concerned with the study of experiments whose outcome is not
known in advance ( a priori). However, all likely outcomes are assumed to be finite and
known before the conduct of the experiment

ƒ For example, tossing of a coin is an experiment. The outcome of tossing cannot be

predicted in advance, but outcome is likely be Head or Tail

The set of all possible outcomes of an experiment are called sample space or population..
Let e1, e2, e3, …… en are the probable n outcomes of an experiment. Then the sample space
is the set Ω = { e1, e2, e3, …… en }
 Sample Space
Examples
Example (1): A fair coin is tossed. The result can be Head( H ) or Tail( T ). The sample
space for the tossing experiment is:
Ω = {H, T}.

Example (2) : A fair coin is tossed twice ( or two coins are tossed simultaneously) the sample
space for the toss is:
Ω = { HH, HT, TH, TT}

Example (3): A six-faced die is thrown. The sample space for the event is
Ω = {1, 2, 3, 4, 5, 6 }
. where digits represent number of dots on each face of the die.

Example(4): Two six-faced dice are tossed. The sample space consists of 36 pairs of
integers:
Ω={(1,1),(1,2)(1,3),(1,4),(1,5), (1,6), (2,1),(2,2)(2,3,),(2,4),(2,5), (2,6),
(3,1),(3,2)(3,3),(3,4),(3,5), (3,6), (4,1),(4,2)(4,3,),(4,4),(4,5), (4,6),
(5,1),(5,2)(5,3),(5,4),(5,5), (5,6), (6,1),(6,2)(6,3),(6,4),(6,5), (6,6) }

Example(5): The sample space for weather forecast can be

Ω = (fair, rainy, cloudy, partly cloudy, hailstorm, snow}
 Events Space
Definition
ƒ A subset E of the sample space Ω is called event space . Symbolically ,

E ⊆ Ω

¾ A six-faced die is thrown. The sample space Ω consists of all numbers that are likely to
be shown i. e,
Ω ={1, 2, 3, 4, 5, 6 }

Example(1): The event space that an even number is shown:

E1={2, 4, 6}

Example(2): The event space that an of odd number is shown:

E2=(1, 3, 5}

Example(3) : The event space that a number greater than 4 is shown

E3={ 5, 6}

ƒ An event set consisting of a single element is called elementary event.

ƒ The empty set φ is referred to as impossible event

 Probabilities
Definition
ƒ Let E be an event space consisting of events e1, e2, e3…..en. Each of the events in the is
assigned a real value (number) denoted by P(e1), P(e2) , P(e3)…..P(en), called
probabilities, such that

P(e1) + P(e2) + P(e3)+……….+ P(en) = 1

ƒ The function P(e) is called probability function.

¾ The probabilities are generally assigned on the basis of experience or judgment. Some
times the probabilities are expressed as percentages, or in terms frequencies of happening of
different events.

Example(1): Chance that a flight will be on time is 90% and would be late is 10% . Thus,
P(on time)= 90/100=0.9 and P(late)=10/100=0.1

Example(2): Based on experience, we say that chance of finding a document on Yahoo is twice that of
finding on MSN, and four times on Google. If p is probability of finding the document on MSN then,
we have :P (msn ) = p ; P(yahoo ) = 2p ; P(google) ) = 4p
By probability axiom , p + 2p + 4p =1 or p=1/7
Thus, we have P(msn)= 1 / 7, P(yahoo)= 2 / 7, and P(google)=4 / 7
 Probability Theory
Equiprobable Events
ƒ In several cases the probabilities are not known, but it is expected that all events are equally
likely to occur . Therefore, all the elementary events are assigned equal probabilities.

ƒ Let E be event space for equiprobable events

E ={e1, e2, …….en}

Then, P( e1 ) = P( e2 ) = P(e3 ) =…….= P( en ) = p , say, where p is some constant

Since by definition

P(e1)+P(e2)+P(e3)+…+P(en)=1

it follows that

p +p + p+…..+ p ( n terms ) = 1. i.e n.p= 1 or p = 1/n

Thus, for equiprobable n possible events, P(ej) = 1 / n ( 1 ≤ j ≤ n)

¾ The function P(n) is called uniform probability function.

 Equiprobable Events
Examples
Example (1) In tossing a fair coin heads and tails have equal chance. The event set is {H, T}.,
where H refers to occurrence of head and T to occurrence of tail. Therefore,
P(H)=1/2;
P(T)=1/2

Example (2) When six-faced die is thrown, all numbers, 1 to 6, have equal chance of being
shown. Therefore, each event has probability 1/6. Thus, probabilities of showing different
numbers are
. P(1)=1/6, P(2)=1/6, P(3)=1/6, P(4)=1/6, P(5)=1/6, P(6)=1/6

Example(3): An array stores n random keys. Assuming that a search key has equal chance
of being in any of the array cells, the probability of finding the key in a given cell is 1/n

Example(4): A six-faced die is tossed. The event set that a odd number is shown consists of
the elements 1, 3, 5. The probability that an odd number would be displayed is 3/6=1/2
 Probability Theory
Law of Addition
Two events A, B are said to be mutually exclusive provided that if A occurs then B cannot
occur; conversely, if B occurs then A cannot occur. For example, if a six-faced die is tossed
then showing of 1 and 6 are mutually exclusive events because if 1 is shown then 6 will not be
shown and vice versa.
ƒ According to Law of Addition of Probabilities, If A and B are two mutually exclusive
events with probabilities P(A) and P(B) ,then probability that A occurs OR B occurs is given
by the sum of probabilities P(A) + P(B). Simply stated ,

P(A OR B)= P(A)+ P(B)

ƒIn general, if A1, A2, A3…Ak are mutually exclusive events then

P(A1 OR A2 OR A3…..OR Ak)=P(A1)+ P(A2)+P(A3)+….+P(Ak)

 Law of Addition
Examples
Example (1) : A six-faced die is tossed. What is the probability that an even number would
be show.?
The likely even numbers are 2,4,6. Each has the probability of 1/6. Thus, according law
of addition
P( Even Number)= P(2 OR 4 OR 6)= 1/6 + 1/6+ 1/6 =1/2

Example(2): A die is thrown. What is probability of showing a prime number or a number

greater than 5 ?
(1) Events set that a prime number would be shown is A= {2,3,5}. Each event has the
probability of 1/6. Since these are mutually exclusive events, probability that a prime
number would be shown is given by
P( Prime Number) =P( 2 OR 3 OR 5) = 1/6+ 1/6+ 1/6 =1/2
(2)Events set that a number than greater 5 would be shown is B={6} . The Probability that
a number greater than 5 would be shown is
P( Number greater than 5) =1/6
(3) Combining the above results, the probability that outcome would be a prime number or a
number greater than 5 is
P(Prime Number OR Number greater than 5)=P(Prime Number) + P(Number greater than 5)
=1/2 + 1/6
= 2/3
 Probability Theory
Law of Multiplication
ƒ Two events A , B are said to be stochastically independent if occurrence of A is not
influenced by the occurrence of B. Conversely, occurrence of B is not influenced by
occurrence of A.

ƒ For example, if a coin is tossed several times the result of previous toss does affect the
result of current toss, and result of current toss would not influence the result of next toss.
We can say that the events of successive tossing are stochastically independent events.

ƒ According to Law of Multiplication, if A and B are stochastically independent events then

the probability of occurrence A AND occurrence of B is given by the product of the
probabilities P(A).P(B). Mathematically,

P( A AND B) = P(A).P(B)

ƒ In general, if A1, A2, A3, .Ak are stochastically independent events then

P( A1 AND A2 AND A3… AND Ak ) = P(A1) .P(A2 ).P(A3 ). P(Ak )

 Law of Multiplication
Examples
Example (1): A coin is tossed three times. What is the probability of occurrence of three heads?
The three tosses are independent events.
In each of three tosses the probability of showing a head is 1/2. Thus,
After the first toss, P( Head )= 1/2
After the second toss, P( Head AND Head)= P( Head )..P( Head ) =1/2 . 1/2= 1/4
After the third toss , P( Head AND Head AND Head) = 1/2 . 1/2 . 1/2 = 1/8

Example(2): A six faced die is tossed three times. What is probability that three sixes would
be shown or three fives would be shown consecutively.
In each toss the probability of showing 5 or 6 is 1/6.
(1)According multiplication law, the probability of showing three consecutive sixes is
P(6 AND 6 AND 6) =(1/6).(1/6).(1/6)= 1/216

(2) Again according to law of multiplication, the probability of showing three fives
P(5 AND 5 AND 5) =(1/6) .(1/6). (1/6)=1/216

(3) According to law of addition , the probability that 6 is shown three times OR 5 is shown
three times, is obtained by adding the above probabilities
P( (6 AND 6 AND 6) OR ( 5 AND 5 AND 5 ) )= P ( 6 AND 6 AND 6) + P(5 AND 5 AND 5)
=1/216 + 1/216 = 1/108
 Probability Theory
Expected Cost
ƒ Let c1, c2, …..cn be the costs associated with events e1, e2, …en . Let the probabilities for
the occurrence of events be P(e1), P(e2)…..P(en) , then the expected cost (EC) is defined as

EC = c1.P( e1) + c2.P( e2 )……+cn.P (en)

Example: Suppose a coin is tossed 2 times. Assume that the costs associated with events
are :
if two heads appear cost is 2
if one head appears cost is 1
if no heads appear (equivalently only tails show) cost is 0
• The event space S for the experiment is Ω = { HH, HT, TH, TT}
Probability that event HH occurs = 1/2.1/2= 1/4
Probability that event HT occurs =1/2.1/2=1/4
Probability that event TH occurs = 1/2.1/2=1/4 By law of multiplication
Probability that event TT occurs =1/2.1/2=1/4

• The four events have associated costs of 2, 1,1, 0 with probabilities 1/4, 1/4. 1/4, 1/4.
Thus, the expected cost of the tossing experiment = 2.(1/4) + 1.(1/4) + 1.(1/4) + 0.(1/4)
= 1/ 2 + 1/ 4 + 1/ 4 = 1
 Searching an Array
Expected Cost
ƒ Consider the problem of performing linear search on an array of size n .

ƒ It is assumed that the search key has equal probability of being found in any
of the n array cells. Thus, the probability that key is in any of the n cells is 1/n

Cell A[1] A[2] A[3] - A[n-1] A[n]

Probability 1/n 1/n 1/n - 1/n 1/n

ƒ Assuming that unit cost of searching one cell of the array is c, the probabilities and the
costs associated with search operation up to each cell are summarized in the table

Search Key location Probability Cost of Searching

First cell 1/n c
Second cell 1/n 2c
Third cell 1/n 3c
- -
nth cell 1/n nc

ƒ Thus, expected cost of search is

Texpected(n) =1c.1/n + 2.c.1/n+…….+(n-1).c.1/n + n.c.1/n
= [1+2+3+….+n ].c./ n
= c. (n+1)/2 (Using arithmetic summation)