Discrete Maths by Ashish C

DISCRETE MATHEMATICS - IIITB
Prof. Ashish Choudhury

Computer Science and Engineering
IIIT Bangalore-IISc
INDEX
S.NO TOPICS PAGE.NO
Week 1
1 Introduction to Mathematical Logic 5
2 Logical Equivalence 19
3 SAT Problem 29
4 Rules of Inference 42
5 Resolution 57
6 Tutorial 1: Part I 70
7 Tutorial 1: Part II 86
Week 2
8 Predicate Logic 99
9 Rules of Inferences in Predicate Logic 112
10 Proof Strategies I 133
11 Proof Strategies II 144
12 Induction 157
Week 3
15 Sets 199
16 Relations 214
1
17 Operations on Relations 234
18 Transitive Closure of Relations 247
19 Warshall’s Algorithm for Computing Transitive Closure 263
20 Tutorial 3 273
Week 4
21 Equivalence Relation 289
22 Equivalence Relations and Partitions 299
23 Partial Ordering 308
24 Functions 328
Week 5
27 Countable and Uncountable Sets 363
28 Examples of Countably Infinite Sets 376
29 Cantor’s Diagonalization Argument 395
30 Uncomputable Functions 413
31 Tutorial 5 421
Week 6
32 Basic Rules of Counting 435
33 Permutation and Combination 452
34 Counting Using Recurrence Equations 472
35 Solving Linear Homogeneous Recurrence Equations – Part I 489
36 Solving Linear Homogeneous Recurrence Equations – Part II 503
2
Week 7
39 Solving Linear Non-Homogeneous Recurrence Equations 547
40 Catalan Numbers 562
41 Catalan Numbers – Derivation of Closed Form Formula 575
42 Counting Using Principle of Inclusion-Exclusion 590
43 Tutorial 7 606
Week 8
44 Graph Theory Basics 617
45 Matching 632
46 Proof of Hall’s Marriage Theorem 647
47 Various Operations on Graphs 659
48 Vertex and Edge Connectivity 675
49 Tutorial 8 689
Week 9
50 Euler Path and Euler Circuit 703
51 Hamiltonian Circuit 721
52 Vertex and Edge Coloring 733
Week 10
55 Modular Arithmetic 778
3
56 Prime Numbers and GCD 790
57 Properties of GCD and Bézout’s Theorem 805
58 Linear Congruence Equations and Chinese Remainder Theorem 821
59 Uniqueness Proof of the CRT 834
Fermat’s Little Theorem, Primality Testing and Carmichael 847

60
Numbers
Week 11
61 Group Theory 861
62 Cyclic Groups 874
63 Subgroups 886
64 More Applications of Groups 903
65 Discrete Logarithm and Cryptographic Applications 918
Week 12
66 Rings, Fields and Polynomials 937
67 Polynomials Over Fields and Properties 951
68 Finite Fields and Properties I 971
69 Finite Fields and Properties II 982
70 Primitive Element of a Finite Field 994
71 Applications of Finite Fields 1010
72 Goodbye and Farewell 1027
4
Discrete Mathematics
Department of Mathematics and Statistics
International Institute of Information Technology, Bangalore
Lecture -01
Introduction to Mathematical Logic
Hello everyone, welcome to this lecture on introduction to mathematical logic.

(Refer Slide Time: 00:24)
The plan for this lecture is as follows. In this lecture we will discuss mathematical logic and
applications. We will discuss propositional logic and we will discuss compound propositions and
logical operators.
5
So to begin with let us ask the question what is logic or more precisely what is mathematical
logic. So on a very high level, mathematical logic is the science of reasoning. Namely it tells you
how to verify or how to conclude whether a statement or mathematical statement is true or
mathematical statement is false. To be more precise consider this theorem statement, which is a
very straight forward theorem statement, which says that, for every n ≥ 0 the summation of first
n(n+1)
n numbers is .
2
So this is the mathematical statement and now we want to verify whether this statement is true or
false. Well, it turns out that statement is true because we have something called what we call as
proof and there are several proofs for proving this statement to be true, so one of the simplest
proofs is the proof by mathematical induction. So the proof by mathematical induction basically
argues that the statement is obviously true for n = 1.
And then we show that if you assume that the statement is true for any arbitrary n = k then we
also end up showing that the statement is true for n = k + 1 as well. And these two statements
constitute a proof and based on the above two points we conclude that our theorem statement that
we made here is true. Now the question here is on what basis we can say that these two
statements that we have written here in this highlighted rectangular box constitutes a correct
proof.
6
How do we argue that indeed this is a correct proof? So on a very high level mathematical logic
or the science of reasoning is what helps us to conclude that two statements that we have written
here indeed constitute a valid proof for proving that this statement is correct.
It turns out that there are several applications of logic, So for example, we use it extensively in
Boolean algebra, which forms the basis of computer architecture. We also use it for program
verification and validation. So what I mean by program verification, here is the following.
Suppose we have written a software for particular application. How do we verify whether our
software is performing its intended task?
One way of verifying that is we run the program or the software for various inputs and verify
whether it is giving the correct output or not and if it is giving the correct output for various
inputs that we have tried then we believe that the program is indeed performing its intended task.
But in this process there might be a possibility that there is some bad input which we never tried,
for which the program may not be performing or may not be giving you the right output. So this
way of verifying the program is not the right approach specifically for softwares or programs
which are meant for very critical applications. So for example, if your software is developed for
an Aeroplane application then we require the software to be completely robust. It should be
foolproof. It should perform these actions always correctly.
7
So there we use mathematical logic to verify whether the software is indeed performing its right
task or not. So there is a huge body of work, a discipline of computer science, which we call as
program verification and validation where the goal is to verify whether the given software or the
program is performing its correct intended application or not. So there also we use mathematical
logic.
In the similar way we use mathematical logic for hardware verification. So for instance, if you
have a motherboard and there are several chips or various units which are embedded there which
are very small in size. So how do you verify whether the embedding that you have done on your
motherboard is correct or whether it is performing the right task or not. So again we use
mathematical logic to do hardware verification. Mathematical logic is very useful in artificial
intelligence.
In fact there is a programming language, which is called as PROLOG which is used extensively
in AI applications and the basis for PROLOG is mathematical logic and we also use
mathematical logic for proving theorem in the form of automatic theorem proving., where in
automated theorem proving we write computer programs to prove whether a statement is true or
false, so there also we use mathematical logic extensively. So it turns out that mathematical logic
is very important, very significant.
8
And there are various types of mathematical logic, the basic form of mathematical logic is called
as propositional logic and a generalization of propositional logic is called as predicate logic.
So let us begin our discussion on propositional logic. So we first define what is a proposition. So
informally, proposition is a declarative statement which is either true or false. But it cannot take
both the values simultaneously that means it has to be either true or it has to be either false. So
for instance consider the statement that New Delhi is the capital of India. It is a declarative
statement because it is declaring something about the city called New Delhi. It is declaring that
the city New Delhi is capital of India or not.
And indeed this statement can be either true or false because if you take the city New Delhi then
either it will be the capital of India or it will not be the capital of India, but it cannot happen that
it is simultaneously capital of India as well as not the capital of India. So, that is why it is a
proposition. In the same way if I make a statement like Bahubali was killed by Katappa or then
this is a declarative statement because it is declaring something about a character called Bahubali
and Katappa and the statement is indeed true. Because we now have approved whoever has seen
the movie Bahubali part 2, we now have the witness that indeed in that movie the character
Katappa killed Bahubali. So this is also a declarative statement which can be either true or false
and hence it is a proposition. Whereas the statement X+ 2 is equal to 4 this is not a proposition.
Well, it is a declarative statement because it is declaring something about X, 2 and 4.
9
But we do not know what is the value of X. Depending upon the value of X this statement can be
either true or it can be either false. So it can simultaneously take values true as well as false. So
hence, it is not a proposition. Now let us next define what we call as propositional variables and
these propositional variables are typically denoted by lower cap letters. So for instance p, q, r etc.
So, what is the propositional variable? It is a variable which represents an arbitrary proposition.
That means it is a placeholder to store an arbitrary proposition and the truth value of this
propositional variable depends upon the exact proposition which we assign to these variables p,
q, r. So for instance in your programming language, say for instance, in your C programming
language, we define variables like integer variables.
So if I make a statement declaration, like int x that means I am declaring here that x is an integer
variable and this variable can store any integer value. It can store values 1, 2, 3 any integer value.
In the same way a propositional variable it is a placeholder or an arbitrary proposition and
depending upon what exact proposition we assign to that variable the variable can take the truth
value either true or false.
Now let us next define compound propositions. So a compound proposition is a larger

proposition or a bigger proposition, which is obtained by combining many propositions using
10
what we call as logical operators. So the simplest form of the logical operator is the ¬ operator,
which is an unary operator because it operates on a single variable and the truth table or the truth
assignment for this negation operator is as follows.
So imagine p is a variable propositional variable and this propositional variable can take two
values either true or false. So T here stands for true and F stands for here false. So this negation
operator is defined as follows. If the variable p is true then the ¬ of p will be false whereas if the
variable p is false then the ¬ of that variable will be true. That is how this unary operator
negation is defined.
So we will be using this notation for denoting the negation operator. In various other books there
are different notations which are used for the same negation operator. In some books, they use
this tilde symbol ~p to denote negation of p. In some books they also use this p complement that
means we put this bar on top of p ( 𝑝 ). So there are various notations for the same negation
operator, but we will be extensively using the notation that is there in this table (¬𝑝).
Now we define another logical operator which is called as the conjunction and it is also called as
AND, logical AND. We denote this operator by this notation ( ˄ ) again in some books they used
a notation p dot q. But I will be using this notation p and this symbol q and this is a binary
operator because it operates on two propositional variables. So if your p is a variable and q is a
variable then p conjunction q is another propositional variable.
And a truth value of p conjunction q is defined as follows; p conjunction q is defined to be true

only when both the variable p as well as the variable q are simultaneously true. If any of them is
false then the conjunction of p and q is defined to be false. The next logical operator is the
disjunction operator which is also called as OR operator denoted by this notation ( ⋁ ) and it is
defined as follows.
If any of the variables p or q is true then the disjunction of p and q is defined to be true. But if
both p and q is false then disjunction is defined to be false. And it turns out that there are plenty
of other logical operators that we can define on propositional variables. By the way why I am
11
calling them as logical operator? Because it is operating on propositional variables, that is why I
am calling it as logical operator.
And the result of any logical operator will be either true or false because we are in two valued
logic where the propositional variable of the result of applying the logical operator can take only
one of the two possible values. Now here is the question. How many distinct logical operators
are possible operating on two propositional variables? That means suppose I have a propositional
variable p and a propositional variable q.
My question is how many distinct types of logical operators I can define on these two
propositional variables. So you can think in your mind and try to come up with an answer.
Let us try to derive the answer for the question. So to begin with you should understand that each
logical operator has a distinct truth table. That means it does not matter what is the name of the
logical operator that you give. You can call it conjunction, disjunction, x, y, z, alpha, beta,
gamma. You can call it by any name. Once you fix the name of a logical operator operating on
variables p and q then it has a distinct truth table. And what do I mean by a distinct truth table?
Well, since we are trying to define a logical operator on two variables and the variables p and q
can take these four combinations. p can be true q can be true p to be true q to be false p to be
12
false q to be true and p to be false q to be false. So we have these four possible combinations and
here o is denoting an arbitrary propositional variable. My goal is to identify how many different
types of arbitrary logical operator o I can define here.
That is my question here. And the question mark here below is what will be the truth value of p
operation q for the various truth assignments of p and q. That means here the first question mark
says what can be the possible value of p operation q if both p and q takes the truth value true.
The second row here denotes what is the value of this operation operator p operation q if p takes
the value true q takes the value false and so on.
It turns out that since we are dealing with mathematical logic where each variable can take only
two possible values then each of this possible question marks can be either true or false that
means my operation o can be such that for p to be true and q to be true the result of p operation q
can be either true or it can be either false. In the same way my operation o can be such that for p
to be true and q to be false the operation p operation q may result in true or false.
And this holds for all the four possible rows. It means that if I consider the number of distinct
truth tables here, I can construct at most in fact exactly 16 distinct truth tables because for each
of the possible rows here for each of the possible question marks here I have two possible
options and each of them is independent of each other. Each of them are independent. So that is
why I have to tell 16 number of distinct truth tables are possible.
Sorry for this rendering issue here. This parenthesis should come down here. So since each row
should take the value true or false that is how I get a total number of distinct truth tables to be 16.
And what I have done here is I have written down the 16 distinct truth tables which can be
possible. I am calling those distinct 16 distinct truth tables by various operators. So operator 1
denotes a truth table where for p to be true and q to be true the result is true. For p to be true q to
be false the result is true for p to be false q to be true the result is true and for p to be false q to be
false the result is true. That is one possible truth table and I am saying that that corresponding
operator is operator O1. In the same way, I have operator O2, O3, O4 and operator O6. And now
I can give fancy names to each of these operators. I can give some name conjunction,
13
disjunction, XOR, exclusive OR, exclusive NOR etc etc.
But I cannot give more than 16 distinct names because I do not have more than 16 possible truth
tables. So that is why there are 16 possible logical operators distinct logical operators, which are
possible with two propositional variables.
Now let us define another logical operator, which is the conditional statement. This is also called
as if then statement and we use this notation p → q and the truth table for p implies q is defined
as this. So you can see that p implies q is true for three possible combinations and it is defined to
be false only when p is true, but q is false. Now, the question is why the truth table of p → q is
defined like this?
That means why p → q is true, even if both p and q are false or why p → q is defined to be true
even if p is false, but q is true. So to understand that let me give an example, a very simple
example. Suppose we make this logical statement. Suppose there is a PM candidate called John
and he makes the election promise that if he becomes the prime minister then good days will
come for the country.
This is a pre-election promise which John makes. Now the question is when exactly will you
consider this logical statement to be false? It turns out that this logical statement will be
14
considered as a false statement only when good days does not come even if John becomes the
PM. For all other cases the statement will be considered as a true statement; that means if John
does not become the PM at the first place then I do not care whether good days come or not.
Overall, the statement will be considered as a true statement because John is not breaking his
promise. We will be saying that John is breaking his promise only when John becomes PM but
still good days are not coming for the country and that corresponds to the truth assignment p to
be true and q to be false and as a result for that combination I define p implies q to be false.
There are other interpretations of this if then statement.
So the usual interpretation of p implies q is this if p then q where there are other interpretations
like q follows from p. So q follows from p means if you ensure p to be true we can denote we
can come to the conclusion q. p is sufficient for q tells that if you ensure that p is happening or p
is true then the conclusion is q, right? q whenever p is another way of interpreting if p can then q.
Now, the question is why p implies q denotes q is necessary for p and why p implies q denotes p
only if q.
So these are the two common forms of p implies q, which is a common source of confusion for
the students.
15
So, let us try to understand that why p implies q or why if p then q can also be interpreted as q is
necessary for p and why it can be interpreted as p only if q. So to understand that let me first
claim here that the statement p → q and the statement ¬q →¬p are both logically equivalent.
What I mean by both are logically equivalent is that you have the truth table of p → q and this is
the truth table of ¬q → ¬p.
And you can see that both of them have the same truth table. That means both p → q as well as
¬q → ¬p have the same truth table. So if p is true, q is true then p implies q is true and same is as
the case for ¬q → ¬p. That means row wise the first row here and the first row here are the same.
Row wise the second row here and here are same. Row wise the third row of the two tables are
the same and row wise the fourth rows of both the tables are the same.
And that is why I can say that p → q and ¬q → ¬p are both logically equivalent. Now consider
this statement. I make the statement my statement is I will go to pub only if it is a Friday and say
the statement I will go to the pub is represented by p variable p. So p is a propositional variable
denoting the statement I will go to the pub and q is another propositional variable denoting the
statement it is a Friday. My claim is that this statement can be represented by p → q. Why so?
Because if you try to understand the logical meaning of this English statement, then what I am
trying to say here is that if it is not a Friday definitely I will not go to the pub. Because that is
how the statement “only if it” is interpreted and no Friday implies no pub can be represented by
¬q → ¬p because I am representing it is a Friday by the variable q. So it is not a Friday will be
represented by ¬q and I will not go to the pub will be represented by ¬p because p as per my
definition denotes I will go to the pub.
So in that sense, Friday is a necessity or it is a necessary condition for thinking of going to the
pub. That means I will think about going to the pub only if it is Friday. If it is not a Friday
definitely I would not go to the pub. So that is why Friday is a necessary condition here. But
Friday is not a sufficient condition for me going to the pub. Because it might be possible that
there is some Friday on which I am ill or I may have some personal work for which I am not
going to the pub.
16
That means in this case being Friday is not a sufficient condition; that means you cannot simply
conclude that since it is a Friday definitely this guy will go to the pub and that is why this
statement I will go to the pub only if it is a Friday will be represented by p → q. It would not be
represented by q → p because q → p denotes that if it is Friday then I will go to the pub, but that
is not what I want to represent here.
I want to state here the necessity of the condition that it is a Friday then only I can think of going
to the pub and that is why the statements of the form that q is necessary for p or the statements of
the form p only if q are represented by p → q.
So whenever we write a conditional statement of form, if then, p → q, then the p statement or

whatever statements you have for the propositional variable p they are called as hypothesis or
antecedent or premise and q denotes the conclusion or the consequence. It turns out that in
mathematical logic the statements in p and in q need not be related in the English sense. So, for
example, if I make a logical statement that if the earth is triangular in shape then Dr. Ashish
namely myself is the director of IIIT Bangalore.
So in English Language, this is an absurd statement. This is a completely incorrect statement.

But in mathematical logic this is a correct statement because earth is definitely not in triangular
17
shape. So this is false. And remember for p → q I do not care what is the q part. As soon as my p
part is false I can conclude that p → q is an overall true statement. That means even though the
conclusion here namely Dr. Ashish is the director of IIIT Bangalore, which is actually a false
conclusion in English, it does not matter whether it is true or false since my premise is false here
namely since our earth is not triangular in shape, the overall statement is a true statement
logically true statement and that is why in mathematical logic the statement p and the statement q
may not have any relation with each other. They are just logical statements.
So that brings me to the end of this lecture. To conclude in this lecture we started our discussion
on mathematical logic. We saw various applications of mathematical logic. We started our
discussion on propositional logic. We defined what is a proposition? We defined compound
propositions and how compound propositions can be formed from simple propositions using
logical operators. We discussed various logical operators, like disjunction, conjunction, negation,
if then statement and so on. Thank you.
18
Lecture -02
Logical Equivalence
Hello everyone. Welcome to this lecture on logical equivalence. So, just a quick recap. In the last
lecture we discussed about propositional logic, various logical operators.
And how do we form compound propositions from simple propositions using logical operators.
In this lecture, we will discuss about logical equivalence and logical identities.
19
So, remember if p then q is represented by p → q and truth table of p → q is this. Then the
converse of p → q is denoted by q → p and it is easy to see that the truth table of q → p or the
converse is this. The inverse of p → q is denoted by ¬ p → ¬ q and its truth table will be like this
and the contrapositive which is very important for p → q will be the statement ¬ q → ¬ p.
And if you see closely, the truth tables of the converse of p → q and the truth table of p → q,
they are not same. If you see the truth table of p → q and inverse of p → q are also not same. But
if you see the truth table of p → q and its contrapositive they are same; that means the first row
of both the tables are same. The second rows of both the tables are same. The third rows of both
the tables are same and same and same as fourth row.
And that is why I can say that p → q and negation q → negation p are the same statements, they
are logically equivalent.
20
We will come back to that point later but let me first define a bi conditional operator or a bi
conditional statement which for which we use this notation ↔ that means an arrowhead which
has an arrowhead at both ends. And this bi conditional statement is used to represent statements
of the form p if and only if q or in short form p if and only if q says another way another form of
representing if and only if is iff.
So very often for mathematical and for various theorem statements, you must have seen
conditions like prove that this is true if and only if this holds right? So wherever we are making
statements of that form, we are actually making statements of the form p bi-implication q.
Another equivalent form of this bi conditional statement is the conjunction of p implies q and q
implies p.
So you can see that row wise, the first row of both the tables are same, the second row of both
the tables are same. The third row of both the tables are same and the fourth row of both the
tables are same. Hence I can say that this bi conditional statement is same as the conjunction of p
→q and q →p. Now p →q means p is sufficient for q right? And q →p means p is necessary for
q. So that is why this bi conditional statement also represents a statement of the form that p is
necessary and sufficient for q.
21
Now let us next define tautology, contradiction and contingency. So a tautology is a proposition
which is always true, irrespective of what truth value you assigned to the underlying variables.
So, for example, if I consider this compound statement namely the disjunction of p and ¬ p, then
this will be always true; that means if p is true, then this is true and even for p equal to false this
statement is again true.
That means it does not matter whether your p is true or false; this statement is disjunction of p
and ¬ p will always be true and hence this is a tautology. Whereas a proposition is called a
contradiction if it is always false irrespective of what truth value I assign to the underlying
variables. So an example of contradiction is p conjunction ¬p. So you can verify that if p is false
then this statement is false.
And even for p equal to true this overall statement is false that means this statement is always
false for every possible truth assignment of p and hence it is a contradiction. Whereas a
contingency is a proposition, which is neither a tautology nor a contradiction that means it can be
sometime true it can be sometimes false. I cannot say that it is always true or it is always false.
So for instance, if I take the statement p conjunction q then for p equal to false and q equal to
false this overall statement is false. But for p equal to true and q equal to true, the statement is
true. So, that is why it is a contingency.
22
Now, we want to define what we call as logically equivalent statement. So before trying to
understand what are logical equivalent statements? Remember in algebra and in mathematics,
you often come across expressions of this form. We say for instance that a2 + 2ab + b2 is equal to
(a + b)2. That means these two expressions are the same expression. What do I mean by same
expression? Well, by that I mean that whatever value you assign to a and b, the left hand side and
right hand side will give you the same answer.
That is why these two expressions are the same expression. In the same way in mathematical
logic if we have a compound proposition X and a compound proposition Y then I say that they
are logically equivalent and I use this notation ≡. This is not an “equal to” notation, this is
representation of equivalence, this is also called as an equivalence notation. So I say that X and
Y are logically equivalent if they have the same truth values. What I mean by that is I mean that
if X is true then Y is true if X is false then Y is false that means it never happens that X and Y
takes different truth values.
More formally X is logically equivalent to Y provided the X bi-implication Y is a tautology,

right? Because if X bi-implication Y is a tautology, then it means that whenever X is false Y has
to be false whenever X is true Y has to be true. It cannot be possible that X and Y takes different
values because if X and Y takes different values then the bi-implication of X and Y will be false
and a tautology means that this statement is always true.
23
So the statement will be true only when both the sides of this expression or the compound
propositions on both the sides take the same truth value.
So there are various standard logical equivalent statements which are available which are very
commonly used in mathematical logic and they are also called by various names. So for instance,
the conjunction of p and true is always p that is called this law is called as the identity law. In the
same way we have this double negation law which says that if you take the negation of negation
of p then that is logically equivalent to p.
We have this De Morgan’s law which is very important which says that if you have a negation
outside then you can take the negation inside and split it across the various variables and if you
have conjunction inside then it becomes disjunction and vice versa. We also have this
distributive law this says that you can distribute the disjunction over conjunction and so on. How
do we verify whether these logical identities are correct?
Well, we can verify using the truth table method namely we can draw, we can construct a truth
table of the left hand side of the expression, we draw the truth table of the right hand side of the
expression and verify whether the truth tables are the same. So for instance, if you want to verify
the De Morgan’s law, so the first part of the De Morgan’s law says that the negation of
24
conjunction of p and q is logically equivalent to negation p disjunction negation q. So what you
can do is you can draw the truth table for the left hand side here.
You can draw the truth table for the right hand side part here. And you can easily verify that the
rows of both the tables are equivalent, they are same and that is why I can say that these two are
logical equivalent statement and now I have given a name namely De Morgan’s law to this
logical identity. However, the truth table method of verifying logically equivalent statement has
a limitation.
Namely, the limitation here is it works as long as the number of variables the number of
propositional variables which are there in your identity or the statement this is small. So in all
this logical identity that I have written down in this table, there are at most three propositional
variables and if I try to draw the truth table of a statement having 3 variables, and there will be
only 8 rows which are easy to manage.
But imagine I have a logical identity which has a 20 number of variables then the number of
rows and that truth table will be 220 and definitely you cannot draw such a huge table. So that is
why it is infeasible to verify the logical equivalence of statements using the truth table method
and that is why what we do here is we use some standard logical equivalent statements.
So for instance, these are some of the standard logical equivalent statements, which we use to
simplify complex expressions and verify whether those complex expressions are logically
equivalent or not and this is something similar to what we do in our regular maths. In regular
maths if we have two expressions and if you want to simplify one expression and convert it to
another expression then we have some well-known rules which we can always use to do some
substitution in our process of simplifying the expressions.
So we are trying to do the same thing even in the mathematical logic. If you are given a very
complex expression X, a compound proposition X, which you want to show to be logical
equivalent to Y and you do not want to involve the truth table method, then our goal will be to
simplify the expression X and keep on doing the simplification till we can convert it into the
25
expression which has the same form as Y.
During this conversion process or the simplification process we can use this well-known logical
identities by just quoting their names. We do not have to separately prove the De Morgan’s law
because it is a well-known identity we can simply say that okay, we are using the De Morgan’s
law and hence we are substituting this part with this part and so on.
There are various other standard logically equivalent statements, so these are some of them they
do not have any name but they are some well-known logical equivalent statements which we can
use while doing the simplification.
26
So now let us do an example here. Suppose I want to prove that my LHS expression and RHS
expression, they are logically equivalent so this is my statement X this is my statement Y. Well
in this case I can use the truth table method because my expressions X and Y involve only 2
variables and I can draw truth table which will have only 4 rows, but what I want to do here, I
want to demonstrate here is that without even drawing the truth table, I can show that the
expression X is logically equivalent to expression Y by using logical identities.
So here is the proof that expression X is equivalent to expression Y. I start with my expression
X. What I can say is that this expression X is equivalent to this new expression and why this
expression X is equivalent to this new expression because I can apply the De Morgan’s law
twice. So what I can do is I can take this negation first inside, so that is why I get this negation p
and this negation is will be now present outside this bracket.
So that is why again I can apply the De Morgan’s law and this negation when it goes inside the
negation negation p becomes p and then this negation also goes to q. And this conjunction gets
gets converted into disjunction. Now what I can say is that this expression which I have derived
from the expression X can be further converted into this expression because I can apply the
distributive law.
The distributive law says that you can always split the conjunction over disjunction, so that is
27
what I am doing. Now I can say that this expression negation p conjunction p is equivalent to the
value false. So we have this identity and I am not quoting the name of the identity but this is the
well-known identity. So I can substitute this conjunction of negation p and p is false and
whatever there is left over here and then I can apply identity law which says that the disjunction
of false with any proposition is the proposition itself.
Now you can see that I started with X and I kept on applying various laws and I keep on doing
the simplification and then finally I can derive expression Y and hence I come to the conclusion
that starting with X, I can conclude a statement Y. And hence the statements X and Y are
logically equivalent. So that is how we can derive new logically equivalent statement from old
statements by applying well known logical identities and why this is called a proof, because at
each step we are doing the following.
At each step we are deriving a new statement, a new true statement from the collection of
existing true statement and this sequence of steps which I have done here constitutes what is
called as a proof that indeed X is equivalent to Y. So that brings me to the end of this lecture.
Just to summarize, in this lecture we introduced new logical operators namely the bi conditional
operator, we introduced the terms tautology, contradiction contingency.
We defined what we call as logical equivalence of two statements. Two compound propositions
are called logically equivalent to each other if they say take the same truth values or formally bi-
implication of X and Y is a tautology. We discussed various well known logical identities which
we can very quickly prove using truth table method and then we saw that how this well-known
logical identities can be used to prove the equivalence of complex compound propositions by the
simplification method where our goal will be to keep on simplifying the expression X and
convert it into expression Y, thank you.
28
Lecture -03
SAT Problem
Hello everyone. Welcome to this lecture on SAT Problem.

Just to recall in the last lecture. We discussed tautology, contingency, contradiction, we

discussed about logical identities, logical equivalent statements. In the plan for this lecture is
as follows. In this lecture, we will introduce the satisfiability problem which is also called as
the SAT problem and we will discuss about the conjunctive normal forms.
29
So what is the satisfiability problem? So first let us first define, what do we mean by a
satisfiable proposition? So imagine you are given a compound proposition X. So this is an
example of compound proposition X. The compound proposition X will be called satisfiable
if it is true for at least one truth assignment of the underlying propositional variable. So in this
case, this is my expression X and it turns out that if I make p to be true and r to the true and q
to be true then the overall expression X becomes true.
Because for this truth assignment this p disjunction ¬q becomes true. The disjunction of q ¬r
becomes true, the disjunction of r ¬p also becomes true and conjunction of true, true, true is
overall true. So, I have one truth assignment for p, q and r for which the statement X becomes
true. Of course there might other possible assignments of p, q and r as well for which it might
be true.
The question here the definition say that even if you have one true assignment for which X
becomes true, my statement X will be called as satisfied. An alternate definition here for
satisfiability is the following. I will say that my compound proposition X is unsatisfiable. I
stress unsatisfiable if and only if negation of X is the tautology. And why so? Because if X is
unsatisfiable that means it is always false, right?
Because satisfiable means at least one truth assignment is there for which X will be true, but
unsatisfiable it means it is always false or in other words it is a contradiction and if it is
contradiction then you take the negation of that it will be always true. That is why this
alternate definition is correct. Now the SAT problem is the following. You will be given a
compound proposition X here and you have to verify whether the proposition X is satisfiable
or not.
That means you have to give me a yes or no answer, yes means yes there exists a true
assignment for which the proposition X is true, no means ok there is no truth assignment for
which X is true. And this is one of the widely studied problems in computer science. There
are plenty of applications of this SAT problem we will see one such application in this lecture
and it is believed to be a hard problem, what do I mean by hard problem?
That means by hard problem informally I mean here that we do not have efficient algorithms
practical, algorithms to verify whether the given proposition X is satisfiable or not. What do I
30
mean by practical? By practical here roughly I mean that it gives you the answer very
quickly, and of course we do not have any proof yet that this is indeed a hard problem. It says
that we do not have any efficient algorithm as of now to verify whether a given arbitrary
proposition is satisfiable or not.
But we strongly believe that it is indeed a very difficult problem to solve and as I said there
are plenty of applications, we will see one of the applications namely the computer aided
Sudoku puzzle solver.
Before that let me introduce what we call as Conjunctive Normal Form or CNF form for a
compound proposition. The motivation for studying the Conjunctive Normal form is that if
you are compound proposition X is given in its CNF form then it is relatively easier to verify
whether X is satisfiable or not, so what do I mean by a conjunctive normal form? In some
books, they also called this as product of sum.
So here is an example of an expression which is in conjunction normal form. You have

several parentheses here inside parenthesis, you have disjunctions of variables and each
parenthesis is connected by this conjunction operator. That is why it is called product of sum,
outside you have the products, you have products of various brackets here and each bracket
you have the sums.
So now let us formally define what do we mean by a conjunctive normal form? So an

expression X is said to be in its conjunctive normal form if it is expressed as a conjunction of
31
clauses, so you can see here you have the conjunctions appearing here. Each such parenthesis
here is called a clause. Now, the question is, what is the clause? Well, it turns out that the
clause is a disjunction of literals, because informally in each clause, inside each clause you
have disjunctions of various variables.
Those variables might be either without the negation operator or with negation operator. So
that is why we define a clause to be disjunction of literals. Now the next question is what is
the literal? And my definition of literal here is a literal is either a propositional variable or the
constants true or false. So here T stands for true F stands for false, and these are my
constants.
So my definition of literal here is it can be either a propositional variable or it can be one of

these two constants true or false. So let us verify whether this expression X which we have
written here is indeed in its conjunction normal form as per this definition that we have given
here, that is what we want to check quickly. This is my X here, so the definition says that X
should be a conjunction of clauses.
So we have conjunctions appearing here, we have to verify whether indeed we have the
conjunction of clauses appearing here or not. So we have to verify whether each of the
expressions within the parenthesis is a clause or not. What is the definition of clause? It says
it should be a disjunction of literals. So let us check whether the first sub expression, namely
the disjunction of p and ¬q is a clause or not.
So p is a literal and negation q is also a literal remember because what is the definition of a
literal? A literal is a propositional variable. So negation of q is also a propositional variable.
Do not think that propositional variable just means the positive form of that variable. So if q
is a propositional variable negation of q is another propositional variable. I can represent it by
r. So you can imagine that I have the disjunction of p and r appearing here and that is why it
is a literal.
And in the same way the second sub-expression namely the disjunction of q and ¬r. It is also
a clause so, this is the propositional variable and negation of r is another propositional
variable. In the same way the third sub expression the disjunction of r and negation p is also a
32
clause. So I have three clauses here and we have conjunctions of these three clauses and
hence this expression X is in its conjunctive normal form, it satisfies my definition.
So now let us answer this question, is this expression the disjunction of p and negation q is in
its Conjunctive normal form? So you might be wondering well there is only one clause here
and as the definition says that the expression is in its CNF form only if it is a conjunction of
clauses. So we do not have any second clause here we do not have any conjunction appearing
outside this expression.
And hence it might look that this expression is not in its conjunctive normal form, but that is
not a right argument. Indeed this expression is in its conjunctive normal form. Because if you
see closely this expression can be, logically equivalent to this expression. You can verify that
and I can put a parenthesis around this T and I can further say that this expression (T) is
logically equivalent to (T disjunction T)
Now this is a Clause this first sub expression and this second sub expression is also a clause. I
have conjunction appearing between these two clauses. So even though there is no explicit
second clause here. I can always interpret that I have this constant true which is appearing in
conjunction with this expression and hence this expression the disjunction of p and negation q
is also in its conjunctive normal form.
Now let us ask whether this expression p is in its conjunctive normal form? So remember the
expression the compound p itself can be interpreted as a compound proposition. So again, it
might look that there are no clauses here and there and there are no parentheses here. So that
is why this is not in its conjunctive normal form. Again, that is an invalid argument. This
expression is indeed in its conjunctive normal form.
Because I can always state p to be equivalent to the disjunction of p and the constant false
and this is a clause and that is why the expression p is already in its conjunctive normal form.
Now the question is, well if my expression X is not given in its conjunctive normal form can
I convert it into another expression which is in conjunctive normal form and which is
logically equivalent to my original expression X and answer is yes.
33
You give me any expression X. If it is already in its conjunctive normal form then well and
good but if it is not and it is conjunctive normal form, then there is an algorithm which I can
use and convert my expression X into another expression Y, where Y will be logically
equivalent to X and Y will be in conjunctive normal form.
So let us see that algorithm. So the input here is some expression X which may or may not be
in its conjunctive normal form. The first step here is the following. I will check the
expression X and wherever there is an occurrence of bi-implication. I will substitute that bi-
implication by using this logical identity. I can use the logical identity that the bi-implication
of p and q is logically equivalent to the conjunction of p → q and q → p.
So by applying this law repeatedly I can get rid of all the occurrences of bi-implications in
my expression X and the new expression that I will obtain will be logically equivalent to my
original expression. Remember at each step of the substitution my goal will be to substitute
something with a new thing that a new thing should be logically equivalent to the old thing,
otherwise the whole process will give you an incorrect answer.
Next what I do is by applying the previous step in my expression, there will be no more
occurrences of the bi-mplications, but what I might have done is I might have introduced
some implications and there might be already some existing implications symbols in my
expression. So I have to get rid of all of them and how do I get rid of all of them? By
applying this logical identity.
34
So there is a well-known logical identity which says that p →q is logically equivalent to the
disjunction of ¬p and q. So you apply this law repeatedly in your expression and get rid of all
the implication symbols. Next what I do is I apply the De Morgan’s law wherever applicable.
That means wherever there is an expression of the form negation and inside I have a
disjunction, what I will do is I will take the negation inside and split the negation over the
disjunction and same way I do over the conjunction.
So wherever there is a scope of applying the De Morgan’s law, apply it. And then finally you
apply the distributive law wherever applicable; that means you distribute your disjunction
over conjunction. Well, I am not going to distribute conjunctions over disjunctions because
remember finally I have to bring my expression in the conjunctive normal form and
conjunctive normal form means I should have this conjunction of clauses.
So that is why I apply the distributive law where the disjunction is distributed over
conjunction. If I apply the other distributive law then I may not get the expression in the
conjunctive normal form and now if you apply these four steps there is always the guarantee
that you get a logical equivalent expression which is in its Conjunctive normal form.
Now let us see some fun application of the SAT problem, of course as I said there are plenty
of advanced applications of the SAT problem. The SAT problem is encountered very
frequently in AI applications. But I am going to demonstrate application the SAT problem
with this fun application, which is solving the Sudoku Puzzle. So what is a Sudoku puzzle?
So you will be given a partially complete nine 3 X 3 sub-grids.
35
So what do I mean by sub-grids here? So this is sub grid number 1, then this is sub grid
number 2, sub grid number 3 and so on. You are given 9 such sub-grids, okay? Well you can
generalize it to any n X n. I am taking here the case of 9, 3 X 3 sub grids, but it can be always
generalized to higher dimensions, this is just for simplicity. So each 3 X 3 sub-grid here will
be called as a block.
Some of the cells here, so each cell will be either filled or it might be blank, so some of the
cells here are already filled and that is why it is a partially complete collection of 3 X 3 sub
grids, rest of the cells are incomplete here. So what the solution of this Sudoku puzzle
demands, it demands from you the following you have to fill the blank cells here. And you
can fill the cells the empty cells with any number in the set 1 to 9 but there are some
restrictions.
The restrictions are the following, you have to ensure that once you fill the empty cells here
then each row should have the numbers 1 to 9 exactly once, each column after filling all the
empty cells should have each of the numbers 1 to 9 exactly once and each block namely each
3 X 3 sub-grid after filling all the empty cells should have all the numbers 1 to 9, of course
exactly once. So these are the restrictions for it.
So for instance, this is a problem instance which might be given to you and you have to come
up with a solution. Solution means you have to come up with possible values, which you can
fill in this empty cells satisfying the above restrictions, it turns out that one of the solutions is
this. So very often we write computer programs which take problem instances of Sudoku
puzzle and come up with possible solutions.
Of course, it might be possible that the given problem instance do not have any solution that
means my partially complete problem instance is such that I cannot satisfy all these three
conditions simultaneously. In that case I will say that no solution is there. So a Sudoku solver
or a computer program which tries to solve a Sudoku puzzle will try to come up with one
possible assignment; if at all it is there which satisfies all these three conditions
simultaneously.
36
So now you can think in your mind that how exactly you will proceed to solve an instance of
Sudoku puzzle. You will think in your mind, okay, what if I fill this cell with 1 and then what
if I fill this cell with 2 and so on and then for each possibility you will check whether I am
successfully able to satisfy all the three conditions if not then let me go back and change one
of the possible assignments and then repeat the process.
That is the rough algorithm you will follow in your mind and that is what precisely computer
program for solving the Sudoku puzzle will try to emulate.
So what I am going to do here is I am going to represent an instance of Sudoku Puzzle using

a compound proposition and I will show how I can convert an instance of solving this Sudoku
puzzle into an instance of SAT problem. So the first step here will be that I will do some
encoding here. I will encode this given problem instance using propositional variables, so I
introduce a propositional variable p(i, j, n).
So this entire thing is a propositional variable, p is not only propositional variable. My entire
propositional variable is p(i, j, n) and this propositional variable p(i, j, n) will be true provided
n is assigned to the cell number i, j. It is like saying the following if in my mind I am thinking
that ok I am going to assign the value one here then that means the propositional variable p(1,
1, 1) is true, otherwise p(1,1,1) is false.
So since I am already given some filled cells here, in this particular case, for instance if you
take the fifth row and the first column, it is already occupied with the value 6. That is why the
37
propositional variable p(5, 1, 6) is true. And since 6 is already there in the first column of the
fifth row, I cannot put 6 in any of the remaining columns of the fifth row. That is why all the
remaining propositional variables p(5, 2, 6), p(5, 3, 6) p(5, 9, 6) will be automatically false.
So that is how I can encode the initial conditions which are given to me by this propositional
variable. So now remember a solution for the Sudoku puzzle has to satisfy three conditions,
namely, each of the values 1 to 9 should occur exactly once in each of the rows. Each of the
values 1 to 9 few occur exactly once in each of the columns and so on. So let me try to
encode the assertion that each row should contain the numbers from 1 to 9.
And I want to represent this criteria, this condition, this restriction by a compound
proposition using the propositional variable that I have introduced here. So my claim here is
that this expression namely this disjunction of 9 variables, the variables are here p(i, 1, n), p(i,
2, n) up to p(i, 9, n) I am focusing on row number i here where I of course ranges from 1 to 9
because I have 9 rows here.
So my claim here is that the disjunction of these 9 variables represent a fact that or represent
a condition that my row number i should contain the number n in one of its columns. So
remember I would like the number n to be either available in the first column of the ith row or
in the second column of the ith row or in the ninth column of the ith row, how do I express this
condition ? By this compound expression.
So that means now if I make my n to be 1, this disjunction of 9 variables represent that I want
the number 1 to appear in at least in exactly in one of the columns of the ith row, it could be
either in the first column or in the second column or in the ninth column. In the same way the
disjunction of this 9 variables represent that I want a number n to appear in one of the 9
columns of the ith row.
And disjunction of this 9 variables represent that I want a number 9 to appear in one of the
columns of the ith row. So I am fixing the ith row and I am stating the fact that my ith row
should have 1 appearing somewhere, 2 appearing somewhere, n appearing somewhere and 9
appearing somewhere by this various expressions. And now if I take the conjunctions of all
this expressions separately, that represents the fact that I want to assert here that my row
number I should have all the numbers from 1 to 9 appearing somewhere, that is what the
38
conjunction of this 9 sub expressions will represent. So now what I can say is that if I iterate
over i = 1 to 9, so remember whatever I did till now is with respect to the row number i. But
now I have 9 possible rows so my i can be 1, my i can be 2, my i can be 3, and my i can be 9.
So if I iterate over various possible values of i and if I take the conjunctions of all this
disjunctions separately then that represents my first restriction of the solution or the first
assertion that my solution should satisfy namely each of the rows should have the numbers 1
to 9 somewhere.
So in the short form I can represent whatever we have discussed till now by this compound
proposition. So I am iterating over the row i =1 to 9 and column j = 1 to 9 and possible values
in the cell from 1 to 9 and I want the value n to appear somewhere in the ith row in one of the
columns of the ith row. In the same way we can do a similar exercise and come to the fact or
come to the conclusion that if I want to state the assertion that each column should have the
occurrence of 1 to 9 somewhere;
Then that can be represented by this compound proposition using exactly similar exercise that
we have done for deriving the compound proposition to represent the assertion about our row
condition. So I am not going to do this I leave it to you as an exercise and what is our third
assertion the third condition that our Sudoku puzzle should satisfy? Well our Sudoku puzzle
should satisfy the condition that each three cross three sub-grid should contain the numbers 1
to 9.
39
Again I leave it as an exercise for you. You can verify that this condition can be represented
by this compound proposition. It might look complicated but again if you do it slowly for
each of the blocks you try to derive the condition and then you iterate it over all possible 9
blocks, you can verify that this compound proposition represents the assertion about the 3 X 3
sub-grid that your Sudoku puzzle should satisfy.
Finally, we also need to ensure the following. We need to ensure that our solution for the
Sudoku puzzle should assign only unique values to each cell. It should not happen that a
solution says that a particular cell should take the value say 8 and simultaneously the same
cell should take value 7 that should not happen because that is an invalid solution. Once a cell
is occupied with the value it cannot take any other value.
The reason we have to separately state this condition is. You might try to separately find the
solution satisfying the row condition. You might try to find a solution satisfying your column
condition and then in the same way you might try to find the solutions satisfying your block
condition. Now what may happen is that the solution you try to find for the row condition
might conflict with the solution that you are trying to find for the column solution.
That means the row solution may say that okay, you assign the value say 1 to the first row
and first column, whereas the column condition says that you assign the value 2 to the first
row first column that should not happen. So that is why we have to explicitly state this and it
is very easy to verify that this condition can be represented by this compound proposition.
This compound proposition says that once you have assigned the value n to a cell i, j then you
cannot assign a different value n’ to the same cell i, j.
Where n and n’ or n and n’ are different. So now we have four compound propositions I have
the compound propositions say X, I have to compound proposition Y, I have the compound
propositions Z and I can call this compound proposition as A. Now the goal for a Sudoku
puzzle solver is the following it has to find the truth assignments for the propositional
variables p(i, j, n) and how many such propositional variables are there?
There are 729 propositional variables because i ranges from 1 to 9, j ranges from 1 to 9, n
ranges from 1 to 9. Namely I have to find the truth assignments for each of the 729 cells that
is why I have 729 variables. Some of these propositional variables might have already taken
40
truth assignments because of the initial conditions. So remember some of the cells of the
problem instance might be already filled that means it has already fixed the truth values for
the particular cells.
Those cells I cannot fill with any other value, apart from those propositional variables I have
to find the propositional truth assignment for other propositional variables, so that the
conjunction of X, Y, Z and A becomes true. That means now I have to verify whether this
compound proposition which is the conjunction of X, Y, Z and A is satisfiable or not. So now
you can see that an instance of Sudoku puzzle is reduced to an instance of SAT problem if
this expression is satisfiable.
That means the problem instance, the Sudoku puzzle problem instance that is given to you is
indeed solvable otherwise, it is not solvable. So that is all you can represent, you can encode
real world problem instances wherever you have to find problem solutions right for
computational problems into an instance SAT problem and then you can solve the SAT
problem. The solution for the SAT problem will give you the solution for the computational
problem that you are interested to solve.
So that brings me to the end of this lecture. Just to summarize, in this lecture we introduced
the satisfiability problem there are plenty of applications of the SAT problem. We saw a fun
application of the SAT problem namely the Sudoku puzzle solver, thank you.
41
Lecture -04
Rules of Inference
Hello everyone, welcome to this lecture on rules of inferences.

Just to recap in the last lecture, we introduced the SAT problem, we saw an application of the
SAT problem namely the Sudoku puzzle solver, the plan for this lecture is as follows. We will
introduce what we call as valid arguments, we will see rules of inferences and we will discuss
some fallacies.
42
So, what we do mean by valid arguments in propositional logic? Suppose we are given a bunch
of statements like this the statements are; if you know the password then you can login to the
network and it is also given that you know the password therefore I am concluding that you can
log on to the network. This is an argument which is given to you and we have to verify whether
this argument is logically correct or not.
Then consider another bunch of statements which also looks like an argument, the statements
that are given here are the following it is given that if you have access to the network then you
can change your grade. When it is also given that you have access the network therefore I can
conclude that you can change your grade. So, here you have a bunch of statements before the
conclusion.
So those bunch of statements are called as premises. If you might be given one premises or two
premises or multiple premises and based on those premises I am trying to derive a conclusion.
So whatever is there before therefore is called a premise and whatever is there after therefore is
called as conclusion. So before going into verifying whether these arguments are mathematical
or logically correct or not, we want to argue here or we want to check here is that is there any
similarity in the arguments.
Well, if I view these two arguments these two set of arguments in the English language sense
43
then they are different because we are talking about different things. But if I try to extract out
these two arguments, then there is a similarity that both these arguments have a common
structure, they have a common template and what is the common template here the template is
the following.
If you see the premise, the premise is of the form p → q and p and the conclusion is of the form
q, why so? So you can see that, I can say that p represents the statement you know the password.
I say that p represents the statement, you know the password and q represent a statement you can
log on to the network. So that is why in the first argument the statement if you know the
password then you can log on to the network can be represented by p → q.
And it is also given that we know the passwords. So I have used the statement variable q to
represent a statement, you know the passwords sorry, I have used the variable p to represent the
statement you know the password therefore the second premise which is given to me is p. And
what is the conclusion I am trying to do I am draw here, I am trying to draw the conclusion that
you can log on to the network which represented by q here.
So I can say that this English language argument form can be represented by this template where
in the premise I have I am given the statement p → q and p and therefore I am drawing the
conclusion q. In the same way for the second set of arguments as in the second argument it is set
of statements which are given to me they can be abstracted by this argument form why so? Again
I can say that, let p represent the statement you have access to the network and let q represent the
statement you can change your grade.
I have used p to represent you have access to the network and a conclusion, I am trying to draw
here is you can change your grade, which is q. So again, the second argument the second set of
English statements which are given in the argument can be also abstracted by this common form.
So this common form where I do not worry about what is my p what is my q I have just a set of
premise and I can corresponding conclusion, I call it as an argument form.
Now what I want to verify is whether this argument form is valid or not, whether it is correct or
44
not by valid, I mean whether it is correct or not. What do I mean by argument form is valid my
definition here is I will say my argument form is valid, if I can say that the conjunction of
premises implies conclusion is a tautology, that means if I can prove that if all the premises are
true then my conclusion is also true.
Because remember, anything of the form p → q is false only when p is true, but q is false. So that
means I want to verify here that indeed q follows p or not, the capital Q follows from capital P
that means all the conditions in my capital P are true whether I can come to the conclusion
capital Q that is the case that this argument form will be a tautology. Because for all other three
cases when p is capital P is false and when capital Q is false by default P → Q is true.
I have to only worry about a condition then my capital P is true and my capital Q is false I have
to verify whether this is happening whether if it so happens that capital P is true, but capital Q is
false and then I cannot say that capital P → capital Q is the tautology in which case I will say by
argument form is invalid. So that is my definition of valid argument why I am interested in
argument form here is that if I know how to verify whether a given abstract argument form is
valid then it does not matter how do I instantiate my variable p, q and so on.
My p could be the statement if you know the password my q could be you can log on to the
network, my p could be if you have access to the network my q could be you can change your
grade and so on. If I know that this abstract form is valid then irrespective of the exact statements
which I substitute for p, q my overall corresponding English language argument will be valid. So
instead of individually verifying whether this argument the first set of English statement is valid
or not, the second set of it is English statement is valid argument or not.
Instead of individually verifying them what I focusing here is I am focusing on the validity of the
abstract argument form. If I can prove that abstract argument form is valid then it automatically
implies that this is valid, this is valid, and so on.
45
So in general, my abstract argument form is the following form. I will give a set of premises,
namely n premises and each of them can be a compound proposition, all of them can be
compound propositions and I will be given a conclusion which is also a compound proposition
and my definition of a valid argument is the following. I will say that the above argument form is
valid if the conjunction of premises implies the conclusion is a tautology.
If that is the case or if my argument form is valid and I will use the term that Q is a logical
consequence of P1, P2, Pn, that means if you ensure all together that P1, P2, Pn, are simultaneously
true. Then I can come to the conclusion Q, if my argument form is a valid argument form. If my
argument form is not valid that means if this implication is not a tautology, then I cannot say that
Q is a logical consequence of P1 to Pn.
46
Now how do I check whether a given argument form is valid well? My definition says to check
whether a given argument form is valid or not you check whether this implication is a tautology
or not. The conjunction of premises, implication, the conclusion is a tautology or not and now
you have the definition of tautology means that this expression. So I call this expression X
implies, I represent this conjunction of P1 to Pn by capital P, I have to check whether P → Q is a
tautology or not. P → Q will be a tautology if this statement is always true.
Remember that P → Q is always true for the case when P is false, it does not matter whether Q is
true or false in both the cases it will be true and if Q is true, it does not matter what is P. P → Q
is always true. The only case I have to verify is when P is true, I have to check if it is so happens
that P is true but Q is false. If that is the case, then this is not a tautology, okay? So that is the
only case I am interested in.
So it turns out that I can verify whether the above implication is a tautology or not by using the
truth table method that method I can always apply but truth table method will be infeasible for
large arguments if I am given a large number of premises each involving many numbers of
variables. So in general, I use rules of inference we can use rules of inferences which are very
simple argument forms whose validity can be easily established using the truth table method.
And we give some names to this simple argument forms and then the idea is that we use this
47
simple argument forms as a building block for proving the validity of large arguments. So when I
will be proving the validity of the large arguments, I will not be using the truth table method, but
I will be doing some kind of simplification where I will identify some parts of the argument form
which I know is already true based on the validity of smaller argument forms, I will have some
well known rules applied.
Now some well known names and if I can identify some portion in my complex argument form, I
can easily say well that part is true so I do not we have to worry about that. That is a whole idea
of rules of inferences. It is used for proving the validity of complex large argument form where I
use simple argument forms whose validity has been already established using some well known
methods say truth table method.
So, I will be listing down some standard rules of inferences each of them can be easily proved
using the truth table method, the most popular here is what we call as Modus ponen. So what
Modus ponen says is that if you are given the premises p and p → q, you can come to the
conclusion q. To verify whether this is a valid argument form, you have to verify whether p
conjunction p → q overall implies q is tautology or not.
And you can easily verify that this is a tautology using the truth table method, I am not going to
do that it is an exercise for you the truth table will have 4 rows right p to be true, q, to be true p
48
to be false, q to be false and so on and you can verify that for all possible cases or of all possible
four combinations this overall expression is always true and hence I can always conclude the
conclusion q from the premises p and p → q.
Now why it is called Modus ponen and well there are some reasons for that I am not going to
details. There is another well known rule of inferences, which is called as Modus tollen. It says
the following that if you are given the premises ¬ q and p → q then you can come to the
conclusion ¬ p. So even though it is called by a different name it is given a different name and it
might look a different argument form I can view it as a special form of Modus ponen.
So let us see how. You are given the premise ¬ q and remember the p → q is logically
equivalent to ¬ q → ¬ p. Because ¬ q → ¬ p is the contrapositive of p → q and contrapositive is
always logically equivalent to the original implication and your conclusion here is ¬ p. So if you
see closely here the new thing that I have written here is of the same form as Modus ponen right,
same form. Why?
Well you can say that negation of r is nothing but q some other variable and since negation of r is
¬ of q is r, so I can say r → s and I can say ¬ of p is denoted by s. And hence I am denoting the
conclusion s and this is nothing but Modus ponen, which I know is a valid argument form. So
even though Modus tollen is looking structurally different because you have the negation
appearing here if I rewrite everything and do the substitution here I can bring this Modus tollen
into the form of Modus ponen.
And, hence I do not have to separately prove by truth table that this Modus tollen is a valid
argument form. So valid argument form I can use it in any simplification. In the same way I have
this transitive law, which is also called as hypothetical syllogism which says the following. If
you are given the premises p → q and q → r, you can draw the conclusion p → r and again, I
have to verify here whether the conjunction of p → q and q → r overall giving the conclusion p
→ r is a tautology or not.
And you can verify that it is indeed a tautology. I am not doing that, there will be 8 rows because
49
you have 3 variables and you can quickly verify that in each of the rows the final result is true.
We have disjunctive syllogism that says that you have if you are given disjunction of p and q and
you will given the premise ¬ p then you can draw the conclusion q and so on. You have the
addition law,, simplification law, conjunction.
And there is another law called as a resolution which again is a special form of Modus ponen, we
will come back to this resolution later. Why this is an equivalent form of Modus ponen because
of the following of sorry it is well, I can apply Modus ponen and I can apply the transitive law
and show that by applying these two laws, I can get the resolution. I am not going into the
details, I leave it as an exercise for you.
So now imagine that I am already given some well known rules of inferences. So whatever rules
I have stated here, they are called as rules of inferences. I do not have to separately prove them
they are now valid argument form and that is why I can use them at my discretion. So now I have
to verify complex looking arguments using these rules of inferences. So imagine I am given this
English argument, I am given a bunch of four statements, four premises and based on these four
premises, I am trying to draw conclusion here.
So the first thing that I have to do here is I have to write down the abstract argument form here
and for doing that I have to convert each of the English statements into some compound
50
proposition. So what I do is I assign various propositional variables to represent different
statements here. So suppose I use the variable p to represent a statement that it is not sunny this
afternoon. So it is a declarative statement, it is a proposition.
I am assigning the propositional variable p for that. In the same way say I assign the
propositional variable q to represent a statement it is colder than yesterday. My second premise is
we will go to swimming. So this statement has not occurred till now. So I am assigning a new
variable r here for that and it is given here that we will go to swimming only if it is sunny this
afternoon.
Well, I have already assigned a variable p to denote the statement it is not sunny this afternoon,
so it is sunny this afternoon will be represented by ¬ p. Now my third premise is if we do not go
swimming then we will take a canoe trip. So I have already used the variable r to represent we
will go swimming. So ¬ r will represent we will not go swimming and this statement we will
take canoe trip is coming for the first time.
So, I will present I use introduce a variable s for that and it is easy to see that the fourth premise,
I do not have to introduce anything for this statement if we take a canoe trip but I have to
introduce a variable t for representing the statement we will go home by sunset. What is the
conclusion I am trying to draw? We will be home by sunset I am using the variable t for that. So
the first step here is I have assigned truth variables for various statements involved in this
argument form.
Now, I have to write down the abstract argument form. What is the first statement? It is not
sunny this afternoon and it is colder than yesterday. So we have an occurrence of ‘and’ here
right? So that is why it is p conjunction q. What is the second statement? We will go swimming
only if it is sunny this afternoon. So remember whenever we have an occurrence of only if, then
whatever is occurring after only if is the necessity condition, this is the necessary condition not a
sufficient condition.
So you go back to the lecture where we have introduced the implications operator, right? So this
51
statement will be represented by r → ¬ p that means ¬ p is a necessary condition for r that means
if it is not sunny this afternoon definitely I will not go to the swimming that is what is the
interpretation of this statement. So that is why it will be represented by r → ¬ p.
What is the third premise here? The third premise is a statement of the form if something then
something. That is why it is negation r → s and my fourth statement is again a statement of the
form if something then something, I have used a variable s and t that is why it is of the form s →
t and what is the conclusion? I am trying to draw the conclusion t. So now I have to verify
whether this argument form is valid or not as per the definition, I have to verify whether the
conjunction of p and q and r → ¬p and ¬r → s.
And s → t overall implying t is the tautology or not, that is what I have to verify. If it is the
tautology, then I will say that this is a valid argument otherwise, I will say it is an invalid
argument. Well, I can use the truth table method here to verify whether this is a tautology or not,
but how many variables I have? I have p, q, r, s, t, 5 variables. So 32 rows, well it still
manageable but might be time consuming.
So what I will do instead is I wont touch the truth table here, but still I will be able to show you
that how we can verify whether this argument form is valid or not using well known standard
rules of inferences. So here is the proof, so I start with some premise here. So remember I have
52
to show that if premises are true, I have to check whether if premises are true then conclusion is
true.
I have to verify that, then only this is a tautology that is a definition of valid argument. So I am
assuming that all my premises are true that is, this is true, this is true, this is true, this is true, and
then I have to check whether my conclusion is also true or not. So since p and q is true, I am
starting with that, I apply the simplification law and why I am writing it as one, one because the
first statement is I am starting with the premise p and I am giving a number to that the number
that I have given to that statement that premises the conjunction of p and q.
So I can say that since, my premise p and q is true. I can apply the simplification law on that and
come to the conclusion p. Now, I can say that since my premise r → ¬p is true, I can apply the
contrapositive law on that and come to the conclusion p → ¬r; that means if r → ¬p is true, of
course p → ¬r will be true; that is what is the interpretation of this statement.
And I am giving this number 3 to this new statement, which I am deriving from already known
statements. Now what I can say is, I can apply the Modus ponen on statement number two and
statement number three. So I have derived p to be a true statement, I have derived p → ¬r to be a
true statement and now I can say that if both statement two and statement three are true then by
applying the Modus ponen and on that I come to the conclusion negation r, that is a new true
statement, which I can apply to my bunch of true statements.
And now I can say that if negation of r is the true statement and anyhow I am given the premise
¬r → s which I am assuming to be true based on these two statements by applying the Modus
ponen, I can come to the conclusion s and again, I can say that if s is a true statement and if my
premise s → t is the true statement based on these two things I can come to the conclusion t. So
what I have done here.
At each step, I have used already known true statement which might be either given already as
part of premises or which I might have derived in some previous step to derive new statements
which will be true and by doing this process I ended up coming to the conclusion t and hence I
53
can say that this argument form is a valid argument form that means if I assume all the premises
to be true, based on those things by applying rules of inferences properly at each step, I keep on
deriving new conclusions.
And, I end up coming to the final conclusion which is there in my argument form and hence my
argument form here is a valid argument form and now you can check here that I have not at all
touched truth table here, I never said okay since p and q is true then both of them will be true and
then go into the following propositions and so on. I never did an argument of that form, I just
used whatever premises are given to me and I kept on deriving new conclusions and ended up
showing you the final conclusion.
So that is how we use rules of inferences to prove whether complex argument forms are valid or
not.
Now there are some well known fallacies which are incorrect arguments but on a very high level
it might look a valid argument but they are very subtle incorrect arguments. So there I will be
showing you two common incorrect arguments or two common fallacies. The first fallacy is that
of affirming the conclusion. So consider this argument form: your premises are p → q and q and
you are drawing the conclusion p.
54
So to verify this you have to verify whether p → q and q, implies p is a tautology or not. Well, it
is not a tautology, the problem here is the following consider this following argument, I make the
premise, I give you the premise that if you solve every problem of Rosen’s book, you will learn
discrete maths and it is already give also given that you have learnt discrete maths, okay?
Now based on these two premises, I am trying to draw the conclusion that you have indeed
solved all the problems of Rosen’s books, is this a valid argument? Is this a valid reasoning?
Well, this is an invalid reasoning because it might be possible that even without solving any of
the problems of Rosen’s books you have learn discrete maths by some other mechanism say for
instance by watching the NPTEL videos, this discrete maths course and without even touching
any problem of Rosen’s books.
So this argument, by this English argument forms in this argument form. So let p represent a
statement at you solve every problem of Rosen’s books and q represent a statement at you will
learn discrete maths. So that is why this is p → q. Another premise that is given is you learn
discrete maths that means it is given q to the true therefore the conclusion that I am trying to
draw here is that you solved every problem of Rosen’s books, which is p.
And as I am giving here, this is an invalid argument here, because it might be possible that you
have learned discrete maths even without touching any problem of the Rosen’s books. So this is
called as fallacy of affirming the conclusion this is different from your Modus ponen and
remember your Modus ponen was p → q and p, therefore q, this is different, right?
Modus ponen is a valid argument form, but this fallacy of affirming the conclusion is incorrect. It
may or may not be true. So let us see another fallacy here, this is the fallacy of denying the
hypothesis the argument form here is p → q, negation p and these are the two premises and the
conclusion you are trying to draw is negation q. So an instantiation of this abstract argument
form is the following say again, my premises are if you solve every problem of Rosen’s books,
you will learn discrete maths.
You do not solve every problem of Rosen’s books. So these two premises come under this
55
abstract form p → q and ¬p. And therefore I am drawing the conclusion you would not learn
discrete maths. Again, as you can see, this is a false argument because you might have learned
discrete maths by just watching NPTEL videos if they are very good without even solving any of
the problem of Rosen’s books.
So these two are very common fallacy might look very similar to Modus ponen and but they are
not valid argument form. So that brings me to the end of this lecture, just to summarize in this
lecture we have introduced argument forms we have defined valid argument forms when do we
say an argument form is valid. We have seen various rules of inferences and how we use rules of
inferences to verify complex argument forms are valid or not. Thank you.
56
Lecture -05
Resolution
Hello everyone, welcome to this lecture on resolution just a quick recap.

In the last lecture we discussed about valid arguments, argument form when exactly we say an
argument form to be valid and so on and we also saw various rules of inferences. The plan for
this lecture is as follows in this lecture we will discuss about resolution which is an important
influence rule and based on resolution we will see a proof strategy which is called as proof by
resolution refutation.
57
So to begin with, let us try to understand what exactly is the resolution rule. It is a very
important inference rule and it is used extensively in this programming language called
PROLOG. So recall I said that PROLOG is an important programming language, which is used
in AI applications. So what exactly is this resolution rule? So it says the following, imagine you
are given two clauses. So C1 is the clause and C2 is another clause.
And the important property here is that I have a literal L which is present in positive form in C1
and negative form in C2. So you can imagine that C1 is a huge clause consisting of one or more
literals, one of the literals is L. So just to recall a literal is propositional variable or the constants
True or False. So what I am saying here is that we have two clauses C1 and C2. In C1, we have
some literal L and the same literal is available in a negation form in C2.
The remaining portion of C1, I am denoting it as C1’ and the remaining portion of C2, I am
denoting it as C2‘. So you have 2 such clauses and what this resolution rule says is the following.
It says that, if it is given that the clause C1 and C2 are true, then based on the truth of these 2
clauses we can conclude, conclusion C1‘ ˅ C2‘. So in some sense you can imagine that resolution
rule is something equivalent to cancellation rule.
That means you can cancel out the literal L if it is available in positive form in C1 and negative
form in C2 and whatever is left in C1 and C2 you take the disjunction of that will be the
58
conclusion of C1. and C2. So in some way you are actually simplifying your clause C1 and C2. So
in argument form the resolution rule can be stated as follows. So this is the argument form of
resolution inference rule.
It says that if you are given the clauses C1 and C2 where, C1 is C1‘ ˅L and C2 is C2‘ ˅¬ L. Then
based on these two premises, you can conclude the conclusion C1‘ ˅C2’. I stress that to apply the
resolution rule you need C1 and C2 to be clauses. That means C1 and C2 have to be compound
propositions which are available in the form of clause. It should not be available in a different
form.
So the conclusion that we can draw from the resolution rule namely the disjunction of C1‘ and
C2‘ is also called as the resolvent of the clauses C1 and C2. That means after resolving the clause
C1 and C2 we are getting the resolvent C1‘ ˅C2‘. And remember as per our definition of argument
forms since we are saying that our resolution is a valid argument form.
And a definition of valid argument form is that conjunction of premises implies conclusion
should be a tautology. That was our definition of a valid argument. Then since we are saying that
resolution as a valid inference rule we will prove that, assume for the moment resolution is a
valid inference rule, it means that we can say that the conjunction of clauses C 1 and C2 where C1
and C2 have the common literal L available in positive as well as in negative form in C1 and C2
respectively implies the disjunction of C1‘ and C2‘ is a tautology. It will always be a true
statement we will prove that very soon. So, that is the resolution. So now we want to prove that
indeed the resolution is the, indeed resolution in principle that we are stating here is a valid
argument form. So what we have to prove is we want to prove this statement that indeed this
implication is a tautology. So for that we assume that a left hand side of this implication namely
the conjunction of C1 and C2 is true.
Why we are assuming it to be true because remember we want to show that this implication is a
tautology and this implication is true for all other cases. Remember the truth table of implication
of false → false is true, false → true is true and in the same way true → true is true. So for these
three cases by default this implication is always a true statement we have to consider the fourth
59
case when your left hand side of this implication is true and we have to show in that case the
right hand side of the implication is also true.
That will prove that this implication is indeed a tautology. So that is why I am assuming here that
the left hand side of your implication is true. So now I can split my proof into two cases
depending upon whether my literal L which is available in positive form and negative form in C1
and C2 respectively is true or not. So if L is true since I am assuming that this whole conjunction
is a true statement and since L is true, that means this portion here, this portion of your left hand
side. Since I am assuming it to be true this has to be true right the disjunction of C2‘ and negation
of L has to be true because then only the overall conjunction can be true. But since I am
assuming L to be true negation of L will be false. And if negation of L is false then in order that
is the overall C2 should be true. I require that C2‘ should be true.
And if C2‘ is true then you take the disjunction of C2‘ with anything, say with C1‘ the overall
disjunction will always be true. So that proves that this implication is a tautology for case one
that means if you assume your left hand side is true of this implication and if L is true, then I
draw the conclusion that even RHS is also true. Now take the second case when L is false, so
these are the two possible cases with respect to the literal L, can be either true or it can be either
false.
So if you literal L is false then I focus on C1 this is your whole C1. And since I am assuming that
this overall conjunction is true, this overall conjunction will be only if the individual clauses here
are true. But if I focus on the clause C1 I am assuming case two where L is false. Then in order
that C1 is true, C1‘ has to be definitely true. Because if C1‘ is also false and if L is also false your
C1 can never be true.
But I am assuming that my left hand side is true. So now if C1‘ is true, I take the disjunction of
C1‘ with anything the overall disjunction will be true. So that proves that even for case 2 my
RHS is true and that proves that, this implication that I have stated here is indeed a tautology.
And since it is a tautology as per my definition of valid argument form I can say that resolution
is indeed a valid inference rule or it has a valid argument form and hence the corresponding
60
inference rule is indeed valid. Resolution is a valid argument form.
So we have seen already how to find or how to resolve a pair of clauses, now next we want to see
how exactly we resolve a set of clauses where we may have more than two clauses. So imagine
you are given a set of n clauses and I would be interested to resolve the clauses in this set, which
is often called as a resolvent of the set of clauses. So the idea remains the same that means we
will keep on finding two clauses from this collection and keep on resolving them and we stop till
we cannot proceed further.
So basically we build what we call as a resolution tree and in the resolution tree we can keep the
n clauses that are given in my set S at the root level, that means by default we can imagine that
the clauses C1, C2 Cn each of them belongs to the resolvent of my set of clauses S because I can
always conclude C1, I can always conclude C2 and I can always conclude Cn from my set of
clauses in S.
Now, next what I have to do is the following I have to resolve a pair of existing clauses which
are already there in my tree and whatever is the resolvent I obtain by resolving the pair of
clauses, which I have resolved that will be treated as a new clause which will be again added to
my tree.
61
And then I go to this step and again pick two clauses which I can resolve, I find a resolvent and
again I add them to the resolution tree and I stop this process when I cannot find any more
clauses which can be resolved at that step I stop. That is how I can find a resolvent for a set of
clauses. I stress here that there is no restriction at each step regarding the choice of the clauses
which you can pick for resolving in what order you have to resolve the clauses and so on.
As long as you are picking two clauses which can be resolved and adding the resolvent to the
tree you are fine.
So let me give you an example to show how exactly we compute the resolution of a set of
clauses, so imagine you are given here compound propositions p → q, r → s, p and r. So the first
step will be that I will be converting this compound propositions into their corresponding clause
form because as of now p → q is not in its clause form. But by applying the rules of logical
equivalence, I can convert p → q into ¬ p disjunction q and so on.
So now I obtain clauses C1, C2, C3, C4 and this will be my set S. So now here is how I can build
my resolution tree at the root I can pick, I can keep all the clauses that are there currently in my
set S. And now I keep on picking clauses, pair of clauses, which I can resolve. So for instance, I
can resolve these two clauses by cancelling p and ¬ p and the resolvent will be q which will be
now added to the tree.
62
Next, I can resolve r from these two clauses and a resolvent will be S and now you can see that I
can no longer find a pair of clauses which can be resolved further and I stop here and hence I will
say that the resolvent of the set of clauses consist of the conjunction of clause q and the
conjunction of the clause s. So that is how we actually built the resolution of a set of clauses.
Now, I will be discussing two key properties regarding the resolution of a set of clauses based on
which we will see a very nice proof strategy which we call as proof by resolution refutation. So
the first property here is the following, imagine you are given a set of n clauses. Now, the claim
here is that the empty clause or the constant False, or the false statement you can imagine you
can interpret it in different manner.
So this is the constant False which is also denoted by the notation ϕ in some text that I will be
interchangeably using the constant F as well as ϕ for denoting the false statement or the false
constant. So the claim here is that a constant false will belong to the resolvent of set of these
clauses if and only if this, the set of clauses in S is unsatisfiable. What does that mean? That
means that if the conjunction of the n clauses is always false that is what it means when I say that
the set of clauses in S is unsatisfied.
If the set of clauses is unsatisfiable that means no truth assignment of the clauses C1 to Cn can
63
satisfy to make it true. That means it is always false. So the statement here is that if the set of
clauses in S is unsatisfiable, then when you build the resolution tree for resolving the set of
clauses in S, you will see that, the constant F appears in the tree. So due to interest of time I am
not going to give you the proof for this but you can easily verify that this is indeed true.
And in fact later on we will demonstrate the statement with an example. Now, based on this
statement I can prove another statement which states the following. So here again you are given
with the clauses C1 to Cn and suppose C is another clause. Now the statement says that, the
clause C belongs to the resolvent of the set of clauses in S if and only if the set of clauses in S,
along with the clause ¬ C is unsatisfiable.
So this union here means that I am adding the Clause ¬ C to the set of clauses in S. That means I
am basically taking the conjunction of the existing clauses in the set S and the clause ¬ C. So the
statement says here that the clause C will belong to the resolvent of S, if I build a resolution tree,
for the set of clauses in S, I will see a node with label C and the statement says that this is
possible if and only if the conjunction of the clauses in S along with ¬ C is false.
And by the previous statement I know that a set of clauses is unsatisfiable if and only if the
constant False belongs to the resolvent of the set of clauses. So when can it be possible, so now
my set S is modified to C1 to Cn and along with that ¬ C. So when can I say that the set S
consisting of the n clauses along with negation of C is unsatisfiable well, that will be
unsatisfiable if and only if the resolvent of C1, C2 and Cn and negation of C, gives me constant
False.
That means, I can say that a constant False belongs to the resolution of resolvent of the set of n
clauses along with negation of C. So that is the second statement.
64
So based on these two properties, I can next discuss what we call as proof by resolution
refutation, so in the proof by the resolution refutation the goal is the following you are given an
argument form and you have to verify whether this argument form is valid or invalid. Namely, I
have to check whether the conjunction of premises implies conclusion is true or not, or
equivalently whether Q can be concluded logically from the conjunction of my premises.
Now the first thing that I do in the proof by resolution refutation is I can convert my premises as
well as conclusion into its equivalent clausal form because the premises P1, P2, Pm may not be
available in clausal form, so I have to convert them into the clausal form in the same way my
conclusion also need not be available in the clausal form. So I convert it into it is clausal form
and now my goal is to verify whether the equivalent argument form where everything is in the
form of clauses is valid or not.
Namely, I have to check whether C belongs to the resolvent of this set of clauses. So this is my
set S, I want to check whether I can conclude C from this set of clauses in S which is equivalent
to saying whether I want to check whether the clause C belongs to the resolvent of the set of
clauses and for that as per this property, I have to check whether the conjunction of my premises
along with the negation of the conclusion is unsatisfiable or not.
And that will be unsatisfiable only if the resolvent of my premises along with the negation of the
65
conclusion is empty or not. Actually it is not exactly equivalent to checking whether the
resolvent is empty or not is actually check equivalent to checking whether ϕ belongs to the
resolvent of this or not, because the resolvent of this set of clauses may consist of many clauses.
Among those clauses I have to check whether one of the clauses is empty close or not.
If that is the case then it will show that this collection of clauses is unsatisfiable which will show
that this argument form is a valid argument form.
So, let me demonstrate it with an example that will make things clear, so I am given a bunch of
premises you are given premise number 1, 2, this is your 1, 2, 3 and 4 premises and this is your
conclusion and I want to verify whether this is a valid argument form using proof by resolution
refutation method. The first thing I will do is, I will convert each of these English statements into
an abstract argument form where everything will be in terms of propositional variables.
So for doing that, I will introduce propositional variables to represent various statements. So let p
denote a statement today is Friday and q denote the statement I will go to a movie, if that is the
case then the first premise can be represented by p → q because this is a statement of the form if-
then. Then let r denote the statement today is bright and s denote the statement I will go outside,
then the second premise can be represented by r → s.
66
I have already introduced the variable p for denoting the statement that today is Friday, so the
third premise is p. I have already introduced a variable r to denote the statement today is bright,
so the fourth premise will be r. And what is the conclusion I am trying to do well I am trying to
do a conclusion that the disjunction of q and s. So I have to verify whether this argument form is
valid or not.
Well you can use truth table method or you can keep on applying Modus Ponen and
simplification rule and so on to verify whether these argument form is valid or not. But goal of
this exercise is we will show that how this argument form is valid or not, how we can check that
using the resolution refutation method.
The first thing that we have to do is we have to convert the premises as well as the conclusion
everything in the form of equivalent clauses. So p → q is not in the form of clause, so we have to
convert p → q into its clause form and I can rewrite p → q in the form ¬ p or q by the way if you
are wondering how we are converting statements into its equivalent clause form that is nothing
but converting statements into its conjunctive normal form.
If you have to conjunctive normal form equivalent of the original statement that is nothing but
the clause form of your original statement. So, if I convert p → q and to conjunctive normal form
I get ¬ p or q, r → s is equivalent to ¬ r or s, p is anyhow in it is conjunctive normal form r is
67
anyhow in its conjunctive normal form and my conclusion is already in it is CNF. So that is the
equivalent clause form.
Now I have to verify whether this is a valid argument form or not and the proof by resolution
refutation says the following you take the premises. That is your set of clause s and you add the
negation of the conclusion to that and what will be the negation of the conclusion. The negation
of the conclusion will be negation of q ˅s. I can apply the De-Morgan’s law and take ¬ q, take
the negation inside and these are the two clauses corresponding to my conclusion which I am
adding to my resolution tree.
And now I have to, so the four clauses was your set s and these two clauses correspond to your
negation of conclusion and now I have to find a resolution of this set of clauses s union of ¬ c
and check whether I get the conclusion false or not. Again, I can pick any pair of clause and keep
on resolving, so what I do is I choose clause number c1 and c3 to resolve because p is available in
positive and negative form, I can cancel out.
And resolvent of c1 and c3 will be q. Then, let q be the clause which I pick and negation of q is
the second clause which I am choosing here and I can cancel them and I obtain False because if
you take q and ¬ q and cancel out your left with nothing and that is a false conclusion. Since, I
have obtained a false conclusion that means this argument form in it is clausal form is indeed
valid.
Remember you might be wondering here that I am not using c2 and r and negation of s in my
resolution process. That is not necessary when I am constructing, when I am doing the
resolution refutation proof, my goal is to arrive at the false conclusion as soon as possible for that
it is not necessary to touch upon each and individual clauses in my tree, it might be possible that
just by using two clauses at the first place I can arrive at a conclusion false.
That will complete the process I need not have to touch upon the other clauses that is not
necessary. So, in this particular case just by resolving c1 and ¬ q, c1 and c3 I can come to the
conclusion q and then by resolving q and ¬ q, I can come to the conclusion. Well, you can do the
68
proof other way around as well in the sense that you can do the proof differently as well that
means instead of say resolving c1 and c3 first, you can do the following as well, you can resolve
this c2 and negation of s you can cancel out and you can come to the conclusion negation of r.
And then you can choose this r and negation of r also to derive at the false conclusion that is
another resolution refutation proof. So there can be multiple resolution refutation proof or
proving the validity of the same argument you just have to come to the conclusion ϕ if at all it is
possible well if you cannot come to the conclusion ϕ even after repeatedly applying or resolving
pair of clauses that shows your argument form is not valid.
So in this case the argument form is valid, the clausal form of the argument form is valid that
shows that the original argument form is also valid. So that brings me to the end of this lecture
the references used for today's lecture are the chapters in the Rosen’s book and just to summarize
in this lecture we have introduced the resolution refutation proof strategy, which is based on the
resolution inference rule.
The resolution inference rule can be considered as an equivalence form of proof by cancellation
where you pick two clauses where you have a literal available in positive form in one of the
clauses and negative form in other clause. You can cancel both literals and whatever is left in
both the clauses the conjunction of that you can consider as the resolvent or the conclusion of the
two clauses which you have simplified.
This is a very powerful proof mechanism, which is very extensively used in programming
language PROLOG, thank you.
69
Lecture -06
Tutorial 1: Part I
Hello everyone welcome to the first part of tutorial one, so let us start with question number
1.
So in this question the goal is the following. You are given two propositional variables p and
q representing the propositions you drive over 65 miles per hour and you get a speeding ticket
respectively. Then your goal is the following using this two propositions p and q you have to
represent the following statements and by compound propositions using logical connectives.
So the first statement that we want to represent here is you do not drive over 65 miles per
hour, which is very simple this statement is nothing but negation of p because p represents
the statement you drive over 65 miles per hour, so you want to represent the negation of that.
The second statement is that you want to represent here is you will get a speeding ticket if
you drive over 65 miles per hour, so this is some form of if-then statement. This is the if part
this is the conclusion. So the if part here is if you drive over 65 miles per hour which is p and
the conclusion here is you will get a speeding ticket.
70
That is why this statement will be represented by p → q. The third statement that we want to
represent here is you drive over 65 miles per hour only if you will get a speeding ticket. So
this is a statement of the form only if, so recall p → q also represents p only if q, or
equivalently q is necessary for p. These are the various forms for p → q. So this condition is
the necessity condition here.
So you can write it, you can write the statement either in the form of ¬ q → ¬ p. Why ¬ q →
¬ p? Because remember when I say only if part that means whatever is there after only if part
if that condition is not satisfied, What is that condition, you will get a speeding ticket, which
is your statement q. So the equivalent form of only if is that if that negation of that thing
happens, then whatever is there before only if that does not happen.
And what is there before only if is the statement that you drive over 65 miles per hour, which
is your p. So that will not happen. So that is why this statement can be represented by ¬ q →
¬ p and remember that the contra positive of an implication is equivalent to the original
implication. So what will be the contra positive of this implication?
So it will be negation of negation of p implies the negation of negation of q and if you take
negation inside you get p → q. The last statement we want to represent here is driving over
65 miles per hour is sufficient for getting a speeding ticket. That means whatever is there
before your sufficient part that is your if statement. If you ensure that then whatever is
thereafter sufficient that will be ensured that will happen. So this is equivalent to p → q. So it
is a very simple straightforward question.
71
In question 2 the goal is the following, you have to write down the converse contrapositive
and inverse of the following statements. So just to recall, if you are given an implication p →
q then the contrapositive of that is ¬ q → ¬ p, the converse of that implication is q → p and
inverse of that implication is ¬ p → ¬ q, that is the definition.
Now the first statement here is if it snows today I will ski tomorrow. So this is your p part
and this is your q part. The statement then is not explicit sorry the word then is not explicitly
given here but it is present implicitly here. So this is your p → q part. So the contrapositive of
this will be ¬ q → ¬ p and ¬ q will be if I do not ski tomorrow, ¬p will be it did not snow
today.
The converse will be q → p; that means if I ski tomorrow then it snows today, the inverse
will be ¬ p → ¬ q. So ¬ p means it does not snow today ¬ q means I will not ski tomorrow.
The second statement here is a positive integer is a prime only if it has no divisors other than
1 and itself. So let us first identify the implication here. What is the form, what is the
implication here in what?
So this is an only if statement. So you can represent whatever is there before only if as p
whatever is there after only if as q and the implication that is here is p → q. The
contrapositive will be ¬ q → ¬ p. So the negation of q will be the number has a divisor other
than one and itself because q means it has no divisors. So negation of q means, it will have a
divisor other than one and itself. And p is the number is a prime so negation of p will be it is
not a prime, so straight forward.
72
Similarly the converse will be q → p so q is the number has no divisors other than one and
itself, p is the number is a prime. The inverse will be ¬ p → ¬q so ¬ p means the number is
not prime. If the number is not prime then we have to conclude ¬ q, ¬ q means it will have a
divisor other than one and itself. So that is how you write the contrapositive of converse and
inverse of implications.
The question 3 is asking you to do the following. You are given, a set of compound
propositions and you have to draw the truth table it is a very straightforward question here.
So the first compound proposition is this conjunction of two implications. What you have to
do is it is a compound proposition involving two variables p and q so you have one column
for all possible values for p one column for all possible values of q and then what I am doing
here is for simplification, I am separately writing down the column for p → q. I am separately
writing down the column for ¬ p → q and then finally I am separately writing down the
column for conjunction of these two things namely p → q and ¬ p → q. So let us first begin
with the column for p and q. I have written down the various possible truth assignments that p
and q can take.
Based on this, the column for p → q will be this so remember p → q takes the value false
only when p is true and q is false. For all other possible assignments p → q is always true
now the next column is for ¬ p → q and the column will take these values. So let us focus on
the entry when this statement is false. So this statement will be false when LHS is true, but
RHS is false.
73
That means your q is false and p is false. Because if p is false then this negation of p will
become true, and that is why this true implies false will lead to the value false for all other
three combinations, this implication will always take the truth value as true. Now I have the
columns for p → q and negation p → q what I have to do is I have to take the conjunction.
And remember conjunction of two things two variables will be false if any of them is false.
So the truth value for the conjunction will be as follows, since I have true here and true here
and true is true, but I have false here and true here so conjunction will be false. I have true
here and true here, they are conjunction will be true. I have true here and false here and the
conjunction will be false.
That is how you can build a truth table for this compound proposition. The second compound
proposition for which I am supposed to draw the truth table is this, it involves several bi-
implications. So I can do the similar stuff which I have done for the first part. I will be
separately writing down the column for p. I will be separately writing down the column for q
and I will be separately writing down the columns for each individual portions of this
compound proposition, and this will be the final column, which I am interested in.
So the columns for p and q are straight forward based on the truth values that I am giving to p
and q the column for this bi-implication, the first bi-implication is this. So remember bi-
implication is going to take the truth value true if both the LHS and RHS have the same truth
value. If both of them are false or if both of them are true the bi- implication will take the
value true.
So that is why wherever you have a mismatch, the bi-implication takes the value false;
otherwise the bi implication takes the truth value true. Similarly you can draw the column for
the p ↔ q, wherever there is a mismatch you get the false value otherwise it is true. And then
you have to take the bi-implication of this column and this column. Wherever they are
matching, they will take the truth value. In fact, they are matching at all positions all the four
positions and that is why it will be always true.
74
Now, the fourth question is the following. You are given three propositional variables p, q, r
denoting these three statements. And using p, q, r and various logical operators, connectives,
you have to write down compound propositions representing some given English statements.
So the first statement is the user has paid the subscription fee but does not enter a valid
password.
So you might be wondering what will be the logical connective operator for representing
‘but’. So the but here should be treated as some form of and or conjunction here. That means,
you can equivalently pass the statement saying that user has paid the subscription fee and he
has not entered a valid password. And I know that the variable r represents the statement user
has paid the subscription fee.
The variable p represent the user enters a valid password, but I want to represent here, denote
here, state here that he has not entered a valid password, that is why it will be ¬ p. The second
statement is access is granted whenever the user has paid the subscription fee and enters a
valid password. So let us parse first this statement. So remember ‘whenever’- is a form of
Implication.
That means this part is going to be a premise and this is the conclusion. So that means if this
is a form of if-then, so whatever is there after whenever that is the if-part, if that thing is
ensured then whatever is there before whenever that is the conclusion, that you can conclude.
Now in the premise here a conjunction is involved because my premise consists of
conjunction of two things.
75
And now you have all the details to write this statement in the form of a compound
proposition, so my premise here is the conjunction of two things, the user has paid the
subscription fee which is denoted by variable r and he has entered a valid password which is
p. That is the conjunction of these two things is the premise if both these two things are
ensured the conclusion that I can draw is that access will be granted and access is granted
means q.
The last statement is the user has not entered a valid password but has paid the subscription
fee then access is granted. So this is a form of if-then statement. Whatever is there after then
that is the conclusion whatever is there before then that is a premise, but in the premise you
again have an occurrence of ‘but’ and remember but is nothing but conjunction. So you can
represent this statement in this form.
The premise is the conjunction of two things namely user has not entered a valid password,
so user has entered a valid password is p, so he has not entered a valid password is ¬ p. But
he has paid the subscription fee that means, the statement that he has paid the subscription
fees will be represented by r and as I said ‘but’ should be treated as a form of conjunction so
premise is ¬ p conjunction r.
The conclusion that I want to draw here is that in that case access is granted. Access is
granted is denoted by q.
76
So now let us consider the first part of question 5. I would not be discussing the second part
of the question 5. I will be just discussing the first part the second part I am leaving for you as
an assignment. So the question basically says the following that you are given a set of
statements and you have to verify whether the set of statements constitute a consistent system
specification and what exactly I mean by a consistent system specification?
Well by that I mean that is it possible to simultaneously satisfy all those conditions that
means is it possible that each of the individual conditions simultaneously hold. Or logically
the conjunction of all this specification should be satisfiable. Because if the conjunction of all
those specifications is satisfiable that means there is some truth value which you can assign
which can satisfy each of those individual specification.
So what are the specifications that are given to you? You are given here 5 statements, 5
specifications. The first thing we have to do here is we have to convert this thing into an
abstract argument form or sorry in the form of compound propositions. So for that we will
introduce variables to represent the various statements here. So let p represent the statement
the system is in multi user state and let q represent a statement it is operating normally.
So the first statement is a form of if and only if, that means bi-implication. So the first
statement can be represented as p ↔ q then the second statement or the specification requires
two new variables q and r. So let q represent the statement proposition the system is operating
normally and let r represent the proposition the kernel is functioning. Then the second
statement is a form of if-then statement which can be represented by q → r.
Then the proposition kernel is not functioning can be represented by ¬ r because I have
already used the variable r to represent a statement the kernel is functioning. So the kernel is
not functioning will be represented by negation of r but I have to introduce a new variable to
represent proposition the system is in interrupt state because I have not encountered it earlier.
And this is a disjunction because it is an or statement. So this statement can be logically

represented as distinction of ¬ r and s. I do not need any new variable for the fourth statement
because I have already used a variable p to represent the proposition system is in multi user
state, so system is not in the multi user state will be represented by ¬ p.
77
And I have already used variable s to define, represent a proposition the system is in an
interrupt mode. So the system is in interrupt mode will be represented by s and this is an if-
then statement so it will be ¬ p → s. And the last statement is the system is not in the
interrupt mode. So system is an interrupt mode is represented by s. So this statement will be
represented by negation of s.
So now I have to verify whether all this 5 compound propositions can be satisfied
simultaneously. So if at all negation of s has to be true, that means the variable s should be
false. Now if variable s is false and I want this implication ¬ p → s to be true that is possible
only if p is true otherwise I will not be able to satisfy the statement ¬ p → s.
Now since s is false in order to satisfy this disjunction namely the disjunction of ¬ r and s, my
r has to be false because if r is true and s is also false, then this disjunction can never be
satisfied, so my r has to be false. Now my r is false then in order to satisfy this proposition q
→ r, my q has to be false because if q is true and r is false then this proposition q → r can
never be satisfied.
And now if my q is false, then this bi-implication p ↔ q can be satisfied provided, p is false.
But here is a contradiction. In order to satisfy ¬ p → s, my p should be true but in order to
satisfy p ↔ q, my p should be false. But p cannot simultaneously take the value true as well
as false that means I can conclude that there is no possible truth assignment for p, q, r and s,
which can simultaneously satisfy or ensure that all the five statements here are true that
means this system specification is not consistent. You cannot have a system where all this
five conditions simultaneously, hold. The similar exercise you can do for the part b of
question five. You are given a set of statements about a system namely a set of specifications
and you have to convert those specifications into compound propositions and then you have
to check whether the conjunction of those compound propositions is satisfiable or not.
78
Ok now let us go to question 6, we will discuss part a and part b of question 6. Part a of the
question is we have to verify whether the conjunction of p → r and q → r is logically
equivalent to the implication p or q → r. So again you can use truth table method you can
draw the truth table for that LHS part here, you can draw the truth table for this RHS part and
then check whether both the truth tables are same or not.
We will not do that we want to apply various identities, rules of inferences and so on. So
what we will do is we start with the LHS namely p → r conjunction q → r. Somehow I will
try to bring it into my RHS part. So what I can do is I can replace this p → r by ¬ p or r
because I know that p → q is equivalent to the disjunction of ¬ p and q.
And the same rule I can apply here for converting q → r into a disjunction of ¬ q and r. Now
what I can do is. I can apply the distributive law and simplify the conjunction of these two
clauses I can bring it into this form. Because if indeed I apply the distributive law the
disjunction goes inside and r also goes once with ¬ p and once with ¬ q. Now what I can do is
I can apply the De Morgan’s law and write this conjunction of ¬ p and ¬ q in the form of
negation of this whole disjunction.
And then again, I can apply this law namely, ¬ p or q is equivalent to p → q here. So you can
imagine that, this whole thing is some s and r. So this is the form of ¬ s or r and this is
equivalent to s → r and then you can substitute back s to be p or q. And what is this? This is
nothing but your RHS that means starting with LHS by keep on simplifying it we can convert
79
it into our RHS form and hence I can conclude that my LHS and RHS are logically
equivalent.
The second part of this question is we have to verify whether (p → q) → r is logically

equivalent to p → (q → r). So I have explicitly added the parenthesis here. Because the
parenthesis says in what order the implication is going to be applied. It turns out that these
two statements are not logically equivalent and we can prove it by a counter example namely
we can give we can demonstrate a truth assignment when for that particular truth assignment
the two statements take different truth values.
There could be many such counter examples, if at all you want to show that the two
statements are not logically equivalent, even if you show one of the counter examples that is
sufficient. So the counter example or the truth value that I will demonstrating here is when p
q and r all takes the truth value false. In that case, the left hand side part namely p → q → r
will be considered as false implies false and then that implies false.
But false implies false is true and true implies false is false. So that means for the truth
assignment that I have considered here, this implication is going to take the value false. Now,
what about this implication; this implication is p → q → r. So p is false and q → r is this false
implies false. The false implies false is true and false implies true is true.
That means what I have demonstrated here is that for the truth assignment when p, q and r are
all false the two expressions have different truth assignments or truth values. One of the
expressions takes the value false the other expression takes the value true and hence they
cannot be logically equivalent because as per the definition of logical equivalence both of
them should have the same truth value.
So I am leaving the other parts of question 6, you can verify similarly whether the two
statements are logically equivalent or not?
80
So let us go to question 7, in question 7 we are defining a concept which we call as the dual
of a compound proposition. Dual of a compound proposition is denoted by this notation s*
and what exactly is the dual? How do we construct a dual of a compound proposition? What
we have to do is wherever we have an occurrence of conjunction in s we replace them by a
disjunction.
Wherever there is a disjunction we replace them by a conjunction. Wherever there is an

occurrence of the constant true, we replace them by constant false and wherever there is an
occurrence of false I replace them by constant true. If you apply these 4 rules throughout the
expression s then the resultant expression that you obtain is called as s*. So the first part of
the question is you are given some statements and you have to construct their duals. So here
is one of the compound propositions.
So what I have to do is remember while forming the dual I do not change the literals, the
literal remains in their original form. I just have to change the conjunctions and disjunctions
and the constants. So this distinction becomes conjunction and this disjunction becomes
conjunction that is all. This will be the dual. In the same way the dual of the second statement
will be this disjunction goes to conjunction, this disjunction goes to conjunction and this
conjunction goes to disjunction.
The third statement you have now some constants also involved. So the conjunctions and
disjunctions are converted vice versa and now you have false getting converted to true and
true getting converted to false, that will be the dual. Now the b part of question 7 was you to
81
ask the following it says when is it possible that the dual of the statement is exactly equal to
the original statement?
I stress I am asking here exactly equal that means structurally, formula wise it is exactly the
same proposition as the original proposition I am not saying logically equivalent and answer
is very simple its, s* will be equal to s only when s is a single literal and that too different
from the constants true or false, why so? Because if the compound proposition s has any
occurrence of conjunction, disjunction, constant true, constant false then when you form the
dual of that statement you will get a different expression.
Because all the conjunctions will be replaced by disjunctions, disjunctions by conjunction,

true by false, false by true and so on. But if your statement s is, a statement just a single
literal say p or say if s is equal to ¬ p then in that case s* will be same as your original s. For
all other cases s* can never be equal to s. The third part of question 7 ask you to show that if
you take the dual of a statement and then again take its dual you will get back the original
expression and it is very simple.
Because if you take the dual twice then when you are taking the dual first time all the
disjunctions get converted to conjunctions and then again when you take the dual second time
conjunctions are converted back to disjunctions. The same thing happens for conjunctions
when you take the dual first time they get converted to disjunctions and again when you take
the dual second time they get converted to conjunction.
Similarly if you have any occurrence of constant true first time when you form a dual they
get converted to constants false and again when you take the dual they get back to constant
true and similarly for false this is very straightforward.
82
The last part of the question 7 is the following. You are given two compound propositions P
and Q and they are logically equivalent it is given to you and it is also given that P and Q
contains only conjunction, disjunction and negation there is no occurrence of implications
and bi-implication. No occurrences of implication, bi-implication. In that case, we have to
show that the dual of P and dual of Q are also logically equivalent.
So here is how we can prove that. So since P and Q are logically equivalent their negations
also will be logically equivalent because if the negation of P and negation Q are not logically
equivalent that means they take different truth values then how can it is possible that P and Q
are logical equivalent. When I say P and Q are logically equivalent that means they take the
same truth value.
Whenever P is true Q is true whenever P is false Q is false. That is a definition of logical

equivalence. Now what I can do is let me apply the De Morgan’s law both to the expression
(¬ P) as well as to the expression (¬ Q). In this process what will happen is the following : all
the occurrences of disjunction gets converted to conjunction all the occurrences of
conjunction gets converted to disjunction all occurrences of constant true get converted into
false and similarly all occurrences of false get converted into constant true.
This will be effect of applying the De Morgan’s law and remember the ¬ P is logically
equivalent to ¬ Q then even after applying the De Morgan’s law to ¬ P and ¬ Q the resultant
expressions will remain the same because I am not doing anything fancy here. I am just
83
applying some standard identity even after applying De Morgan law, ¬ P will be equivalent
to ¬ Q.
But the next thing that I do is, in ¬ P and in ¬ Q each occurrence of the atomic proposition pi
or a literal pi is replaced by negation of that literal. That means wherever you have an
occurrence of small p, in p and q you replace them by ¬ p and you do this both in negation of
P as well as in negation of Q, right. In the same way wherever you have an occurrence of
negation of propositional variable p in this expressions negation of p and negation of q you
replace them by positive p.
Now, if you closely observe what we have done is by applying the De Morgan’s law and by
replacing small p by this negation of small p and by negation of small p by positive p we
have obtained expressions which are P* and Q* and throughout this process the logical
equivalence of my LHS part and RHS part is maintained. Because I started with two logical
equivalent statement P and Q their negations will be logically equivalent.
Then individually if I apply the De Morgan’s law in the ¬ P part and ¬ Q part the resultant
expressions will be still logically equivalent and now if I apply this substitution of replacing
each atomic proposition by its negative form simultaneously in the left hand side part and in
the right hand side part the resultant expressions will be still logically equivalent. But in this
whole process starting with P and Q, I have got down P to P* and Q to Q* and the logical
equivalence of both the LHS part and RHS part is retained.
And that shows that if you start with two logically equivalent statement and if you take their
duals, their duals also will be logically equivalent and this is a very powerful result because
what it says is that if you have some well known identity established with respect to P and Q
then the same law is applicable even for P* and Q* namely if you have proved a law say law
X. I do not know what, law X could be anything.
This says that hey expression P is equivalent to expression Q then you can get another form
of the law X where you say that P* is equivalent to Q*. You do not have to separately write
down as law X.2 or law X.3 for the dual part. It comes automatically for free because of this
result that the dual of two equivalent statements are also equivalent. So with that, I conclude
the first part of tutorial 1. Thank you.
84
85
Lecture -07
Tutorial 1: Part II
Hello everyone, welcome to the second part of the first tutorial.

So we will start with question 8, in question 8 we are defining a functionally complete set of
logical operators, if you are given a set of logical operators, we say it is functionally complete, if
every compound proposition can be converted into a logically equivalent proposition involving
only the logical operators, that is given in your collection. So the first part of question 8 asks you
the following.
So here we want to prove that the set of these three operators is functionally complete. That
means any compound proposition you can represent just by using these three operators, so how
do we prove this? Well if I am given a proposition which indeed involves only these three
operators, I do not have to do anything. But what about a compound proposition where I have an
occurrence of implication?
In that case what I can do is I can use this logical identity that p → q is logically equivalent to
86
negation of p disjunction q and I can substitute p → q by this RHS expression. And by applying
this rule repeatedly wherever I have an occurrence of implication I can remove those
implications and I will now have an equivalent formula where everything is represented only in
terms of conjunction, disjunction and negation.
What if my expression has bi implication (↔) symbol? I do not have to worry, what I have to do
is I can use the identity that the bi implication is nothing but the conjunction of two individual
implications and I know that each individual implication can be replaced by these two sub
expressions. So now you can see that my original expression is converted into an expression
where every operator is either conjunction, disjunction and negation.
So that shows that if you have these three operators namely a conjunction, disjunction and
negation, you can represent any statement, any compound proposition and hence this is a
functionally complete set of logical operators. Now the second part of the question says that I do
not need both conjunction and disjunction to be there. It is sufficient if I just have a disjunction
and negation operator and I can represent every statement.
So what I have to do is from the first part of the question, I know that if you have an occurrence
of implication you can represent them by just using negation and disjunction, this is what we
have shown. What we have to now worry about is how do I represent even a conjunction, namely
a proposition where conjunction is involved by an equivalent proposition where I have just
occurrences of disjunction and negation.
And this is how we can prove that imagine you have an expression of the form conjunction of p
and q. This is logically equivalent to negation of negation of p, conjunction negation of negation
of q and then by De Morgan's law, this is nothing but equivalent to negation of this entire
expression, namely disjunction of ¬ p and ¬ q. So now you see that even if you have an
occurrence of ‘and’, in your expression you can substitute that expression by another expression
where you have only occurrences of negation and disjunction.
And which shows that just your negation operator and disjunction operator are functionally
87
complete. You can represent any statement. The third part now says that you have to show that
only the negation operator and the conjunction operator are functionally complete and here we
have to show how we can represent a disjunction in terms of conjunction and negation. And
again, we have to do similar work which we have done for the previous part, I can represent any
disjunction in this form where I have just occurrences of negations and conjunction.
So that shows that just two operators either conjunction along with a negation or disjunction
along with negation is sufficient to represent any expression that you are interested in.
Question 9 asks you to show whether this long expression long compound proposition is
satisfiable or not. So what I do here is this expression is already in its conjunctive normal form
and what I have done here is I have written down the various clauses that are involved in this
compound proposition. And I have to worry and I have to think that how is it possible that I can
simultaneously satisfy all these six clauses.
Well, if this expression is satisfiable then there might be many truth assignments which can
satisfy all the six clauses, our goal will be to find at least one of them, so let us try to do that. So
what I do here is if I ensure that r is true then that will ensure that my clause C1 will be true, I do
not worry what is p and q. If I ensure r is true the disjunction of r with everything will be overall
true.
88
And if I assume r to be true, what happens here is if I go to clause number 4 here, if r is true, then
negation of r will be false. Negation of r will be false then what I have to do is in order to satisfy
clause number 4, I have to make negation of p to be true or negation of s to be true. So let me
make negation of p to be true for that I have to ensure that p is false because if I ensure p is false
negation of p will become true then overall this expression C4 will become true; that is why I
have put tick mark here; that means this clause will be satisfied now.
And due to the same truth assignment p equal to false the clause C2 also will be satisfied because
I have an occurrence of ¬ p. That means with r equal to true and p equal to false, I will be able to
satisfy clause number C1, clause number C2, clause number C4. Now my negation of r will be
false because I am assuming here r is true and I am assuming p to be false, then to satisfy clause
number 6, I am left with only one option.
Namely I have to ensure negation of s is false, then only clause 6 can be satisfied and negation of
s is false means, so what I am doing here is I am not trying to satisfy clause 6 as of now. I am
trying to satisfy clause number C3 first and to satisfy clause number C3 what I observe here is
that p is already taking the value false here and q is not taking any value as of now, I have not
assigned any value to q.
But what I observe here is that if I ensure that negation of s is true, then clause C3 will be
satisfied and negation of s is true means s is false. And if I ensure s is false, my clause 6 also gets
satisfied because I have an occurrence of ¬ s, I do not have to worry what is p and ¬ r at all. That
means assigning these values to r, p and s, I am able to satisfy all the clauses except clause C5.
Clause C5 is not yet satisfied because I have assigned p to be false and I have assigned r to be
true, so the negation of r are also false. So the only way I can satisfy clause number 5 is I give
the value true to q in clause number C5 and that means I have found at least 1 truth assignment
which can satisfy all these 6 clauses. So now you can see that if your expression is in its
conjunctive normal form, you can run this mental algorithm, you can try to individually satisfy
each clause at a time and try to come up with a truth assignment which can satisfy all the clauses.
89
So in question 9a, there are some other expressions also which are given to you and you have to
verify whether they are satisfiable or not, I am leaving that for you. So let me go, sorry this is not
9b, this is question number 9a.3, sorry for this numbering. So the question basically asks you to
show the following: it says that imagine you are given an algorithm which can check whether a
compound proposition is satisfiable or not.
You do not have to worry about the details of that algorithm. Imagine a box is given to you, you
feed some compound proposition to that box and it gives you and yes no answer. Now using that
algorithmic box, you have to come up with another algorithm which should tell you whether any
input that you feed to that algorithm is a tautology or not. So we first prove a very simple fact
here regarding satisfiable statements and tautology.
The claim here is that if you are given a compound proposition X then it is a tautology if and
only if negation of X is unsatisfiable and it is very simple, and this is an if and only if statement.
So, let me prove it that if X is a tautology that is always be true, then what about the negation of
X the negation of X can never be satisfied, you can never find a truth assignment which will
make negation of X true because if negation of X is also true and X is also tautology.
Then this is not possible simultaneously. On the other hand you assume that if negation of X is
90
unsatisfiable then I have to prove that X is a tautology, we prove it by contrapositive. So,
showing that negation of X is unsatisfiable implies X is tautology is equivalent to showing that if
X is not a tautology the negation of X is satisfiable. Our goal is to show p → q and this is
equivalent to showing ¬ q implies ¬ p.
So if I want to show p → q it is equivalent to if it is enough if I showed ¬ q →¬ p and what is

negation of q? Negation of q is X is not a tautology and what is negation of p? That negation of
X is satisfiable and indeed this implication that if X is not a tautology then negation of X is
satisfiable is a true implication. Because if X is not a tautology it means it is not the case that X
is always true.
That means there is one truth assignment for which X is false. For that specific truth assignment,
what about negation of X? For that specific truth assignment negation of X will be true because
for that assignment X was false. So that proves the implication of this theorem statement in the
other direction as well and that is why this is a condition. So now we can utilize this theorem to
get an answer for our question.
So as I said earlier you are given an algorithm I call that algorithm as Asat, which takes a
compound proposition and it gives you yes no answer. It gives you an answer one if X is
satisfiable, it gives you the answer zero if X is not satisfiable. Using this I design another
algorithm which I call as algorithm tautology Atau, which will take some compound proposition
and it will give me an answer yes if X is a tautology otherwise it will give me an answer no if X
is not a tautology.
And I am allowed to use this existing algorithm Asat. What I am going to do is my algorithm A
tautology will do the following: it will first find a negation of my input X. So I call that
expression as Y and I give the compound proposition Y as an input to my algorithm Asat. The
algorithm is that will give me a yes no answer. It will give me the answer 1, if Y is satisfiable it
will give me the answer 0 if Y is not satisfiable.
What I have to do is, I have to use this response that I am getting from an algorithm Asat to
91
decide the outcome of the algorithm Atau. And my output is the following, if my Asat says that Y
is satisfiable, I will say that X is not a tautology, whereas if Asat says that Y is not satisfiable, I
will say X is a tautology. That means I will just give the reverse answer, opposite answer which I
got with respect to the expression Y from the algorithm Asat.
And this is because of the theorem statement which we have just proved now. We have proved
that if Atau X was a tautology then Y will be unsatisfied. That is why I am just flipping or
complimenting the bit or the response which I am getting for the expression ¬ X from the Asat
algorithm. What will be the running time of the algorithm Atau? The running time will be almost
the same as your algorithm Asat, plus the running time that you need to convert your expression
X into expression Y.
That means if your algorithm Atau is going to take 1 hour and converting expression X to
expression Y takes the 10 minutes then the running time of Atau will be 1 hour 10 minutes, you
are almost proportional to the running time of Asat.
In question 10, you are given a set of premises and a conclusion and you have to verify whether
this is a valid argument or not. So what we first do is we convert statements into propositions, so
I introduce the variable p here and this is a simple proposition, then the second statement to
represent that I introduce another variable q because Randy works hard is already represented by
92
p and the second statement will be represented by then p → q.
For the third statement I need another variable r here to represent a Randy will not get the job.
And then the third premise q → r, the conclusion that I am drawing is Randy will not get a job.
The argument from here is very simple, you are given three premises and a conclusion is r. Let
us see whether this argument form is valid or not, so what I do is I apply Modus Ponen on the
first two statements here.
The first two premises here and come to the conclusion q. And then I apply again Modus Ponen
on q and third premise and draw the conclusion r. That means this is a valid argument form, a
valid conclusion because I can draw the conclusion from my premises.
Now in question 11, you are given the following, you are given that this argument form is valid
where you are given a set of n premises and (n + 1)th premises is q and the conclusion is r. Now
you have to show if this is the case then the argument form where only p1 to pn are the premises
and the conclusion is q → r is also valid. Again, there are several ways to do this, you can use a
truth table argument and so on, we will avoid that.
Since we are given that this argument form is valid as per the definition of valid argument, I can
say that conjunction of p1 to pn and q → r is a tautology that means it is never possible that your
93
left hand side is true and RHS is false, that is not going to happen. That means if my LHS is true,
RHS is also true. That means if the conjunction of p1 to pn and q is true then r is also true and as
a result I can say that this implication is also true.
Because if the conjunction of p1 to pn and q is true, that means this part is definitely to this
conjunction of p1 to pn is true. And since q is true here r is also true, then true implies true is
anyhow true and true implies true is anyhow true. But if I closely see here, what does exactly this
implication means? If I say that this implication is always true then another form of the
implication is that you have the premises p1 to pn and the conclusion is q → r.
And if I want to say that this argument form is valid, this is equivalent to asking whether the
(conjunction of p1 to pn) → (q → r) is tautology and that is what we have proved here. That
means if this has been given, this argument form is given to be a valid argument form, then this
new argument form is also valid.
Now, let me go to question 12. I will not be solving part A here, I will be focusing on part B, part
A I am leaving for you. You have to show using resolution whether the following argument is
valid. So, these are the premises here, these are the three premises and this is the conclusion. So
as usual the first thing that we will do is we will introduce propositional variables and bring
everything in the form of compound propositions.
94
So, if I introduce the variables here like this, it is up to you what form of the variable you use. I
am using it is not raining as ¬ r. You could have used r, it is not raining. In that case, it is raining
and becomes negation of r and so on and you can use any name, you can use A, B, C for
propositional variables, just for my convenience that I am using these names. Now if I use these
propositional variables then the argument form here is the following.
And I have to show whether this argument form is valid or not. The first thing I have to check is
whether this argument form is in its clausal form or not, that means everything the premises and
conclusion everything is in the form of clauses or not and in this case yes, this is clause C1, this is
clause C2, this is clause C3 and this is clause C4. What is the resolution refutation method? The
resolution refutation method says that you take the set of clauses which are your premises and to
that you add the negation of your conclusion.
Remember we have to check whether s union negation of the conclusion if the resolvent of this
thing is false or not if that is the case then I say that C is a logical conclusion from my set of
premises in the set s, that is what is the proof by resolution refutation, so I have added a negation
of the conclusion and now I have to resolve. So, I take the first two classes here, I cancel out u
here and I get a resolvent disjunction of ¬ r and the ¬ w.
Then I choose the resolvent and the next clause here and I cancel out r and r here and I get
negation of w and now if I take negation of w and w I cancel them and get false. So since I am
getting the resolvent to be empty that means this is a valid argument.
95
So now let us next go to question number 13 and in question number 13, we have to use
resolution to show that the following compound proposition is not satisfiable and as per the
properties of resolution basically to show that the conjunction of these clauses is unsatisfiable. I
have to show that the constant F belongs to the resolvent of the above clauses. So let us build a
resolvent or resolution tree for this set of clauses here.
So I can pick the first two clauses and resolve p and then I can pick the last two clauses and I can
again resolve p. So the resolvents are now added to the tree and now I can pick these two
resolvents for resolving and I obtain the conclusion, the constant, F which shows that the
conjunction of these four clauses is not satisfied. So that is how you can actually prove whether a
kind compound proposition is satisfiable or not you have to bring everything in the clause form
and then build a resolvent, resolution tree and then arrive at the constant F.
96
Let us go to the last question for the first tutorial. So again, you are given a set of English
statements and you have to verify whether it is a valid argument or not. So what I do here is as I
am repeatedly doing it I will be introducing propositional variables and converting each
statement into some compound proposition, you are free to use any variable name here, I am just
using these variables for my convenience.
After converting everything, this is the argument form here and I am going to use proof by
resolution refutation because I find it very comfortable because we just have to keep on
canceling clauses here. Cancelling literals in two clauses and keep on doing the simplification till
you either get an empty conclusion or you cannot resolve further. You do not have to worry
about logical identities, De Morgan law etc etc. That is why resolution refutation is a very very
powerful proof mechanism.
So these are your set of premises converted into their equivalent clauses, so in this case I have to
convert some of the premises into their corresponding clause form or cnf form. In fact I have to
convert everything because none of the premises are available in their cnf form except this ¬ p.
Negation p is available in its cnf form; everything else has to be converted, after converting I
have added all the premises, so that is my set of clauses or premises s. And what is the
conclusion?
97
The conclusion that I am trying to draw is negation of e which is already in its cnf form but I
have to add the negation of C in the resolution tree to do the resolution refutation proof
mechanism. And now I have to resolve, so I start with the first two things and cancel out p and
then I cancel out a, then cancel out w, then cancel out i, next I cancel out m and then after
canceling out e I have left with nothing, empty.
So I got a resolvent to be empty and that shows that this is a valid argument. Well you could
have shown that this argument form is valid by using simplification, rules of inferences, Modus
Ponen etc etc not stressing that you have to only use resolution refutation. Just that I find the
resolution refutation to be a simpler proof mechanism. And again the tree that I have constructed
here need not be the only tree when you are building the resolution, when you are doing the
resolution refutation proof.
You might pick the pair of clauses to resolve in any arbitrary order. It is just that some trees lead
you to the empty conclusion very soon; some trees might lead you to the empty conclusion after
a long time. So depending upon which two clauses you cleverly use at each stage that will
determine how fast you reach to the empty conclusion that is all. So with that, I conclude the first
tutorial. Thank you.
98
Lecture -08
Predicate Logic
Hello everyone welcome to this lecture on predicate logic.

Just to recap, till now we had extensively discussed propositional logic how do we form
compound propositions, how do we verify whether argument forms are valid, various ways of
verifying argument forms are valid or not using rules of inferences and so on. The plan for this
lecture is as follows: in this lecture we will discuss predicate logic and its motivation, we will
discuss about quantification mechanisms and we will discuss logical equivalence in predicate
logic.
99
So let us start with the motivation of studying predicate logic, so it turns out that even though
propositional logic is very interesting it cannot represent all kinds of mathematical statements
that we are interested in. So for instance consider this declarative statement at x is greater than 3,
which is a declarative statement but it is not a proposition because until and unless the value of x
is assigned, we cannot decide whether the statement is true or false.
Whereas, the definition of proposition is, it is a declarative statement for which we know that it
is either true or false but not simultaneously it cannot take the value true or false. So that is not
happening for the first statements of this form. So the question is how exactly we can represent
statements of this form because in general when we are writing mathematical theorems and
statements, very often want to characterize properties about arbitrary sets, arbitrary domains
without worrying about the underlying values which are going to be taken in the domain and so
on. So for that we need predicate logic, so it turns out that there are two parts of the above
statement which we are interested to represent: the first part is the variable x which is the subject
of the statement, because in this statement we are trying to say something about the property of
x.
We want to say that x is some value which is greater than 3, we do not know whether that is true
or not because that depends upon the exact value that x is going to take. But we want to
characterize the property that whatever is the value of x it is greater than 3 or not. So greater than
100
3 is the property for the subject x. So the way we represent these statements in the predicate logic
is we introduce a function which I call as a predicate function.
And this predicate function typically they are represented by capital letters and the choice of
exact capital letter is up to you. You can use M, A, B, C anything and why we are using capital
letters to differentiate from propositional variables for which we use lower case letters. So the
propositional variables they are represented by lower case letters predicate function variables,
they are represented by capital letters.
So this is a function of variable x, where x can take any value and this P(x) becomes a
proposition when we assign a concrete value to x. So for instance if I assign the value 4 to x, then
the proposition that I obtained is p1, which is 4 is greater than 3. Now the statement 4 is greater
than 3 is indeed a proposition, it is a declarative statement and we know it is a true statement that
means there is no ambiguity about the truth value or the truth status of p1.
Whereas P(3) is another proposition, say p2 which is the statement 3 is greater than 3. Now this is
a proposition because this is the declarative statement and the truth status of p2 is confirmed here,
it is a false statement, there is no more ambiguity left. Now we can see how we can represent or
how we can deal with statements of this form where we have some abstract variable and we want
to state or declare some property about the variable using these predicate functions.
101
It turns out that we can define multi-valued predicate functions right? So the previous example
was for statements where we had only one subject namely the subject x but now you might be
dealing with statements where you have multiple subjects. For example, we want to represent
declarative statements of the form x equal to y + 1 + 3. So this is not a proposition because until
and unless we do not assign values to x and y, we do not know what is the status of the resultant
proposition.
So I can represent this statement by a predicate variable, I use the predicate variable capital P and
it is a function of two variables x and y. When I assign the value 4 to x and zero to y, I get the
proposition say p1 that 4 equal to 0 + 1 + 3 and this is a true statement, this is a true proposition.
So I will say that P(4,0) is true whereas if I assign the value x equal to 3, y equal to 0 then this is
a false proposition.
So now you can see that this predicate variable is acting as an abstract function as a placeholder
and it can represent multiple propositions depending upon what exact values you assign to the
corresponding variables, which are there in the corresponding predicate function. In the same
way you can have three valued predicate functions. So, for example, if I want to represent x
equal to y + z and I can introduce a predicate function P of 3 variables and depending upon the
values which you assign to x, y, z you get propositions, which can be either true or false.
102
So now once we have predicates we will be interested to convert them into propositions because
then only we can apply the rules of inferences, rules of mathematical logic and do something
meaningful with the resultant propositions. It turns out that there are two methods of converting
your predicates into propositions. The method number one is you assign explicitly, manually the
values to your underlying variables, but it is very less interesting.
We are not interested in this method and this is what we were doing till now in all the examples
that I have demonstrated. I have manually assigned the values to my underlying variables and
convert the resultant predicates into propositions. We will be interested in method 2, which we
call as the quantification method. Because we will be interested in representing quantified
statements of the form that something is true for all values in my domain, something is true for
some values in my domain something is not at all true for any of the values for my domain.
So we will be interested in these three forms of quantifications and we will be discussing about
various mechanisms how to deal with such statements and so on. So we will be first starting with
two forms of quantifications; namely the universal quantification and existential quantification.
103
What is a universal quantification? Well whenever we want to assert that a property is true for all
the elements of my domain, then I use universal quantification. So very often you might have
encountered this notation ( ꓯ ) for all x in your theorem statements. You encounter this whenever
we say that some property is true for all integers all real numbers. So very often we use this
notation for all x.
And say we want to say that all integers satisfy something, something, that something, something
is nothing but a predicate function. You want to say that some property P is true for all the x
values in your underlying domain. If that is the case then we use this notation. So this expression
for all x, P(x) is true, if the property P holds for every value x in your domain that means for
simplicity assume your domain has m possible values of x.
Well your domain could be infinite as well say your domain could be the set of integers that is
possible because you might want to assert a property which is true for all integers. But just for
simplicity I am assuming here that my domain consists of m number of elements. Then the
quantification for all x, P(x) is logically equivalent to the conjunction of m propositions, where
proposition P(x1) denotes that property P is true for x1, P(x2) denotes that property P is true for x2
and so on and P(xm) denotes that property P is true for xm.
104
That means if this conjunction on your right hand side, if this is false, then for all x, P(x) is not
true, that will be false. That means even if you find one counter example or a bad witness, your
counter example is nothing but a bad witness, even if you find a bad witness, at least one bad
witness for which the property P fails, I can conclude that for all x, P(x) is false.
So before proceeding further, I would like to stress here on the significance of the domain here.
Whenever we are making quantified statements, it could be any form of quantification, it is very
important that you clearly and explicitly say what is the domain of x. Say for instance I define a
predicate P(x) this, says that x2 is greater than zero. This is a predicate function which will be
true if x2 is greater than 0, it will be false if x2 is not greater than 0.
Now, suppose someone says that, is it the case that for all x, P(x) is true, if someone asks this
question, how will you verify that? Well, it depends upon what exactly is the domain, if the
domain does not include 0, that means you are considering a domain where x can take any value,
except 0, then yes this statement for all x, P(x) is true because the property P namely x2 greater
than 0 will be true for every x except 0.
But the same predicate P(x) will not satisfy the condition for all x, P(x) for all x, P(x) will be
false if zero is included in the domain that means your x can take the value zero. So it is very
important to explicitly mention what exactly is your domain, if you do not specify your domain
105
then universal quantification does not make very sense. The logical meaning will completely
change as soon as you change the domain.
That means for the same P(x) for all x P(x) may become true for one domain, but as soon as you
change the domain for all x, P(x) may become false.
Now, let us go to the next form of quantification which we call as existential quantification and
this quantification asserts that a property is true for at least one element of my domain. Here I am
not interested to state that my property is true for every value in the domain. I want to stress that
it is true for at least one value of the domain, well it might be true for every element of the
domain that I am not worried about, I am interested to assert that it is true for at least one
element of my domain.
And this is represented by this expression there exists x ( ꓱ ), so this notation stands for there
exists. So whenever property P is true for at least for some x in your domain the expression, there
exists x, P(x) becomes true. Again for simplicity assume your domain consists of some m
number of elements, then I can say that there exist x, P(x) is logically equivalent to disjunction of
this m propositions where the first proposition is property P is true for x1, second proposition is
property P is true for x2 and so on.
106
That means if your RHS is false and when can be RHS false? When this disjunction is false and
when can this disjunction in your RHS will be false? When the property fails or the property
does not hold for any of the x values; that means P(x1) is false, P(x2) is false and like that P(xm)
is false, that means for none of the x values in your domain the property P is true, in that case the
statement there exist x, P(x) will be false. But even if at least one of the x values satisfies the
property P then the statement there exists x, P(x) becomes true.
Now let us define what we call as bounded and free variables. So a variable is called a bounded
if there is a quantifier, which is applied on it. So for example if I write an expression of this form
then x is bounded because the quantification there exists is applicable on this x, but what about
this y? This is acting as a free variable here. There is no quantification applicable on this variable
y whereas consider this expression, there exists x, P(x) disjunction for all x, Q(x).
Now question here, am I talking about 1 x or am I talking about 2 x, it turns out there are 2
different variables, the 2 x s are represented by the same x here, which is the common source of
confusion here. So there are two different variables and two different quantifiers are applied on
it. The there exists quantification is applicable on this x which is the subject for predicate P and
the quantification for all is applicable on the second x variable, which is the subject for
proposition Q.
107
So what you can do is to avoid confusion, you can either put x explicitly the brackets to denote
that there exist is applicable on P(x) and for all is applicable on Q(x) or it is recommended to use
different variables, if you do not want to put parentheses to avoid confusion. Use different
variables for different predicate functions, if they are applicable with respect to different
quantifiers. What is the scope of a quantifier?
The scope of a quantifier is that part of the expression over which the quantifier is applicable. So
for instance, if I take this expression, this example; the scope of this quantifier is only limited to
P(x), it is not applicable to for all x, Q(x), no. In the same way the scope of this for all quantifier
is applicable only to this Q(x), it is not applicable to P(x). So that means what is a free variable?
A variable is free if it is outside the scope of every quantifier, right?
So if I take the first expression here, the scope of this there exists a quantifier is this x, but if I
take this variable y, it does not come within the scope of this there exist, because there is no
quantification which is applicable over this y and that is why this y is a free variable here. And if
you have expressions involving predicate functions where you have free variables, then it is
completely ambiguous, you cannot make any meaning out of that.
So now we can define logical equivalences even for the predicate world, we can define
predicates, we can have quantified statements and we can have two different expressions and we
108
can verify whether the two expressions are logically equivalent or not. So for instance if I want
to verify whether the expression in your LHS part and the expression in your RHS part, they are
logically equivalent or not we can verify that.
And, see I have explicitly used parenthesis here to distinguish the quantification of this for all x
and this for all x. So how do we prove logical equivalences involving predicates? Well, what was
our definition of logical equivalence in the propositional world? If you have two compound
propositions x and y we said that x is logically equivalent to y if and only if x bi-implication y is
a tautology.
That was the definition of logical equivalence, that means x and y takes the same truth value,
whenever x is true y is true, whenever x is false y is false. But now we are talking about arbitrary
domains, arbitrary predicate functions and so on. Intuitively, we will say that my expressions
involving predicate functions are logically equivalent if they are equivalent with respect to any
possible domain.
That means it should not happen that the two expressions are equivalent for one domain but there
is some bad domain for which the two expressions are not equivalent. Even if there is one bad
domain for which the two expressions are not equivalent, the overall expressions will not be
considered as logically equivalent expressions. But now the problem here is that in this particular
case I am not specifying what is the domain of x and I cannot do that.
Because whenever I have specified logical equivalences involving predicates, my predicate could
be over any possible arbitrary domain. Because I want to state here that this condition holds, so
the way you can interpret this statement is I want to prove that my LHS and RHS are equivalent.
It does not matter what is the domain of x that is what is the meaning of logical equivalence in
the predicate world.
So you might be wondering that without even talking about the domain, how can I verify
whether the LHS and RHS are true for every possible domain, the way we do it, we say that
because it has some arbitrary domain, that arbitrary domain could be the set of integers it could
109
be anything and my property P, my property Q could be arbitrary properties, I am not explicitly
specifying what are property P and Q.
So I am considering arbitrary domains and arbitrary predicates P and Q and for simplicity I am
taking a finite domain, but whatever I am saying here, you can generalize it to infinite domains
as well. Now our goal will be to show that with respect to this arbitrary domain and arbitrary
predicates P and Q, the LHS expression and RHS expression have the same truth value. If we do
that then that establishes the logical equivalence of this expression, because I am considering an
arbitrary domain for which I am showing the equivalence here.
So let us start with the LHS part here, what was your LHS part? The LHS part says that for all x
conjunction of P(x) and Q(x) is true. Now if you remember for all x, P(x) conjunction Q(x) is
true if the property P and Q is simultaneously true for every x value in your domain that means
this conjunction should be true. Now what I do is, I just shuffle the terms here, I bring all the P
terms together and all the Q terms together.
So remember this P(x1), P(x2) all these are propositions. So what I am saying is that for all x,
P(x) and Q(x) is true, that means P(x1) and Q(x1) is true, the proposition P(x2) and Q(x2) is true
and so on and now I am shuffling around the individual propositions. Now if the conjunction of
all the P prepositions is true that is equivalent to saying that for all x, P(x) is true and the same
way if the conjunction of all the Q propositions are true that is equivalent to saying that for all x,
Q(x) is true and this is nothing but your RHS expression.
In the same way we can prove De Morgan's law involving quantified statements, so for instance
we can prove that if you have a negation symbol outside for all x, you can take the negation
inside and for all become there exist and you take the negation and put it before your predicate
P(x) and the dual property will be if you have a negation before there exist, you can take the
negation inside, there exists become for all and property P is replaced by negation of property P.
You can prove that using the same thing, so you can say that for all x, P(x) is equivalent to P(x1)
and P(x2) and so on. That means negation of for all x, P(x) is nothing but negation of this entire
110
conjunction and then you can apply De Morgan's law of propositions and you can take the
negation inside each of them and the conjunctions get converted into disjunction, so you get
negation of P(x1) disjunction negation of P(x2) and so on.
And since we want to assert that negation P(x1) is true, or negation of P(x2) is true and so on, that
is equivalent to saying that the negation of P property is true for at least 1 value of x in my
domain. Same thing you can do for the second dual form of De Morgan's law. So that brings to
the end of this lecture, the references for this lecture is Rosen’s book.
And to summarize in this lecture, we started discussing predicate logic, we saw the motivation of
predicate logic and we discussed how we can represent quantified statements. We saw two forms
of quantifications namely the universal quantification and existential quantification. Universal
quantification is true when the property P is true for all the values in your domain. Existential
quantification is true when the property is true for at least one value on your domain.
And, we saw some logical equivalences involving statements having predicate functions. Thank
you.
111
Lecture -09
Rules of Inferences in Predicate Logic
Hello everyone, welcome to this lecture on rules of inferences in Predicate Logic.

Just to recap in the last lecture we started discussing about predicate logic, the motivation for
predicate logic and then we saw two forms of quantifications namely existential quantification
and universal quantification. The plan for this lecture is as follows; in this lecture, we will see
how to translate English statements using predicates, then we will see rules of inferences in
predicate logic and then we will discuss arguments in predicate logic.
112
So let us begin with an example where we are given an English statement and we want to
represent it using predicates and we will be encountering this situation again and again where we
will be given English arguments and then we have to verify whether they are logically correct or
not and for that we have to convert those English statements, into the predicate world. So the
example that we are considering here is the following.
I want to represent a statement that every student in course number CS201 has studied calculus.
If you are wondering what is this CS201 well at my institute IIIT, Bangalore the course number
for discrete maths course is CS201 and say my domain is the set of all students in a college. So
since I am considering for instance IIIT, Bangalore, my domain is the set of all students in IIIT,
Bangalore but it could be any domain.
So I want to represent a statement or assertion that in a college every student in course number
CS201 has studied calculus. So, how I am going to represent it using predicates. So the first thing
here is that we have to understand how to logically interpret this statement. So for instance
imagine you have a domain say consisting of three students, well your domain will be very large
but just for simplicity I am assuming here that my domain has three students and say I have class
CS201.
So the property that I want to infer or the fact that I want to represent from this logical statement
113
is the following: I want to say that if say x1, x2, x3, x4 and xn are my students of the college, then I
want to represent here the fact that if x1 has studied or if x1 has enrolled in course number CS201
then he has studied calculus. In the same way I want to state that if x2 has studied or in if x2 has
enrolled for course number CS201, then it has studied calculus.
In the same way I want to represent that, if x3 has enrolled for CS201 then it has studied calculus.
So when I am saying that every student in my domain who is enrolled for CS201 has studied
calculus the interpretation of that is that I am making a universal statement, a universally
quantified statement where I am saying that all for every student x in my domain, if student x has
enrolled for CS201, then student x has studied calculus.
That is what is the logical interpretation of the statement that every student in CS201 has studied
calculus, I am making an assertion about every x from my domain, okay? So now I have to
introduce some predicates here to represent the statement at every student x in my domain if
student x is enrolled for CS201 then it has studied calculus. So, let me first introduce a predicate
here S(x) while you can use any predicate variable but I am using S(x) for my convenience.
And, remember in the predicate world we use variables in capital letters for denoting predicate
functions. So, S(x) will be true if student x has enrolled for CS201 where as S(x) will be false if
student x in your domain has not enrolled for CS201 and let me introduce another predicate here
I am denoting it as C(x) and it will be true if student x in your domain has studied calculus else, it
will be false.
And, I do need these two predicates here because I want to assert or relate properties of a student
x with respect to whether he has studied calculus or not and whether he has enrolled for CS201 or
not. So that is why I have introduced two predicate functions here. Now coming to the question
how do I represent a statement that every student in CS201 has studied calculus? So I am writing
down here two expressions.
One expression is for all x, S(x) → C(x) this represents that for every x in the domain here
domain is the set of all students in my college, if student x has enrolled for CS201, then he has
114
studied calculus, whereas the other expression the right hand side expression here denotes that
every student x in the college has enrolled for CS201 and studied calculus. Now an interesting
question here is whether the statement that I want to represent is represented by the first
expression or is it represented by the second expression?
Very often students do think that it is the second expression which is representing the statement
every student in CS201 has studied calculus but that is not the case.
So, let me demonstrate that why the second expression is an incorrect expression and it is the first
expression which represents the statement every student has studied calculus in CS201. So
consider a college where the student domain has three students Ram, Shyam, and Balram and say
in that college, all the students except Balram has enrolled for calculus; so that is why S(Ram) is
true, S(Shyam) is true and S(Balram) is false.
So remember S(Ram), S(Shyam) and S(Balram), they are now propositions because I am
assigning the values x equal to Ram, x equal to Shyam, x equal to Balram and as soon as I assign
concrete values to my predicate variable, the predicate gets converted into a proposition and say
in the same domain Ram, Shyam, and Balram all of them have studied calculus that means the
proposition C(Ram) is true C(Shyam) is true and C(Balram) is true.
115
Now you can see here that in this domain indeed the assertion that every student in CS201 has
studied calculus is true because you check Ram has studied, Ram has enrolled for CS201 and
indeed he has studied calculus. Shyam has enrolled for CS201 and indeed has studied calculus
but Balram he is not enrolled for CS201 so I do not care whether he has studied calculus or not.
My assertion was that every student in CS201 has definitely studied for calculus or not.
I do not care about the students who are outside CS201; they may or may not have studied
calculus that is not conveyed through this statement. Now, let us consider the two expressions our
goal is to identify whether it is the expression number one or expression number two which
represents the assertion that every student in CS201 has studied calculus.
So if I consider the first expression which is for all x, S(x) → C(x) and if I substitute x equal to
Ram, x equal to Shyam and x equal to Balram then this universally quantified statement gets
converted into conjunction of three propositions. Why conjunction of three propositions because
recall from the last lecture a universally quantified statement is true, if it is true for every x in the
domain.
And, my x in the domain are Ram, Shyam and Balram and it is an implication statement, so it
will be conjunction of three implications. Now, with respect to the truth values that have been
assigned to S(Ram), S(Shyam), S(Balram) and C(Ram), C(Shyam) and C(Balram). In my domain
it turns out that each of the implications is true. Indeed S(Ram) is true and C(Ram) is true, so true
implies true is true.
Now S(Shyam) is true, C(Shyam) is true, so true implies true is also true and S(Balram) is false,
so I do not care whether C(Balram) is true or false, false implies anything is true and the
conjunction of true, true, true is of course true, so you can see that the expression for all x, S(x)
→ C(x) indeed turns out to be true with respect to this domain where the assertion that every
student in CS201 has studied calculus is true.
Whereas consider the expression, second expression, namely for all x, S(x) conjunction C(x). So
if I substitute the different values of x, I get conjunction of three propositions here and each
116
individual proposition is conjunction of two propositions namely S and C. If I assigned a truth
values, if you check the truth values that we have assigned for the proposition S and proposition
C for Ram, Shyam and Balram, it turns out that the first compound proposition here is true
because both S(Ram) and C(Ram) are true.
The second conjunction here is also true because S(Shyam) is true and C(Shyam) is true, but
S(Balram) is false and C(Balram) I do not care whether this true or false because false
conjunction with anything is false and hence the over all expression is false and indeed the
second expression here should turn out to be false here because the second expression here
denotes the assertion that every student of the college has enrolled for CS201 and he has studied
for calculus.
But that is not what we want to assert here, our assertion that we are interested to express is that
if at all a student x has enrolled for CS201 then he has studied calculus. So the summary here is
that even though there is no explicit “if then” statement given here the statement of the form
every student in CS201 has studied calculus has an implicit, it has an implicit, “if then else” form
and the second thing here is that this is a universally quantified statement because I am making a
statement about every x in my domain.
So even though the statement is not given of the form for all students that word for all is not
explicitly given here you have to understand that it is implicitly hidden here and that is why the
quantification that we have used in this predicate is for all x.
117
Let us see another example, so my domain is still the students of my college and I want to
represent the statement that some student in class CS201 has studied calculus and let me retain
the same two predicates S(x) and C(x) from the previous example. So again, we have to
understand whether this statement is universally quantified or is it existential quantified whether
it involves any kind of “if then” or not and so on.
So if you see here closely, it turns out that this statement some student in CS201 has studied
calculus means that I want to represent a fact that for some x in my domain, so I have multiple x
values possible from my domain I want to represent the assertion that for some x from my
domain the x satisfies two properties simultaneously namely the same x has enrolled for CS201
and the same x as studied calculus.
That means the property that x is enrolled for CS201 and has satisfied calculus hold
simultaneously for the same x was from my domain and this is true for at least one x because I
am saying here that it is true for some x I am not saying it is true for all x. So it turns out that this
statement or this assertion will be represented by this existentially quantified statement namely
there exists some x in my domain such that the property S(x) and C(x) are simultaneously true for
the same x.
And, I have explicitly put the parenthesis here because this existential quantification it is
118
applicable both over the predicate S as well as C here. If I do not put the parenthesis here then
you get ambiguity whether x is within the scope of, where the occurrence of x in both S(x) and
C(x) is within the scope of there-exist or not. So that is why to avoid confusion I have explicitly
added parenthesis here because I want to represent the fact that it is for the same x that both S(x)
and C(x) holds simultaneously.
Now an interesting question here is why cannot we represent this assumption by this second
expression there exists x such that S(x) → C(x) might look that this second expression also can
represent the same assertion but that is not the case because if you closely see here this second
expression, this expression becomes true even for an x who is not enrolled for CS201 that means
if you have say some x1 such that S(x1) is false.
Then even for such an x1 this existential quantification becomes true because since S(x1) is false,
it does not matter whether C(x1) is true or false the overall implication will be true because false
implies anything is true.
So to make my point more clear, our goal is to identify whether it is the first expression or the
whether it is the second expression which represents my assertion that some student in CS201
has studied calculus or not and again consider a college which has three students Ram, Shyam
and Balram and say for that college none of the students has enrolled for CS201 and say only
119
Ram and Shyam has studied calculus while Balram has not studied calculus.
Now you can check here that indeed in this particular college the assertion some student in
CS201 has studied calculus is false. For this particular domain because there is no student in
CS201 at the first place itself, it does not matter whether they have studied calculus or not. That
means if expression one represents my assertion, then that expression should turn out to be false.
Whereas if expression 2 represents my statement; then the second expression should turn out to
be false with respect to this domain. Let us check whether it is expression 1 or whether it is
expression 2 which turns out to be false with respect to this particular truth assignment, so if I
consider expression number 1; the expression number 1 is an existential quantified statement,
which has a conjunction involved.
Now if I expand x and give it values Ram, Shyam and Balram I get that this expression is
logically equivalent to disjunction of three statements, why disjunction? Because remember an
existentially quantified statement is true if it is true for at least one x value in the domain, and
now you can check with respect to the truth values that have been assigned to x variable in S
propositions and C proposition this expression turns out to be the disjunction of false, false and
false which is overall false.
And, that is what we want because indeed in this particular domain the assertion that some
student in CS201 has studied calculus is false and that is what expression number one also tells
us. But what about expression number two? The expression number two is for all x, sorry for the
typo here, it should not be for all x it should be there exist x. The second expression is there exist
x.
So, again if I expand this there exist statement since it is an existential quantification, it will be
disjunction of three propositions where each proposition is an implication, S(x) → C(x) and x can
take values Ram, Shyam and Balram. Now you can check here that each of the individual x
compound propositions here are true, with respect to the truth values that have been assigned.
S(Ram) → C(Ram) will be true because S(Ram) is false and false implies anything is true.
120
S(Shyam) → C(Shyam) will be true because S(Shyam) is false and false implies anything is true.
S(Balram) → C(Balram) is also true because false implies false is true and disjunction of truth is
always true that means even though the assertion that some student in CS201 has studied calculus
is false with respect to this domain, the second expression turns out to be true with respect to this
domain. That tells us that it is not the second expression which represents the assertion that we
are interested to state here. It is the first expression which is the correct expression, so these two
examples are very important, it tells you the significance that where to use implication and where
to use conjunction, whenever you have assertions of the form “some” definitely, and some
properties are involved here, then you have conjunction involved whereas in the previous
example it is a universally quantified statement we are an implicit if then was present.
Now, let us take another example to make the concepts more clear here you are given an English
argument a set of English statements and you have to convert everything into predicates and your
domain here is a set of birds because I am stating several properties about birds here, so my
domain is set of birds. So whenever you are given English arguments you have to first identify
what is the domain.
The domain may or may not be explicitly given to you here it is not explicitly given but by
identifying the statements we find out that we are making statements about birds here, that is why
121
the domain will be set of birds. So the first statement is all hummingbirds are richly coloured. So,
let me introduce predicates B(x) and C(x) here. So B(x) will be true if the bird x is a humming
bird.
Whereas the predicate C(x) will be true if and only if the bird x is richly coloured that is the
definition of my predicates B(x) and C(x) and that is the case and this statement will be
represented by for all x, P(x) → C(x) because an equivalent form of this statement is for all birds
x, if bird x is a hummingbird then it is richly coloured. That is what is the interpretation of this
statement.
And, then you can check here that indeed this implication, this universally quantified implication
represents this equivalent statement. The second statement is no large birds live on honey. So I
have to introduce a predicate L(x); where L(x) will be true if and only if the bird x is a large bird
and my predicate H(x) will be true if and only if the bird x lives on honey that is the
interpretation of the predicates L(x) and H(x).
Now again, if you closely see here, there is a universal quantification involved, okay? So let us so
there are two forms of the same statement, I can represent this English statement either by this
first expression as well as by the second expression. So let us see the second expression, why?
The second expression is the representation of this English statement. If you see here closely, if
you interpret it closely the logical form of this interpretation of this statement is the following.
I want to represent that for all birds x. If bird x is large, then it does not live on honey that is what
is the logical interpretation and indeed this expression represents this statement, whereas the
second expression is arrived as follows, so for the moment forget about this negation which is
present outside. Let us forget about this negation for the moment, let us try to understand what
exactly there-exists x, L(x) conjunction, H(x) represent.
This represents that, there is some large bird some bird x which is large and lives on honey. That
is what will be the interpretation of this expression but this is not what I want to represent; I want
to represent that there is no such bird exist which is simultaneously large as well as lives on
122
honey and that is why I have put a negation outside. If I put a negation outside that means this
property is not possible which is indeed what I want to represent, okay?
Now, if you closely see if I apply the rules of equivalence for predicates here if I apply the De
Morgan’s law for predicates, which I have discussed in the last lecture. Then I can take this
negation inside and when I take negation inside they are exists gets converted into “for all” and
this negation will also go with L. So, I will get negation of L(x) and this conjunction gets
converted into disjunction and now you know that negation p OR q is logically equivalent to p →
q.
So I can further rewrite this expression as this and that is how I get the second expression. So you
can get the second expression by reinterpreting this statement in the form that for all birds if bird
x is large, then it does not live one honey or you can first arrive at this first expression and then
apply the De Morgan’s law and apply it to get into the second expression. So both the
expressions are correct.
You can use either the first expression or the second expression to represent the statement that no
large birds live on honey.
Now what about the third statement? So I do not need to introduce new predicates here because I
123
have already introduced the predicate H(x) over to represent that bird x lives on honey and I have
already introduced the predicate C(x) to denote that bird x is richly coloured. So dull in colour
will be negation of C(x). Now the question is, is this universal quantified statement or existential
quantified statement?
It turns out that it is a universally quantified statement because I am making or asserting this
property for all birds, I am not saying it just for some specific bird, right? I am trying, so you can
imagine that another way to re-interpret this statement is I am making the statement that for all
birds x, if bird x does not live on honey then it is dull in colour. So there is “if then” involved
here and it is a universal quantified statement.
And that is why this will be represented by this expression and what is the last statement that
hummingbirds are small, again I do not need any new predicate here, hummingbirds is
represented by the predicate B(x) and L(x) was used to represent that bird x is large so negation
of L(x) will represent that the bird x is small and again this conclusion is about all hummingbirds,
it is not about a specific hummingbird, right?
And again this property, another way to reinterpret this English statement is that for all birds x, if
bird x is hummingbird then it is a small bird. So that is why there is an implicit “if then” involved
here that is why this English statement will be represented by this expression. So, that is how you
can convert your English statements into predicates.
124
Now let us try to understand Nested Quantifiers, so there is very often we encounter statements
where we need to have a nested form of quantification and this is similar to nested loops in
programming languages. So let us see an example here, so say the predicate M(x, y) is defined in
such a way that it is true if person y is the mother of person x, that is the definition of the
predicate M(x, y).
And, I want to represent a statement that every person in this world has a mother. So my claim is
that this can be represented by this expression for all x there exist y such that M(x,y) is true and
this is an example of nested quantification. You want to say that you fix a value of x, for that
fixed value x there exists some y, you change x then for the new x there might be another y, you
change x then for the new x you have another y such that this property M(x, y) is true.
And why this is the expression representing every person in this world has a mother; well, this is
equivalent to saying that for all person x, there is some y, such that person y is the mother of
person x which is indeed what is represented by this expression. Now when you are dealing with
nested quantification the order of the quantification matters a lot because if you change the order
of the quantification then the logical interpretation of the statement changes completely.
For instance if I write an expression there exist y for all x, M(x,y); where M(x,y) is as defined
above, the interpretation of that is you have there exist coming outside first, that means you want
125
to say that there is some person y, such that all the x are related to that y. Namely the same y is
the mother of all persons x in the world, that is not what we want to interpret. This statement
some person is the mother of all persons in the world and every person in this world has a
mother, they are two different logical statements.
And hence they are represented by two different nested quantifications. So that is why swapping
of quantifications are not always possible, it is possible only when you have the quantifications
of the same type occurring throughout the expression. That means if you have an expression of
the form for all x for all y or a sequence of quantifications which are of the same type then it does
not matter whether it is y appearing first or whether it is x appearing first.
You can conveniently swap the order of the quantification and both LHS and RHS will be
equivalent if you want to check that well for all x for all y can be considered as follows if you
expand the for all x and for all y then it will be considered, imagine that x takes values from x1 to
xn and y takes values from y1 to ym, right? I can expand this left hand side in this form and
everything is conjunction here.
And, then I can swap and can shuffle around all the P(y1) first all of so I can shuffle around all
the expressions of the form anything P of anything followed by y1 and take them together and
then followed by conjunctions of all P anything of y2 and so on and that will be equivalent to the
second expression right and this shuffling around is possible because everywhere AND is
appearing and it satisfies the associative law.
But if you have an expression where you have quantifications of different form, then this kind of
swapping may not be possible. The logical interpretations might be completely different.
126
So, now let us see some more examples here how we can start translating statements using the
help of nested quantification. So suppose I want to represent a statement that if a person is female
and is a parent then this person is someone's mother. So we have to first identify or define the
predicates that we have to use here and here again, the domain is not explicitly given but you can
imagine here that the domain is the set of all people.
So let me introduce this predicate F(x) which is true if person x is female and I also need a
predicate P(x) to represent that person x is parent and I had already introduced a predicate M in
the previous slide which I am retaining here. So first of all, this is a universally quantified
statement because I am making a statement about all persons here, I am not making a statement
about some specific person, I am making a statement about all persons.
So that is why this will be a universally quantified statement and this is an if statement of the
form if-then your premise is for all person x in the domain I want to state that if the person x is
female and if the person is a parent, so that is why conjunction of F(x) and P(x) then for the same
x there exist a y, a person y such that x is the mother of y and you see how carefully I have put
the parentheses here.
If I do not put the parentheses then the expression becomes ambiguous it will not be clear that
whether it is x which is appearing first and then followed by y and so on. So x is occurring on a
127
higher level and for each x there will be some y. Similarly if I want to represent statements of the
form that every person has exactly one best friend, so this statement has two parts.
The first part is that each person has at least one best friend definitely, thats the first part of this
statement : is one best friend in fact each person x has at least one best friend y and the second
part is the same person x has no other best friend z, where z is different from y and this is true for
all x that is what is the logical interpretation of this statement. So let me first introduce the
required predicates here, so I introduce a predicate B(x, y) which is true if person y is the best
friend of person x that is a definition of my predicate B(x, y).
And, now you can see here that since I have identified the two parts of this English statement, the
first part is that for every person x there is some y such that y is the best friend of x and I want to
state that for the same x there is no different person z different from y who is also the best friend
of x that should not be possible, so that is why the left hand side represents the first part of this
expression represents that person x has at least one best friend.
And the second part of the expression represents that person x has the possibility of a second best
friend as well I want to avoid that and that is why I put a negation in front of that if I put the
negation in front of that then that rules out the possibility that there is no second person z
different from y who is also the best friend of x because of the occurrence of this negation.
And then conjunction of both these conditions will represent what I am interested to assert. Of
course now, if you do want to apply the De Morgan’s law of quantifications, you can take the
negation, this negation that is here, and you can take it inside and then conjunctions get converted
into disjunctions and so on and then you can apply the rule that negation P OR Q is equivalent to
P → Q and this is another equivalent form of the same expression.
128
Now let us do some rules of inferences for quantified statements, so which are very important the
first rules of inference is universal instantiation and argument form of this universal instantiation
is if you are given the premise that for all x, P(x) is true, then you can come to the conclusion that
the predicate P is true for some element c in the domain, where c is some specific element that
you are interested in that you want to explicitly specify.
And, this is because since the premise for all x, P(x) is true that means property P is true for
every x in the domain, property P is true for every x in the domain. So of course, it will be true
for the element c as well, ok, whereas universal generalization has a different argument form, so
what exactly universal generalization is used for so imagine you want to prove that a property P
is true for every x in the domain that means you want to prove or assert that for all x, P(x) is true.
How do you do that? One option could be that you check whether property P is indeed true for x 1
or not, x2 or not, x3 or not and so on, where x1, x2, x3 etc are the various values in your domain
but this becomes infeasible if your domain is infinitely large. So to prove statements of the form
that prove that something is true for every x in the domain where domain is infinitely large, very
often we encounter statements of the form that prove some property is true for every integer x.
How do we prove it? We cannot take each and every integer and show that indeed the property is
true for every integer that you have chosen. So to prove statements of that form, what we do is we
129
pick some arbitrary element of the domain when I say arbitrarily element of the domain that
means there is no specific property of that element, it is just some arbitrary element and show that
the property P is true for that arbitrarily chosen element; if it is true for that arbitrarily chosen
element, you can come to the conclusion that P is true for any element in the domain because the
sample point that you have chosen was arbitrary. So the argument form here is if you show or if
you know the premise that property P is true for element c where c is some arbitrarily chosen
element then you can come to the conclusion that for all x, P(x) is true. So this is called universal
generalization.
Now the duels of these rules are existential instantiation, which says that if you have the premise
there exist P(x) then you can conclude at proposition P(c) is true where c is some non arbitrary
but unknown element, I stress here that you may not be knowing what exactly is the element but
you will be knowing that since P is true for some x in the domain, let c be the x for which it is
true what exactly is that c you may not know that whereas existential generalization says that if
you know that property P is true for element c in the domain where c is some fixed element,
which you are aware of, that means you have a witness c explicitly for which the property P is
true, then you can come to the conclusion that there exist x, P(x) is true.
130
So these are four popular rules of inferences which we use involving which we use while dealing
with quantifications. So now let us do an example to verify how to verify whether argument
forms are valid or not, in predicate logic. So here you are given two premises and conclusion
here. So I am retaining the same predicates S(x) and C(x) that we have defined in some earlier
slides.
So the first statement is : every student in CS201 has studied calculus. So that is represented by
for all x, S(x) → C(x) that is your first premise and the second premise is Srinivas is a student in
the CS201 course that means the property S(x) is true for x equal to Srinivas; that means
S(Srinivas) which is now a proposition is true, that is your premise. I said this is now a
proposition because you have now assigned a value x equal to Srinivas.
The conclusion you are drawing here is that Srinivas has studied calculus that means you have to
show that C(Srinivas) is true. So, let us see whether this argument form is valid or not, so you are
given the premise for all x, S(x) → C(x) so what you can do is you can apply the universal
instantiation and you can substitute x equal to Srinivas and get the proposition S(Srinivas) →
C(Srinivas) to be true.
You are also given the premise S(Srinivas) to be true, now what you can do is you can think that
this is now P → Q a proposition and a proposition P both these premises are true so you can
131
apply Modus Ponen and come to the conclusion that C(Srinivas) is true.
So that leads us to the Modus Ponen and Modus Tollen rules. These are the generalizations of
Modus Ponen and Modus Tollen to the predicate world. Modus Ponen says the following if you
are given the premises for all x, P(x) → Q(x) and if P is true for some element c in the domain
then you can come to the conclusion Q(c) and then same way Modus Tollen is generalized.
So that brings me to the end of this lecture. Just to summarize. In this lecture we saw how to
convert English statements using predicates and logical connectives, we saw some rules of
inferences using predicate logic and we saw how to verify whether a given argument form
involving predicates is a valid argument form or not, thank you.
132
Lecture -10
Proof Strategies-I
Hello everyone, welcome to the first part of proof strategies.

The plan for this lecture is as follows we will introduce various proof strategies that we will be
encountering in this course namely direct proofs and we will see some forms of indirect proof
namely the proof by contrapositive, vacuous proof and proof by contradiction.
133
So several form of theorems often involved a statement of the form, you have to prove an
implication, which is universally quantified, say you have to prove statement of the form that for
all integers x, if x satisfies some property then it has this condition or this extra property and so
on. So those statements will be represented by this universally quantified implication and unique
proof mechanisms to prove universal quantifications of this form.
So, how do we prove a universally quantified implication, we cannot take each and every x value
in the domain and check whether P(x) → Q(x) is true or not, if my domain is infinitely large and
that is why in the last lecture we saw that you can apply universal generalization to prove
universally quantified statement, namely you can pick some arbitrary element c from the domain,
where c has no extra property you do not know anything about c it just some arbitrary element
and you show that the implication P(c) → Q(c) is true.
If you show that this implication P(c) → Q(c) is true then based on universal generalization, you
can conclude that this universally quantified implication is also true. That means now our
problem boils down to proving statements of the form a proposition implies another proposition,
because P(c) is a proposition Q(c) is also a proposition because we have substituted x with c and
we want to prove whether this implication is true or not.
134
So now our goal will be to see various proof mechanisms for proving statements of the form p →
q where both p and q are propositions. So we will start with the direct proof method as the name
suggests it is direct because in this proof method we start assuming that my premise p is true and
logically I show that my conclusion also will be true that is why this method is called a direct
proof method.
A very simple illustration of this proof method is the following, say I want to prove the statement
that if n is an odd integer then n2 is odd. First thing that you have to understand here is that this
statement is about all integers n and even though the word all is not explicitly given here this is a
universally quantified statement and say, O(n) is a predicate which is true if and only if n is odd.
So the statement that I want to prove here is the following, O(n) → O(n2) is true for every integer
n in the domain and I want to use a direct proof method. So, what I do in the direct proof method
I assume my premise to be true, so I pick some arbitrary integer n here and assume it is an odd
integer. Under that assumption I have to show that the square of the same odd integer is also odd.
So since n is an odd integer. I do not know the exact value of n that is important right because it
is an arbitrarily chosen element from my domain but since it is odd, I know that I can write it in
the form 2k + 1, where k is also some integer it could be positive it could be negative depending
135
upon what is my n. Now if I take the square of the same n, by rearranging the term, I get n2 is of
this form.
And, now if I substitute (2k2 + 2k) by another integer k’, then I come to the conclusion that n2 is
2 times some integer plus 1 meaning n square is also odd that means starting with the premise I
can directly come to the conclusion that is why this is a direct proof method.
However, it turns out that it is not always possible to directly prove that p → q is true and for
situations like that, we need to have mechanisms which are indirect, we still want to prove p → q
but not based on the direct proof method and there are various proof mechanisms under this
category of indirect proof. So, we will see each of them and as I said, indirect methods are used
where we cannot apply the direct method.
So for instance if I want to prove the statement that for all integers n, if 3n+2 is odd then n is odd
then I cannot prove this statement using the direct proof method even though this statement is
true, because if I start with the direct proof method I will say let 3n + 2 be odd, where n is
arbitrarily chosen then you can say that 3n + 2 since it is odd it is some two times k plus one.
And after that I do not know how to proceed and come to the conclusion that n is also odd my
proof might become very complicated.
136
So that is why I need additional methods here which are indirect methods. So let us see the first
indirect method which is called proof by contraposition and idea behind proof by contraposition
is the following: your goal is to prove the validity or the fact that p → q is true we do that by
instead showing that negation of q → negation of p is true and why this is a valid proof method
because we know that p → q is logically equivalent to ¬ q → ¬ p that means both p → q as well
as ¬ q → ¬ p takes the same truth value.
If p → q is F then so is ¬ q → ¬ p, if p → q is true then so is ¬ q → ¬ p. So now let us take the

same example which we discuss in the last slide we want to prove that if 3n + 2 is odd then n is
odd for every integer n and we want to prove it by contrapositive so what is the p part here the p
part here is if 3n + 2 is odd the q part here is n is odd.
That is what you want to prove what will be negation of q, the negation of q will be n is even and
what will be negation of p? Negation of p will be 3n + 2 is even. So now our goal will be to
verify the implication in the reverse direction. We want to verify that if n is even then can we
come to the conclusion that 3n + 2 is also even. If we can prove that then that is equivalent to
proving our initial original implication.
137
So now I assume n is an even integer and it is arbitrary, why arbitrary? Because, we are still
applying universal generalization because we are now trying to prove an implication which is
universally quantified. So, I cannot take each and every n in the domain and prove this
implication that is why I am taking an arbitrary n which is an even number. Since n is arbitrary,
but I know it is even I can write it in the form 2k; where k is some integer which could be either
positive or negative.
And then it is easy to see that 3n + 2 can also be represented as an integer of the form 2k’; where
k’ is some integer and hence my implication in the contrapositive direction is correct true and
that is why my original implication is also true so now you can see the proof is so convenient.
There is another indirect proof method for proving p → q, which is called vacuous proof and this
is based on the idea that your implication p → q is always true if p is false irrespective of what is
q, q could be true q could be a false it does not matter if your premise p is false then definitely p
→ q will be true. So for instance imagine P(n) is a predicate which is defined over the set of
integers and P(n) represents the statement that if n is greater than 1 then n2 is greater than n that
is the definition of P(n).
Now, we want to check whether P(0) is true or not. So remember P(0) is now a proposition,
which is obtained by substituting n equal to 0 in your predicate P(n). What is the proposition
138
P(0)? The proposition P(0) is, if 0 is greater than 1 then 0 square is greater than 0. So, this is your
p part, this is your q part you want to prove p → q is true or not. So, now if you check the p part
it is a false proposition because 0 is not greater than 1 then it does not matter what is q whether it
is true or false.
In fact in this case q is false, the overall implication is true because F → F is defined to be true
that is the truth value of implication that means I can say that the statement P(0) is vacuously true
because the premise is false and it does not matter what is the conclusion, even though the
conclusion is false, the overall implication is a true statement here. So here I am using a vacuous
proof method.
We can prove p → q even by contradiction method, which is an indirect proof method for
proving p → q and this is based on the idea that p → q is logically equivalent to p conjunction ¬
q → F, you can easily verify that you can draw the truth table of p → q and you can draw the
truth table of this RHS expression and both the truth tables are same.
So, the idea here will be that if our goal is to prove that p → q is true then instead of assuming q
to be true you assume that negation of q to be true and add it to the premise p and come to the
false conclusion. If that is the case, then that is equivalent to showing that indeed q follows from
p that is what is the basic essence of this proof method. So for instance if I want to prove this
139
statement that if 3n + 2 is odd then n is odd we had already proved it by proof by contrapositive
but let us see a proof by contradiction method.
So this part is your p part this part is your q part and the proof by contradiction will proceed as
follows. So you are assuming p to be true but you are assuming q to be false and then based on
these two statements, you have to come to a false conclusion. If you do that, that means whatever
you assumed about q is not correct that is what is the essense of this proof by contradiction
method.
So since you are now assuming n to be even that means n is some 2k and since n is arbitrarily
chosen because we are proving a universally quantified statement and we have chosen n to be
arbitrary because we are applying the universal generalization here. So we do not know the exact
value of n except that it is an arbitrarily chosen even integer that is why n will be some 2k and
that gives us the conclusion that 3n + 2 is even.
So, now you can see that how do we get a contradiction here? So we started with the premise 3n
+ 2 to be odd and we assumed n to be even based on these two premises or these two statements,
I come to the conclusion that 3n + 2 is even, how can that be possible simultaneously that 3n + 2
is odd as well as 3n + 2 is even that is not possible simultaneously these two things cannot exist
simultaneously.
That means the problem due to which this situation has occurred is because you made this
incorrect assumption that n is even, that means n has to be odd and that shows that p → q is true
and this follows this structure. So p was the statement that 3n + 2 is odd to that you added the
negation of the conclusion namely n is not even and based on these two things you come to a
false statement a false situation that both 3n + 2 is odd and 3n + 2 is even that means both p and
¬ p, which is not simultaneously possible that means this is p and this is ¬ p which is equal to
false, so you come to the false conclusion.
140
It turns out that we can use the proof by contradiction method even to prove that a single
proposition p is true, we use proof by contradiction in the previous slide to prove the truth of an
implication namely p → q, but you can use the proof by contradiction method even to prove that
a statement a single statement p is true, and this is based on the following idea : this is based on
the idea that negation of p implies conjunction of r and negation of r is logically equivalent to ¬ p
→ F that means your goal is to show p is true.
But in this proof method what we do is we instead assume that p is false and if we assume p is
false then we have to show that based on that we come to a false conclusion that means we come
to a scenario where both r as well as a statement negation of r is true and if that is the case that
means if negation of p → F is true, then that can be possible only if p is true, because if negation
of p turns out to be true then true → false can never be true.
So the only way negation of p → F can be true is if negation of p is also false and if negation of p
is false that means my statement p is true so that is the proof mechanism here. So you start with
the negation of whatever you are supposed to prove and from that you should be able to logically
show that both a positive statement and the negative statement simultaneously can be derived.
141
So let us see how this proof method is applicable to prove that √2 is irrational, so what I do here:
is this is the statement p that I want to prove, my proposition p which I want to prove here is that
√2 is irrational, I assume a negation of that, that means on contrary I assume that the √2 is
rational that means I am now assuming negation p is true and based on that I have to come to a
false conclusion; that means from negation p I have to come to a false statement, which will
show that p is actually true.
Now since I am assuming √2 to be rational that means I can express it in the form a/b some
integer a over some integer b where the greatest common divisor of (a,b) = 1 that is a definition
of a rational number and say this statement is r that means from negation of p I have derived the
statement r, I will show that from the same statement negation p I will derive the conclusion
negation of r which will establish that negation p is not possible.
So let us see how we can derive negation of r as well from negation of p. So, since √2 is of the
form a over b by taking square on both sides I get a square equal to 2b2 and now if a2 is equal to
2b2, so a2 is equal to 2b2 means a2 is even, because it is two times some integer b2 then if a2 is
even I can prove that a is also even. So I am not separately proving it you can easily verify this
that if a2 is even, it implies a is even.
142
You can easily verify that you can prove it by contrapositive you can show that if a is odd then a2
is also odd. So, I am not separately proving that. So I come to the conclusion that a is even and if
a is even then I can write it in the form 2 times some integer c. Now, if a is some 2c then I get
that b2 is also some 2c2 because if a is 2c, then a2 will be 4c2.
So 4c2 is equal to 2b2 that means b2 is equal to 2c2 and now b2 is 2c2 then by applying the same
property here that if b2 is even I can prove that b is also even by applying the same rule here. So
say b is even and b is even; that means b is some 2 times an integer d but if a is some 2c and b is
some 2d that means what can you say about the G C D of a and b.
The G C D of a and b will be definitely more than 1, in fact 2 divides both a as well as b because
both a and b are even that means I can say I can come to the conclusion that G C D of a and b is
not 1 and that is the negation of r because r represented the statement that GCD of a and b is 1.
Whereas negation of r represents, G C D of a and b is not 1. So you can see that now assuming
negation of p namely the statement which I wanted to prove I can logically conclude that √2 is
some a over b where GCD of a and b is 1.
And simultaneously, I can conclude that √2 is some a over b where GCD of (a, b) is not 1, which
is not going to happen; how can it be possible that both r as well as negation of r holds and r and
negation of r means false that means from p you can come to the false conclusion, that means we
have shown here that ¬ p → F is true, if this is true this is possible only if p is true.
So that brings me to the first end of this lecture. In this lecture, we introduced various proof
methods, our main motivation is to prove implications because we often encountered universally
quantified implications and to prove that, by applying universal generalization we have to prove
implications involving propositions. So we have introduced a direct proof method for proving
implications.
And, we have seen some indirect proof methods like proof by contrapositive or proof by
contradiction and vacuous proof method to indirectly prove whether p → q is true or not. Thank
you.
143
Lecture -11
Proof Strategies-II
Hello everyone, welcome to the second part of proof strategies.

In this lecture we will continue our discussion on the proof strategies that we started in the
previous lecture.
144
We will start with how to disprove a universally quantified statement. So till now we were seeing
various proof methods, where we wanted to prove universal quantified statements. But
sometimes we also encounter statements where we want to prove that a universal statement,
universally quantified statement, is not true. To disprove that a universally quantified statement
is not true we have to give what we call as a counterexample.
And, by counterexample we mean some element c from the domain such that the statement P is
not true for the element c, because if the statement P is not true for the element c. Then the
universal quantification for all x, P(x) will turn out to be false. So, very simple example of this is
if I make a statement that every positive integer can be expressed as the sum of squares of 2
integers. So this property is my property P and I am making this statement for every x.
So, now I have to verify whether the statement P is true for every integer x or not. Even if I can
find one integer for which this property is not true, I can say that ꓯ x, P(x) is false here and a
very simple counterexample is 3 here, you can check that 3 can never be expressed as a sum of
squares of 2 integers. You can never do that. So that is how we disprove universally quantified
statement.
However, I would like to stress here that there is nothing called proof by example for proving
universally quantified statements. If you want to prove that your statement P is true for every
145
element in the domain, then you cannot say that okay, I am showing it for some x, where x is
explicitly chosen by you, it is not arbitrary. You are not choosing your x arbitrary, you are just
taking your x specifically and that is an example for you.
And you show that a property P is true for that specific x and then you happily conclude that P is
true for every x in the domain. What if there is some bad x which you have not verified for the
property P. So that is why when we try to prove universally quantified statement the witness c is
chosen arbitrarily, it is not chosen in some specific well defined way.
There is another proof mechanism which is called proof by cases. This is also called exhaustive
proof. So if you want to prove an implication of the form p →q, where p and q are propositions,
are propositions and if you are proposition P can be decomposed into conjunction of various
other propositions say n propositions, then p → q is logically equivalent to p1 → q, conjunction
p2 → q and so on.
That means here p1, p2, pn are the various cases for your proposition p and your goal was to
prove p → q. So to do that, you show that for various cases for p that sub case implies q is true.
That is why it is the essence of the proof mechanism. So for instance, if I want to prove a
statement that ꓯ n, if n is an integer, then n2 ≥ n . This is a universally quantified statement.
146
And now if I apply the universal generalization, you will be choosing an arbitrary n and you will
be trying to prove that this property is true for that arbitrarily chosen integer. But now that
arbitrarily chosen integer can be positive, it can be negative, it can be 0. You have 3 possible
cases right for this arbitrarily chosen n and that is why you split the proof into three cases and
you show that p → q is true for any of those three cases does not matter which case you are in.
So if you take the case n equal to 0 the statement is true, namely p → q is true. If you take the
case when your arbitrarily chosen n is positive, then also this statement p → q is true and if your
arbitrarily chosen n is negative, then you show that in that case also this p → q is true and since
these are the three possible cases for your arbitrarily chosen n and you have chosen that p1 → q
is true p2 → q is true and p3 → q is true.
And there are only three possible divisions or cases for your p, you can conclude that p → q is
true. So that is your proof by cases method.
There is also something called without loss of generality, which we very often encounter while
writing the proof. In short form they call it as w.l.o.g. So let me demonstrate we have what
exactly I mean by without loss of generality. So imagine I want to prove that if x and y are
integers and both x, y and x + y are even, then x and y are both even. That is a statement I am
making and again, this is a statement involving for all x for all y.
147
Because; I am making a statement for all integers x and all integers y. So, let me try to prove it
by contrapositive. So the contrapositive means a negation of the condition appearing after then.
So I wanted to prove both x and y are even, so, that is my q part. So negation of q will be either x
is odd or y is odd. Because both means AND negation of AND means OR and in the same way
this is my p, so negation of p will be x, y is odd OR x + y is odd.
So this is what I want to prove now and now I can apply proof by cases because I am taking
arbitrary x and arbitrary y here and my premise is that either x is odd or y is odd. So now I have
three possible cases. Case 1, when x is odd y is even, so let us prove it for that part. So since x is
an arbitrary odd number, I can write it in the form x equal to 2k + 1, I can say y is equal to 2
times of m because y is even.
Then x times y turns out to be even, but x + y turns out to be odd that means the statement turns
out to be correct. So this is case 1 for the premise x or y is, one of x or y is odd. The Case 2 could
be when x is even, y is odd that also satisfies my premise but it turns out that I do not need to
explicitly write a proof for Case 2, because this case will be symmetric to your Case number 1.
Why it is symmetric? Because for Case 1 you chose x to be arbitrary and y to be arbitrary.
They were not specific values and even for Case 2 your x is arbitrary and your y is arbitrary that
means whatever argument you gave for Case 1, the same argument is applicable as it is for case
2 as well. So there is no need to separately write down these 3 statements for case number 2. You
just have to swap the rules of x and y for whatever proof you have given for Case number 1 and
that is possible that subsumed for in case 1 because x and y were arbitrarily chosen.
So that is why there are not 3 cases for this premise instead, there are two cases only. Case 1
when exactly 1 of x or y is odd. This is different from a previous case 1, my previous case 1, I
was explicitly taking the cases x odd, y even, x even, y odd and my third case would have been
both x and y are odd. But what I am doing now is I am clubbing both case 1 and case 2 into one
new case 1, where I say that exactly 1 of x and y is odd which one I do not know.
148
It could be either x or either y and that is why I can say that I do not know which one of x and y
is odd without loss of generality. Let it be x and for that particular case this argument is given.
This argument automatically subsumes the case when y is odd as well and now my second case
will be when both x and y are odd. So these will be now the two subcases for my premise x or y
is odd. There will not be three separate cases because two of the sub cases are subsumed by one
sub case here.
We also need proof mechanisms for proving existential quantified statements and there are two
variants for that. Variant one is called constructive proof where we give a specific example
specific witness and show that specific witness satisfies my conditional property. Because when I
want to prove an existentially quantified statement, I have to show that a statement is true for at
least one value in the domain.
There might be multiple values, but I just have to show one value for which the statement is true
that I can do by giving a constructive proof by showing an explicit example. So for instance here
is an example of existentially quantified statement that says that there exists a positive integer
that can be written as the sum of cubes of positive integers in two different ways. So there might
be several such positive integers which can satisfy this positive property.
149
My goal is to show whether at least one such integer exist or not and indeed 1729 is one such
number of course, as I said there can be multiple such integers and you can check that 1729 can
be written as the summation of 103 and 93, as well as the summation of 123 and 13. That means it
can be expressed as a sum of cubes of two numbers in two different ways.
If you are wondering why I am showing this car here, some of you might know the famous story
about this number 1729, so this is also called Ramanujan Number. So the story goes says that
Ramanujan and Mathematician Hardy were travelling in a car in London and there was this car
with number 1729 in front of their car. So Hardy said that is nothing interesting about this
number 1729. But the great Ramanujan said no it is an interesting number, because it can be
expressed as the sum of cubes of two numbers in two different ways.
So that is one way of proving existentially quantified statements, another way of proving
existential quantified statements is what we call as non constructive proof and here we do not
show any concrete example or witness from the domain for which the statement is true. We do
not do that, but we logically argue that definitely at least one element from the domain exists for
which the property is true.
So let me demonstrate this proof mechanism by this statement I want to prove that there exists
irrational numbers x and y. Such that xy is rational, again there might be multiple x and multiple
150
y satisfying this condition, but I have to show the existence of at least one such x, y pair. It turns
out that I cannot explicitly show that x, y pair, but I can guarantee I can logically argue that
definitely such x, y per exist. How?
So consider the number x equal to √2. We already proved that it is irrational. We proved it in our
previous lecture using the proof by contradiction method, we assumed x to be, we assumed √2 to
be rational and then we came to the conclusion that both r and negation r exist where r is some
statement. So we already proved that √2 is irrational. Now, what can you say about the number
(√2)(√2).
There can be only 2 possibilities either this is rational or this is irrational. Only 2 possibilities, we
cannot have any third possibility. So, these are the two possible cases. Now if √2, raised to
power √2 is rational then I got my witness x, y. My witness in that case will be x to be √2 and y
to be √2. Whereas, if √2 to the power √2 is irrational then my x y candidate is the following I
will take x to be √2 to the power √2 and y to be √2.
Because if √2 to the power √2 is irrational and I raise it to again power √2, then this is equivalent
to saying (√2)2, which is 2 and 2 is rational. Now you might be wondering how it constitutes a
proof, we do not know for surety whether it is case 1 or whether it is case 2 which is true, we
cannot verify whether √2 power √2 is rational or irrational but what we are logically arguing is
that either it will be this case or this case.
And, it does not matter whether it is case 1 which is true or it does not matter whether it is case 2
is true in both cases I can show you an x,y pair satisfying the conditions that I am stating in this
theorem statement and that is why this is a non-constructive proof. You cannot say that my x and
y is always √2 and √2. We do not know or you cannot say that my x is √2 power √2 and y is √2.
It depends whether I am case 1 or case 2, but I do not know whether it is case 1 which is always
true or whether this case 2 which is always true. But I know definitely logically that one of them
is true. So that is what we do in non-constructive proof method, we do not give you the explicit
151
witness, but we logically argue that the statement that we are making is true for some witness in
the domain.
We also encounter proof statements where we have to prove the uniqueness of something and
namely we have to show the uniqueness of some element which satisfies a given property of the
theorem statement, and such proofs involve two parts. The part 1 will be the existence of some
witness, say x that means we have to show that definitely the property that is mentioned in the
theorem statement holds for some x in the domain.
And the second part is we have to show that apart from x there is no other witness y which also
has the same property mentioned in the theorem statement. This is equivalent to saying that if
you had a different witness y from x, where x has already satisfied the property p then y cannot
satisfy the statement to be proved or equivalently if y satisfies the same property, which is given
in the statement then definitely y equal to x.
These are equivalent and both of them represent this part 2. This is something similar to what we
did in the lecture where we represented English statements by predicates, there we represented a
statement of the form that every person has exactly one best friend. So there we first logically
represented that a person has at least one best friend and we negated in the same expression that
apart from that friend he cannot have a second best friend.
152
So that is the part 2 here. Part 1 is you show that some witness is there; part 2 you show that no
other witness is there. So this is an example of a statement where you have to show the
uniqueness of something. The statement is if a and b are real numbers again, this is a universally
quantified statement because we are making the statement for all real numbers a and b. The
statement is if a is not equal to 0 then there is a unique r such that the condition a times r + b
equal to 0 holds.
So part 1 of the proof will be we have to show that, at least one r is there which satisfies this
property. That is a part 1 of the proof and indeed an r satisfying this property is minus b over a
and minus b over a is well defined, because a is not equal to 0. That is why this condition a is not
equal to 0 is important. If a is equal to 0, then this property that a times r + b equal to 0 may not
hold.
Now, we have to argue that apart from this value of r namely minus b over a, there is no other
integer value possible satisfying the same condition and this simply follows from the fact that if
you have another witness r’, which also satisfies the same property. Then r’ also will be minus b
over a and minus b over a is nothing but r; that means you cannot have anything different from
minus b over a which can satisfy this condition a times r + b equal to 0. So this is an example of
uniqueness proof.
153
We have another proof mechanism called backward reasoning and this is an interesting proof
mechanism. So imagine your goal is to prove a statement q to be true. The proof strategy that is
involved here is instead of proving q to be true; we will try to find out statement p instead, which
is true. Such that p → q is true. So we will not be proving q here, our goal will be to instead find
this true statement p.
Such that p → q is true and if p → q is true and p is true that is possible then that is possible only
if q is true, because if we are showing p to be true and simultaneously implies q to be true, then
that is possible only if q is also true that is a proof mechanism here. So let me demonstrate this
proof mechanism by this theorem statement by proving this theorem statement we want to prove
that for every distinct real numbers x and y their arithmetic mean is always greater than the
geometric mean and again this is a universally quantified statement. What we will do is instead
of proving it for every x and y we will try to prove this property for an arbitrary x and arbitrary y
that is the statement q. My statement q is that for an arbitrarily chosen x and y arithmetic mean is
greater than geometric mean, that is what I want to prove.
So our goal will be to find the true statement p involving this arbitrary x and y, such that p → q
is true. So here is how we find the statement p. What we do is we keep on going backward
starting with q, till we arrive at a statement p which is true and then we will try to see whether we
154
can come back all the way from that p to our goal q. So as I am saying that we have to go
backward.
So we argue that in order that arithmetic mean is greater than geometric mean the condition x + y
over 2 should be greater than the square root of x + y why? Because, this is your definition of
arithmetic mean and geometric mean and this is the statement q, which you want to prove. Well
this will be true, provided the square of x + y is greater than 4 times x times y and for this to
happen, the square of x - y should be greater than 0.
But this is always true because you are given that your arbitrary x and y are distinct here. That is
what is the condition here and for any arbitrary distinct x and y, it does not matter whether x is
greater than y or y is greater than x. The square of their difference will always be positive which
is a true statement and this is the statement p which we are interested to find a true statement p
which is always true.
Now the actual proof in the backward reasoning will be to reverse this argument and show that
starting with p, assuming p to be true, you can come all the way to the conclusion q. That will
complete the proof; until and unless you do not do this backward reasoning, your proof is not
complete you have shown p to be true assuming, q to be true. That is not what you want to prove
you wanted to prove the truth of statement q. So the actual proof will be as follows.
We start with the true statement p, we know that what all distinct x, y the square of their
difference is always positive, from that we can come to the conclusion that the square of their
sum is greater than 4 times xy. From that we can come to the conclusion that x + y over 2 is
greater than the root of xy and from that I can come to the conclusion that arithmetic mean, is
greater than the geometric mean.
So that brings me to the end of this lecture we have seen in this lecture some other proof
mechanisms or indirect proof mechanisms and we have seen proof mechanisms for proving
existential quantified statement namely the constructive method and the non-constructive
methods and we also saw the backward receiving method. So we have seen lots of interesting
155
proof mechanisms, we will be encountering all of them in various forms throughout this course.
Thank you.
156
Lecture -12
Induction
Hello everyone. Welcome to this lecture on proof by induction.

So just to recap in the last lecture we have we started discussing extensively about various proof
mechanisms, which we used to prove different kind of statements. In this lecture, we will
continue our discussion on proof strategies and we will introduce a very important proof
mechanism namely proof by induction which we will be using extensively in this course. We
will be seeing two forms of proof by induction namely proof by regular induction and proof by
strong induction.
157
So what is proof by induction? So you must have encountered proof by induction several times.
It is generally used to prove universally quantified statements namely statements of the forms
such as for all positive integers n factorial is less than or equal to nn. For all positive integers n
𝑛 (𝑛+1)
and the summation of first n numbers is and so on. So it is used to prove all this
2
universally quantified statements and what is the argument form of induction proof?
So imagine P is a property or a predicate and you want to prove that the property P is true for all
values of n starting from b onwards. So for instance, if I take the first statement here, the
property P here is that n factorial is less than equal to nn, that is the property P and we want to
prove it is true for all positive integers. In the same way for the second statement a property P is
𝑛(𝑛+1)
that summation of 1 to n is and so on.
2
So there will be some base case or some starting value and we want to prove that the property P
is true for all values of n starting from b onwards. So the argument form for the proof by
induction is as follows. So, these are your premises, namely it will be given to you or you will be
proving explicitly that the property P is true for the element b in the domain. So the proposition
P(b) is true and you will also prove that for any k greater than equal to b if the property P is true
for the element k in the domain then the property P is true even for the element k + 1 in the
domain.
158
So based on these two premises a proof by induction concludes the conclusion that the property
P is true for all n greater than or equal to b. So now the question is this argument form valid?
Because that is what we typically do in proof by induction, in proof by induction these are the
two things which you prove. You prove what we call as the base case and then we prove the
inductive step and based on that we conclude at the property based P is for all n greater than
equal to b. So the question is that is this a valid proof mechanism is this argument form valid or
not.
So to understand that why prove by induction is a valid proof mechanism, let me give you an
analogy. So you imagine that you have an infinite ladder and I want to make the conclusion that
all steps of the ladder starting from b onwards can be climbed, if these two premises hold. So,
what are the two premises here? It is given to me that definitely you can climb step number b and
it is also given that, if you can climb step number k then it is guaranteed that you can climb step
number k + 1.
So these are the two conditions given to you and my claim is that if these two conditions are true
then I can conclude that all steps starting from b onward can be climbed, that is what is an
analogy for proof by induction. So the property P in this example is that you can reach step
159
number x or step number b or step number k. So; the property that I want to prove here that you
can reach all the steps here.
So the way we prove that proof by induction is a valid proof mechanism is as follows. So assume
that the argument form of proof by induction is invalid and from the definition of invalid
argument this means that I have true premises that means, the statements in the premises are true,
but the conclusion is false. If that is the case that means there are definitely some steps which
cannot be climbed.
So say these are the set of steps which cannot be climbed and this set exists because I am
assuming that the conclusion is false. So since the conclusion is false that means definitely there
is at least one step which I cannot climb and there might be many such steps. So I am a
numerating all such steps which are unreachable. Now among these steps, I focus on the least
indexed step, which cannot be climbed and I call it kmin and again this index kmin is well defined
because this kmin is the least value from a set of values.
Now, I can say that definitely kmin is greater than b this is because I am assuming my premises
are true and my premises are true means step b can be climbed; that means that is a true
statement, that means definitely kmin cannot b. So kmin can be anything after b onwards, but if the
property P is false for kmin; that means if the step number kmin is unreachable; that means the step
number kmin - 1 is reachable; or the property P is true for the element, P(kmin -1).
But this gives a contradiction, because if step number kmin - 1 is reachable; then since my premises
are true; it gives me the guarantee that step number kmin can also be can be reached that means
the property P is true even for the element kmin, because my premise is true but then it is not
simultaneously possible that kmin is true as well as kmin is false that means whatever I assumed,
that means, I assumed that means my assumption that argument form is invalid is an incorrect
assumption and that means the proof by induction is a valid proof mechanism.
So that is a very simple proof that indeed just by proving these two statements, you can come to
the conclusion that the property P is true for all values, all values starting from b onward. So in
160
the proof by induction the starting case is called as the base case, that means the first few values
for which the proposition of which is the predicate is true, though they are called as the base
cases.
There may be multiple base cases; we will see, it is not necessary that there is just one base case
and the second premise that you are proving here is the inductive step. Where assuming that the
property P is true for any value of k starting from greater than equal to b onward. You prove that
the property P is true even for the next element of the domain that is inductive step.
It turns out that very often people make subtle mistakes in proof by induction. So here let me
demonstrate one such subtle mistake. So imagine my property P is that P(n), my property P(n) is
that n is equal to n + 1 and I am making a universally quantified statement that the property P is
true for all values of n greater than equal to zero. So my base case here is zero. Now, suppose
someone tries, but definitely this is a false statement, here is the induction proof, which is given
in an attempt to prove this statement to be true.
So, let the statement be true for some n equal to k; that means we are proving trying to prove the
inductive step then the statement is also true for n equal to k + 1 and why so because if the
statement is true for n equal to k, so that means P(k) is true and P(k) is nothing but the
proposition k is equal to k + 1 and we want to prove that the statement is true even for k + 1 and
161
the predicate P(k+1) is nothing but the proposition k + 1 equal to k + 2 and if you see this
implication is logically true because P(k) is false.
Why P(k) is false? Because k is not equal to k + 1. So I have false here my premise is false and if
my premise is false over all the implication will be true that means assuming the statement is true
for n equal to k and I am able to show that the statement is true for n equal to k + 1 as well. Now
since we have proved the inductive step you might be wondering whether we have proved that
the statement is true for all values of n.
Well, the mistake in this proof is that you have not proved the statement for the base case; you
have just proved the inductive step here. Assuming that the property P is true for n equal to k you
have proved at a statement is true even for n equal to k + 1, but what about the base case? There
is no starting case for which you have proved the property to be true. You have not proved the
statement to be true for the base case and the base case here is n equal to zero and proposition for
the base case n equal to zero or P(0) is the statement that zero equal to one which is definitely a
false statement.
So, that is why this is an incomplete induction proof, you just prove the inductive step you have
not proved a base case and that is why this proof is not acceptable.
162
It turns out that there is another form of induction, which we call as strong induction. So this is
your argument form for the regular induction where you are given a base case and in the
inductive step assuming that, the predicate P is true for k you prove it to be true for k + 1. In the
strong induction, the difference is in the inductive step. So the difference is that in the regular
induction, the truth of the proposition P(k+1) has to be established by just using P(k) that means
when you want to prove that P(k+1) is true, you are just given the hypothesis or the premise that
P(k) is true. You are not told anything about what is P(k – 1), P(k – 2) and so on. Whereas in the
strong induction, which we have for which argument form is given in the right hand side part
when you are establishing the truth of proposition P(k+1), while doing that you can assume that
the statement P is true for all values in the domain starting from b up to k that is the difference.
The difference is in the inductive hypothesis.
However it turns out that both forms of induction are equivalent that means if you have a proof
by regular induction, then you have proof by strong induction for the same property P, whereas if
you have a proof by strong induction for the property P, then you can find an equivalent proof
for the same property P, but using regular induction. We will prove we will establish this
equivalence towards the end of this lecture but you might be wondering that why, what some
motivation of strong induction.
163
The main motivation of strong induction is that it simplifies your proofs several times. In many
cases, it is possible that you cannot apply the regular induction directly, but by using strong
induction, using the help of strong induction the proof is simplified a lot. So let me demonstrate
this. I prove what we call as the fundamental theorem of arithmetic and the fundamental theorem
of arithmetic says that you take any positive integer starting from one onward it can be expressed
as product of prime factors or prime powers, basically.
So what the state informally the statement here is any positive integer n can be expressed as
product of powers of prime numbers and if you are wondering what are prime numbers, well a
number is prime if it has no divisor other than the number itself and one, other than the number
itself and one. So, this is the formal statement, if you want to prove this statement and we will
prove this statement using induction and we will be using strong induction.
So since this is a universally quantified statement, we have to prove a base case and we have to
prove the inductive step. The base case is when n is equal to one and it is easy to see that if n is
equal to one then one can be written as 20. I stress here the statement does not need that b2, b3, b4,
everything should be 1, the powers of prime it can be zero as well. So I can express 1 as 20 and if
you want you can further write it as 30, 50 and so on that means the base case is true.
Now I go and prove the inductive step and while proving the inductive steps since I am using
strong induction, my inductive hypothesis will be that assume that the statement is true for all
values of n or all integers n from one to k onwards; I do not know what exactly is the prime
power factorization of 1, 2, 3, 4 up to k, I do not know the exact prime power factorization, but I
am just assuming that this statement is true for all numbers in the range 1 to k.
And now assuming this, I have to show that even the statement is true for n equal to k + 1 as
well. This is your inductive step. Now there can be two cases possible with respect to k +1. So
now you can see that I am applying here proof by cases. So within the inductive step, I am
applying the proof by cases depending upon whether k + 1 itself is a prime or it is a composite
number. So there can be only two possibilities.
164
If k + 1 it is a prime number then my statement is true because I can write k + 1 as k + 1 raise to
power 1. Thats all, into of course 20, 30 and so on. So that means my statement is true for n equal
to k + 1 whereas case two could be where my k + 1 is a composite number that means it has
some divisors and let p and q be the factors of k + 1. Now since p and q are factors of k + 1 both
of them are upper bounded by k.
It cannot be possible that p is k + 1 or q is k + 1 because if p is k + 1 that means the number itself

is a factor of itself which implies that k + 1 is a prime number not a composite number. Since it
is a composite number its factors will be definitely less than equal to k. But I do not know the
exact values of p and q, I just know that factors p and q exist, the exact values not known. But I
know the range of p and q because k + 1 is an arbitrary integer here.
Now since p is less than or equal to k and q is less than or equal to k and in my inductive
hypothesis I am assuming that the statement is true for every value of n equal to 1 to k, that
means the statement is true even for p and the statement is true even for n equal to q. Since the
statement is true for n equal to p, that means the number p has a prime power factorization that
means it can be expressed as product of powers of prime.
I do not know what exactly are those prime powers, but I know it is expressible as product of
prime powers. So let this expression be the prime power factorization of p and in the same way
since the statement is true for n equal to q that means the number q also has its own prime power
factorization. Now based on this, since my k + 1 is equal to p times q because p and q are factors
of k + 1, I can say that the prime power factorization of k + 1 can be obtained by combining the
powers of two from the individual factorizations of p and q. Similarly, combining the powers of
three from the individual factorizations of p and q and so on, that means I have proved that even
there exist a prime power factorization for the integer k + 1 and that proves my inductive step.
So, you can see that how the strong induction simplifies my proof.
The reason I am using strong induction is because I do not know the exact values of p and k, I
cannot say that definitely p is k or q is k say p is k by 2 and q is something else. In that case, if I
am trying to give a regular induction proof, I cannot use the fact that the statement is true only
165
for n equal to k, I need here the fact that the statement is true for all values of n up to k and that
simplifies the proof a lot.
Let me give you another example of strong induction and a statement here is imagine that in
India the only postal stamps which are issued are of denomination rupee 4 and rupee 5. Now the
statement I am trying to make here is that each denomination or each postage of rupees 12 or
more can be expressed in terms of only 4 rupees stamp and 5 rupees stamps that is the statement
I am making here.
So here my base cases will be as follows; I have four base cases, and you might be wondering
why four base cases it will be clear very soon. So, I am showing here that if you have an amount
of rupee 12, then you can express it by taking three stamps of 4 rupee. If you have an amount of
13 rupees, then you can represent it by taking two stamps of 4 rupee and one stamp of 5 rupee
and in the same way you can represent any you can, represent a denomination 14 and you can
represent the postage amount of 15, so that these are my base cases.
Now I want to prove the inductive step where I want to prove that the statement is true for k + 1
assuming that the property P is true for 12, assuming that the property P is true for 13, the
property is true for 14, the property is true for 15 and the property P is true for any denomination
166
equal to k. Assuming all these things I have to show that the denomination k + 1 is also
expressible in terms of 4 rupee stamp and 5 rupees stamp.
So the idea here is that the postage of k + 1 can be written as summation of postage for k - 3 and
a 4 rupee stamp, that means you take one 4 rupee stamp and whatever way you can represent the
postage of k - 3 to that representation, if you add a stamp of 4 rupee denomination, then you can
get a representation for postage for k + 1 and this works provided k - 3 is greater than equal to 12
that means your k - 3 has to be 12 and that is why we have here four base cases.
If you do not have four base cases here, if you have say base case of only 12 then this proof does
not work. So now you can see that how the proof is simplified if I assume a strong induction
proof and I have multiple base cases.
What I will do is I will show that the same statement can be proved using a regular induction
where while proving the inductive step I am just using the fact that the premise is true for n equal
to k and here I will be just proving one base case. I do not need four base cases by base case here
will be that the postage of 12 rupee can be represented by taking three stamps of 4 rupee. While
proving the inductive step I assume the hypothesis that the statement is true only for postage of
rupee k.
167
So since the statement is true for postage of rupee k that means I can represent the amount of
rupee k by taking x numbers of 4 rupee stamps and say y number of 5 rupees stamp. I do not
know the exact values of x and y because my k is arbitrary here remember, when I am proving
the inductive step I am taking my k to be arbitrary because I am trying to prove a universally
quantified statement and to prove a universally quantified statement my value of k has to be
arbitrary because I will be applying the universal generalization.
Now while applying the inductive step I have to show how can I represent postage of k + 1. Now
by proving that inductive step I take two cases, two possible cases depending upon whether x is
zero or nonzero. If x is non zero that means at least one 4 rupees stamp was used to represent my
postage of rupee k and what I can do is the following. I can take x - 1 number of stamps of rupee
4 and take y + 1 numbers of stamps of rupee 5 and that will together give me a postage for rupee
k + 1.
And this is possible because I am assuming that x is greater than equal to 1. So what I am saying
here is say for instance x is equal to 2 and say y is equal to 3 then instead of taking two 4 rupee
stamps, now you take one 4 rupees stamps in that process now you have reduced the postage by
4 rupee and you have to take care of one rupee more because now you are trying to find a
denomination of k + 1.
So you have reduced by four that means you have to take care of 4 + 1. So you have to take care
of 5 rupees postage which you can take care by adding one extra stamp of five rupees to the
number of stamps of 5 rupees, which you might have used for representing the postage of k
rupees that is the idea here; that is case number 1.
168
Case number 2 is; when x is zero that means when you represented the postage of k rupees no
stamp of 4 rupees was used. Well, in this case what we can say is that the denomination case
definitely 15 because if only stamps of 5 rupees are used to represent my amount or postage of
rupee k, that means k is a multiple of 5 and the statement is for any denomination from 12
onwards.
So the smallest multiple of 5 is 15 that means my y is at least 3 here, I have at least three 5 rupee
stamps which are used to represent my postage of k rupee. So what I have to do is I have to
represent now a postage of k + 1 rupee. So what I can do is, instead of taking now y number of
stamps of 5 rupees, I will take y - 3 numbers of 5 rupees; y - 3 numbers of 5 rupees and I will
take four stamps of 4 rupee that will overall give me a postage of rupee k + 1.
And this is possible because y is greater than equal to 3. So, I can reduce the number of five
rupees times by three. So the idea here is since you are reducing the stamps of 5 rupees by 3 you
are subtracting 15 from k and you have to take care of one more rupee postage because you want
to represent k + 1. So, you have to take care of a postage of rupee 16, which you can take care by
taking or purchasing four stamps of 4 rupees that is the idea here.
So, now you can see here that in the inductive step I am just assuming that the statement is true
for n equal to k, I am not using the fact that a statement is true for all n equal to 12 up to k but in
169
that case my base case will be one and the proof will be divided into two cases which was not the
case for proof by strong induction. So, depends upon your convenience if it is convenient to give
proof by strong induction, you can go for proof by strong induction otherwise you can use proof
by regular induction.
Now as I said earlier that any proof given by regular induction is equivalent to proof by strong
induction and so on. So what we will do here is we will prove that if you have a predicate P(n)
and if the universally quantified statement for all n, P(n) is true. Then we will show that any
proof for proving this universal quantification by induction can be converted into a proof by
strong induction and vice versa.
So one direction is very simple if this universal quantification is proved using a regular induction
proof then it automatically can be treated as a proof by strong induction. Because in a proof by
strong induction, you are not forced to use all the premises, namely you are not forced to use
P(b) as well as P(b+1) as well as P(b+2) as well as P(k). you have free to use any of these
premises to establish that P(k+1) is true.
Well, if you can just use P(k) to prove P(k+1) that is also can be created as a proof by strong
induction, so this is easy. What we will now show is that if you have proved this universal
quantification using strong induction, then I can find a proof for proving the same universal
170
quantification, but using regular induction. How do I do that? So let me define a predicate Q(k)
and Q(k) is defined to be the conjunction of propositions P(1), P(2) up to P(k).
So as per my definition Q(1) is same as P(1), the proposition Q(2) is the conjunction of
proposition P(1) and P(2) and in the same way the proposition Q(k) is the conjunction of
propositions P(1) up to P(k). From this I can conclude that the universal quantification for all n
P(n) is logically equivalent to the universal quantification for all n Q(n) that means if your
property P is true for all values of n in your domain then so is the property Q.
And this follows from the way I have defined my predicate Q(n). So what I will show here is; if
you have a strong induction proof for proving your LHS then I can convert it into a proof or I
can get a proof using regular induction to prove my RHS and since both LHS and RHS are
logically equivalent that means I have now given a regular induction proof to prove my original
property.
So let us see a regular induction proof for proving my universal quantification that for all n Q(n)
is true. So my base case will be Q(1) and Q(1) is true because as per my definition Q(1) is P(1).
And I am assuming here that there is a strong induction proof for universal quantification for all
n P(n). In that strong induction prove, there will be a base case, the base case will be P(1) and
since P(1) is true, I can conclude that Q(1) is true.
Now let me prove the inductive step for this regular induction proof; for the regular inductive
step I will make the inductive hypothesis that Q(k) is true. I am not making the hypothesis that
Q(2), Q(3), Q(4) up to Q(k – 1) is true, I am just making the hypothesis that Q(k) is true but if
Q(k) is true then as per the definition it means that the conjunction of P(1), P(2) up to P(k) is true
that is the definition of proposition Q(k).
And, since I have a strong induction proof for the property P, in the strong induction proof I can
conclude that if P(1), P(2), P(k) are simultaneously true then the property P is true for even k + 1
that is a guarantee that the strong induction proof the existing strong induction proof gives to me.
171
But if P(k + 1) is also true then I can say that the conjunction of P(1) and conjunction of P(2)and
conjunction of P(k + 1) is also true.
And this is nothing but Q(k + 1) that means starting with the assumption or the hypothesis that
Q(k) is true, I established the truth of proposition Q(k + 1) and that means I have given a regular
induction proof for proving the universal quantification involving the predicate Q. Internally
while proving this implication I use the fact that I have already a strong induction proof for the
universal quantification involving the predicate P.
So that brings me to the end of this lecture, just to summarize in this lecture we introduced the
proof by induction mechanism, we saw two forms of induction proof namely the proof by
regular induction and proof by strong induction and we also discussed that they are equivalent to
each other. Thank you.
172
Lecture -13
Tutorial 2: Part 1
Hello everyone, welcome to the part 1 of tutorial 2. So let us start with question number 1.
Here, you are supposed to find out whether the following argument is valid or not. So you are
given some premises and conclusion. So the first thing that we have to do is we have to convert
everything in terms of predicate functions. So we introduce appropriate predicates here. So of
course, the domain is explicitly not given here. But domain, the implicit domain here is the set of
students.
So the first statement here, the premise here is some math majors left the campus for the
weekend. So it is easy to see that this is an existential quantified statement, it is not making an
assertion about all the math majors. But let us first decide what are the predicates that we need
here. So the assertion is about math majors. So let M(x) be the predicate which is true if the
student x is a math major.
173
And we are saying something regarding whether he has left the campus for the weekend or not.
So that is the second property for the subject x. So that is why I introduce a predicate the W(x)
which is true, if the subject x or if the student x is left for the weekend. And I am making a
statement that there is some student x for which both these conditions are true, so that is why this
is an existentially quantified statement with conjunction inside.
The second statement here or the premise here is that all seniors left the campus for the weekend.
So this is a universally quantified statement. And if you see clearly or closely here, the
interpretation of this statement is that, if a student x is senior then he has left the campus. So
there is an implicit implication here and that is why this premise can be represented as ꓯx, S(x)
→ W(x).
The conclusion that I am making here is, some seniors that means existentially quantified
statement, are math majors. That means at least one student is there for which the property that
he is a math major as well as, he is a senior are true. Now we have to verify whether this is a
valid argument and as per the definition it will be a valid argument if, based on the premises I
can draw the conclusion for every possible domain.
However, it turns out that this is not a valid argument and we can give a counterexample. You
can give multiple counter examples here. Even if you show one counter example that is sufficient
to show that this argument form is not valid. So the domain that I consider is the following
imagine you have a college where you have 3 students x1, x2, x3. And say with respect to those 3
students the status of the 3 predicate functions are as follows.
For x1, the property M is true, W is true, but S is false. Student x2, the property M is false,
property W is true and the property S is true and so on. Now you can verify that with respect to
this domain and this assignment, the premises are true. Indeed there exists a student for which
the property that he is a math major and he has left for the weekend, he has left the campus for
the weekend are both true.
174
Namely x1 is one such student. And similarly the second premise namely all seniors have left the
campus is also true. So who are the seniors here? The seniors are x 2 and x3. And indeed x2 has
left the campus and x3 has also left the campus. So both your premises are true but what about
the conclusion, is there any student who is a math major as well as senior? Well, it turns out the
answer is no. That means my premises are true here, but my conclusion is false and that is why
this is a invalid argument.
Let us see question number 2. In this question, you are given two defined or two predicates
which are defined for you. I(x) denotes that a stamp collector has stamp x in her collection and
F(x, y) denotes that stamp x is issued by country y and you have to express the statement that this
collector has exactly one stamp issued by each African country. So I am making a statement
about a specific collector and I want to state that, for each African country, she has exactly one
stamp issued by that country in her collection. So, of course my domain here is set of all of
African countries. So if you recall from the lecture whenever we face this scenario where we
want to represent a property p is true for exactly one element of the domain then there are two
things which we have to represent. The first thing; that the property is true for at least one
element of the domain.
In this case, the property is that for every African country, there is one stamp at least issued by
that country, which is there in the collection of the collector. That is the first part here, which is
175
represented by this expression. So, this expression means that for every African country y, there
is at least one stamp x, such that the stamp x was issued by that country y and the collector has
that stamp x.
For the moment forget about what is there in the remaining part of the expression forget it. Just
focus on this part of the expression. But this is not what we want to represent because I cannot
stop with this expression because this expression also means that there might be multiple x
values for the same y, for the same country y where those other x stamps are also issued by the
same country y and the collector has those other x stamps in her collection.
That is not what we want to represent. We want to represent that exactly one value of x or one
stamp x is there for each country y. So that is why we have to put this second part of the
expression. For the moment forget about this negation. And whatever is there before the
conjunction forget about that as well. The second part of the expression denotes that, there can be
other stamps x’ issued by the same country y and which is also there in the collection of the
collector and you see I have very carefully put the parentheses here.
The scope of this y is still covered by this ꓯy, the scope of this universal quantification is carried
over to this y as well. And the scope of this x’, this x’ is again within it is a nested quantification
here, there exists x prime it is nested quantification falling within this ꓯy. So the second part of
the color expression denotes that there might be other stamps x’ issued by the same country y
which can be there in the collection of the collector.
But I do not want that to happen that is not what I want to represent. So that is why I put a
negation here and if I put a negation that means there cannot be any other stamp x’ different from
x, which is also there in the collection of the collector and x’ was issued by the African country
y. And that is why the conjunction of these two things represents the required statement.
Of course, you can simplify this, apply the De Morgan’s law and take this negation inside
convert everything, make everything in the form of an implication and so on that also you can do
but even if you write this expression, that is correct.
176
Part a of question 3, asks you to do the following. It asks you to give an example of a predicate
P(n) over the domain of non-negative integers such that the proposition P(0) is true, but the
universal quantification ꓯn P(n) → P(n+1) is false. So if you want to make P(n) → P(n+1) to be
false, ꓯn that does not mean you have to make it false for every value of n in the domain.
Remember the meaning of this universal quantification is that it will be true for all the universal
quantification will be true if it is true for every value of n in the domain. But even if it is false for
one value of n in the domain that shows that this universal quantification is false. So here there
can be multiple examples of such a predicate P. A very simple example is the following.
Say my property P is that integer n is even. When I substitute n = 0, the resultant proposition is
that 0 is an even integer, which is a true proposition but what about the statement P(0) → P(1). It
is false, because P(0) is, if 0 is even → 1 is odd. Which is clearly a false implication and that
means, since P(0) → P(1) is false, ꓯn P(n) → P(n+1) is automatically false. It does not matter
that P(1) → P(2), this is true.
Because P(1) is false, P(2) is true, false → true is true. Whereas P(2) → P(3) is false and so on.
So I have a statement here which is, for which this universal implication is not coming out to be
true for every value of n in the domain and that is why this is an example of such a predicate.
177
The part b of the question is an opposite of part a here. You are asked to give a predicate Q, such
that Q(0) is false, but the universal implication Q(n) → Q(n+1) is true.
So now my example here is that property Q(n) is defined that integer n is positive. It turns out
that Q(0) is false, because Q(0) is the proposition that 0 is positive and definitely 0 is not
positive. So, this proposition is false, but it turns out that Q(0) → Q(1) is true. Because Q(0) is
false the false implies anything is true and now any statement of the form Q(n) → Q(n+1)
everything will be true, that means now I can say that this universal quantification is true.
Now let us see question number 4. Here I have to show or I have to either prove or disprove that
the left hand side expression implies the right hand side expression. So you see the left hand side
expression, I have explicitly added the parenthesis here, so the x within the P and x within the Q
are different here whereas in the right hand side the x both within P and Q are the same. Because
both of them are covered by the same ‘there exist’.
Whereas, in the left hand side, the first x is covered by the first ‘there exist’ (ꓯ) and the second x
is covered by the second ‘there exist’ (ꓯ). The informal way to interpret the statement is if you
are given that property P is true for some element in the domain and if you are given that
property Q is true for some element of the domain, then can you conclude that both P and Q
property are true for some element of the domain.
178
And this need not be true. I can give you a very simple counter example, imagine a domain
where you have two values of x possible and say property P is true for x 1, but false for x2
whereas, Q is false for x1 and true for x2. In this case, you can check that your left hand side is
true, because indeed the property P is true for at least one value of the domain and indeed the
property Q is true for at least one value of the domain.
But that does not mean that it is the same x for which both P and Q are true. Individually P might
be true for some x and Q might be true for a different x. That does not mean ꓯan x for which
both P and Q property are true and which is happening in this case. So this is not a correct
statement. What about the part b is the implication in the reverse direction. It says that if you are
given that ꓯsome x value in the domain for which both property P and property Q are true.
Then you can conclude that individually the property P and Q are true for some value in the
domain. So we can prove this and the way we prove this is as follows. So since you are given, so
to prove that this implication is true, we have to show that if I assume left hand side is true, then
I have to show that the right hand side is also true. Because for all other cases an implication
always turns out to be true that means by false implies anything is true and so on.
So assume your left hand side is true, that means there exists some x value in the domain for
which both property P and Q are true. I do not know the exact value of that x, because my
domain could be very large. But I can say that that element x for which the left hand side is true
can be represented by c. So this is your existential instantiation. So I know that proposition P(c)
is true as well as the proposition Q(c) is true. I stress the value of c is not known here.
It is an arbitrary element, but it exists. Now since the conjunction of the two propositions P(c)
and Q(c) is given to be true. This is possible only if the individual propositions P(c) and Q(c) are
true. And if the proposition P(c) is true, that means I can say that existential quantification,
ꓯP(x) is true. And in the same way since the proposition Q(c) is true, I can say that the
existential quantification ꓯQ(x) is true.
179
Both of them are true, that means the right hand side is true. That means assuming left hand side
to be true I can conclude the right hand side is true and hence this identity is a correct identity.
The 5 question is a very interesting question. It asks you to show that there are infinitely many
prime numbers and there are several interesting proofs possible for this statement, let me show
one of them. So I am trying here a proof by contradiction. So that the statement I want to prove is
there are infinitely many primes but I assume a contradictory statement that there are only a
finite number of primes.
Say n number of primes, n could be anything it could be 2, 3 or 4 anything. Now what I do is I

define a new number Q which is the product of my finite number of primes, which I am
assuming to exist plus 1. Now what can I say about the number Q. There can be 2 possible cases.
Now, I apply the proof by cases here. My Q could be a prime number itself, my Q could be a
composite number and there cannot be any third case possible with respect to Q.
It turns out that, if Q is a prime number, then definitely Q is different from all your numbers
prime numbers P1, P2, Pn that are the only prime numbers you assumed to exist. That means now
you have found a new prime number. That means your listing of P1 to Pn is not an exhaustive
listing of all the prime numbers that exist. So you got a contradiction. Whereas it might be
possible that Q is a composite number.
180
If Q is a composite number I can show that none of these prime numbers P 1, P2, Pn will be a
factor of Q. None of them will divide Q. On contrary, assume say for instance P1 divides your
number Q. Now, if P1 divides Q, that means P1 is a factor of Q, and we know that P1 is a factor
of the product of P1 to Pn, because that has P1 in it. So that means you have now a number P1, a
prime number P1 which divides both Q as well as the product of P1 to Pn.
That means it will divide the difference of Q and the product of P1 to Pn. But the difference of Q
and the product of P1 to Pn is 1. That means P1 divides 1. But that is not possible because P1 is at
least 2, because you are assuming that P1 to Pn are primes and a smallest prime that is possible is
2. And 2 cannot divide 1. That means we have shown here that P1 cannot divide your number Q.
In the same way we can show that P2 does not divide Q.
In the same way we can show that Pi also does not divide Q. And in the same way I can show
that Pn does not divide Q. But Q definitely has a prime factor because that comes from my
fundamental theorem of arithmetic you take any number it can be expressed as product of prime
powers. That means it has definitely one prime factor, say P. But at the same time I am showing
that P cannot be P1, it cannot be P2, it cannot be Pn.
That again shows that I am missing a prime number P in my listing of prime number. That means
my list of prime numbers P1 to Pn which I assumed is not the complete list. So that is a
contradiction I will get in case 2. It turns out that very often students just give the following
argument which is an incorrect argument. They say that for surety since Q is not divisible by P1,
since Q is not divisible by P1, Q is not divisible by P2, Q is not divisible by Pn.
They end up with the conclusion that Q is definitely prime. That is not correct, let me
demonstrate that. Imagine that these are the only prime numbers which you assume to exist. Now
your Q in this case will be the product of all these prime numbers plus 1. And as per your
argument Q should be always prime, because it is not divisible by 2, it is not divisible by 3, it is
not divisible by 5, not divisible by 7, not divisible by 11 and 13.
181
But it turns out that Q is composite here, where the prime factors of this composite Q are 59 and
509. And these are the 2 primes which are not there because missing from your list of exhaustive
prime numbers, which you are assuming to exist. And that is why in case 2 we cannot simply
stop with the argument that Q is also a new prime number which I am finding because it is not
divisible by P1, P2, Pn.
The correct argument is that we will show that Q will have at least 1 prime factor, which is not
present in the list of prime numbers, which I am assuming to exist which is demonstrated by this
example.
Now, let us see question number 6 and 7 together. We will first equation number 6 and the
solution of question 6 will be used for question number 7. The question 6 says you have to prove
that there exists at least one real number among a set of n real numbers which is greater than
equal to their average. I stress here that a1 to an are arbitrary here. You cannot show concrete
values of a1 to an and prove this statement for those concrete values and conclude that this
statement is true.
This is a universally quantified statement. So how do we prove it? We have to take arbitrary
values of a1 to an and prove the statement with respect to those arbitrarily chosen values of a1 to
an. What we do here is we give proof by contradiction. So our goal is to prove that average of a1
182
to an is less than equal to some ai. But instead, I assume that each of the individual numbers
among these n numbers is less than their average.
That means the first number is less than their average. The second number is less than the
average of the n numbers and similarly the last number is less than the average of the n numbers.
That is a contradiction. Now if I add this n equations I get this inequality. And if I substitute the
value of the average by this formula I come to the conclusion that the summation of n numbers is
less than the summation of n numbers which is not possible which is a contradiction.
That means assuming this contradiction leads to a false conclusion that means the statement is a
true statement. That means you take any n real numbers, any n arbitrary real numbers, they could
be positive, negative, they may be the same, different. At least one of them will be greater than
or equal to their average. Based on this I want to solve question 7. In question 7, you are given
the following.
You are given the numbers 1 to 10 which are placed around the circle in any arbitrary order.
Maybe in ascending order, descending order, maybe the odd numbers first, next even numbers
and so on. So the order is not given. It is an arbitrary order. And the question says that it does not
matter in what order you arrange the numbers 1 to 10, there always exist 3 integers in that
arrangement which will be in consecutive locations.
Such that the sum of those three numbers will be greater than or equal to 17. So I stress here this
is with respect to any arbitrary arrangement of the numbers 1 to 10. So pictorially, you can
imagine that you are given this arbitrary circular ordering of 1 to 10 where, a1 can be any number
from 1 to 10, a2 could be any number from 1 to 10 and so on. I have to show that once I freeze
this arbitrary ordering.
In this arbitrary ordering, there exist collections of 3 integers, in 3 locations such that their sum is
greater than equal to 17. And I want to take the help of question number 6, whatever I have
proved in question number 6. So what I do here is since the question involves sum of 3 numbers
183
what I do here is once I freeze this circular arrangement of 1 to 10, I take the following sums, I
take the sum of first 3 numbers namely a1, a2, a3.
That is my S1. In the same way, I take the sum of next 3 numbers namely a2, a3, a4. I call it S2. I
take the sum of a3, a4, a5 that I call it as S3 and in the same way I take the sum of a10, a1, a2, that
will be my last sum namely S10. And what is my goal? The question says that either S1 is greater
than equal to 17 or S2 is greater than equal to 17 or sum Si is greater than equal to 17. That is
what I want to prove here.
Because I have taken the different possible sum of 3 consecutive numbers in this circular
arrangement. Now what I can do here is I can interpret S1, S2 up to S10 as 10 possible values.
That means let n = 10. Now, what can be what will be the average of these sums S1, S2, S10. It
does not matter what are the values in your circular arrangement. If you take the average of these
10 sum values, then in the denominator, you will have 10. Because n is 10 but when you take the
sum of S1 up to S10 each number in this arrangement will occur three times because a1 will be
occurring in S1. a1 will be occurring in S10 and a1 will be also occurring in S9.
In the same way, a2 will be occurring in S2, it will be occurring in S3 and S4, and so on. So each
of this value a1 to a10 will be occurring thrice when you take the average and when taking the
average you will be adding S1 to S10. And if my claim here is if you add S1 to S10 each of this
value say a1 to a10 will be occurring thrice. And possible values of a1 to a10 each of them belong
to 1 to 10 and only once they occur.
That means i know that if you add the value say a 1 to a 10 you are basically adding the numbers
1 to 10. And the summation of the numbers 1 to 10 is nothing but 55. That means I know that it
does not matter in what order the numbers are arranged. If I define sums like this and take the
average it will be 16.5. And from previous question I know that either S1 is greater than equal to
the average of S1 to S10 or S2 is greater than equal to the average of S1 to S10 and so on.
That means at least one Si is there which is greater than equal to 16.5. And each Si is an integer
because Si, S1 is the summation of three integers S2 is the summation of three integers. In the
184
same way, S10 is the summation of three integers, so each Si is an integer. So what is the smallest
integer, which is greater than equal to 16.5, well it is 17. So that shows that either S1 is equal to
17 or S2 is 17 and so on. So that solves your question number 7.
185
Lecture -14
Tutorial 2: Part II
Hello everyone. Welcome to the second part of tutorial 2.

So we start with question number 8. So here you have to use proof by induction to show that if
you are given n arbitrary positive real numbers, where n is some power of two. Then their
arithmetic mean is greater than equal to geometric mean and this is true for any collection of n
arbitrary positive real numbers provided n is some 2k. So this is a universally quantified
statement because I am making this statement for all n where n is equal to 2k.
So I have to prove a base case and I take the base case where k equal to 1, k equal to 1 my
statement true for two positive real numbers. This is two if you remember the proof mechanisms
we use a backward proof mechanism to prove that arithmetic mean of any two positive real
numbers is greater than equal to their geometric mean. So the base case is true. Now assume the
statement is true for any collection of n numbers n positive real numbers where n is 2k.
186
And, since it is true for n equal to 2k that means this expression or this inequality holds. The left
hand side is your arithmetic mean. The right hand side is your geometric mean. The geometric
mean will be the (2k)th root of the product of a1 to an which can be rewritten and the form that is
given here. Now, I want to prove the statement to be true for next higher power of n, next higher
power of n is 2(k+1).
So to do that what I do here is the following let me define x. So you are given now a collection
of 2(k + 1) numbers which you can split into two parts. You can consider the first collection of 2k
numbers and the next 2k, next 2k numbers in the list. So this part has 2k elements, this part also
has 2k elements this is your list a1 to an. So what I do here is I define the quantity x and y here.
So x is the Arithmetic mean of the first 2k elements in my collection and my y here is the
arithmetic mean of the next 2k elements and the collection, you can verify here. Now what I
know is that I can treat x and y as two numbers they will be positive real numbers and I know
that from my base case the arithmetic mean of x and y will be greater than equal to the geometric
mean of x and y.
So if I expand this, the arithmetic mean of x and y will be as follows; so this is your x, this is
your y. The arithmetic mean will be x + y over two, so one over two I am taking outside and
geometric mean will be (xy)1/2 namely the square root of x times y. So this is your x and this is
your y, this is what I get from the base case. Now, what I do is I apply the inductive hypothesis
on my right hand side.
Since I am assuming my inductive hypothesis to be true, I know that arithmetic mean of any 2k
elements is greater than equal to its geometric mean. So the portion that I have circled here it is
an arithmetic mean of 2k numbers, so that is greater than equal to the geometric mean of those 2k
numbers. In the same way the y here can be considered as arithmetic mean of 2k numbers.
And that will be greater than equal to the geometric mean of those 2k numbers the one over two
outside remains as it is. Now what I can do is I can take this 2k 1/2k , 1/2k appearing in the
exponent all together outside and multiply the first 2k numbers and the next 2k numbers, this is
187
just plain simplification. But, now if I rearrange everything or reinterpret everything the left hand
side is nothing but the arithmetic mean of 2k+1 numbers.
Because the arithmetic mean of 2k+1 numbers here will be a1 + a2 …+ a2k + a2k+ 1 upto a2(k + 1)
whole over 2(k+1) which I can rewrite in this form and your right hand side expression is nothing
but the geometric mean of the same 2(k + 1) elements. So now you can see that I am using the base
case here as well as the inductive step here to prove the inductive hypothesis here to prove my
inductive step. So that completes your question number 8.
In question 9, you asked to prove that every positive integer n can be expressed as sum of
distinct powers of two basically this is a fundamental fact that we learn that you take any positive
integer, it has a binary representation and a binary representation of that number is nothing but
sum of distinct powers of two. So that is what we want to prove here, you want to prove it can be
always possible to represent any positive integer is.
And the powers of two here will be distinct which is equivalent to saying that every positive
integer has a unique or distinct binary representation. So we will prove it by induction because
this is a universally quantified statement my base case will be n equal to 1. If my integer n is 1,
then I can represent this as 20. Namely the binary representation is zero, the binary representation
zero corresponds to 20 here.
188
If n is equal to one the binary representation is one here and a binary representation one
corresponds to 20 here. Let us take the inductive hypothesis here assume the statement is true for
n is equal to k that means you give me any integer k where k is arbitrary, it can be expressed as
the sum of distinct powers of 2 or it has a distinct binary representation. Making this hypothesis,
assuming this hypothesis to be true I will prove the inductive step and will show a unique binary
representation for the integer k + 1.
So, how do I proceed here? So I use proof by cases. Case one, if k is even; now if k is even and
since k has a binary representation a unique binary representation namely k is expressible as sum
of distinct powers of two and let that sum of distinct powers of two will be this. My claim here is
that 20 is not present in this existing binary representation or sum of distinct powers of two in the
representation of k because k is even.
If k is even you cannot have 20 also present along with other powers of two when you express k
as the sum of distinct powers of two because that will imply your k is odd. So since 20 is missing
in the representation of k and I want to represent k + 1 what I can do is I can take the
representation of k and to that I add a new power of two namely 20 that will give me the
representation of k + 1.
And since all the existing powers of two in the representation of k are distinct and none of those
powers were zero by adding this 20, I am not violating my condition, which I want to prove here.
My case two is when k is odd. Now, if k is odd then it follows that k + 1 will be even and if k + 1
is even that means it is divisible by 2 and k + 1 suppose it is l and l will be a number which is
definitely less than equal to k.
Since it is less than equal to k that means from the inductive hypothesis, it follows that there is a
binary unique binary representation for l and namely l can be represented as sum of distinct
powers of two and let that sum of distinct powers of two is this. Now what I know is k + 1 is
nothing but two times l and two times l can be obtained by just incrementing all the powers of
two that we had in the representation of l.
189
Since the different powers of two in the representation of l were distinct each of them
incremented by one will still give me distinct powers of two and now if I sum this new powers of
two that will give me the integer k + 1. So now you can see here that when I am proving it for
the case when k is odd, I am using a strong induction because I do not know what I cannot say it
definitely l is equal to k, l is k + 1 over 2. So it is any value in the range 1 to k.
So I have to use the inductive hypothesis; I have to assume it is true for all integers in the range 1
to l and that is why it is proof by strong induction.
Now come to question number 10. In question number 10, you are given the following, you are
having a party and n guests in a party and in the party of n people we call a guest to be a
celebrity, this is my definition of a celebrity, if every guest in the party knows the guest Gx while
Gx does not know any of the other guest. If that is the case then I will call the guest Gx as a
celebrity and our goal here is to find out whether there exist a celebrity in the party or not.
And for doing that given here a primitive namely you are asked to ask questions of the form
Knows(Gi, Gj). So if you ask, guest number i whether he knows guest number Gj then you can
get the 0, 1 answer depending upon whether guest number i know is guest number j or not that
190
means this is the only operation allowed to you. You can ask guest number i well, you know
guest number j or not and you can tell you whether he knows the guest or not.
And vice versa you can ask guest number Gj, you can ask guest number j whether he knows
guest number i or not and depending upon whether he knows or not you get answer 1 or 0. So the
question first part of the question ask you how many celebrities can be there in a party. Well, the
first thing is it is not necessary that there are exist definitely a celebrity in the party. It might be
possible that all the n people know each other. In that case none of them is a celebrity.
Because in that case there exist no celebrity, because everyone knows each other because the
condition of the celebrity is that the celebrity should not know any of the other guests. But if
everyone knows everyone then how can the celebrity be possible? So it turns out that if at all a
celebrity is there, there can be only one celebrity you cannot have two celebrities you cannot
have a celebrity Gx as well as a celebrity Gy simultaneously.
Because if Gx is a celebrity then he should know Gy and if Gy is a celebrity then he should not
know Gx, but since Gx is a celebrity, Gy knows Gx, because which gives you a contradiction. So
you cannot have two simultaneous celebrities possible in a party. If at all there is a celebrity you
can have exactly one celebrity.
So now in this question, we want to prove that in order to find celebrity in a party, it is sufficient
to make at most three times n - 1 number of calls to this Knows primitive. That means you can
ask at most, it is sufficient to ask at most three times n - 1 questions, asking various guests
whether they know other guest or not to find out whether a celebrity exists in the party or not. So
we will prove it by induction, before proceeding when I say three times n - 1 definitely for n
greater than equal to two.
Because it does not make any sense this expression three times n - 1 becomes 0; if n equal to 1.
So my claim here is that it is sufficient to make three times n - 1 number of calls in any party
consisting of two or more people to find out whether a celebrity exist or not. So we start with the
base case imagine you have only two guests G1 and G2. So to find out whether there exists a
191
celebrity or not, you just have to ask two questions, whether G1 knows G2 and whether G2 knows
G1 or not.
And two is definitely right for n equal to two the expression three times n - 1 is three times 2 - 1
is 3 and so you are able to find out the celebrity within the allowed limit here. So assume the
statement is true for n equal to k that is my inductive hypothesis that means assume you have an
arbitrary party consisting of n arbitrary guest where n is equal to k and three times k - 1 questions
or calls for Knows primitive or sufficient to find out the celebrity.
Now in that party if a new guest comes where the new guest is denoted by Gk + 1, I have to prove
that I can still find out whether the celebrity exist or not, by making three times k number of
calls. So this is assumed to be true, I have to show this. So here is my algorithm to find out the
celebrity among this k + 1 guests. I first ask the new guest who has joined the party whether he
knows the guest Gk or not and there could be two possibilities.
If indeed the new guest Gk + 1 knows the guest number Gk. Then I can rule out the possibility of
guest number k + 1 to be a celebrity, because he knows someone and as a celebrity he is not
supposed to know anyone. So Gk + 1 cannot be a celebrity and I have already asked one question
here. Now what I do is, since Gk + 1 cannot be a celebrity if at all a celebrity is there he will be in
the remaining group of k people.
So I check whether there exists a celebrity among the remaining group of k people and from my
inductive hypothesis these many calls namely 3 times k - 1 calls for the Knows primitive or three
times k - 1 questions are sufficient to check whether a Gx celebrity exists or not in the remaining
group of k people. Now there can be two possibilities; if in the remaining group of k people no
celebrity exist then I can simply say that there is no celebrity in the overall group of k + 1 people.
Whereas if I find a celebrity Gx in the group of first k people, I cannot say that he is also a
celebrity even if I include the k + 1 th guest, because I have to check whether the guest Gx who is
the celebrity among the first k guest knows Gk + 1 or not. So I have to now ask two more
192
questions; mainly I have to ask the guest Gx whether he knows Gk + 1 and the same way I have to
ask the Gk + 1, whether he knows Gx or not.
And then only I can confirm whether Gx is a guest in the whole group of whole bunch of k + 1
people or not. So in this whole process how many questions I am using/asking. So, I have
already asked one question here to check whether Gk + 1 was a celebrity or not. Now, I have one
question here, one question here. So total three questions there and in the group of k people, I
will be requiring, I will be making three times k - 1 number of calls.
So the total number of calls that I need here is summation of three and three times k - 1, which is
three k that is case one.
Case 2 would be the following; that guest number k + 1 does not know Gk. If the guest number k
+ 1 does not know guest Gk, then definitely Gk cannot be a celebrity because if at all Gk was a
celebrity then he should be known by everyone but k + 1 does not know Gk. So what now I have
to do is I now have to focus on the remaining k people excluding Gk. So I am excluding Gk
throwing out Gk and now I am left with only k people.
And I have to check whether it there exists a celebrity in the group of these k people that I can do
by making three times k - 1 number of calls. This follows from my inductive hypothesis and
193
there could be two possible cases. If in this remaining group of k people no celebrity exist then I
can say that no global celebrity exists. By global celebrity means a celebrity in the whole bunch
of k + 1 people.
Whereas if I find a celebrity Gx in this reduced bunch of k people, I have to check whether that
guest whether that celebrity knows Gk or not, eliminated party who cannot be the guest. Because
then only I can confirm whether Gx is the global celebrity or not. So now again in this case the
total number of questions that I am making that I am asking is three plus three times k - 1, which
is 3 k.
So in both cases, I have shown that it is sufficient to make three times k number of calls to find
out or check the possibility of a celebrity and that completes our inductive step and that proves
that the claim that I made here is correct.
In question 11, we are supposed to use strong induction to prove that √2 is irrational. Just to
recap we already proved that √2 is irrational using a proof by contradiction using proof by
contradiction, but here I am asking you to do the same thing to show the same thing using strong
induction so before starting the strong induction proof we have to first identify the universal
statement which we are trying to make.
194
Remember, an induction is used to prove a universally quantified predicate. So first we have to
identify what exactly is the predicate here. So the predicate P(n) here is the following; P(n) is the
predicate that √2 is not equal to n/b for any positive integer b and I want to prove that this
universal quantification is true using strong induction because if this universal quantification is
true, that means that √2 is not equal to one over any b and it is also not equal to two over any
integer b, it is also not equal to three over any integer b and in the same way it is not possible to
represent √2 in the form of any n over b and if √2 is not representable in the form of any n over b
that shows that as per the definition of rational numbers √2 is irrational. So how do I prove this
universal quantification? I start with the base case.
I start with the base case, my base case will be when n is equal to 1 that means √2 is not equal to
one over any b where b is a positive integer and this is obviously true because we know that the
value of √2 is greater than one and one over any positive integer will be strictly less than or equal
to one. So your √2 will be 1.44 something, something and on your right hand side is you have
integers of the form one or one over two, one over three one over four, and so on.
So your left hand side is always greater than right side. So that is why your base case is true here.
Now assume my inductive hypothesis is true, that means √2 cannot be represented in the form of
one over b, √2 is cannot be represented in the form of 2 over b and in the same way √2 is not
cannot be represented in the form of k over b.We want to prove that a statement is true even for k
+ 1.
Now to prove the statement is true for k + 1, I will be using a proof by contradiction and that is
allowed because overall I am using an inductive proof mechanism where I have to now prove
that this proposition P(k + 1) is also true, that I can prove using contradiction with the help of
induction. So, since I am using proof I contradiction I will assume that the proposition P(k + 1) is
false; that means √2 can be represented in the form k + 1 over sum positive integer b such that
the G C D of k + 1 and b is 1.
And now I recall the proof that I used to prove that √2 is irrational using the contradiction
method. So, since √2 is now assumed to be of the form k + 1 over b, I can get the conclusion that
195
(k + 1)2 equal to 2 b2, which means that k + 1, is even. So I can prove this and I can prove that if
the square of a number is even then the number itself is even. So the same thing we did even for
our earlier proof to prove √2 is irrational.
So I am not separately proving that, so since I come to the conclusion that k + 1 is even say 2 s
and if k + 1 is even then I also get the conclusion that b is even namely 2 t. that means I can say
that √2 can be represented in the form 2s / 2t, two cancels out and I get the conclusion that √2 is
of the form s/t where s is less than equal to k, this is because I started with √2 equal to k + 1 and
k + 1 is 2 s.
So, s will be definitely less than equal to k because your k + 1 is 2 s. So s is basically k + 1 over
two, so definitely s is at most k and that means the proposition P(s) is false because P(s) means
that √2 is not equal to s over any b. That is what is the proposition P(s), but I am getting the
conclusion here that √2 is some s over positive integer that means √2 can be represented in the
form s over some positive integer, that means P(s) is false here.
So, which gives me a contradiction because I assumed at P(s) is true and P(s) is true means this
is true, but I get here a conclusion that √2 is equal to s/t. So these two things contradict each
other.
196
In question 12, I am supposed to find out the number of diagonals in an n sided polygon and I
want to prove that it is n times n minus three over two, of course for all n greater than equal to
three and I will prove it by induction. So, let us first define the predicate which we want to prove
here. So the predicate here is that P(n) is true if the number of diagonals in n sided polygon is n
times n minus three over two and using induction we want to prove that for all n greater than
equal to three the property P is true for n.
Of course my base case will be three because I am making this statement to be true I am
assuming, I am making the claim statement is true for n greater than equal to three onwards. So,
of course the statement is true for any polygon with three sides because a polygon with three
sides is nothing but a triangle and you do not have any diagonal in a triangle. Assume the
statement is true for any polygon with k sides.
I now want to prove that the statement is true even for the polygon with k + 1 sides. So here is a
polygon with k + 1 sides and I want to count the number of diagonals here. So if I add the side, if
I add the vertex number one and with vertex number k, these are non adjacent vertices that
constitutes one of the diagonals in this polygon of k + 1 sides. Now, if I focus on this polygon;
this is now a polygon with k vertices or k sides.
And it has these many diagonals namely k times k minus three over two diagonals and these
diagonals also will constitute a diagonal of the overall polygon with k + 1 sides. So, I already
found these many diagonals in the overall polygon but now the question is that is are not the only
diagonals. I still have diagonals which I have not included in my list and these diagonals are the
diagonal obtained by connecting vertex number two with k + 1, the vertex three with vertex k +
1, vertex i with vertex k + 1 and like that vertex k - 1 with vertex k + 1.
None of this diagonals in this blue color where denoted by this blue color are included currently
in my enumeration process. Now how many such diagonals are there, which I have not included
yet namely the blue color was they are k - 2. Namely, I cannot count k + 1, 1, this side cannot be
considered as a diagonal this is not a diagonal and this is not a diagonal. So the remaining
possibilities are k - 2 number of diagonals, which are denoted by blue color.
197
And this gives me the total number of diagonals in the overall polygon and it comes out to be
what you want to show for your inductive step, that brings me to the end of tutorial number two.
Thank you.
198
Lecture -15
Sets
Hello everyone, welcome to this lecture on sets. The plan for this lecture is this follows, we will
introduce the definition of sets, we will introduce various set theoretic operations and we will
discuss various set theoretic identities.
199
So, what is the definition of a set? A very high level definition is, it is an unordered collection of
objects and what I mean by unordered collection of objects here is that ordering of the elements
in the set does not matter. So, for instance, if I have a set consisting of the elements 1, 2 and 3
and then it does not matter whether I list them as 1, 2, 3 or whether I list them as 3, 2, 1 both will
be the same sets.
It turns out that the elements of the set need not be related. So, for instance, if I have a set
consisting of entities, Narendra Modi, Manmohan Singh, Ashish Choudhury and 100 it is a valid
set as far as the definition of a set, because the definition does not say anything regarding the
properties of the elements of the same. We use some well known well-defined notations for
representing sets.
So, we use this notation ∈ for a belongs to A. So, this notation “belongs to”, whenever a is an
element of set A we use this notation. And throughout this course, we will follow the notation
that we will be using small letters for elements of the sets, and we will be using capital letters for
the sets.
200
Now, how do we express a set? There are two well known methods. The first method is the
Roster method, where we specify the elements of the set within braces. So, for instance if A is a
set consisting of 4 elements, then I have listed down the elements of the set A and this is a
convenient way of representing a set provided the number of elements in the set is small. If the
number of elements in the set is extremely large then it will not be feasible to write down or list
down all the elements of the set explicitly.
So, that is why we use the second form or second way of expressing a set which is also called as
the set builder form and what we do here is that instead of listing down the elements of the set,
we write down or state the general property of the elements of the set, which is specifically
specified by a predicate function. So, for instance here is a set A consisting of elements 1, 3, 5, 7,
9 in the Roster method.
The same set can be expressed in the set builder form where I can specify that A is the collection
of all odd positive integers x which are less than 10. That means I am basically stating the
properties of all the elements of the set A. So, I do not need to explicitly write down all the
candidate values of x satisfying this property. I am just specifying the general property and you
can imagine that this general property is a predicate function.
201
So, you can imagine that in the set builder form we specify the predicate function, which is
applicable on all the elements of the set and this is the most popular method of representing a set
specifically if we are dealing with an infinite set.
We often encounter some special sets. So, a null set or the empty set is one of them and this is
the notation ϕ which we use to represent the null set. This is also called as phi set or phi set and
it is a set which has no elements. So, you can imagine that a directory which has no files inside it
is an example of an empty set. Another special set which we encounter is the singleton set and it
is a set which has a single element in it.
Now an interesting question is that are these two sets the same? So, I have the set ϕ and I have a
set which has an element ϕ and it turns out that these two are different sets. If I consider the set
ϕ, then it is a set which has zero content, it has no element in it. Whereas if I consider the set
specified by this notation { ϕ } namely we have the braces, within the braces we have this ϕ and
this is a singleton set because it has one element and hence it has non-zero content.
So, analogy here is you can imagine that ϕ is an example of an empty directory, which has no
files inside it. Whereas this notation { ϕ } namely the set specified by this parenthesis within
which you have this ϕ can be interpreted as it is directory which has a sub directory or which has
specifically an empty subdirectory within it and clearly these two things are different. So, very
202
often people get confused.
They think that the set ϕ is equal to the singleton set consisting of element ϕ that is not correct
they are two different sets as soon as I put a parenthesis around ϕ the meaning completely
changes.
So, now we will introduce some definitions in the context of sets. So, we start with what we call
as equality of sets. So, intuitively if I have two sets A and B they will be called, or they will be
considered equal sets if they have the same elements. That means if I have any element present
in the set A it is present in B and in the same way any element which is present in B is also
present in A there is nothing which is extra present in A or which is extra present in B.
So, this is formally stated by the following definition. We say that the sets A and B are equal
provided the following statement is a tautology namely for all x, of course the domain of x here
is the set of elements in A and B which is not explicitly specified here. So that expression says
you take any x from the domain if it is present in A then it should be present in B and vice versa
because this is a bi-implication.
And this bi-implication will be tautology only when if left hand side is true right hand side
should be true and vice versa. The next definition is a subset of a set. So, if I have two sets A and
203
B then the set A is called a subset of the set B and for denoting that we use this notation ( ⊆ ),
provided the following holds, you take any element in the set A it should be present in B that
means it should not happen that there is something in A which is not there in B.
This is stated formally by saying that the following expression should be a tautology namely for
all x in the domain, if x is in A then it should be present in B. I do not care what happens if x is
not present in A. I do not care for those elements x. I am interested only for those elements x
which are present in A. My requirement is they should be present in B. So, my claim is that the
empty set is always a subset of any set does not matter whether the set A is empty or not.
The empty set is always a subset of any set and if you are wondering why this is the case then
you apply this definition on the set ϕ, then ϕ will be a subset of A. If the following implication is
a tautology namely for all x, if x is an element of ϕ it should be an element of A. And it turns out
that this implication is vacuously true because what is the premise here, the premise here is that
element x belongs to ϕ.
But that is false because ϕ is an empty set and x belonging to ϕ is defined to be a false statement.
It is a false statement because ϕ does not have any content. So, this statement is vacuously true.
This implication is vacuously true and that is why ϕ is always a subset of any set. You also have
what we call as proper subset of a set. So, A will be called a proper subset of B and for this we
use this notation ( ⊂ ).
So, you see here that this equal to symbol which was present in the notation for subset is missing
in this notation. So, we will say A is proper subset of B provided there exist at least one element
in B which is not in A. So, we still have the requirement at everything in A should be present in
B, but it might be possible that A is equal to B. If A is equal to B then in that case also we will be
saying A is a subset of B.
But when I say proper subset, by the word proper here, I mean that there is something extra in B,
which is not there any A. So, more formally the following expression should be a tautology for
every x in the domain if x is present in A it should definitely be present in B. So, this is the
204
requirement of subset, this captures the fact that A is a subset of B plus I need something extra.
That is why conjunction and what is the extra thing here, there should be some element y in the
domain which should be present in B, but it should be absent in A. At least one such y should be
there. If no such y is there but only this part is true, then the proper subset definition turns out to
be the same as that of a subset.
Next we define the cardinality of a set. So, we say that cardinality of a set S is n and for that we
use this notation. We use this to vertical bar symbols ( | | ) within S to denote its cardinality and
we write it is equal to n provided there are n elements in S where n is some non-negative integer.
So, n could be 0 or 1 or it should be some value belonging to the set of natural numbers or it
should be a non-negative integer.
So, if the cardinality is some n where n is a non-negative integer then we say that S is a finite set
else we say it is an infinite set. That means if we cannot express the number of elements in a set
by any non-negative integer then the cardinality of the set will be considered as infinite.
205
We next define what we call as the power set of a set and we use this notation P(S). So, you are
given a set S. And if I take the collection of all subsets of this set S, then that itself is a set
because I am just listing down the subsets of S and the elements here the elements of P(S) are the
subsets of S. So, if I list down all the subsets of S the resultant set is called as the power set. So,
let us try to find out that what will be the power set of ϕ.
So, it turns out that a power set of ϕ will be a singleton set consisting of the empty set, because
empty set is always a subset of itself or any set. What will be the power set of this singleton set
which has ϕ as its element, it turns out that the power set will have two elements because ϕ is a
subset of any set. So, we have ϕ subset of a singleton set consisting of ϕ and the singleton set
consisting of ϕ by default is always a subset of itself.
So, we have two subsets of the singleton set consisting of the element ϕ. Now a very
fundamental fact here is that, if the cardinality of your set is n where n is some non-negative
integer, then the cardinality of the power set will be 2n. There are 2 to the power n possible
subsets of a set consisting of n elements and this is a very interesting fundamental fact which can
be proved in several ways.
But what we will do now is we will try to prove it using the methods that we have learnt till now,
namely the proof mechanisms that we have seen till now. And since this is a universally
206
quantified statement, applicable for all n, a natural choice here to apply the proof by induction
namely we will prove the statement by induction on the value of n.
So, my base case will be n equal to 0, we start with base case n equal to 1. Of course, you can
start with base case of n equal to 0. If n is equal to 0 that means your set is the empty set and
hence its power set will have only one element. But I start with base case n equal to 1. So it is
easy to see that if my set has only one element then the power set will have two elements.
Namely there are two subsets of this singleton set the set ϕ and the set itself.
Now let us prove the inductive step and for that let me assume that the inductive hypothesis is
true. Namely I assume that the statement is true for any set consisting of k elements that means I
am assuming that if my set S has elements a1 to ak, then the number of subsets of this set S is 2k.
So, all these blue circles are the subsets of S and there are 2k such circles, that is my inductive
hypothesis.
Now I am increasing the number of elements in my set S by 1 more, I am adding 1 extra element.
A new element, which is not there in the existing set and now I have to find out the number of
subsets of this new set S. So, it turns out that all the old subsets of the old set S are going to be
subsets of this new set S. Because if I do not include ak + 1 in a subset of S, then that is still a
valid subset of S.
207
That means whatever sets for the subsets of old S they are still the subsets of new S and how
many such old subsets I have, I have 2k subsets because that is coming from inductive
hypothesis. And now what about subsets of this new S, which has the element ak + 1. It turns out
that those new subsets I can form by taking these old subsets and adding the new element ak + 1 to
it.
There is a 1 to 1 correspondence here. This is because in each of this old subsets the element ak +
1 was not present and if I add ak + 1 to that, that becomes a valid subset of S, and all these new
subsets, which I have constructed they have not been counted earlier and how many new subsets
I can form? It is exactly the same as the number of subsets of old S which is 2k. And it turns out
that any subset of S can be either of this type or of this type.
Namely it either will have the element ak + 1 or it will not have the element ak + 1. If it does not
have the element of ak + 1, that means those subsets are of the type this. Whereas the ak + 1 is
present, then those subsets are of type this. So, overall, how many subsets I obtain? 2k + 2k which
is 2k+1 and that proves my inductive step.
So, now let me introduce some set operations and most of you will be familiar with this. This is
the union ( ∪ ) of two sets and it is the collection of all elements x in the domain which are
208
present in either A or present in B. Of course, it might be present in both of them because this
disjunction will be true if the condition x belongs to A is simultaneously true as well as condition
x belongs to B is also simultaneously true.
Then we have the intersection of two sets ( ∩ ) and it consists of all the elements x in the domain
which are present in both A as well as in B. That is why we have a conjunction here. That is,
both the conditions x belonging to A and x belonging to B should be true. These are two
fundamental operations. Then we have the set difference is A - B is called as the set difference
and A - B consist of all elements from the domain which are present in A but not in B.
And „but‟ is represented by conjunction. Then we have this operation called A complement ( 𝐴 ).
And A complement is defined with respect to a universal set which you can imagine as kind of a
bigger set. So, if you subtract a set A from the universal set whatever is left, it is called as A
complement, denoted by this notation A bar (A). Now this is an important operation A cross B
which is called as the Cartesian product of A and B.
And what exactly this set is, well it consists of all ordered pairs of the form (a, b). Where the first
component of the ordered pair should be from the set A and the second component of the ordered
pair should be from the set B. So, if I collect all such ordered pairs then the collection of those
ordered pairs will becomes called A x B. It is important to note here that the order matters a lot
here when I take the Cartesian product when I am saying A x B, then the first component of the
ordered pairs should be from the set A and the second component should be from the set B
whereas if I take the Cartesian product of B and A then it will be collection of all ordered pairs
with the first component in the set B and the second component from the set A. So, it is not
always the case that A x B is equal to B x A that can happen only in some special cases.
Namely we can very easily show that the Cartesian product of A and B and the Cartesian product
of B and A are same provided one of the following three holds : either your A should be an
empty set because if A is the empty set then it does not matter what is B, A x B will always be
empty and B x A also will be empty. Same holds if B is equal to empty set. Whereas if A is equal
to B in that case also A x B and B x A will be same.
209
Because, all ordered pairs of the form (a, b) will also be encountered in the Cartesian product of
B x A. So, these are the only three cases when the Cartesian product of A and B will be same.
Otherwise the Cartesian product of A and B need not be same. So, let me demonstrate these
operations with some example here. So, I take these two sets A and B. Union means I pick all the
elements which are both in A and B.
So, since 1, 3, 5 are already there in A, I do not have to separately write down 1, 3, 5 again
because the definition of set says that I will be listing down the elements of the set only once
even if it is appearing multiple times. The same way the intersection here, so, what are the
common elements? I have 1 present in both A and B, I have 3 present in both A and B and I have
5 present in both A and B and it is a set 1, 3, 5, which will be considered as the intersection,
because the elements 1 3 5 satisfies this predicate condition in the definition of A ∩ B. What will
be A - B? So, the definition of A - B means all the elements which are only in A, but not in B.
That means I have to subtract out all the elements which are in B from A as well. So, 1 and 1
cancels out, 3 should not be included, 5 should not be included; that means I am left only with 2
and 4.
So, it is only the elements 2 and 4 which satisfies the definition of A - B. Now if I take the
Cartesian product of A and B. That means I will be now taking all elements of the form (a, b)
where a‟s will be coming from the set A and b's will be coming from the set B. So, these are the
elements which I have listed down in A x B whereas if I take the Cartesian product of B and A,
then it will be collection of all ordered pairs of the form (b, a).
Where b comes from B and a comes from A. And now, you can check that A x B is not equal to
B x A. So, for instance, you have the element (3, 1) present in B x A, but (3, 1) is not present in
A x B and there are many such elements which are there in one set, but not in the other set.
210
Now, there are some well known set identities which are available. We have some names also for
these set identities where each of these identities basically state that a set in the left hand side and
the set in the right hand side are the same. And we can prove them and assuming that these are
true we have associated names with them and whenever we want to simplify expressions
involving set, we can call these set identities.
So, the question here is how do we prove a set identity, if an identity is given to us, how do I
prove that the two sets A and B which are given in the left hand side and in the right hand side
they are same. So, for that we have to understand here that two sets A and B are equal if they are
respectively subsets of each other. Because the definition of A equal to B was for all x: x implies
211
x belonging to A bi-implication x belonging to B.
Now, if I split this bi-implication, this means if x belongs to A it should belongs to B as well, and
bi-implication can be splitted into conjunction of two implications. Now this condition means of
course everything is with respect to for all x, this condition means A is a subset of B and this
condition means B is a subset of A. That means to show that two sets A and B are equal, I have
to show that A is a subset of B and B is a subset of A.
Namely I have to prove two implications. And the two implications are I will be taking some
arbitrary element x and assuming if it is present in A I have to show it is present in B. Why I am
taking x to be arbitrary here? Because remember I want to prove this universal quantification.
So, since this statement has to be proved for every x in the domain. I cannot take every x in the
domain and prove this implication to be true.
And that is why I apply the universal generalization here. Where the, universal generalization
says that to prove this universal quantification, you prove the universal quantified statement to be
true for some arbitrary x in the domain. If you prove it to be true for arbitrary x in the domain
you can conclude it is true for every x in the domain. So, that is why I am taking my x here
arbitrary. So, to prove that A and B are subsets of each other these are the two implications I
have to show.
So, let me demonstrate what I said with respect to this example. I want to prove the De Morgan's
law. The De Morgan‟s law is there are two variants of De Morgan‟s law. I am proving one of
them. It says that if you take the complement of intersection of A and B that is same as the union
of A and B, of course, here everything is with respect to some universal set, everything is with
respect to some universal set U.
Because whenever a compliment is coming into picture, we have to assume that there is some
universal set U. So, I have to prove that everything here is also present in here. And everything
in the right hand side set is also present in the left hand side set. So, these are the two things you
have to prove. So, let me prove that everything in the left hand side set is also present in the right
212
hand side set. So, let x be some arbitrary element present in the A ∩ B.
Since the x is present in the A ∩ B, that means it is not present in the A ∩ B because that is a
definition of this complement operation. Whatever was present in A ∩ B, if I separate it out that
will give me the compliment; that means I can say that for those x which are present in the
complement of A ∩ B this condition holds. The negation of this condition holds.
Now what I can do is I can apply the definition of intersection. Since x is present in A ∩ B, that
means x is present in both A as well as in B. And now I can apply the De Morgan‟s law of
predicate logic. So, what I can do is I can take the negation inside and this conjunction gets
converted into disjunction. And negation gets splitted over the individual expressions.
But if I see here the negation of this statement namely negation of x belonging to A means x
belongs to A because that is coming from the definition of A. In the same way this thing, that
negation of x belonging to B means, x belongs to B because that is the definition of B. And now,
I apply the definition of disjunction here.
If x is present in A or if x is present in B, that means it is present in the A ∪ B. And throughout

my x was arbitrary, that means I have shown that you take any member of the set A ∩ B, it will
be present in the set A ∪ B. And now I can show the other way around as well.
I take an arbitrary element x present in the A ∪ B and by applying simple rules of logic here and
using definition of A and B and negations. I end up with the conclusion that the element x will be
also present in A ∩ B. So, that is how we prove the set identities. We have to show that both the
left hand side set and the right hand side sets are same.
And for that I have to prove that both the sets are individually subsets of each other. So, that
brings me to the end of this lecture, just to summarize in this lecture we introduce the definition
of sets we introduced set theoretic operations and set theoretic identities.
213
Lecture -16
Relations
Hello everyone, welcome to this lecture on relations. Just to recap, in the last lecture we
introduced the definition of sets and various set theoretic operations, we also saw various set
theoretic identities. The plan for this lecture is as follows. In this lecture we will introduce what
we call as a relations, we will see their various properties, we will discuss how to represent
relations and we will see some special types of relations.
214
So, what is a relation? So, let me begin with this example, you consider this table and people
who are familiar with databases, they very well know what is a table. Table basically consist of
several columns and each column have some entries. So, I call this table T which has two
columns, column number 1 and column number 2. In column number 1, you have some countries
and in column number 2, you have some cities listed.
Now if your general knowledge is good then it turns out that the elements you can spot here that
elements of the first column and the second column they are related by some relationship. And
the relationship here is that, in the first column I have listed some of the countries and in the
second column I have listed the capital of the corresponding countries. Of course, I can add
multiple entries in this table and I am not doing that.
Now how do we mathematically interpret this table? Is there any mathematical interpretation or
mathematical abstraction by which you can define this table? Well the way I can mathematically
interpret this table T is as follows. I can imagine that I have a set A which is defined to be the set
of all countries in the world. Well I do not know right now how many exact numbers of countries
in the world definitely it is more than 200.
So, A has more than 200 elements. Whereas B is another set, which is defined to be the set of all
cities in the world. Again this is a well defined set because we know the list of all cities in this
215
world, so both these sets are well defined. Now if I take the Cartesian product of A and B, what
will I obtain? The Cartesian product of A and B will be a set of the form (a, b), where a will be
some country.
Namely, it will be belonging to A and b will be some city. As of now when I take A x B, there is
no relationship between the elements a,b, I am just picking some country and some city, country,
city. I have listed down all possible pairs of the form country, city and this will be an enormously
large set. Now if I take a subset of that A x B, a special subset of that A x B and call it T and
what is that special subset?
Namely I take a subset where the ‘a’ component is a country and ‘b’ component will be the
capital. And if I take only components or pairs of the form (a,b) of this form, where a is the
country and b is the capital and list down all such (a, b) pairs I obtain this table T. So, you can
imagine that this table T here is nothing but a special subset of A x B. So, to demonstrate, my A
x B could also consist of elements of the form Afghanistan, New Delhi, it will also have
elements of the form India, Cairo and so on. I am not taking those elements, I am taking only
those elements (Afghanistan, Kabul) from A x B, I am taking only the element (India, New
Delhi) and so on. So, I am taking a subset of A x B and only those subsets which have a special
relationship among the, a component and the b component.
So, that is a loose definition of a relation. A relation here is basically a subset of A x B, if I am

considering two sets A and B and of course whatever I have discussed here can be extended
where I have multiple sets. What do I mean by that? In this example I had only two columns, C1
C2. And C1 was having entries from set A and C2 was having some entries from B. What if I
have a database consisting of 3 columns?
Say there is a third column as well, where the third column denotes population. So, those entries
will be coming from a set C, what if I have a fourth column which denotes another feature of the
table, say the climate or the temperature of the respective countries. So, those entries will be
coming from another set D and so on. And the table with some specific entries will be considered
as a subset of the Cartesian product of all the big sets A, B, C, D from which the elements in
216
your column C1, C2, C3, C4 are occurring here.
So, that is how we are going to define a relation, so we will focus in mostly on binary relations
and by binary relations I mean, we will be working with two sets A and B, but whatever we are
discussing here can be generalized for extended for any number, it can be generalized for n-ary
relations which are defined over n sets. But for this course and for most important cases, we will
be focusing on binary relations.
So, how do we define a binary relation? So, we are given two sets here, call them A and B and
they need not be different, I stress here, they can be the same, definition does not say that they
have to be different sets, because we are defining a relation in an abstract fashion. Then a binary
relation from A to B is a subset of A x B. So, I have highlighted the term from A to B by a
different color because the order of the relation matters.
So, if you are defining a relation from A to B then the relation should be a Cartesian product of
A x B. Whereas if your relation is from B to A, then that relation should be a Cartesian product
of B x A or it should be Cartesian product of B and A, it should be a subset of B x A. So, the
order matters here a lot. Now when I say a subset my relation R could be empty as well, that
means my table could have zero entries, that is also possible, that is also a valid table.
217
So, it is not necessary that a relation always should have at least one element of the form (a,b) it
could be empty as well. So, we use some notations here when dealing with relations. So if the
element (a, b) belongs to the relation R then I will be often writing this expression aRb, I will be
saying a followed by capital R followed by b, to denote that a is related to b whereas if the
element (a,b) does not belong to the relation R, then I will strike off R in this expression. So, as I
said here when I am defining the binary relation my sets A and B could be the same, they could
be different. So, I can define a relation from the set to itself so let me demonstrate this. So,
imagine I have a set A consisting of the elements {1, 2, 3, 4} and I define a relation R consisting
of all elements of the form (a, b) we are both a and b are from the set A such that a divides b.
So, basically I am defining a relation over A x A. So A x A here will have many elements here, it
will have (1, 1) it will have (1, 2), (1, 3), (1, 4). It will have the elements (2, 1), (2, 2), (2, 3), (2,
4), (3, 1), (3, 2), (3, 3), (3, 4) and (4, 1), (4, 2), (4, 3), (4, 4). So, I will be taking only those
components, (a, b) from this collection A x A where the first component a divides the second
component b.
So, it turns out that I will be taking only (1, 1) because one divides one. I will be taking (1, 2), I
will be taking (1, 3), I will be taking (1, 4). But I would not be taking (2, 1) in the relation R,
because for (2, 1), a is 2, b is 1 and 2 does not divide 1. I will be taking (2, 2) but I would not be
taking (2, 3) because 2 does not divide 3, I will be taking (2, 4) and so on. So, the elements of the
relation are listed down here.
I could define another relation, I could define a relation R’ collect consisting of all (a, b) such
that a < b. That is another relation and that will be consisting of other pairs, it might be
consisting of other pairs different from the pairs which you have listed down in R. So, it depends
upon the property which you want to be satisfied by the elements of the relation that defines that
tuples or the pairs which will be present in that relation.
218
So, now an interesting question is that if you have a binary relation defined from the set A to B
how many such binary relations can you define? Can I define any number of binary relations or
is there an upper bound on the maximum number of binary relations that I can define? Namely
how many tables I can form with two columns, where the first column can take entries from the
set A and a second column can take entries from the set B.
Well it turns out that I can form 2mn number of binary relations provided A has m number of
elements and B has n number of elements. And this simply comes from the observation that you
take any relation R, it is nothing but subset of A x B. So, you have this bigger set A x B
consisting of all elements of the form (a, b) where a is from A and b is from B, you pick any
subset of this A x B that gives you a relation.
So, the number of binary relations is nothing but how many different subsets of A x B you can
form. Namely the cardinality of the number of relations will be the same as the cardinality of the
power set of A x B and what will be the power set of A x B, so you recall the theorem that we
proved in the last lecture. The last lecture we proved that if the cardinality of a set S is n, then the
cardinality of the power set is 2n.
There will be 2n subsets of a set consisting of n elements. Now what will be the cardinality of A
x B? Well the cardinality of A x B will be m times n. Why? Because it is the collection of all (a,
219
b) pairs, where a is from A. So, how many different values of a you can have m and how many
different values of b you can have, n. So, you have m times n number of (a,b) pairs, which will
be present in A x B.
So, your cardinality of A x B is mn and that means the number of subsets of A x B that you can
form is 2mn. That means these 2mn is the maximum number of tables that you can form with 2
columns, where the first column takes entries from the set A and the second column takes the
entries from the set B.
Now the next question is how do we represent binary relations? So, there are some well known
methods for representing binary relations, the first method is the matrix representation. So, since
we are dealing with binary relations the matrix representation here will be an m x n matrix, m
because there are m possible elements from the set A and n columns because I have n possible
elements from the set B.
And I will be denoting this matrix by MR, where R is the relation. And what will be the entries of
this matrix. So, it will be a Boolean matrix where the entry number ith row and jth column will be
1 provided the element (ai, bj) is present in the relation R. So, what I am saying here is that
imagine you have constructed a table or a relation R, so it will be either consisting of 0 number
of rows or some number of rows and each row will have 2 columns.
220
So, if you have a row where an element ai is present in the first column and bj is present in the
second column, that means you have (ai, bj) present in the relation R, then in the matrix
representation what you will do is you will go to the ith row, jth column and you will put an entry
1. Whereas if the entry number (ai, bj) is not there in your database or in the relation R if it is not
present, then the ith row and jth column entry will become 0.
That means you want to denote there that element ai is not related to the element bj as per your
relation R. So, that is the matrix representation, so if I want to now look back into the question
that how many binary relations I can form, well it turns out to the answer for how many different
or distinct Boolean matrices of dimension m x n you can construct because each Boolean matrix
will correspond to one binary relation.
You cannot have two different relations or two different tables represented by the same Boolean
matrix that is not going to happen. So, this is the matrix representation for representing a
relation, we have another representation which we call as the directed graph representation. So,
what do we do in this representation, we draw a graph and by graph I mean a collection of
vertices and edges.
The vertices will be the nodes a1, a2, am, and b1, b2, bn and it will be a directed graph that means
the edges here will have a direction associated, how the edges are added in this graph? So, if this
graph represents the relation R, and if in the relation R the element ai is related to the element bj,
that means in the database of R, you have a row with first column being ai and the second
column being bj, that means ai is related to bj.
Then what you will do is you will take the node ai here and bj here and you will add a directed
edge from ai to bj, so the edge here denotes that ai is related to bj. The direction here matters, if
the edge from ai to bj is present that does not mean that the edge from bj to ai is also present. That
depends whether bj is related to ai or not, you might have a relation where only ai is related to bj.
But bj is not related to ai in which case the reverse direction edge may not be present. And now
221
you can see here that the matrix representation and the directed graph representation they are
equivalent to each other. If you have the entry ai, bj in the matrix representation 1, then you will
have the edge from ai to bj, whereas if in the ith row and jth column of the matrix, you have the
entry 0, then this edge from ai to bj will not be present.
You might be wondering why we have two different representations. We will be seeing that
depending upon how we want to prove or whether what kind of properties we want to prove
regarding the relations, the representation matters a lot. There might be cases where if we use the
matrix representation then the arguments become very simple, whereas there might be cases
where we want to deal with the directed graph representation.
Now, we would define some special types of relations. So, the first special type of relation is a
reflexive relation, it is a relation defined from the set to itself. And when do we say that the
relation is reflexive, as the term reflects here signifies, the relation will be called a reflexive if
every element from the set A is related to itself as per the relation. That means you take any
element from the set A, the element A should be related to itself as per the relation.
If this is true for every element a, from the set A then I say that my relation is reflexive. Even if
there is one a, one element a, which is not related to itself in the relation, then it will not be
considered as a reflexive relation. So, now if I want to interpret the matrix representation of a
222
reflexive relation, it is easy to see that its relation R is reflexive. Then the matrix MR will be n x
n matrix.
Because the relation is defined from the set to itself and if it is reflexive then all the diagonal
entries will be 1. Because I will be having (a1, a1) present in the relation, I will be having (a2, a2)
present in the relation and in the same way I will be having (an, an) also present in the relation.
So, since (a1, a1) is present that means the entry (1, 1) will be 1, (a2, a2) is present that means the
entry (2, 2) in the matrix will be 1.
Since the an is related to an that means the entry number (n,n) in the matrix will be 1 which is
equivalent to saying that all the diagonal entries will be 1. There might be additional entries in
the relation apart from these reflexive entries that is also fine. The definition of the reflexive
relation says that you only want the guarantee that every element from the set A should be
related to itself as per the relation.
There might be other elements which are related to the elements of A as per the relation R, I do
not care about those elements. If I focus on the graph representation of reflexive relation, then it
will be a special type of graph where I will be having a self-loop at each node of the graph.
Because a1 is related to itself, that means I will be having a loop or a directed edge from a1 to a1.
Since a2 is related to itself, I will be having an edge from a2 to itself and so on.
So, now let us see some examples of reflexive relations. So, I have a set here A consisting of 2
elements {1, 2} and I have given you many relations over this set. We have to find out which of
these relations are reflexive relations. So, let us start with the relation R1. It is reflexive because
indeed the element (1, 1) is present in the relation and the element (2, 2) is present in the relation.
So, this satisfies the definition of reflexive relation.
My relation R2, it also satisfies the definition of reflexive relation because (1, 1) is present (2, 2)
is present and I have (1, 2) is also present here, but that is fine because even the element (1, 2)
does not violate the truth of this universal quantification for the relation R2. What is this
universal quantification? It says that if 1 is present in the set A then (1, 1) should be in your
223
relation.
If 2 is present in your set A then (2, 2) should be present in your relation. That is all, it does not
say whether anything additional is present or not. Now I come to the relation R3, my relation R3
is not a reflexive relation, because (2, 2) is not present here. That means if I take this definition
of reflexive relation here, this universal quantification is not true, because indeed I have an
element 2 in my set A, and 2 is present in A, but (2, 2) is not present in R3, this is not happening.
That means this universal quantification is false and that is why R3 is not an example of reflexive
relation and what about R4? Again, R4 is not a reflexive relation over A. Because I have 1 present
in A, but (1, 1) is not present in R4. If I consider this implication, this implication is false because
1 is present in the set A, but (1, 1) is not present in R4.
Because R4 is empty in the same way 2 is present in A, but (2, 2) is not present in R4, that means
this implication is also false. And since both the implications are false here this universal
quantification for this relation R4 is not true. That is why relation R4 is not an example of
reflexive relation. Now here is an interesting question for you: can it happen that the set ϕ or the
relation ϕ is a reflexive relation over a set A.
So, remember ϕ is also a relation because ϕ is a subset of Cartesian product of any A x A. So, the
question here is: is it possible that ϕ is a valid reflexive relation over some set A? Might look no,
but the answer is yes. If A is empty by itself then ϕ is a valid reflexive relation over A. This is
because if A is ϕ, then A x A will also be ϕ. And the universal quantification which is there in
the definition of reflexive relation will be true for the relation R equal to ϕ.
So, you take R equal to ϕ, so remember ϕ is always a valid subset of A x A and for the relation R
equal to ϕ this universal quantification is true for a being ϕ, because the definition says for every
a belonging to ϕ. But no element belongs to ϕ, so vacuously this implication will be true for the
relation R equal to ϕ. So, we can have ϕ as a valid reflexive relation provided the set over which
the relation is defined is an empty set. But if A over which the relation is defined is non empty,
then ϕ can never be a valid reflexive relation.
224
Now let us define another special relation defined from the set to itself which is called the
irreflexive relation. And the requirement here is that you need that no element should be related
to itself in the relation that means you take any element a from the set A, so this universal
quantification over the domain is the set A. You take every element a from the domain or the set
A, (a,a) should not be present in the relation
Or the element should not be related to itself. So, it is easy to see that if your relation R is
irreflexive, then none of the diagonal entries should be 1 in the relation. So, the matrix for your
irreflexive relation will be an n x n matrix. Because the relation is defined from the set A to itself
and (a1, a1) is not there in the relation, that means the entry number (1, 1) in the matrix will be 0.
Similarly (a2, a2) is not there in your relation.
That means the entry number (2, 2) in your matrix will be 0 and so on, that means the diagonal
entry will be just consisting of 0’s or equivalently in terms of the graph representation no self
loops will be present, because a1 will not have any directed edge to itself, a2 will not have any
directed edge to itself and so on. So, again, let me demonstrate irreflexive relations here, so my
set A is {1, 2} and I have taken the same 4 relations here.
It turns out that relation R1 is not irreflexive because you have both (1, 1) and (2, 2) present.
Similarly R2 is not irreflexive, R3 is also not irreflexive because you have (1, 1) present here,
225
whereas R4 is a valid irreflexive relation because no element of the form (a, a) is present in R4.
Now it might look that any relation which is reflexive cannot be irreflexive or vice versa but or
equivalently can we say that is it possible that I have a relation which is both reflexive as well as
irreflexive defined over the same set A.
Well the answer is yes because if you consider the set A equal to the empty set, and if you take
the relation R, which is also the empty relation. That is the only relation possible over an empty
set A then this relation R is both reflexive as well as irreflexive. It is reflexive because at the first
place there is no element present in your set A and hence there is no chance of existence of any
(a, a), present in the relation R equal to ϕ.
And due to the same reason since no element is present in the set A you do not need any (a,a) to
be present in R. So, the relation R equal to ϕ satisfies the implication, this universal implication
given in the definition of reflexive relation as well as irreflexive relation vacuously. So, we can
have a relation defined over a set which can be simultaneously reflexive and irreflexive and that
can happen in the special case when the set is an empty set.
If A is non empty, then definitely you cannot have a relation which is both reflexive as well as
irreflexive.
226
Now let us define symmetric relations, so this relation can be defined from a set A to B where B
is might be different from A. So, the relation is from A to B and we say it is symmetric, so as the
name suggests symmetric we want here the following to hold, whenever a is related to b as per
the relation R, we need that b also should be related to a and that is why the term symmetric here
and of course this universal quantification is the domain of a is A and domain of b is B.
I stress here this does not mean that you need every element of the form (a, b) and (b, a) to be
present in the relation R, this is an implication. The implication here says that if (a, b) is present
in R, then only you need (b, a) to be present in R. If (a, b) is not present at the first place in the
relation, then I do not care whether (b, a) is there or not. I do not need (b, a) to be present, so the
implication puts the restriction that this condition should be there should be true only if (a, b) is
there in the relation.
So, it is easy to see that the matrix for a symmetric relation will always be a symmetric matrix,
because if you have ai R bj, that means the i, jth entrry will be 1 and since my relation is
symmetric, that means I will also have (bj, ai) to be present. That means if I take the transpose of
MR, then in the jth row and ith column, the entry will be 1. Equivalently in terms of directed graph
representation, if I have a directed edge from the node ai to bj and since my relation is symmetric,
the edge from bj to ai will also be present. So, again let us do this example, I have set A = {1, 2}
and I am defining various binary relations from A to A itself. That means in this case my A is
equal to B here. Now which of the following relations are symmetric. So, it is easy to see that the
first relation is a symmetric relation because this condition is true here.
I can say that since (1, 1) is present in the relation, I also have (1, 1) which can be interpreted as
(b, a), also present in the relation. Due to the same reason since (2, 2) is present in the relation
which can be interpreted as a being 2 and b being 2, I also have (b, a), present in the relation.
Similarly the relation R2 is a symmetric relation, the relation R3 is also a symmetric relation
because I have (1, 1) present in the relation.
And for symmetric relation (1, 1) also should be present in the relation, which is the case. Turns
out that ϕ, is also a symmetric relation here. It satisfies the requirement of symmetric relation
227
because at the first place there is no (a, b) present in my R4. That means vacuously this
implication that this universally quantified statement is true for R4. And that is why R4 is also a
valid symmetric relation.
But R5 is not a valid symmetric relation because I have (2, 1) present in my relation but (1, 2) is
not present in the relation. So, here is a question for you, can I say that every reflexive relation is
also a symmetric relation? So, remember reflexive relation means every element of the form (a,
a) will represent in R. And apart from that I might have something additional also present in the
relation.
So, if you are given a relation which is reflexive can I say that definitely it is also a symmetric
relation and the answer is no. Take the example where A is equal to {1, 2} and let me define a
relation R consisting of (1, 1), (2, 2) and say the element (1, 2). This relation is a reflexive
relation, but this is not a symmetric relation. But this is not symmetric because you have (1, 2)
present in the relation, but you do not have (2, 1) present in the relation.
Now the next special relation is the asymmetric relation and the condition here is, if you have a
related to b in the relation, then you demand that b should not be related to a. And again this is an
implication that means this should hold only if (a, b) is present in the relation at the first place, if
(a, b) is not present in the relation, vacuously this statement will be true. So, in terms of matrix
228
notation the property of matrix for an asymmetric relation will be as follows.
You take any i, jth entry, ith row and jth column, you can have at most one of the entries i, j or j, i
being 1 in the matrix. You cannot have both entry number i, j 1 as well as j, i also 1. Because that
will mean that you have (ai, bj) present in R, and (bj, ai) also present in the R, which goes against
the definition of asymmetric relation. This automatically means that the diagonal entries will be
0.
Because if you have (a, a) present in the R, then that violates the universal quantification here,
that serves as a counter example because you have (a, a) present in the R and this (a, a) can be
treated as again (a, a) with a and b so here a only is playing the role of both a as well as b. So,
you have (a, b) as well as (b, a) both present in this relation R and that serves as a counter
example for this universal quantification and hence your relation will not be asymmetric.
So, none of the diagonal entries will be 1. In terms of graph representation, if you take any pair
of nodes (ai, bj) then either you can have at most one edge, that means you can have either the
edge from ai to bj or from bj to ai or no edge between ai or bj. So, this is a wrong statement here,
so either edge ai to bj or no edge, that is also fine. Because if at the first place there is no
relationship between ai and bj then that vacuously satisfies this universal quantification.
So, again, here I am taking A and B to be the same sets and I have given you some relations. So,
let us see which of these relations are asymmetric. The first relation is not asymmetric because
you have (a, b) as well as (b, a), only (1, 1) being present in this relation which serves as both
(a,b) as well as (b, a). Due to the same reason R2 is also not an asymmetric relation because you
have both (a, b) as well as (b, a) present here.
Some a and b is there for which this universal quantification is not true. Your relation R3 is also
not asymmetric because you have (a, b) here written as well as (b, a) also present. Whereas the
relation R4 is an asymmetric relation over the set A, because at the first place there is no (a, b)
present in this relation R4, so R4 vacuously satisfies this universal quantification and R5 is also an
asymmetric relation because you have only (a, b) present in this relation but no (b, a).
229
The next special relation is antisymmetric relation and the requirement here is the following.
You want that if both (a, b) and (b, a) are present in your relation, that means if you have a case
where an element a is related to b and b is also related to a, then that is possible only if a is equal
to b. Contra-positively if a is not equal to b, then you can have either (a, b) present in the relation
or (b, a) present in the relation or none of them being present in the relation.
That means for distinct elements, you cannot have simultaneously a R b as well as b R a. That is
what is the interpretation of this condition. So, in terms of matrix properties if you focus on ith
row and jth column where i and j are distinct, then only one of those entries can be 1. Of course
both of them can be 0, that is also fine, because that means that neither a R b or nor b R a.
The condition demands that if at all a and b and b and a are both present in the relation, then that
is possible only when a and b are same, if they are different and you cannot have both (a, b) as
well as (b, a) present in your relation. In terms of graph theoretic properties if you have 2 distinct
nodes ai and bj, then you cannot have an edge simultaneously from ai to bj as well as from bj to ai,
that is not allowed.
Well, it is fine if you have no edge between these two nodes, that satisfies, that does not violate
this universal quantification. So, here are some examples, relation R1 is an antisymmetric relation
230
because you have (a, b) present here namely (1, 1) and you also have (b, a), present here namely
(1, 1), but the implication should be that 1 equal to 1 which is true, same holds for the element
(2, 2).
So, this is an example of an antisymmetric relation. But R2 is not an example of antisymmetric

relation because you have a case here namely you have distinct (a, b) such that both (a, b) as well
as (b, a) are present in your relation. R3 is an example of an antisymmetric relation and R4 is also
an example of an antisymmetric relation because it satisfies this universal quantification
vacuously.
R5 is also an example of universal quantification, because you have (a, b) present here, but the
(b, a) is not present in the relation R5, that means the premise of this implication is vacuously
true for R5 and that is why this R5 is not violating this universal quantification.
So, we have symmetric relation, asymmetric relation and antisymmetric relation. These are the
definitions here and people often wonder that there is some relationship among these three
different notions here, this some people think that something which is not symmetric will be
asymmetric and similarly they try to conclude some relationship between the symmetric
property, asymmetric property and antisymmetric property. But it turns out there is absolutely no
relationship. You might have a possibility where you have a relation which satisfies all the three
231
properties, namely if I take the set A and I take the relation R to be empty, then the relation R,
which is an empty set here is symmetric as well as asymmetric as well as antisymmetric, it
satisfies all these 3 universal quantifications.
Whereas there might be a possibility that a relation satisfies none of these 3 properties. So, if I
take the set A to be {1, 2, 3}, and I take this relation R then it is not so, let us see which of these
three properties it satisfies, so it is not symmetric because you have (a, b) here, but no (b, a).
Only 1 is related to 2, but 2 is not related to 1, so it is not symmetric. It is not an asymmetric
relation because you have (a, b) here and simultaneously (b, a) also, then this (2, 3) is there as
well as (3, 2) is there.
So, it violates the universal quantification for asymmetric relations and it is not an asymmetric
relation because you have (a, b) as well as (b, a) both being present here, even though your a and
b are different. Then 2 is not equal to 3 but still you have (2, 3) as well as (3, 2) present in the
relation. So, there is no absolute relationship among the notion of symmetric, asymmetric and
antisymmetric relations.
Now let us see the last important relation here, which is the transitive relation. And what do we
mean by a transitive relation here, so a relation R is called a transitive relation if the following
universal quantification is true. We want that if at all a R b and b R c in your relation, then a also
232
should be related to c. In terms of graph theoretic properties, if you have an edge from a to b in
the graph of your relation R.
And if you have; a directed edge from the node b to the node c in the graph of your relation R.
Then we need that there should be an edge from a to c as well. And this should hold for every a,
b, c, where the domain of a, b, c are from the sets over which the relation is defined. So, let us
take this example, so consider the first relation it is a transitive relation, of course, so here
everything is defined over a set say {1, 2} and a relation R1 is transitive.
Because you have; (1, 1) present which can be also considered as (a, b) as well as (b, c) as well
as (a, c).
So, again the same is true for (2, 2). But your relation R2 is not transitive because you have a
case here where you have (a, b) present, you also have (b, c) present but no (a, c) is present here.
Namely (1, 1) is not present in your relation. Your relation R3 is also a transitive relation because
you have (a, b) present, (b, c) present and you also have corresponding, (a, c) present.
Your R4 is a transitive relation because it vacuously satisfies this implication because at the first
place there is no (a, b) and (b, c) present in your R4. And your relation R5 also satisfies vacuously
this universal quantification, because you have (a, b) present but there is no (b, c) present that
means there is nothing of the form (1, 2) here or (1, 1) here. That means vacuously this condition
is true for R5.
And that is why R5 is also a transitive relation. So, that brings me to the end of this lecture. Just
to summarize in this lecture we introduced binary relations and some special types of binary
relations. We also discussed the 2 representations that we follow to represent any binary relation.
233
Lecture -17
Operations on Relations
Hello everyone, welcome to this lecture on operations on relations.

Just to recap in the last lecture we introduced the definition of relations and we saw various types
of relations. The plan for this lecture is as follows. We will see various set theoretic operations
which we can perform on relations. Specifically, we will be discussing on the powers of a
relation and closure of a relation.
234
So, it turns out that since relations are nothing but a set so we can perform various set theoretic
operations like union, intersection, set difference on relations as well. So, to demonstrate this, let
us consider two relations R1 and R2, you have a relation R1 which consists of all (x, y) pairs or all
real numbers (x, y) where x < y. So, here my relation R1 is defined over the set of real numbers,
my domain is that of real numbers.
So, R1 will have all tuples of the form (x, y) where real number x is less than real number y and
similarly, relation R2 is another relation defined over the set of real numbers consisting of all (x,
y) pairs where the real number x is greater than the real number y. Now, if I take the union of
these two relations and the union of these two relations is well defined because both R1 and R2
are sets and we can perform the union of two sets.
So, it turns out that the union of these two relations will have all pairs of the form (x, y) where
the real number x is not equal to real number y because the union will have all the elements of R1
and a union also will have all the elements of R2. So, one way of describing the union of the two
relations is that it have all (x, y) pairs where either x < y or x > y. But, if you want to represent
the same if you want to state the same thing in a compact way, we can say that it has all (x, y)
pairs where x is different from y.
235
Whereas, if I take the intersection of these two relations R1 and R2 and it turns out to be an empty
set, because you cannot have real numbers x and y, where x is simultaneously less than y as well
as x is simultaneously greater than y. So, you cannot have any (x, y) pairs satisfying
simultaneously the conditions for the relation R1 and R2. In the same way it is easy to see that if I
take the difference of the relation R1 and R2.
That means, if I subtract R2 from R1, then I will be getting the relation R1 namely it will have
only the elements of the form (x, y) we are x is less than y and similarly, difference of R2 - R1
will be the relation R2.
Now, we can perform another interesting operation on the relations which is called as
composition of relations. So, imagine you are given two relations R and S, the relation R is from
the set A to B. So, R is a subset of A x B. And your relation S is from the set B to C, so S is a
subset of B x C, I am taking here three arbitrary sets A, B, C; A, B, C may be same, they may be
different there is no relationship, they are just arbitrary sets here.
And I am numbering the naming the elements of this set A as a1 to al. Similarly, the elements of
B are named as b1 to bm and elements of C are named as c1 to cn. Now the composition of these
two relations is defined as follows. So, first of all we use this notation S o R, and this means that
236
I am going to apply the relation R first and then the relation S. It is not a relation S applied first
and then the relation R.
We are going to apply the relation R and on top of that we are going to apply the relation S. So,
this will be a relation from the set A to C and that is why these ordering matters are lots. If I am
saying S composed with R, so this operation o is called as the composition operator here and
right now, I am composing the relation S with the relation R. If I write R o S, then that is a
different relation.
That means, I am composing the relation R with the relation S that means here S will be applied
first and on top of that relation R will be applied. So, the notation specifies clearly which relation
is composed with which relation. This S composition R will be a relation from the set A to C.
That means it will have ordered pairs of the form (a, c) and definition here is it will have all
ordered pairs of the form (a, c) provided, you have some element b in the set B such that a is
related to b as for the relation R and b is related to c as per the relation S. So, pictorially, if you
have say ai related to bj as per the relation R and the same bj is related to ck as per the relation S.
Then we will say that, ai is related to ck as per the relation S composition R. So, it is some kind of
transitive property here that transitive pairs or transitive tuples of the form (a,c) which we will be
including in the relation S composition R.
237
So, once the definition of compositions of relations is given, we can define what we call as
powers of a relation and how it is defined. So, imagine R is a relation from A to B that means R
is a subset of A x B. Then the definition of powers of a relation is as follows; R1 is defined to be
the relation R itself and then recursively I define the n + 1th power of R to be the composition of
the relation which is nth power of R with the original relation R.
So, again, I stress here the order matters here, the n + 1th power is defined to be the composition
of Rn with R that means you have to apply the relation R first and then you have to apply the
relation Rn. It need not be equal to the composition of R with the nth power of R that may or may
not be the case because in general the composition of two relations need not be commutative.
Let me demonstrate the powers of a relation with an example. Here I have defined a relation R
consisting of the pairs (1, 1), (2, 1), (3, 2) and (4, 3). So, R1 as per my definition will be the
relation R itself. Now, R2 will be the composition of R with R and my claim is that R2 will
consist of these four pairs {(1,1), (2,1), (3,1), (4,2)}, why so? Let us start with (1, 1); So, you
have (1,1) present in R and the same 1 is again related to 1 itself as per R, therefore I can say that
transitively this is your a here this is your b here and this is your b here and this is your c here.
So, as per the definition (a, c) should be there in R2. So, that is why (1, 1) will be present in R2.
Now take the tuple (2, 1). So, this is your (a, b) and you have the tuple (1, 1) also present in R,
this is your (b, c). So, therefore (a, c) should be in R2, that means (2, 1) will be in R2. If you take
(3, 2) present in R, this is your a, this is your b and you have (2, 1) present in R as well, this is
your b this is your c therefore (a, c) namely (3, 1) will be in R2.
And in the same way you have (a, b) here namely (4, 3) and (b, c) here can be (3, 2) that is why
(4, 2) which is (a, c) will be present in R2 and no other tuples will be there in R2, that is how we
take the powers of relation. If I want to compute R3 then it will be the composition of R2 with
relation R. So, I will take this R2 here first and compose it with the original relation R that means
the relation R has to be applied first and then on top of that the relation R2 has to be applied.
238
So, for instance this is your (a, b) here present in R and you have (b, c) as well present in R2,
therefore (a, c) which is (1,1) will be present in R3. Similarly (2, 1) is present in R and you have
again (1, 1) present in R2. So, that is why (2, 1) is present in R3 , you have (3, 2) present in R and
(2, 1) is present in R2. So, that is why you will have (a, c) which is (3, 1) here present in R3, you
have (4, 3) present in R and you have (3, 1) present in R2.
So, that is why (4, 1) is present in R3 and similarly, if you take the R4 relation, then it turns out to
be the same as R3 and now you can check yourself that after the 4th power you take any power of
the relation R it is going to give you the same relation as R3, you will not get any new tuples
added in the relation R, in the next powers of R.
Now, next what we are going to discuss is a very nice interpretation of the powers of a relation in
terms of a property in the corresponding graph. So, imagine you are given a relation over some
set A consisting of n elements and suppose the relation is represented by a directed graph G.
Now the claim here is the following the ordered pair (ai, aj) will be present in the mth power of
the relation, only ai is related to aj in mth power of the relation, if and only if, there is a directed
path of length m from the node ai to the node aj in the digraph of your relation. So, what do I
mean by directed path here? So, remember G is a graph where you have where you have vertices
a1, a2, ai, aj, an and you have directed edges. So, if there is a direct edge between two node; from
239
one node to another one other say from a1 to a2 then this will be considered as a directed path of
length 1.
Whereas say, if you have a path of the form from a2 you have an edge to a1 and from a1 you have
edge to an, then I will say that a sequence of edges a2 to a1 and a1 to an, this will be considered as
a directed path of length 2 from a2 to an. Like that you can imagine a directed path of length m
from ai to aj, that means your source or the starting point will be ai the end point will be aj and in
intermediate you will have m number of intermediate directed edges in that path.
That is the interpretation of a directed path, later on when we will discuss graph theory in detail
we will formally define what exactly is a path, path length and so on. But right now we want to
prove this nice theorem regarding the power of a relation and since this is a universally
quantified statement because even though the universal quantification is not explicitly available
here, but the statement is for all m >= 1.
So, that is why we will use a proof by induction, induction on the power m to prove this theorem
or the property and the base case will be m equal to 1 and the statement is of course trivially true
for the base case because if at all ai is related to aj in R that means that there will be a direct edge
from the node ai to the node aj in my graph that comes from the definition of the digraph, so the
base case is trivial to prove.
Now, assume that my inductive hypothesis is true, that means I assume that the statement is true
for m = k by that I mean that if ai is related to aj in the kth power of the relation then there exists
a directed path of length k starting at ai and ending at aj in the graph G. Now I go to the inductive
step and I consider an ordered pair, an arbitrary ordered pair, in the k + 1th power of the relation.
So, suppose the ordered pair (ai, aj) is present in the k + 1th power of the relation and the k + 1th
power of the relation is basically the composition of R with the kth power of the relation as per
the definition of powers of the relation. Now, when can it be possible that ai is related to aj in the
k + 1th power of the relation that is possible only if the following two conditions hold. There
240
should be some element a such that ai is related to a and that element a should be related to aj,
then only I can say that ai is related to aj in the k + 1th power of the relation.
But ai related to a means there is a structure of this form in your graph namely there is a direct
edge from ai to a. This should not be a dotted arrow this should be actually a straight arrow
because indeed I have a direct edge from ai to a and the interpretation of a being related to aj in
the kth power means I have a directed path of length k starting at a ending at aj this comes from
my inductive hypothesis.
Now what can I say regarding a path from ai to aj? I can say that if I concatenate these two paths
that means if take the edge from ai to a first and then once I reach the node a from there I traverse
through this path and come to the node aj that will give me a path starting with ai ending with aj
and now what can you say about the length of that path. The length of that path will be k + 1 that
means I have shown here that if the ordered pair (ai, aj) is present in the k + 1th power of the
relation, then in the directed graph of your relation you indeed have a path of length k + 1
starting at ai ending at aj.
Next let us define what we call as closure of a relation. So, what do we mean by this? So,
imagine you are given a set A that may be finite or infinite and you are given a relation R over
the set A that means a relation R is a subset of A x A and I have some abstract property P, it is
241
some abstract property and I am interested to check whether the relation R over the set A
satisfies this property P or not? If it is satisfies the property well and good.
But, that may not be the case R that is given to you need not satisfy the property. So, the closure
of the relation R is defined with respect to this property P. If you change the property P, the
closure will change. So, what is that is the closure of the relation R with the property P. Well, it
is the smallest superset of R which has the property P. Pictorially what I am trying to do here is if
my relation R already satisfies the property P, I do not need to add anything to the relation R.
I do not need to actually add any extra element to the relation R to satisfy the property P. But if
my relation R does not satisfy the property P then I will be interested to introduce new ordered
pairs in the relation R and convert it into another relation S, so that the expanded relation S
satisfies the property P that is what I am trying to do here, this S you can imagine as expanded
version of R and this S is also going to be a relation over the set A itself.
I am including the original relation R that is carried as it is. On top of that I am adding or I may
add few extra elements and try to ensure that expanded R which is S satisfies the property P, but
I am not going to do the expansion arbitrarily; I am interested in the least possible expansion,
least expanded version, what I mean by least? That means this is the minimal expansion which I
need to do in order to ensure that the relation S satisfies the property P that is important.
Otherwise, what is a big deal in expanding the relation R? You keep on adding any arbitrary
number of elements definitely you will more or less soon get an expanded version which will
satisfy the property P. So, we are interested in the smallest possible expansion.
242
So, let us see some examples of this abstract property P and how the resulting closure looks like.
So, my first abstract property is the reflexive property and this gives us what we call as reflexive
closure of a relation. So, you are given a relation R over the set A and I am interested to see
whether this relation R satisfies the reflexive property or not, reflexive over the set A, so that is
what is the reflexive closure of the relation R.
So, how can you construct a reflexive closure of R? Well you just take the union of R with all
ordered pairs of the form (ai, ai) where ai is present in your set A. If your, (ai, ai) is already there
in the relation R then as per the union definition, you will not be including it again. Remember,
union means if (ai, ai) is present in R as well as in this new relation, so this new relation I am
calling it as Δ relation.
So, this Δ relation you can imagine it is consisting of all ordered pairs of the form (ai, ai ) such
that ai is present in A. So, if (ai, ai) is already there in R, it will not be included again but if (ai, ai)
is not present in R then due to the union, due to taking union with this Δ, it will be now added to
the relation R and now you can see that this is your expanded R that may be same as R itself, in
case if your relation R is already reflexive then you are not going to add any extra elements.
So, this expanded R will have the original elements of the relation R plus this expanded R will
satisfy the reflexive property.
243
Now, let us take the case when my abstract property P is the symmetric property. So, again, I am
given a relation R defined over the set A and I am interested to form a symmetric closure of R.
Why I am calling it closure? Because I am trying to put a layer over, I am trying to enclose the
original relation R and get a relation which has the original R as well as it satisfies the property P
that is why it is called a closure.
So, how do I form the Symmetric Closure? So, if you recall the property of symmetric relation
then the requirement here is that if (a, b) or if (ai, aj) is present in R, then I need the guarantee
that (aj, ai) should also be present in R, this is the requirement from a symmetric relation and I
need to include the original relation R, what I am going to do is, I am going to take the union of
R with what I call as the inverse of the relation R.
So, this is the inverse relation and what is this inverse relation? It is defined to consist of all
ordered pairs of the form (aj, ai) such that ai is related to aj in the original relation. So, if in your
original relation ai is related to aj then in the R-1 relation aj will be related to ai and it is easy to
see that if you take the union of R with its inverse then the resultant relation will be symmetric
and it will have the original relation R and this will be the smallest possible expansion of your
relation R which satisfies the symmetric property.
244
Now, what about the transitive closure? So, it turns out that finding transitive closure is not that
simple, why so? Let me demonstrate that with an example. So, you might say that intuitively to
find the transitive closure I add all ordered pairs of the form (a, c). Such that a is related to b as
well as b is a related to c in the relation R because only when I add such order tuples of the form
(a, c) in the relation R, it will ensure that the transitive property is satisfied.
So, let us try to do that, you are given the original relation R and, I am forming, I am taking the
union of the original R and I am adding all the ordered pairs of the form (a, c) which are needed
to ensure the transitivity property. For instance, I need to add (1, 2), if you are wondering why I
need to add (1, 2)? Because I have (a, b) present here, I have (b, c) present here. So, I need (a, c)
which is (1, 2) also to be present which is not there in R so I say let me add it.
In the same way I have (2, 1) present, which is your (a, b) and you have (1, 3) also present which
is (b, c). So, you should add (2, 3) and so on. So, these are the new things which I add and my R’
will be this new relation. It has my original relation R, but now let us stop here and ask whether
R’ is transitive or not. It turns out that R’ is not transitive. So, for instance, you have (2, 3)
present here which is your (a, b) and you have (3, 2) present but you do not have (a, c) namely
(2, 2) present in the relation R’.
245
So, even though you have expanded R and got R’, your R’ is not satisfying the transitivity
property. So, what you can do is? You can again apply the rule and to apply the rule what I do
now is, I take the union of R’ namely the expanded R and add all the extra ordered pairs of the
form (a, c) which were missing in R’ to satisfy the transitivity property. So, see I am adding (1,
1) here, why I am adding (1, 1)? Because I have (a, b) here and (b, c) here but (a, c) namely (1,
1) is not present in R’. So, I am adding here and so on.
And now I call this expanded relation as R’’ and now you can check that is R’’ is indeed
satisfying the transitivity property. So, if you see closely here what happened in this whole
process is the following. First time when I expanded the relation R, I took the union of R with all
the tuples, which will be present in R2. You can verify that as per our definition of R2 the tuples
which I have highlighted here are the tuples in R2.
So, there are tuples which I have highlighted here and I have added with the tuples which were
present in R and nothing but a tuples of R2. In the same way, when I expanded R’ to R’’ the
tuples which I added actually are nothing but elements of R3. And recall R3 is nothing but R2
composition R and so on. It turns out that when we want to find a transitive closure; we need not
be a one step process.
I need to apply this rule namely keep on adding the ordered pairs of the form (a, c), such that (a,
b) and (b, c) are there in your expanded relation and I may need to keep on applying this process
several times and then only I can obtain the transitive closure. So, that brings me to the end of
this lecture just to summarize; in this lecture we introduced some set theoretic operations on
relations and we discussed various closure properties with respect to a given relation. Thank you.
246
Lecture -18
Transitive Closure of Relations
Hello everyone, welcome to this lecture. In this lecture, we will continue our discussion
regarding how to construct a transitive closure of relations.
And for that, we will see some graph theoretic interpretation of transitive closures. We will
discuss what we call as the connectivity relationship in the graph of a relation and then we will
see a naive algorithm for computing the transitive closure.
247
So, let me start by introducing the connectivity relation. So, imagine you are given a relation R
over a set which may or may not be finite. So, your R is a subset of A x A. And I define this
relation R*, which I call as the connectivity relation and this is basically defined to be the union
of different powers of your relation R. It turns out that, as per my definition of R* an element, ai
will be related to aj in this relation R* provided there exists some path of any length of it may be
of length 1, it may be of length 2, it may be of length 3.
I do not care about the length. The guarantee is that there exists at least one path from the node ai
to the node aj in the directed graph of your relation R. And why so, because recall in the last
lecture. We proved this statement by induction. The statement states that if you have element ai
related to element aj in the nth power of your relation R.
Then that is possible only if you have a path of length n, from the node ai to the node aj in the
directed graph of your relation R. Now, if I say that ai is present in R* then it means that either
(ai, aj) is present in R1 or it is present be in R2, and in the same way it will be present in some
power of R. I do not know which power, but since it is present in R* and the definition of R* it is
that it is a union of all powers of R.
So, if ai is related to aj and R*, that means it is present in one of these powers of R say if ith
power. Then as per this theorem statement, which we have proved in the last lecture there exist a
248
path of length i from the node ai to aj. It might be possible that (ai, aj) is also present in say some
other power of R. That is also possible say its present in the kth power. That means, as by the
same statement there exist a path of length k from the node ai to aj.
So, that is why I am not focusing on the path length here. I am just stating here that if at all the
element ai is related to the element aj, in the relation R*, then some path exists from the node ai
to the node aj in your graph of the relation R. Now, we will be focusing on this connectivity
relation where the relation R is defined over a finite set. That means R is defined over a finite set,
consisting of n elements.
My claim here is that R* is nothing but the union of R, R2 and up to Rn because you do not need
to take the union of higher powers of R. Any higher powers of R will be subsumed in the union
of the first n powers of R provided your relation R is defined over a finite set consisting of n
elements. And this is because what can be the maximum path length between any two nodes?
Remember in the graph of your relation R, you have n nodes because now the relation is defined
over a set consisting of n elements. So, what can be the maximum path length? The maximum
path length can be n only because you have only n distinct nodes possible. Of course, you can
keep on traversing along this path again and again that will be considered as a path of a higher
length.
But, what do I mean by maximum path length? By maximum path length I mean here, maximum
path length where path has distinct edges and why distinct edges? Because if I say for instance,
consider Rn+1, that means I am interested to find out whether they are exist a path of length n + 1
between any 2 nodes in the graph G. Well, since I have only n distinct nodes possible, then the
path of length n + 1 is possible only if a node is repeated in the path.
That means say for instance, I have a path of say, n is 3. Now say n equal to 4, So, I can say a
path of length 5 exist between a1 to a2. Because I can go from a1 to a2 that is 1, and a2 to a3 that is
2, and a3 to a4 that is length 3, and then a4 to a1 that is 4. And then again from a1 to a2 that is
249
length 5. But the same path can be considered as a path of length 1 because you have the node a1
to a2.
So, what I can say is that both (a1, a2) will be present in the relation R, and the same (a1, a2) will
also be present in R5, same (a1, a2) will also be present in R9 and so on. So, when I will be taking,
when I will be constructing R*, which will be union of R and R4 then anything which is present
in R5 and R9 and so on. Say for instance this element (a1, a2) which is present in R5, R9 would
have been already subsumed in R itself. I do not need to separately consider those pairs by
considering the fifth power of R and so on.
Because I know that, I cannot have a path of length more than 4 consisting of distinct edges. If at
all there is a path of length more than 4, that means some edges and nodes are repeated. That
means that path by excluding the repeated edges and nodes would have been already counted in
some lower power of R when I would have constructed R*.
So, that is why when my relation R is defined over a finite set consisting of n elements. Then I
have to construct the connectivity relationship I just need to focus on the first n powers of
relation R. There would not be any extra ordered pairs beyond this n different powers of relation
R which are present in R*.
250
So, we have not proved in detail but intuitively this is the statement. So, now what we are going
to prove here is now we are going to see a relationship between the transitive closure and the
connectivity relationship. So, remember in the last lecture we saw an example where we
constructed the transitive closure of a relation. And there we had iteratively applied the process,
the rule that if you have (a, b) and (b, c) in the expanded relation R or in the original relation R,
then you add the element (a, c) and keep on doing this process till we do not need to add any
extra elements of the form (a, c). So now what we are going to do is we are going to formalize
that process by stating this beautiful result that the transitive closure of your relation R is nothing
but the connectivity relation. Now to prove this theorem, we need to prove several things. The
first thing that we have to prove is that relation R, the original relation R is present in your R*.
Because that is one of the requirements of transitive closures, that your original thing should be
original relationship be present in the closure of that relation. But it is easy to see that the
original relation R will be a subset of R*. Because R* is nothing but R union the higher powers
of R. So, your original R will be definitely present in R*. So, all the ordered pairs which were
there in R will be present in R*.
The second thing that we have to prove is that indeed this expanded relation R which is R* is
satisfying your transitivity property. And for proving that this relation R* is going to satisfy
transitivity property what we are going to do is we are going to show the following. You take
any arbitrary (a, b) and (b, c) which are present in R*, then you have the guarantee that (a,c) is
also present in R*.
Why I am going to show it for arbitrary (a, b) and arbitrary (b, c), because the property of
transitive properties, the transitivity relationship demand is that for all a, b, c. if (a, b) and (b, c)
are there in your relation then you need (a, c) to be in your relation. And these need to be shown
for all a, b, c. But we cannot take every possible a, b, c in R* and show this implication to be
true. So, that is why we are showing going to show it for arbitrary (a, b) and arbitrary (b, c).
And then take the help of universal generalization and conclude that the statement is universally
true for all the elements of the domain. So, assume to prove this implication we have to show
251
that the left hand side of the premise of this implication is true and then we have to show that
even the conclusion is true. That is how the definition of implication is given. So, assume that (a,
b) is present in R* that means as per the definition of R*, (a, b) is present in some power of R,
say the jth power because that is the definition of R*.
And in the same way imagine that (b, c) is present in R*, that means it is present in some power
of R say the kth power, there may not be any relationship between j and k, j could be anything k
would be anything. Now what I can say here is that tuple (a, c) will be present in the (j + k)th
power of R. Because that is a definition of (j + k)th power, because (j + k)th power will be
nothing but Rk composed with Rj.
That means you would have applied the relation Rj first, that means you will say that say (a, b) is
there in Rj and on top of that you will apply the relation Rk. So, here b is acting as your
intermediate element. So, (a, b) is present in Rj and (b, c) is present in Rk and we will conclude
that (a, c) is present in the (j + k)th power of R. And (j + k)th power of R will be included in R*
because that is the definition of R*. So, we proved this implication to be true for an arbitrary a, b,
c that shows that the relation R* is transitive.
So, we have proved, we have shown that the two of the requirements of transitive closures are
satisfied by your R* relation. Now we have to prove the important thing. We have to prove that
252
R* is the smallest possible expansion of your relation R which is transitive. And the way we are
going to prove this is as follows. So, we have expanded our relation R to R*. We have shown
that R is present in R* and we have also shown that R* is transitive.
Now we have to show this third property how we are going to do this. We will do this by
showing that you take any transitive relation which includes R that means you take any expanded
version of R which is transitive. In that expanded version of R namely S, R* is present. That will
automatically show that R* is the smallest possible expansion. That means there is no smaller
subset of R* which includes R as well as it is transitive.
That is what we are proving here pictorially. That means it is not the case that you have
something of the following happening, that you have R* and you have say an expanded R say S
which is transitive and which includes R, that is not going to happen, such that S is present in R*
it is not going to happen. We are going to prove it other way around. We are going to show that
you take any transitive relation, which is expansion of R, R* will be definitely present in that
expansion S. And that automatically will show that R* is the smallest expansion that we have to
do.
So, more formally we have to prove that you take any relation S, which is a transitive relation
such that R is included in S. Then the expanded R namely the connectivity relation is included in
253
this expanded relation S. So, to prove this property we are going to take help of a very small fact
regarding the transitive relations and that fact is the following. My claim here is that if S is a
transitive relation then you take any power of that transitive relation, it will be a subset of the
original relation. And since this is a universally quantified statement for all n >= 1 we can
quickly prove it by induction, the base case is obviously true because S is always a subset of S.
Let the statement be true for n equal to 1 to k and now we are going to prove it for n equal to k +
1. So, imagine you have an arbitrary (a, c) element (a, c) present in the k + 1th power of S.
Then as per the definition of Sk+1, you have some intermediate element b such that a is related to
b in the relation S and b is related to c in the relation Sk. Then I apply the inductive hypothesis
here. Since (b, c) is present in Sk and statement is true for n equal to 1 to k. That means this (b, c)
is present in S as well because Sk is a subset of S as per my inductive hypothesis.
Now, if I have (a, b) present in S; and (b, c) presence in S; I can say that (a, c) is also present in
S. Because my base case or the hypothesis of the statement that I am proving is that S is
transitive. So, this is a very straight forward fact regarding the transitive relations. If your
relation is transitive you take any power of that relation, it will be always included in your
original relation.
So, we have proved it for all n >= 1, now, I apply the definition of S*. What is S*? S* is going to
be S1 ∪ S2 ∪ Sn ∪ higher powers. So, each of this power is included in the relation S because
that is what we have proved. We have to for all n >= 1, Sn is included in S. So, if you take the
union of all the powers of S that will be included in S.
And that shows that the connectivity relationship satisfies the property that if your original
relation is transitive then the corresponding connectivity relationship is included in the original
relationship.
254
That is the side result which we are going to retain here. Now coming back to the statement that
we want to prove here. We want to prove that if you take any transitive relation which is an
expansion of R then the connectivity relation of R is going to be included in that expanded R. So,
we have to prove this subset relationship property. So, the definition of subset is that if you have
any (a, b) present in R*, I have to show the same (a, b) is present in S as well.
Provided the hypothesis of this implication is true. So, I take an arbitrary (a, b) and assume it is
present in R*. Then since it is present in R*, it will be present in S* as well. This is because as
per my hypothesis here, R is a subset of S. That means R2 will be a subset of S2 that also means
R3 will be a subset of S3 and so on. You take any power of R that will be a subset of the
corresponding power of S.
That automatically shows that R* is a subset of S*. So, if (a, b) is present in R*, it will be present
in S* as well. But you are also given the hypothesis that S is a transitive relation. And if S is a
transitive relation, then you take S*, it will be a subset of S itself. That means whatever is present
in S* it is bound to be present in S as well. Hence we have shown that if (a, b) is present in R* it
is present in S and we have proved the third requirement of the transitive closure as well.
255
So, this is a very important theorem which we have proved that which we have proved now. And
the theorem is that the transitive closure of the relation is nothing but its connectivity relation.
So, this connectivity relation has got significance here. So, let me show that significance. So,
imagine you are given a relation R and R* is defined to be the union of different powers. So, if
you interpret the connectivity relationship in terms of the directed graph representing your
relation.
And what I can say is that ai is related to aj in this connectivity relationship, if aj is reachable
from ai by some path. Because (ai, aj) present in R* means it is present in some power of R say
the nth power, that means I have a path of length n from the node ai to the node aj. That is what
we have proved and abstractly we can interpret it as if in the graph of your relation R the node aj
can be reached by some path from the node ai.
Now the interpretation of this path depends upon what exactly is your underlying relation. So,
for instance imagine A is the set of all computers in a university and I define a relation between
two computers as follows. I will say that computer ai is related to computer aj, if there exists a
direct link between computer ai and computer aj. That means ai can directly send a message to
the computer aj by the channel or the cable through which the computer ai is connected to
computer aj.
256
Then if I construct the connectivity relation R* for this given relation R then it is easy to see that
R* will have all ordered pairs of the form (ai, aj) where ai is connected to aj may be directly or
through intermediate computers. That means R* basically talks about all interconnected
computers in your university. Now, let me give you another example of this connectivity
relationship.
So, have you ever wondered that how Facebook computes new friends suggestions? So, imagine
A is the set of all Facebook users and I define a relation between two Facebook users as follows.
I call that relation as R. I say that user ai is related to user aj provided ai and aj are mutual friends
over Facebook. Now, if you take the relation R* with respect to this R namely the connectivity
relationship defined over the relation R over the Facebook users.
Then if you have ai related to ak in the relation R*. That means I can say there are series of a
sequence of intermediate Facebook users such that ai is related to the first user then the first user
is related to the second user and so on and the second last user is related to ak. That means in
terms of a graph theoretic property, you can interpret that there is a path of some length from ai
to ak.
That means even though ai and ak may not be mutual friends, what Facebook can think if that it
might be the case that ai has not searched for the Facebook user ak and vice versa. So, let us send
a suggestion to ai that well, you know the user ak or not and similarly to the user ak. Because ai is
directly related to some a, the user a is related to user b, the user b related to user c and say there
is by sequence of intermediate users you have ai related to ak.
And since friendship is by default considered to, be transitive. Of course, there might be
exceptions as well, where a is friend to be b, and b is friend with c, but a need not be friend with
c. That is why Facebook just give you a suggestion that hey looks like that we have found
someone who is related to you by some intermediate users and to do that basically the Facebook
algorithm has to construct here the connectivity relationship of this relation R.
257
So, this connectivity relationship has got a huge amount of significance. Now the question is
how we algorithmically compute this connectivity relationship. So, we will be focusing on the
case when the relation is over a finite set. And recall that in this case R* is nothing but the union
of first n powers of R. So, what we are going to construct is, we are going to construct the
Boolean matrix MR*.
So, it will be an n x n Boolean matrix, representing your connectivity relation R* where the ith
row and the jth column will be 1 provided the element ai is related to aj in the connectivity
relationship R*. So, the question that we want to address here is that you are given the original
relation R namely the Boolean matrix representing the relation R. How you compute the Boolean
matrix representing the relation R* where the relation R is defined over a set consisting of n
elements.
So, here is the naive algorithm for computing the matrix for connectivity relation. We compute
the matrix for different powers of relation R. So, we are already given the relation for R. We are
already given the matrix for the relation R. So, we do not need to compute that. So, that is why
for i equal to 2 to n, we compute the matrix for the next powers of R. And this we can compute
by performing a Boolean matrix multiplication operation, which I will discuss very soon.
258
And here what I am going to do is to compute the matrix for the ith power of the relation R. I am
going to multiply the matrix for the original relation along with the matrix for the (i – 1)th power
of the relation R provided. I have already computed it. And then what I am going to do is, I am
going to take the Boolean disjunction of the individual n x n matrices for the different powers of
R that I have computed.
And that will help me to get the Boolean matrix for this connectivity relation R*. So, now there
are two operations here the Boolean matrix multiplication and the disjunction operation.
So, I am going to define them one by one. So, let us start with this Boolean matrix multiplication
here. So, the goal here is the following. You are given the relation R or equivalently its matrix
representation and say you have already computed the matrix representation for the (i – 1)th
power of R. And now your goal is to compute matrix for the ith power of the relation R. So,
recall as per the definition of the ith power of the relation R, ai will be related to aj, if you have ai
related to some ak in the relation R, and ak is related to aj in the relation R(i – 1), that is what is the
definition here. So, that is what pictorially I have represented here. When that is the case, then
you will say that ai is related to aj in the ith power of the relation R. So, basically this we have to
check whether this structure is present in the graph of relation R or not.
259
If this structure is present then we can say that ai is related to aj. Now, how do we check this
structure? So, we focus on the matrix for the relation R : MR, and for the matrix of the, (i – 1)th
power of R : MRi-1. And my claim is to check whether ai is related to aj or not. It is sufficient to
take the Boolean dot product of the ith row of matrix MR and the jth column of the matrix of (i –
1)th power of R : MRi-1. This is because if you take the Boolean dot product this (i, 1) will be
multiplied here with the entry (1, j).
And multiplying here means conjunction; because remember the matrix MR and the matrix MRi-1,
both are Boolean Matrix. They just say whether something is related to something or not. So, I
can say that if in my graph the element ai is related to element a1, and if element a1 is related to
element aj. Then I can say that element a1 is related to aj. So, that is what is the essence of
checking this first conjunction.
Checking whether (ai, a1) is related in the relation R or not. And whether (a1, aj) is present or
related as per relation Ri-1 or not. That is the case I do not care for the other expressions in this
overall expression. I can simply say that the i, jth entry in Ri should become 1 or I should check
in the similar way that ai is related to say a2. And a2 is related to aj or not. So, that will be the
essence of second conjunction here.
In the same way I should check whether ai is related to ak in the relation R. And ak is related to aj
in the relation Ri-1 or not. And in the same way that will be the essence of this kth conjunction.
And in the same way I should check whether ai is related to an directly. And an is related to aj or
not in the relation Ri-1. If any of these n conjunctions hold, then I can say that definitely ai is
related to aj in the relation Ri.
But if none of these n conjunctions are true then disjunctions of n 0s will be 0s and that is why
the (i, j) th entry will remain 0. So, you can see here that if you are given the Boolean matrix of
the relation R and if you have computed Boolean matrix for relation, Ri-1 then to compute the, (i,
j)th entry. So, let me write down. To compute (i, j)th entry of MR, you need to perform O(n)
Boolean operations.
260
Because you will be performing the dot product of two vectors of size n that will take you order
of O(n) effort. And how many such (i, j) entries are there, that I need to compute in the matrix of
Ri. Here are n2 entries so, that is why it will be order of n3 computation.
Now coming back to the Naive algorithm, the first step was computing the matrix for the
different powers of R starting from the Boolean matrix of the relation R which is given as an
input to you. So, that is why by applying the Boolean matrix multiplication operation that we
have discussed just now we can compute the matrix for the individual powers of the relation R.
And now what I have to check is I have to check whether ai is related to aj in any of these n
powers of R or not.
And for that, I just need to check whether ai is related to aj in MR1 or whether ai is related to aj or
not in the MR2 or whether ai is related to aj or not in the MRn or not. So, I have to just perform
disjunction of n Boolean entries to find out the status of ai, aj in the matrix of R*. So, it turns out
that the first operation here namely computing the matrix of different powers will cost me O(n4)
effort, whereas computing the final matrix of R* will cost me O(n3) Boolean operations. So, now
the question is can I reduce the overall cost here to O(n3).
So, we have this naive algorithm of computing the matrix for connectivity relationship, which
cost me O(n4) efforts. My goal will be to do it with O(n3) Boolean operations, which we will do
261
in the next lecture. So, just to summarize in this lecture, we introduce the connectivity
relationship of our relation. And we saw that the transitive closure of any relation is its
connectivity relation. And we discussed the naïve algorithm for constructing the connectivity
relationship for a relation defined over a finite set. Thank you.
262
Lecture -19
Warshall’s Algorithm for Computing Transitive Closure
Hello everyone. Welcome to this lecture. In this lecture we will continue our discussion
regarding how to compute the transitive closure. So, we will discuss the efficient algorithm given
by Warshall for doing the same.
So, just to recap this was your naive algorithm for computing the connectivity relation. So, you
are given as an input the matrix for your original relation R (MR) and from that we constructed
the matrix for the relation R* (MR*) provided R is defined over a set consisting of n elements. So,
R defined over set A consisting of n elements. We introduced the Boolean matrix operation in
the last lecture.
And we saw that overall to compute the matrix for different powers of R it will cost you n4
Boolean operations. So, now our goal is how we can get an algorithm to compute the matrix for
the connectivity relationship with n3 Boolean operations.
263
So, here is the Warshall’s algorithm for doing the same. So, the first thing that we are going to do
is for simplicity we are going to rename the elements of the set A from a1 to an as 1 to n. So, even
though the set is for instance might be a set of n cities we will be attaching the labels 1 to n to
those n cities. So, we will be now instead of calling a1 to an we will be now calling the elements
of A as 1 to n. This is just for simplicity and understanding.
Now the idea behind the Warshall’s algorithm is that we are going to define a sequence of
matrices where the matrix W0 is the initial matrix namely the matrix for the relation R (MR)
given to you. And from that we are going to construct a sequence of matrices, W1, W2 up to Wn
and we are going to stop with Wn and Wn will be the matrix for your relation R*. So, let us see
the definition of the kth matrix in this sequence.
So, k here ranges from 1 to n. So, the i, jth entry of this kth matrix I am denoting by this notation
(wi,j(k)) . So, you have w i, j or i j, and in the superscript I have is k within the parenthesis. So, I
will have the entry number i, j in different W matrices. So, the version of the W matrix that I am
focusing on will be k. So, that is why k will be superscript and within that kth version of the W
matrix i, jth entry will be denoted by this subscript indices i and j.
So, the i, jth entry in matrix Wk, wi,j(k) will be 1 will be defined to be 1 provided the following
holds.
264
If there is a path of some length, I stress the length is not important. If there is a path of some
length from node i to node j in your original graph such that all internal nodes along the path are
within the set 1 to k. Now what do we mean by internal nodes? By internal nodes I mean the
intermediate nodes in that path.
If there are no intermediate nodes that is fine, I may have a direct edge from the node i to node j.
That is fine. The condition here is that if at all there are any internal nodes along the path from
the node i to j they should be within the set 1 to k. That means only the nodes 1, 2, 3 or k are
allowed as intermediate nodes in that path from the node i to j. If such a path is there I will say
that the i, jth entry in the matrix Wk will be 1 otherwise 0.
So, pictorially, if you have such a path like this, where all intermediate nodes are restricted
within the set 1 to k, I will say the entry is 1 in the i, jth entry Wk matrix is 1. So, couple of
common confusions which has come to the student when we give this definition. First one is as I
said here there is no restriction on this path length. This path could be of length 1, that means it
could be a direct edge in which case it is not violating the condition.
Because the conditions says that if at all the intermediate nodes are there in this path they can be
only node number 1 or node number 2 or node number 3 or node number k. That means the
second restriction here is that it is not necessary that the path should be of length k. That means it
is not required that you should have node number 1 somewhere node number 2 somewhere node
number 3 somewhere node number k somewhere and then you come to node j.
No, that is not the interpretation of this definition. The definition is a conditional requirement or
requirement is that if at all intermediate nodes are there, first of all there can be any number of
intermediate nodes. Of course they cannot be more than k intermediate nodes because the only
intermediate nodes which are allowed in the path which you are considering in the matrix Wk are
the nodes now the nodes 1, 2 up to k.
265
That is the definition of the matrix Wk. So, before proceeding to the Warshall’s algorithm, let me
give you an example to make clear what exactly I mean by these Wk matrices. So, imagine this is
your given relation R. So, the diagram or the directed graph for this relation is as follows. So, the
nodes will be 1, 2, 3, 4. Since 1 is related to 4 you have this directed edge, 2 is related to 1 you
have this directed edge and so on.
We start with the matrix W0, which is the matrix for your relation R (MR) . So, what I have done
here is that I have just added the entry 1 for (i, j) pair if i is an edge to the node j otherwise the
entry is 0. Now let us see the matrix W1 here. How do we compute matrix W1? Forget about it.
The interpretation of W1 matrix is that the i, jth entry will be 1 if a path with only node 1 as
intermediate node is present from i to j.
So, it turns out that if I say for instance the first three columns for the first row then they will be
0, 0, 0. Why so? Because; there is no path from the node 1 to the node 1 where only intermediate
nodes are the node number 1. So, you do have a path here from the node 1 to node 1. So, you can
go via 1 to 4 and then you can go from 4 to 3. And then you can go from 3 to 1.
That is a path from 1 to 1. But what are the intermediate nodes here? So, you started with 1, you
go to 4, you go to 3 and then you go to 1. So, well there is a path from node 1 to 1. But what are
the intermediate nodes? The intermediate nodes here are node number 3, node number 4. But
266
they are not allowed to be considered when you are considering the paths in matrix W1. This will
be considered as a valid path when the possible intermediate nodes are nodes 1, 2, 3, 4.
Namely when you are considering going to consider the matrix number W4, this will be
considered as a valid path. But right now when you are considering paths with respect to the
matrix number W1, this is not a valid path, this is not allowed here. This is not why 1, 1 is 0 here.
In this, due to the same reason if I consider say for instance the entry number 2, 2 it is 0. Because
even though I have a path from 2 to 2, you can go from 2 to 1, you can go from 1 to 4, you can
go from 4 to 3. Sorry here you have no path, sorry. So, the entry number 2, 2 anyhow will be 0.
So, let us see a path which is not allowed as a valid path in matrix number W1. So, if I consider
let me show why this entry 2, 4 is becoming 1 here. So, the entry 2, 4 was 0 in matrix 0, W0.
Because there is no direct edge from 2 to 4, but now since I am allowing node 1 to be included as
an intermediate node, I can see that there is a path from node number 2 to node number 4 going
through this intermediate node 1.
That means once I allowed node number 1 as possible intermediate node, I can get a valid path
and that is why node is entry 2, 4 becomes 1 here and then you can verify that other entries are 0,
will remain 0 here. Now let us see whether there exist, whether I will get any update in matrix
W2. That means now in W2 I am allowing you to include node number 1 and node number 2 as
intermediate nodes between any ai and any aj.
It turns out that if I include node number 1 and node number 2 as possible intermediate nodes the
status of any i, j pair does not change when I go from W1 to W2. But when, I go from W2 to W3,
then the status of this entry 4, 1 changes. So, the 4, 1 entry was 0 in the previous matrix, but now
it is becoming 1 when I am allowing you to include node number 3 as well as a possible
intermediate node and why so?
Because you see here that I do have a path from the node number 4 to node number 1. The path
is as follows, you go from 4 to 3, you go from 3 to 1 then you go from 1 to 4 and what are the
intermediate nodes here? The intermediate nodes are 3, 1. And, 3 and 1 are allowed to be
267
intermediate nodes when you are considering matrix W3. See the idea here is that as I keep on
going to the next sequence of matrix and I am allowing you to include more and more new
intermediate nodes as valid intermediate nodes in the path from i to j.
So, in matrix W2 I was not allowing you to include node number 3. I was only allowing you to
include node number 1 and node number 2. But when I go to matrix W3 I will be giving you
more flexibility. I am giving you the flexibility to even include node number 3 along with node
number 1 and node number 2 as possible intermediate nodes. Now you can check that when I
update W3 to W4, the entry 1, 1 becomes 1 which was 0 in the previous matrix.
That means as long as I allow you only nodes 1, 2 and 3 as intermediate nodes I will say there is
no valid path from 1 to 1 going through only these three intermediate nodes. But as soon as I
allow you to include node 4 as a possible intermediate node I will say that there exist a valid path
from 1 to 1 and now I can stop here with W4. That will be my matrix for R* and this is because
you can check that only those nodes which are reachable from any other node for those particular
entries the i, jth entry will become 1 here in W4. Whereas the 0 entries means that there exists no
path from the node i to node j.
So, now Warshall’s algorithm boils down to the following that how exactly I am going to update
my matrix Wk - 1 to the matrix Wk and as per my definition the i, jth entry in the matrix, Wk-1 is 1
268
provided I have a valid path satisfying the restriction that the intermediate nodes are within the
subset 1 to k – 1 and I want, assuming I have already computed matrix Wk-1, my goal is how to
find out the matrix Wk, where, I am allowing you to include node number k as well, as a possible
intermediate node.
So, the update is very clever. It is done as follows. There can be two possible cases. If your i, jth
entry in the matrix Wk - 1 is 1 then I can simply say that i, jth entry in matrix Wk will also be 1
and this is because any valid path from node i to j where the intermediate nodes are within the
subset 1 to k - 1 can also be considered as a valid path, where the intermediate nodes are
restricted within the subset 1 to k.
And this is because as per the definition of matrix Wk valid paths are not supposed to have all the
nodes in the set 1 to k. The definition says that the path will be valid if at all the intermediate
nodes are within the subset 1 to k. It will not have all the nodes in the set 1 to k. So, it is fine if I
do not include node k in a path from i to j. If such a path excluding node k is also already present
and known to exist in the previous matrix Wk - 1, I can say that that is still a valid path in the
matrix Wk.
And I can check whether the i, jth entry in the matrix Wk - 1 is 0 or 1. If it is 1, I can simply copy
that entry in the new matrix. The second case will be as follows. My claim here is that if in the (k
– 1)th matrix the i, jth entry is 0. That means there was no valid path from the node i to node j
where all intermediate nodes are restricted to the subset 1 to k - 1. No such path was there. Then
what I will check is the following.
I will check whether there exists a path from the node i to k where all the intermediate nodes are
within the subset 1 to k - 1. So, that will be done by checking whether the i, kth entry in the (k –
1)th matrix is 1 or not. That means, I am checking whether there exists a valid path from the
node i to k in the k – 1th matrix. And I will check whether there exists a valid path from the node
k to the node j in the k - 1th matrix.
Now what I can say is if these two paths individually are guaranteed to exist then if I combine or
269
if I merge these two paths then that will be considered as a valid path from the node i to node j
passing through all the intermediate nodes that were there in the previous paths plus a new
intermediate node namely node number k and this will be considered as a valid path for Wk
matrix.
I stress this will not be considered as a valid path for the Wk - 1 matrix because in Wk - 1 matrix
node k is not allowed as a possible intermediate node. Node k can be allowed as a valid
intermediate node only in the paths considered in the matrix Wk. So, intuitively what I am saying
is that in the absence of intermediate node k there was no path from the node i to node j. But as
soon as I include the possibility of having node k as a valid intermediate node I can say that a
path from i to j is there, provided there is a path from i to k and individually a path from k to j
which is equivalent to checking the conjunction of these two conditions. I have to check whether
the entry number i, k and the previous matrix is 1 or not and whether the entry k, j is 1 or not in
the previous matrix. If both of them are 1, I can conclude that the entry i, j in the new matrix will
be 1. That is the intuition for the update in case two and that is a clever intuition.
So, what I can say is that I can summarize the two cases for updating the i, jth entry from the
previous matrix to the new matrix as follows. If in the previous matrix i, jth entry is already 1, I
will say that even in the new matrix the i, jth entry will be 1. Else I will check whether the i, kth
entry as well as the k, jth entry are simultaneously 1 in the previous matrix and if that is the case
I can say that i, jth entry in the updated matrix is also 1.
And what will be the cost for performing these operations? So, this will need constant amount of
effort because you just need to go and check the i, jth entry of the previous matrix and in the
same way you can go and just check the i, kth entry of the previous matrix and the k, jth entry of
the previous matrix we just need to do two matrix lookup, which will be costing you cost amount
of effort. That means to do this update operation you just need to perform a constant amount of
operations. And how many such i, j pairs are there?
There are n2 i, j entries, for each i, j update you have to spend a constant amount of effort O(1)
and that is why for n2 entries the overall update cost will be n2.
270
So, this is different from the way we performed update in the naive algorithm. In the naive
algorithm we were naively computing the higher powers of R from the matrix of the previous
power of R. And each update was costing us n3. But here we are not doing a naive update over
matrices, W0, W1, Wk, Wn have different interpretations. And due to the different interpretation
the update from Wk - 1 to Wk will be now costing us only n2 effort instead of n3 effort.
So, now let us put together everything and get the pseudo code for Warshall’s algorithm. I am
retaining the update operation of the Warshall’s algorithm. How do you update update the k - 1th
matrix to get the kth matrix? So, the input here will be the matrix for your original relation R and
now what I will do is I will run 3 loops each ranging from 1 to n and the update operation that I
am going to perform is as follows.
I am just copying the update operation as it is, removing all the subscripts and superscript
notation. So, W[i, j] denotes the updated i, jth entry and how it is updated? I check whether the
previous i, jth entry is 1 or not or if the i, kth entry as well as the k, jth entry are simultaneously 1
in the previous matrix or not. If that is such case I will update W[i, j] from 0 to 1. Otherwise I
will set W[i, j] to 0 and this I have to do for k = 1 to n.
So, it takes care of the fact that I am doing the update operation n times and for each time I am
271
doing the update for each of the n2 entries of the form i, j. And then finally I will output the W
matrix which will be the matrix for my connectivity relation. And it is easy to see that this
overall operation algorithm will cost you only O(n3) effort. Because there are three loops each
running from 1 to n.
So, that brings me to the end of this lecture. Just to summarize in this lecture we saw the
Warshall’s algorithm for computing the connectivity relationship which is better than our naive
algorithm for computing connectivity relationship. Thank you.
272
Lecture -20
Tutorial 3
Hello everyone welcome to tutorial number 3.

So, let us start the question number 1, here you are given that arbitrary sets A and B.
And the sets A and B as such that, this condition holds namely (A ∩ C) ⊆ (B ∩C)
for any set C that you consider. If that is the case and you have to show that A ⊆ B,
so you are given the premise that the (A ∩ C) ⊆ (B ∩ C) for any set C.
So, since this condition holds for any set C if I substitute C = A in this condition then
I get that A ∩ A ⊆ B ∩ A, but I know that A ∩ A is nothing but the set A, so that
means I can say that my premise, which I obtained by substituting C = A is that is A
⊆ A ∩ B.
Now my goal is to show that A ⊆ B, so for showing that A ⊆ B; I have to show that
you take any element x in the set A, it should be present in the set B as well. So, I am
taking an arbitrary element x and I am assuming it is present in the set A and since, I
have established the premise that A ⊆ (A ∩ B), that means if since the element x is
present in A same element x will be present in the A ∩ B as well.
273
Now, as per the definition of A ∩ B it means the element x is present in A and
simultaneously it is present in B as well, that means definitely it is present in B as
well, so what I have established here is that if I start with an arbitrary element x which
is present in the set A, I have established that it is present in the set B as well where
and since x was chosen arbitrarily here.
This is true for any element x that you choose from the set A and hence I get the
conclusion that A ⊆ B.
In question two you are given the following; you are given 3 sets A, B, C such that
this predicate holds for every element x in the set A and the property here is that if x
∈ A then the implication that x ∈ B → x ∈ C is true and this is a universally
quantified statement, that means this condition holds for every element x in the set A.
Then you have to show, you have to either prove or disprove whether the A ∩ B ⊆ C
or not, so in fact we are going to prove this statement we will prove that you take any
element x which is arbitrarily chosen, and if it is present in the A ∩ B then it is
present in C as well. So, we start with an arbitrary we chosen an element x and we
start by reworking the premise that is given to be true here.
So, the premise here is that this universally quantified statement is true for every
element x. So, what I am doing here is I am rewriting this implication here, so
274
remember the statement p → q is logically equivalent to the disjunction of negation p
and q. So, that is why I have splitted this implication into a disjunction and I get this
equivalent form, then I apply the same rule again over this implication.
So, this implication is now replaced by this disjunction and I put a negation here. Now
you see everywhere I have disjunction, I can apply the associativity property of the
disjunction. I can club together the disjunction of the first two statements here. Now
what I can say is that, the disjunction of these two negations is equal to the negation
of the conjunction of these two statements, this is from the De Morgan’s law.
And again I can use the same rule that I have used here to replace this ¬p ˅ q by p →
q, so this you can interpret as p this you can interpret as q, so you are given ¬p ˅ q; I
can rewrite it as p → q and now what is this condition? This condition is nothing but
it says that if x is present in A and if x is present in B it implies x is present in C.
So, since x was arbitrarily chosen and that means this condition holds for any x from
the set A and the condition that x belongs to A and x belongs to B simultaneously
means x belongs to the A ∩ B and since x belongs to C as well, by the definition of
subset it follows that the A ∩ B ⊆ C. So, this statement is a true statement.
Now in question 3, there are several parts and, you are given a set S, consisting of n
elements where n is not 0. So, implicitly I am assuming here that n is greater than
equal to 1. And now I have to count various number possible relations satisfying some
275
properties. So, the first property here is I am interested to count the number of
relations on this set S which are symmetric.
And just to recall that definition of a symmetric relation is that if you have the
element or the ordered pair (i, j) present in the relation R then the ordered pair (j, i)
should also be present in the relation R that is the requirement for a symmetric
relation. So, what I have done here is I have drawn all possible n2 ordered pairs, these
are the ordered pairs in the set S x S.
So, you can have n2 such ordered pairs and any subset of these n2 ordered pairs will
constitute a relation over the set S. I have to find out how many ways I can take
subsets of this n2 ordered pairs such that resultant subset satisfies the property of a
symmetric relation. So, now among this n2 ordered pairs, I have highlighted this
selected ordered pairs.
So, you can imagine this matrix this is n x n matrix and I am focusing only and the
upper triangular quotient of this matrix and let A be any subset of the highlighted
ordered pairs. So, the highlighted tuples, I am calling it as H and any subset of that, I
am calling it as A, my claim is that if you take the union of the ordered pairs in the
subset A and the ordered pairs in A-1 then the collection of these ordered pairs will
give you a symmetric relation.
So, say for instance, if you pick A is equal to say (1, 1) and say (2, 3), say this is a
subset of H, then A-1 will be the inverse of this ordered pairs inverse means taken by
reversing or swapping the order of the ordered pairs. So, (1, 1) its inverse will be (1, 1)
but the inverse of (2, 3) will be (3, 2) and now if I take the union of these two
relations that will give me a symmetric relation.
So, like that, I take any subset A of the highlighted tuples and to that I add all the
ordered pairs which are found by taking the inverse of the ordered pairs in the set A,
then the collection will be a symmetric relation and that is the only way you can
construct a symmetric relation over the set S because the requirement for a symmetric
relation is that if (i, j) is there if you have included (i, j) then you have to include (j, i)
as well.
276
So, what I am intuitively saying is that in the set A you first decide what are the (i, j)
pairs that you are going to include. once you have decided how many (i, j) pairs you
are going to includes just take the inverse of those (i, j) pairs and the resultant union
will give you a symmetric relation. Now how many ways you can fix your, (i, j) pairs
well those (i, j) pairs I am asking you to pick from the upper triangular portion of this
matrix.
Once you have fixed which (i, j) pairs you are going to select from this upper
triangular matrix just take the inverse of those (i, j) pairs from the lower portion of the
matrix, and that will give you a symmetric relation. That means I can say here that
the number of symmetric relations that I can form is nothing but the number of
subsets A that I can choose from the highlighted tuples.
So, how many elements are there in the highlighted tuples? So, my claim is it is n *
n + 1) / 2 this is, because in the first row you have n number of highlighted, n number
of tuples, in the second row of the upper triangular matrix you have n - 1 number of
ordered pairs and like that, in the last row you have only 1 ordered pair say if I sum
these things I get n * (n + 1) / 2 number of ordered pairs.
And how many subsets of these ordered pairs I can form? That is nothing but a
cardinality of the power set of the tuples in your set H and since the set H has these
many number of elements namely n * (n + 1) / 2, the number of subsets of these order
𝑛 ∗(𝑛 +1)
tuples that I can form is nothing but 2 2 . So, that is the number of symmetric
relations that you can form over a set S consisting of n elements.
277
Now in part b of the question I want to find out the number of relations over the set S,
which are anti-symmetric and just to recap, this is the requirement from a
anti-symmetric relation. If both (a, b) and (b, a) are present in the relation then that is
allowed only if a is equal to b that means a is not equal to b contra-positively if a is
not equal to b then you cannot have simultaneously both (a, b) as well as (b, a) present
in the relation R.
So, now count the number of anti-symmetric relations I do the similar thing here I
have drawn all the n2 possible ordered pairs. Now when you are trying to form an
anti-symmetric relation the first thing to observe here is that the ordered pairs along
the diagonal here can be either present or absent in your anti-symmetric relation that
means if say for instance (1, 1) and (2, 2) are included in your relation then that does
not violates the requirement of anti-symmetric property.
Because if (a, b) and (b, a) simultaneously present then that is allowed only when a is
equal to b and all the ordered pairs along the diagonal elements have a is equal to b.
Mind it, it is not mandatory that all the n ordered pairs along the diagonals should be
present in an anti-symmetric relation, even if none of them is present that is fine that
still satisfies the requirement of an anti-symmetric relation.
The condition is that, if at all both (a, b) and (b, a) are there then a equal to b and
nowhere, it says that for all (a, b) both (a, b) and (b, a) it should be present in your
relation for a equal to b. So, now how many ordered pairs I have along the diagonal? I
278
have n such ordered pairs and for each ordered pair I have 2 option either to include it
or to exclude it in an anti symmetric relation.
Now let us focus on the non diagonal ordered pairs of the form (i, j) and (j, i) where i
and j are distinct, for instance say (n, 1) and (1, n). Now how many ways I can include
these ordered pairs (1, n) and (n, 1) and still satisfy the requirements of an anti
symmetric relation. I have 3 possibilities here, possibility 1 that neither (i, j) nor (j, i)
is included in my relation R.
So, for instance, I do not include (1, n) and I do not include (n, 1) in my relation that
is fine that still satisfies the requirement of an anti-symmetric relation. Option number
two; that I include only (i, j) in the relation but exclude (j, i) remember I cannot have
both (i, j) as well as (j, i) in my relation, because I am considering the case when i is
different from j and if i is different from j then as per the property of an anti
symmetric relation both ordered pairs (i, j) and (j, i) are not allowed to be included.
So, I have only three possibilities include none of them or include (i, j) or include (j, i).
Now how many such (i, j) and (j, i) pairs I have in this matrix where i and j are
different well I have (n2 – n)/2 number of such (i, j) and (j, i) pairs this is because
there are total n2 ordered pairs and from there, I am excluding the elements along the
diagonals.
So, I am left with n2 - n number of pairs, n2 - n number of elements, and if I pair

them in the form (i, j) and (j, i) then I have to divide it by 2. So, for every such pair I
have 3 options and the possibility of including or excluding (i, j), (j, i) pairs is
independent of the possibility of including or excluding the ordered pairs along the
diagonal element when I am forming an anti-symmetric relation.
So, that is why the total number of ways of forming an anti-symmetric relation is how
many ways I can consider the elements along the diagonal which is 2n and how many
𝑛 2 −𝑛
ways I can consider the remaining elements and that is 3 2 , and if I multiply them
that will give me the total number of relations which can be anti-symmetric.
279
In part C of third question, you are supposed to find out the number of relations which
are asymmetric and this is the requirement for asymmetric relation. If at all (a, b) is
there in your relation then (b, a) is not allowed in the relation. This does not mean that,
that for every (a, b) you should have either (a, b) or (b, a) present in the relation, it is
fine if none of them is there in your relation or not.
So, again, I do the same thing here, I consider n2 ordered pairs and see how many
ways I can select subset of these n2 ordered pairs and still satisfy this universal
quantification. So, the first thing to observe here is that now none of the elements or
the ordered pairs along the diagonal are allowed in an asymmetric relation.
Because, if you include (i, i) then that violates this universal quantification because
you have both (a, b) as well as (b, a) present in your relation that is not allowed. So,
none of these, so (1, 1), (2, 2), (i, i) (n, n) none of them are allowed in an asymmetric
relation. Now, let us focus on the remaining elements and again, let us club them into
pairs of the form (i, j) and (j, i) where i and j are distinct and again it is easy to see that
for such (i, j) and (j, i) pairs I have three possibilities here.
I can include none of them and still my relation will be asymmetric. I can include (i, j)
and exclude (j, i) or I include (j, i) and exclude (i, j). So, I have three possibilities here
and how many such pairs, you can have (n2 – n)/2 such pairs. So, as a result since I
have to definitely exclude the diagonal elements the only possibilities I have now is to
consider the non diagonal elements.
280
And, with respect to non diagonal elements I have these many options. So, these are
the total number of asymmetric relations.
In part d, I am interested to find out how many relations I can form which are
irreflexive and the definition of a irreflexive relation is this, it states that for every
element a in the set S (a, a) should not be there in your relation R. That means none of
the elements none of the ordered pairs along the diagonal are allowed because that
will violate this universal quantification.
Whereas if I take any other tuple (i, j) where i and j are different then I can either
include it or exclude it that would not violate the requirement from an irreflexive
relation. So, for instance if (n, 1) is present in my relation that is fine, that satisfies
this universal quantification and I can have (n, 1) as well as (1, n) and both present
here that still satisfies the requirement for this form of irreflexive relation.
So, how many such (i, j) ordered pairs I have here, if I exclude the diagonal elements I
am left with these many ordered pairs and for each such ordered pair I have two
possibilities either include it or exclude it. So, that is why the total number of your
2
reflexive relations is 2𝑛 –𝑛
.
281
Part e, I now want to find the number of relations which are simultaneously reflexive
as well as symmetric and the requirement from a reflexive relation is this and
requirement from a symmetric relation is this. So, if I consider the ordered pairs I
have no choice, but I have to definitely include all the diagonal ordered pairs because
I have to satisfy the reflexive property.
But apart from this, apart from the ordered pairs along the diagonal elements I have
the choice with respect to the elements of the ordered pairs which are not there along
them. So, I consider all (i, j) and (j, i) pairs where i and j are different. So, I have now
two possibilities; I can choose to exclude both (i, j) as well as (j, i) that still satisfies
the requirement of symmetric relation or if I decide to include (i, j) then I am forced to
include (j, i).
Because, if I include only (i, j) but exclude (j, i) then that will violate the requirement
of a symmetric relation. So, with all such ordered pairs of the form (i, j) and (j, i)
where i and j are different I have two possibilities and how many such (i, j) and (j, i)
pairs are there, where i and j are distinct I have (n2 – n) / 2 such pairs and since I have
no option, no choice with respect to the diagonal elements I am forced to include
them.
The total number of relations which are both reflexive and asymmetric is nothing, but
𝑛 2 −𝑛
2 2 .
282
The last part of the question, I have to find out the number of relations, which are
neither reflexive nor irreflexive. So, not reflexive means this universal quantification
should be false that means at least 1 element a, should be there in the set S such that (a,
a) is not present in the relation, then only this universal quantification can become
false.
And, not irreflexive means, this second universal quantification is false, that means
you have at least one element a in the set S such that (a, a) is present in your relation
then only this second universal quantification can be false. So, if I consider this n2
ordered pairs, then what I can say here is that, since my set S is non empty, so I stress
I assumed here n is greater than equal to 1.
So, since my set S is non empty, I cannot have a relation which is simultaneously
reflexive as well as irreflexive, right? It cannot satisfy both, it cannot satisfy both the
requirements of a reflexive relation as well as irreflexive relation. Because, reflexive
relation says that you should have all the elements along the diagonal present in the
relation, whereas the irreflexive property demands that none of the ordered pairs
along the diagonal should be present.
So, you cannot have both these conditions occurring simultaneously in the relation R.
So, I have to exclude all the elements along the diagonal. It turns out that if I try to
find out the number of reflexive and your reflexive relation simultaneously with
283
respect to the options that I have for the non diagonal elements then the counting
might become slightly tricky.
So, instead what I do here is, I find out the number of relations which are either
reflexive, or irreflexive and subtract it from the total number of relations, which I
can form over the set S. What I know here is since I cannot have a relation which is
simultaneously reflexive as well as irreflexive then I can confidently say that, that if I
subtract out the total number of relations which are either reflexive or irreflexive from
the total number of relations that will give me the required answer.
Because there will be no overlap which is possible here, no overlap in the sense there
cannot be any relation which is simultaneously reflexive as well as irreflexive. So, my
goal here was to find out the relations which violate the requirements of being
reflexive and irreflexive. So, to do that what I do is I find out the relations which have
one of these two properties.
That means, it is either reflexive or irreflexive and subtract out the number of set of
relations from the set of all possible relations, which I can have over the set S. The
total number of relations, which I can have over the set S is 2n and if I subtract out the
2 −𝑛
number of reflexive relations which is 2𝑛 which we have calculated.
2 −𝑛
And the number of irreflexive relations which also is 2𝑛 and that will give me the
total number of relations which are neither reflexive nor irreflexive.
284
Now, let us start question 4, again, we have several parts and here again, we are given
a set S consisting of an elements and I assume here n is greater than equal to 1. I have
to find out the number of relations which are symmetric, anti-symmetric and
reflexive. So, these are the requirements from my relation, my relation should satisfy
the property of a reflexive relation.
My relation should satisfy the property of a symmetric relation and my relation should
satisfy the property of an anti-symmetric relation as well. So, I consider the ordered
pairs, n2 ordered pairs. So, since I want my relation to be reflexive, I have no choice
but I have to compulsorily include all the ordered pairs along the diagonal here. Now,
I consider ordered pairs of the form (i, j) where i is different from j and try to see what
I can do with this ordered pairs.
So, that my resultant relation is both symmetric as well as anti symmetric. So, in order
to maintain the symmetric property if I decide to include the ordered pair (i, j) in my
relation, then I need to include (j, i) as well. But as soon as I include (i, j) and (j, i) in
my relation, it will violate the requirement of an anti symmetric relation because
anti-symmetric relation says that if your, i and j are different then both (i, j) and (j, i)
cannot be present in the relation.
That means these 2 conditions cannot occur simultaneously; that means, if I have
ordered pairs of the form (i, j) where i is different from j, I am not allowed to include
such ordered pairs because, that will violate the requirement of symmetric and
285
simultaneously anti-symmetric property. So, that means I have now only one way of
forming the relation.
Namely, I just include all the ordered pairs along the diagonal and that is all, that
relation will be symmetric, anti-symmetric and reflexive apart from that I can not do
anything else I can not include any other ordered pair. So, that is why the total number
of relations which satisfy these three properties simultaneously is 1.
(Refer Slide Time: (26:52)
Part b, I have to find out the number of relations which are symmetric, anti-symmetric
and irreflexive. So, let me write down the requirements; irreflexive means no element,
no ordered pairs of the form (a,a) should be there. Symmetric means, if (a, b) is
included and (b, a) should also be there and anti-symmetric means, if you have
distinct a and b then either (a, b) or (b, a) should be there not both of them, it is fine if
none of them are there.
So, again, I consider the n2 possible ordered pairs here. Now since my relation has to
be irreflexive, I have no choice but I have to exclude, compulsorily exclude all the
ordered pairs along the diagonal elements, all along the diagonal. Now, what about
the ordered pairs of the form (i, j) where i and j are different. So, again, if I want to
include (i, j) then to retain the symmetric property I have to include (j, i).
But since, i and j are different if I have both (i, j) and (j, i) present in my relation then
it will violate the requirement from an anti-symmetric relation, that means again here
286
with respect to the ordered pairs (i, j) where i and j are different I am not allowed to
have the ordered pair (i, j) because if I try to include (i, j) then it will either violate the
requirement of symmetric property or anti symmetric property.
And since the diagonal elements are also not allowed to be included it turns out that
only relation which I can form which is simultaneously symmetric, anti-symmetric as
well as irreflexive is the empty relation namely include no ordered pair; that means
none of the ordered pairs among these n2 ordered pairs should be there in my
relation. Then only my relation can have all the three properties simultaneously and
that is why there is only one relation possible.
Part c of the question, I have to find out the number of relations which are both
symmetric and anti-symmetric. So, this is the requirement from a symmetric relation
and this is the requirement from an anti_symmetric relation. So, now if I consider the
elements, the ordered pairs along the diagonal, for each ordered pair (i, i); I can either
decide I can either choose to include it in my relation or exclude it in the relation, that
will satisfy both this universal quantification.
That means, if I say to decide only to include (1, 1) in my relation that is fine that
does not violate the requirement from a symmetric relation or it does not violate the
requirement from an anti-symmetric relation, because anti-symmetric says that if a
and b are same then fine you can have both (a,b) and (b,a) present in the relation and
287
mind it, I am not enforced to have all the ordered pairs along the diagonal to be
included in my relation, I can just select any subset of them.
What about the ordered pairs of the form (i, j)? Where, i and j are distinct. Again, if I
decide to include (i, j); I am forced to include (j, i) to retain, maintain the symmetric
property, but since i and j are distinct if I cannot have both (i, j) and (j, i)
simultaneously present in my relation because that will violate the anti-symmetric
property. So, again, the conclusion here is that for every ordered pair of the form (i, j)
where i and j are different, I am not allowed to include them.
That means, the only options I have is with respect to the diagonal ordered pairs and I
can take any subset of the ordered pairs along the diagonal that will satisfy the
requirements of a symmetric and anti-symmetric relation and how many such subsets
I can form, I can form 2n subsets. So, this will be the number of relations which will
satisfy simultaneously symmetric as well as anti-symmetric. So, with that we end our
tutorial number 3. Thank you.
288
Lecture -21
Equivalence Relation
Hello everyone, welcome to this lecture on equivalence Relations. And just to recap in the last
lecture we discussed some special types of relations like Reflexive Relations, Symmetric
Relations, Asymmetric relations, Anti Symmetric Relations, Transitive Relations. So, in this
lecture we will introduce a special type of relation called as equivalence Relation and we will see
the definition of equivalence classes.
289
So, what is the formal definition of an equivalence relation? It is a relation R over a set A which
satisfies three properties namely the relation should be reflexive, the relation should be symmetric
and the relation should be transitive. It should satisfy all these three properties. If any of these three
properties is not satisfied, the relation will not be called as an equivalence Relation.
So, let us see an example. So, I define a relation over ℤ here and by relation here is that an integer
a will be related to integer b if 𝑎 ≡ 𝑏 𝑚𝑜𝑑 𝑚.
We say an integer a and integer b are congruent, they are congruent with respect to modulo m if
the remainder which I obtained by dividing a by the modulus m is exactly the same as the
290
remainder which I obtained by dividing b by the modulus m. So, m is the modulus here. I mean
the divisor and you are dividing a by m and b also by m, and if you get the same remainder, then
we say that a and b are kind of equivalent in the sense they have the property that they give you
the same remainder when divided by this modulus m.
𝑅 = {(𝑎, 𝑏): 𝑎 ≡ 𝑏 𝑚𝑜𝑑 𝑚},
Where m is a fixed modulus.
If 𝑎 ≡ 𝑏 𝑚𝑜𝑑 𝑚, → (𝑎 − 𝑏) is completely divisible by m, it gives you 0 remainder.

If 𝑎 ≡ 𝑟 𝑚𝑜𝑑 𝑚 and 𝑏 ≡ 𝑟 𝑚𝑜𝑑 𝑚, → (𝑏 − 𝑎) ≡ (𝑟 − 𝑟)𝑚𝑜𝑑 𝑚 ≡ 0 𝑚𝑜𝑑 𝑚.
Now my claim is that this relation R is an equivalence relation. It satisfies the property of reflexive
relation, Symmetric relation and transitive relation.
So, let us prove that so is the relation R reflexive? Answer is yes. Because, (𝑎 − 𝑎) ≡ 0 𝑚𝑜𝑑 𝑚.
You divide a whatever remainder you obtain by dividing a by m the same remainder you obtained
by dividing a again by m. So, in that sense a is always congruent to a modulo m.
The relation R is also symmetric, we can prove that. For proving the symmetric property, I assume
that consider an arbitrary pair of integers (a, b) where 𝑎 ≡ 𝑏 𝑚𝑜𝑑 𝑚 → 𝑏 ≡ 𝑎 𝑚𝑜𝑑 𝑚.
So, what I have proved here is (b, a) is present in the relation R. That means the integer b is related
to the integer a as per my relation R. So, what I have proved is starting with the premise that (a, b)
is present in the relation R. I can conclude that (b, a) is also there in relation R. That proves my
relation R is symmetric.
291
Now let us prove that the relation R is transitive as well. So, for proving the transitivity property I
have to show that, if I have a related to b in my relation and b related to c in the relation, then I
have to show that the integer a is related to integer c. And I have to show this for any arbitrarily
chosen a, b, c. So, since a is related to integer b, that means a is congruent to b or equivalently (a
– b) is completely divisible by the modulus m.
So, I can say that 𝑎 – 𝑏 = 𝑞1 ⋅ 𝑚. In the same way, since the integer b is related to integer c, that
means integer b is congruent to integer c or equivalently b - c is completely divisible by m. Or in
other words 𝑏 − 𝑐 = 𝑞2 ⋅ 𝑚. Now what I can say here is if I add these two equations here, I get
that 𝑎 – 𝑐 = (𝑞1 + 𝑞2 ) ⋅ 𝑚 . That means a - c is completely divisible by the integer m, which in
other words means that 𝑎 ≡ 𝑐 𝑚𝑜𝑑 𝑚. That means the integer a is related to integers c. And that
proves that your relation R is transitive as well.
292
So, that this is an example of an equivalence relation. So, now let us define equivalence classes.
Imagine R is an equivalence relation over some set A. And now consider an element 𝑎 ∈ 𝐴. Then
the equivalence class of A which is denoted by this notation you have the square bracket and within
that you have the element a. So, the equivalence class of [𝑎] = {𝑏: (𝑎, 𝑏) ∈ 𝑅}, consist of all the
elements from the set A which are related to this element a as per the relation R.
Formally, this equivalence class is a set it will be a subset of your set A. It will be having all the
elements 𝑏 ∈ 𝐴 such that a is related to b. That is equivalence class of an element a. And now this
equivalence class satisfies some very nice properties. The first trivial thing to check here is verify
here is that you take the equivalence class of any element, it will be non- empty.
There will be at least one element which is always guaranteed to be present in the equivalence
class of any element a. And that element is the element a itself, 𝑎 ∈ [𝑎]. Because the element a is
always related as per the relation R because the relation R is an equivalence relation and since it
is an equivalence relation it is reflexive. If it is a reflexive element, every element is related to
itself.
So, the element a will always be present in its equivalence class and hence a equivalence class A
will never be an empty set. Let me demonstrate what exactly equivalence class looks like with an
example. So, I consider this relation R over set of integers ℤ where an integer is related to integer
293
b if 𝑎 ≡ 𝑏 𝑚𝑜𝑑 3. So, m = 3 here, we already proved in the previous slide that this relation is an
equivalence relation.
So, what will be the equivalence class of 0? So, [0], so my a = 0 here, so equivalence class 0 will
have all the elements b, empty all the integers b such that 0 is related to those integers b. And it
easy to see that equivalence class of [0] will be 0, will have definitely 0. Because 0 is related to 0
because 0 is congruent to 0 modulo 3. And equivalence class of 0 will have 3, 6, 9 and these
integers because 0 is related to 3 and you have 0 related to 6 and so on.
In the same way you have 0 related to -3 you have 0 related to -6 and so on. Because 0 is congruent
to -3, 0 is congruent to -6 modulo 3 and so on. So, [0] = {… , −9, −6, −3,0,3,6,9, … }. all the integer
multiples of 3. What about [1]? Definitely the element 1 will be present in the its equivalence
class.
And apart from that we will have the integers 4, 7, 10 and so on. And on the negative side we have
the elements -2, -5, -8 and so on present in the equivalence class of 1. Because all these integers
are related to the integer 1 as per the relation R. [1] = {… , −8, −5, −2,0,4,7,10, … }. In the same
way the [2] = {… , −7, −4, −1,0,2,5,8, … }.
Now if you see closely here, it turns out that [3] = [0] = [−3] and so on will be the same. That
means the equivalence class of all the integer multiples of 3 will be same. In the same way the
[1] = [7] = [−5], any equivalence class of any integer of the form 3 k + 1 are same and so on.
294
So, what we are observing here is that even though we have equivalence class of every integer
possible here. So, these are the various equivalence classes. And this is an infinite list. It turns out
that if we closely look here we find that the two equivalence classes in this sequence are either
same or they are completely disjoint. So, for instance, if I consider [0] and [1], there will be no
common element, there will not be any integer which is present simultaneously in [0] and [1] as
per the relation R. You cannot have an integer b such that 𝑏 ≡ 0 𝑚𝑜𝑑 3 as well as simultaneously
𝑏 ≡ 1 𝑚𝑜𝑑 3.
Whereas if you consider [0] here and [3], they will be exactly same. They will have exactly the
same elements. So, it turns out that this property that 2 equivalence classes are either completely
disjoint or they are completely same is not present, this property does not hold only with respect
to this equivalence relation, this special equivalence relation it holds in general for any arbitrary
equivalence relation which is a very interesting property.
So, more formally we can prove that if you are given any equivalence relation, any arbitrary
equivalence relation over an arbitrary set then a is related to b iff they are equivalence classes are
same and the equivalence classes as [𝑎] = [𝑏] if and only if [𝑎] ∩ [𝑏] ≠ 𝜙. Or in other words if
[𝑎] ∩ [𝑏] = 𝜙, then [𝑎] ≠ [𝑏].
And of course, we can prove we can apply the transitivity property and say that if a is related to b
295
then [𝑎] ∩ [𝑏] ≠ 𝜙. So, there are 2 by implications involved here. I am going to prove one of the
by implications and I leave the proof for the other by implication, for you it is very simple, it
follows the proof of the first by implication. So, I am going to prove this by implication. I will
prove that if R is an equivalence relation and if a is related to b then [𝑎] = [𝑏].
Now since this is a by implication, I have to prove the implication in both the directions. So, I
prove the first implication in the forward direction namely I assume that a is related to b. And then
under this assumption I have to show that they are equivalence classes are same. So, since a is
related to b. So, this is what is given to me and my goal is to show that [𝑎] = [𝑏].
Equivalence class of 𝑎 is a set, equivalence class of 𝑏 is a set. So, I want to prove here that two
sets are equal. So, to prove that two sets are equal I have to show that they are mutually subsets of
each other. That is what is the definition of equality of two sets. So, proving that equivalence class
of 𝑎 is equal to equivalence class of 𝑏 boils down to proving these two things. That [𝑎] ⊆ [𝑏] and
vice versa, given that a is related to b.
And how do I prove that a set is a subset of another set? I prove it by showing that you take any
element 𝑥 in the first set, it is present in the second set. So, I take an arbitrary element 𝑥 belonging
to the first set here. The first set here is [𝑎]. I have to show that the same 𝑥 ∈ [𝑏] as well. How do
I do that? Since 𝑥 ∈ [𝑎], I can say that (𝑎, 𝑥) ∈ 𝑅. That means a is related to the element 𝑥 because
296
that is what is the definition of [𝑎].
Now, I am also given that R is an equivalence relation and if R is an equivalence relation, then one
of the requirements from an equivalence relation is that it should be symmetric. And if relation R
is symmetric and if (𝑎, 𝑥) ∈ 𝑅, then (𝑥, 𝑎) ∈ 𝑅 as well.
Now, I have (𝑥, 𝑎) ∈ 𝑅 and as per my hypothesis here (𝑎, 𝑏) ∈ 𝑅. And since my relation R is
transitive, why? Because my relation R is an equivalence relation. The transitivity property ensures
that (𝑥, 𝑏) ∈ 𝑅. Now since (𝑥, 𝑏) ∈ 𝑅, I can again apply the fact that my relation R is symmetric
because it is an equivalence relation.
So, I get that (𝑏, 𝑥) ∈ 𝑅. And if (𝑏, 𝑥) ∈ 𝑅, then as per the definition of an equivalence class, the
element 𝑥 ∈ [𝑏]. That means starting with the premise that 𝑥 ∈ [𝑎], I have shown that 𝑥 ∈ [𝑏] as
well. Which proves that [𝑎] ⊆ [𝑏].
And I can apply a similar proof to show that [𝑏] ⊆ [𝑎]. So, you start with some arbitrary element
𝑥 ∈ [𝑏] and again applying similar steps that we have done here, we have used here. You can show
that the same element 𝑥 ∈ [𝑎]. And that will show that [𝑎] = [𝑏].
So, that proves the implication in the forward direction this we have done. Now, let us prove the
297
implication in the reverse direction. So, assuming R is an equivalence relation and assuming that
the equivalence class of a and b, are same I have to show that a is related to b. So, for this I start
with some arbitrary element 𝑥 ∈ [𝑎] then as per the definition of equivalence class of a, it means
that a is related to x as per the relation R.
And since it is given that [𝑎] = [𝑏].That means the element x will be present in the equivalence
class of b as well. Then as per the definition of equivalence class, it means that x is related to b as
well. Since b, x is present in my relation, I can say that x, b is also present in my relation. Because
R is symmetric and why R is symmetric? Because my relation R is an equivalence relation.
Now I can apply the transitivity property here on (𝑎, 𝑥) and (𝑥, 𝑏). So, I have (𝑎, 𝑥) present in the
relation I have (𝑥, 𝑏) in the relation and by applying the transitivity property, I get (𝑎, 𝑏) present
in the relation. So, that proves the implication in the other direction. Remember there is another
by implication which I am leaving for you to prove. And that will establish the theorem that we
have stated in couple of slides back.
That brings me to the end of this lecture. Just to recap, in this lecture we introduced the notion of
equivalence relation and we also introduced the notion of equivalence classes. We established
important property that the equivalence classes are either disjoint or they are completely same,
thank you.
298
Lecture -22
Equivalence Relations and Partitions
Hello everyone, welcome to this lecture on equivalence relations and partitions.

Just to recap in the last lecture we introduced the notion of equivalence relation and equivalence
classes. In this lecture, we will continue the discussion on equivalence relations and classes. And
we will introduce the notion of partition of a set and we will see the relationship between
equivalence classes and partitions.
299
So, let us start with the definition of a partition of a set. So, imagine you are given a set C which
may be finite or it may be infinite. Now, what is the partition of this set C? The partition here is
basically a collection of pairwise disjoint, non-empty subsets say m subsets of C which should be
pairwise disjoint such that if you take their union, you should get back the original set C.
So intuitively, say for example, you have the map of India you can say that the various states of
India partition the entire country India into various subsets such that there is no intersection among
the states here. So, in that sense, I am just trying to find out some subsets of the set C such that
there should not be any overlap among those subsets and if I take the union of all those subsets I
should get back the original set C, there should not be any element of C which is missing.
So, more formally the requirements here are the following. Each subset 𝐶𝑖 ≠ 𝜙 that means each
subset should have at least one element. They should be pairwise disjoint. That means if I take any
𝑖, 𝑗 then 𝐶𝑖 ∩ 𝐶𝑗 = 𝜙 and 𝐶1 ∪ … ∪ 𝐶𝑚 = 𝐶. So, one trivial partition of the set C is the set C itself.
I can imagine that C is partitioned into just one subset namely the entire set C or I can decide to
partition C into exactly two halves or I can decide to partition C into three equal sets of equal sizes
and so on. So, there might be various ways of partitioning your set is not a unique way of
partitioning a set. Of course how many ways you can partition a set that is a very interesting
question we will come back to that question later.
300
What we now want to establish here is a very interesting relationship between the equivalence
from an equivalence relation to the partition of a set. So, we want to establish relationship between
equivalence relation and partition of a set. So, imagine you are given a set C consisting of 𝑛
elements. Now what I can prove here is that if R is an equivalence relation over the set C and if
the equivalence classes which I can form with respect to the relation R are 𝐶1 , … , 𝐶𝑚 . Then my
claim here is that the equivalence classes 𝐶1 , … , 𝐶𝑚 constitutes partition of the set C.
So, just to recall, the definition of partition demands me to prove three properties, the first property
is that each of this subset should be non-empty. And that is trivial because I know that each of
these equivalence classes is non-empty because each of these equivalence classes is bound to have
at least one element, 𝑖 ∈ [𝐶𝑖 ] since my relation R is an equivalence relation, it will be a reflexive
relation that means the element 𝑖 will be related to itself. That means none of these equivalence
classes will be an empty set. So, the first requirement is satisfied.
The second requirement from the partition is that the union of the various subsets should give me
back the original set. So, my claim here is that if I take the union of all these m equivalence classes,
I will definitely get back my original set C. And this is because you take any element 𝑖 ∈ 𝐶, it is
bound to be present in at least one equivalence class. Specifically, the element 𝑖 ∈ [𝐶𝑖 ]. So, that
301
means I can safely say that if I take the union of these m equivalence classes, I will not be losing
any element of the set C.
Third requirement from the partition was that the various subsets in the partition should be pairwise
disjoint. So, in this specific case, I have to show that you take any two equivalence classes, they
should be pairwise disjoint and that is easy because in the last lecture we proved that two
equivalence classes are either same or they are disjoint. You cannot have a common element
present in two different equivalence classes which automatically establishes that these subsets are
pairwise disjoint. So, we have proved here that you give me any equivalence relation and if I take
the equivalence classes that I can form with respect to that relation R that collection of equivalence
classes will constitute a partition of my original set.
Now, I can prove the property in the reverse direction as well. What do I mean by that? I claim
here that you give me any partition of a set C, say you give me a collection of m subsets which
constitute a partition of the set C. Then I can give you an equivalence relation R whose equivalence
classes will be the subsets, which you have given me in the partition.
So, I will give you the construction of the equivalence relation and the construction of the
equivalence relation here is very straight forward. So, the required equivalence relation is the
following. You take any subset from the given partition, say the subset 𝐶𝑘 , 𝑘 = 1, … , 𝑚 because
302
you are given m such subsets in your partition. So, with respect to each subset 𝐶𝑘 , what I am going
to do is I take 𝑖 ∈ 𝐶𝑘 and , 𝑗 ∈ 𝐶𝑘 , I add the ordered pair (𝑖, 𝑗) in my relation R.
So, I stress here that there is no special requirements from my, i and j. I am looping over all possible
𝑖, 𝑗 present in the subset 𝐶𝑘 . So, either 𝑖 = 𝑗 or 𝑖 ≠ 𝑗. For every 𝑖 ∈ 𝐶𝑘 and , 𝑗 ∈ 𝐶𝑘 , add (𝑖, 𝑗) in my
relation R. And if I do this for every subset Ck in my given partition then my claim is that the
resultant relation R will be an equivalence relation and its equivalence classes will be the subsets
𝐶1 , … , 𝐶𝑚 .
So, just to demonstrate my point, imagine my set 𝐶 = {1, 2, 3, 4, 5, 6} and a partition of this set is
given to you. So, I am given 3 subsets, 𝐶1 = {1, 2, 3}, 𝐶2 = {4,5}, 𝐶3 = {6}. Let me construct a
relation R as follows. So, I take the first subset here and by iterating over all i, j present in this
subset, I add ordered pairs of the form (i, j).
𝑅=
{(1, 1), (2, 2), (3, 3), (1, 2), (2, 1), (1, 3), (3, 1), (2, 3), (3, 2), (4, 4), (5, 5), (4, 5), (5, 4), (6,6)}.
So, these are the ordered pairs which I have added with respect to the subsets. With respect to the
third subset you might be wondering there is no j present in the third subset. That is why I said
there is no restriction that i should be same, i should be different from j or i should be same as j
also. So, I have to iterate over all possible i, j present in the subset. So, in this subset if I substitute
i = 6 and j = 6, I have to add the ordered pair (6, 6) in my relation R. And now you can check here
that the relation R that I have constructed is indeed an equivalence relation it satisfies the reflexive
properties. It satisfies the symmetric property and it is transitive as well. And if you form the
equivalence classes of this relation R, you will get these three subsets 𝐶1 , 𝐶2 , 𝐶3 .
303
So, let us formally prove this. Now, going to prove that a relation R that I am saying here to
construct indeed will be reflexive, symmetric and transitive. So, let us prove that this relation R
will be reflexive. So, you take any element i from the set C, I have to show that (𝑖, 𝑖) ∈ 𝑅 to show
that it is reflexive. Now since 𝐶1 , . . , 𝐶𝑚 is a partition of the set C, the element i will be present in
one of the subsets in this collection say it is present in the subset 𝐶𝑘 .
Now if it is present in the subset 𝐶𝑘 when I am applying this rule to construct this relation R, I will
see that element i is present in 𝐶𝑘 and I will add the ordered pair (𝑖, 𝑖) in the relation R as per this
rule. So, that shows that you take any element 𝑖 ∈ 𝐶, it is guaranteed that (𝑖, 𝑖) ∈ 𝑅. That proves
that the relation R is reflexive.
304
Let us prove that a relation R that we have constructed here is symmetric as well. And for proving
that I have to show the following. I have to show that if you take any arbitrary (𝑖, 𝑗) ∈ 𝑅, then
(𝑗, 𝑖) ∈ 𝑅. And how do I prove that? So, the first thing to observe here is that if at all you have
(𝑖, 𝑗) ∈ 𝑅. That is possible only because of the following.
You have say, 𝑖 ∈ 𝐶𝑘 , 𝑗 ∈ 𝐶𝑘 , then only you would have added the ordered pair (𝑖, 𝑗) ∈ 𝑅 and
none of these two elements i and j could be present in any other subset in this partition, the given
partition or in the given collection of subsets, because that is the definition of a partition. So, since
𝑖 ∈ 𝐶𝑘 , 𝑗 ∈ 𝐶𝑘 , by applying the rule that I have followed for constructing the relation R, I would
have also added the element (j, i), because I have to iterate over all possible i, j. So, when i become
j and j becomes i as a result I get (𝑗, 𝑖) ∈ 𝑅 and that prove that my relation R is symmetric.
305
Now let us prove that a relation R is transitive and for proving that my relation R is transitive, let
me take an arbitrary ordered pairs. So, I take (𝑖, 𝑗), (𝑗, 𝑘) ∈ 𝑅 and I have to show that the ordered
pair (𝑖, 𝑘) ∈ 𝑅. So, the first thing to observe is that since by construction of my relation R if at all
(𝑖, 𝑗), (𝑗, 𝑘) ∈ 𝑅, that is because all the elements 𝑖, 𝑗, 𝑘 were present in a common subset namely
say subset 𝐶𝑙 .
Because it cannot happen that you have 𝑖, 𝑗 ∈ 𝐶𝑘 and you have or say 𝑖, 𝑗 ∈ 𝐶𝑙 and 𝑘 ∈ 𝐶2 . That is
not possible here. Because that would have been the case then you would have added the ordered
pair (i, j) and (j, i) in the relation but you would have not added the ordered pair (i, k) or (j, k) in
your relation. You would have added the ordered pair (i, j) or (j, k) in the relation only when all
the three elements 𝑖, 𝑗, 𝑘 ∈ 𝐶𝑙 .
Now since you would have iterated over all possible 𝑖, 𝑗 ∈ 𝐶𝑙 , you would have iterated over k as
well and you would have added the ordered pair (i, k) in the relation R as well and that shows that
your relation R is transitive. So, that shows a very nice relationship and a nice property between
the equivalence classes and the partition.
You give me any equivalence relation the corresponding equivalence classes will constitute a
partition. You give me a partition of a set, I will give you an equivalence relation corresponding
306
to those partitions, namely the equivalence relation will be such that its equivalence classes will
give you the same subsets which are given in the partition that you given to me.
So, in other words what we can show here is that the number of equivalence relations what we
have established here actually is that the number of equivalence relations over C is exactly the
same as number of partitions of set C. Because we have established that you give me any
equivalence relation that corresponds to a partition you give me any partition that corresponds to
an equivalence relation. So, the counting the number of equivalence relations in a sense is same as
counting the number of partitions of the sets.
So, that brings me to the end of this lecture. Just to summarize, in this lecture we introduced the
notion of partition of a set and we established formally the relationship between an equivalence
relation, its equivalence classes and the partition of a set.
307
Lecture -23
Partial Ordering
Hello everyone, welcome to this lecture on partial orderings. So, we will introduce the definition
of partial ordering in this lecture. We will discuss Hasse diagram and we will compute with
Topological sorting.
So, what is the partial ordering? So, if you consider a dictionary then the words in a dictionary are
arranged alphabetically or we also say that the words are arranged lexicographically. And there is
a very nice relationship which you can state that holds between the words or relationship holds
with respect to the way in which the words are arranged in your dictionary. So, the relationship
here is, I say that a word a in the dictionary is related to the word b in my dictionary provided the
word a appears before the word b.
So, this alphabetical arrangement of the words can be considered as a relationship. And it turns
out that this alphabetical arrangement of the words in the dictionary satisfies three properties. It
308
satisfies the Reflexive property, it satisfies the Antisymmetric property and it satisfies the
Transitive property.
Reflexive property because implicitly I can always say or I can always assume that a word always
appear before itself. That is not true in the sense of the dictionary, but I can always have this
implicit order. The alphabetical arrangement of the words satisfies antisymmetric properties
because you cannot have two different words such that the word a appears before the word b and
simultaneously the word b appears before the word a.
And this alphabetical arrangement of the words satisfies the transitive property because if you have
the word b, appearing after the word a, and if you have the word c appearing after the word b, then
you can say that the word c is appearing after the word a. So, that sense it is a transitive relation.
It turns out that you can have several such relations which satisfy the property of being reflexive,
antisymmetric and transitive.
So, for instance imagine, I have a big software project. And typically in a big software project you
identify various modules, various components which are independent of each other and each of
them can be executed by separate procedure. So, now imagine that I have several such modules
and I have defined a relationship or a dependency between the modules by a relation R and I say
that module 𝑚𝑖 is related to the module 𝑚𝑗 if there is a dependency on the module j for the module
i. So, I define a relationship R where module i is related to module j provided module j can start
only after module i is over. That means until and unless you are done with the module i, you cannot
start the module j. That is a dependency relationship. Now again I can say here that this dependency
relationship, is reflexive, it is anti symmetric and it is transitive.
It is reflexive in the sense I can always simply assume that a module always depends on itself. It
is an implicit dependency. This dependency relationship is anti symmetric because I cannot have
two separate modules which are dependent on each other. If that is the case if that situation happens
in your software project then it will lead to a state of a dead lock. So, for example, module 1
309
depends on module 2 and module 2 depends on module 1 you cannot start both of the any of them.
So, that is why this relationship will be anti symmetric.
And is relationship is transitive. If module 2 depends on module 1 and if module 3 depends on

module 2, implicitly it means that module 3 depends on module 1 as well. So, I have given you
examples of two relations each of them satisfies the reflexive, anti symmetric and transitive
properties and the essence of both this examples is the following.
You have a well defined ordering among different elements. If I take the first example my elements
were the words of the dictionary and there is a way to ordering, the alphabetical order. If I take the
second example, my elements of the set were the modules of the software project and there is a
well defined ordering.
So, let us now generalize this theory. So, we now have we are now going to define a special type
of relation which we call as a partial ordering. So, you are given a set S over which a relation R is
defined and it will be called as a partial ordering, if the relation is reflexive, antisymmetric and
transitive. In that case, the set S along with the relation R is called a poset. The full form of poset
is partially ordered set.
310
Let me give you some more examples of partial ordering here. So, I consider the set of all positive
integers, so this set ℤ+ is the set of positive integers. And I defined a relation divides which I am
denoting by |. So, my relation R is the 𝑑𝑖𝑣𝑖𝑑𝑒𝑠 (|) relationship.
And I say that (𝑎, 𝑏) ∈ | 𝑖𝑓 𝑎|𝑏 and 𝑎, 𝑏 ∈ ℤ+ . Otherwise, a is not related to b. So, now it is easy
to see that this relationship is reflexive because every positive integer divides itself. This
relationship is antisymmetric, because you cannot have two different positive integers
simultaneously dividing each other if a divides b as well as b divides a then that is possible only
when both the integers are same.
And if | 𝑖𝑓 𝑎|𝑏 and 𝑏|𝑐, then you have 𝑎|𝑐. So, it satisfies the transitivity property. So, this is an
example of a partial ordering. Now, let me define another relation here my relation here is ⊆. My
R here is the subset relationship, which I am denoting by ⊆. And my elements are the elements of
the power set of a set. So, the relation is not defined over the set S. I stress that the relation is
defined over the power set of S.
So, my elements are here subsets of S and I say that a subset A is related to the subset B, if𝐴 ⊆ 𝐵.
That is my relation. Again this relation satisfies the reflexive property because 𝐴 ⊆ 𝐴. It satisfies
the antisymmetric property because you cannot have two different subsets A and B, 𝐴 ⊆ 𝐵 and
𝐵 ⊆ 𝐴, because that is the case that means 𝐴 = 𝐵. And it satisfies the requirement of a transitive
relation. If 𝐴 ⊆ 𝐵 and 𝐵 ⊆ 𝐶, then that means that 𝐴 ⊆ 𝐶. So, this is an example of partial
ordering.
Similarly, if I take the set of integers, ℤ, and if my relationship is the less than equal to relationship
where integer 𝑥 is related to integer 𝑦 provided 𝑥 ≤ 𝑦. Then again this satisfies the reflexive
property, antisymmetric property and transitive property and hence this is an example of a partial
ordering.
311
So, now if you are given an arbitrary poset instead of using the notation R for the relation, I use
the abstract notation ≤. I Stress here that is ≤is just a notation. It is just a substitute for R, it is
notation for R. It does not mean numerical less than equal to, that is very important. That means
when I am writing 𝑎 ≤ 𝑏, that does not mean that a is numerically less than equal to b. That just
mean, that the element a is related to element b as per my relation R.
So, for example, if I take the partial ordering where my relation was the divide (|) relationship
then 2 ≤ 4. Again, do not get confused by the numerical interpretation. Again numerically indeed
2 is less than equal to 4 but less than equal 2 here stands for the divide relationship namely 2 |4,
but 2 is not less than equal to 3. Because, we are not numerically following the interpretation here,
2 does not divide 3. That is why 2 is not less than equal to 3.
Now we also use this abstract notation < and again, this is not a numerical representation. It is just
use to represent the fact that a is related to b but 𝑎 ≠ 𝑏. So, that is a case I use the notation a less
than b. So, for instance, if I take the | relationship we have 2 less than 4, because indeed 2 divides
4, and 2 is not equal to 4, whereas we have 2 not less than 2, even though 2 is less than equal to 2,
because 2 divides 2. But since 2 and 2 are same I cannot say that 2 is less than 2. So, that is the
abstract notation that we are now going to follow for the rest of our discussion on partial ordering.
Less than equal to is not numerical less than equal to, less than is not the numerical less than.
312
So, imagine you are given an arbitrary poset that so this less than equal to is an arbitrary relation
R, which is a reflexive, antisymmetric and transitive. Now you take any two elements from the set
S. They will be called comparable if 𝑎 ≤ 𝑏 or 𝑏 ≤ 𝑎, incomparable otherwise. So, again to
demonstrate these two concepts let us consider the divide relationship, that means you are less than
equal to relationship is the divides relationship.
Then you have 2 ≤ 4, because 2|4. So, 2 and 4 are comparable. Comparable in the sense that they
have there is definitely a relationship between 2 and 4. Either 2 is related to 4 or 4 is related to 2.
But 2 is not related to 3, because 2 does not divide 3, that is why we will say that 2 and 3 are in
comparable. So, in a partial ordering it is not necessary that you take any pair of elements and they
are comparable. You may not have any relationship among them as per the relation R that you are
considered.
So, that brings us to the definition of what we call as a total ordering. And a total ordering is a
special type of poset or partial ordering where you take any pair of elements from your set S, they
will be comparable. That means either a will be related to b or b is related to a and that is why the
name total ordering because you do not have any pair of incomparable elements. Whereas partial
ordering the name partial denotes there, that you have ordering which is only partial. That means
you may have a pair of incomparable elements and your relation R whereas a total ordering means
313
you do not have any pair of incomparable elements. ∀𝑎, 𝑏 ∈ 𝑆, 𝑎 ≤ 𝑏 or 𝑏 ≤ 𝑎. So, that is why
when your poset is a totally ordering, that means your relation is a total ordering then the set S is
called as a totally ordered set. It is not called a partially ordered set.
In partial order set you might have the possibility of existence of incomparable elements. But in a
totally ordered set you have relationship present between every pair of elements in the set. A total
order set is also called as a linearly ordered set or a chain. Why it is called a chain or a linearly
ordered set will be clear soon. So, let us see some examples of total ordering.
If I consider the less than equal to relationship namely, the numerical less than equal to relationship
over the set of integers, then it is a total ordering. You take any pair of integers numerically either
the first integer will be less than equal to the second integer or the second integer will be less than
equal to the first integer. Whereas if you take the divides relationship, where a is related to b,
provided a divides b then this is not a total ordering. Because 2 is not less than equal to 3 and 3 or
also is not less than equal to 2. So, 2 and 3 are incomparable elements.
So, it turns out that we can represent partial ordering or posets by a very specific type of diagrams
which are called as Hasse diagrams. And this is possible provided, this makes this is more
interesting for posets where the relation is to find over a finite set. So, what exactly is this Hasse
314
diagram? So, let me demonstrate this Hasse diagram with this less than equal to relationship which
is the numerical less than equal to relationship defined over the set 𝑆 = {1, 2, 3, 4}.
So, this will be the directed graph for your relationship less than equal to. Since 1 is related to 1, I
have the self loop at the node 1, 2 is related to 2, so I have the self loop at the node 2. Similarly, I
have the self loop at 3 because 3 is related to 3 and I have the self loop at 4, because 4 is related to
4. I have a directed edge from 1 to 2, because 1, 2 is present in the relationship. I have a directed
edge from 1 to 3 because 1 is related to 3 and so on.
So, all the directed edge which are supposed to be present in the relation are there in this graph.
Now what I can say here is that there is no point of explicitly writing down or stating the self loops.
Because I can say that since my relationship is reflexive anyhow, I can always say that the self
loops are implicitly present in my diagram. No need to unnecessarily represent them and make the
diagram untidy.
So, if I remove the self loops and assume that my, self loops are always implicitly present, then
my diagram looks little bit better. Next what I can do is I can remove the transitively implied edges
from this diagram and say that hey, since my relation is anyhow transitive, I can remove the edge
present from the node 1 to 3. Because I can say that since 1 to 2 is present and 2 to 3 is present
anyhow 1 to 3 will be present in my diagram. So, why to again explicitly represented in the
diagram. So, I can remove all the transitively implied edges and my diagram simplifies further.
So, what I am doing is in each stage, I am trying to make my diagram more and more cleaner, tidy
and try to remove all unnecessary information or redundant information, which I am not supposed
to explicitly state in my graph of the relation of a partial ordering.
Now what I can say is that I can say that I make the assumption here that the arrays here, so sorry
it is not arrays, this is arrows. So, I can make the assumption here that the arrows are always
directed from bottom to up and that will take care of the direction of the edges as well and my
graph becomes further simplified to this diagram. And now there is no more of information, which
I can remove from this graph and say that it still represents my original relationship that means
315
what I mean by that is if I take this graph which I obtained by step 1 and then followed by step 2
and followed by step 3 then if you give me just this graph I can reproduce the original graph. How
can I reproduce the original graph?
As per my definition, I will say that arrows are always pointed upwards. Then as per my
assumption the self-loops are always there. But and as per my assumption all the transitively
implied edges are also there in my graph. That means if you give me the graph, this final graph
which will be called as the Hasse Diagram here. If you give me the Hasse diagram here, I can
reproduce the entire original graph for the Partial ordering that you were given here right. So, that
is how you construct a Hasse diagram for partial order.
So, let us see another example here. So, you are given the divide (|) relationship. So, your ≤ is the
divider relationship. So, again, we can start with our directed graph with the nodes 1, 2, 3, 4, 6, 8,
12, I can have all the self loops. I will have the transitively implied edges and so on and then if I
remove all the self loops all the transitively implied edges, and if I remove the direction of the
edges assuming that the arrows are always pointed from bottom to up, then this will be the Hasse
diagram that I will obtain. This will be the minimum piece of information, which I need to retain
in my graph to recover back the original diagram of ≤ or the | relationship over this set 𝑆 =
{1, 2, 3, 4, 6, 8, 12}.
316
Let us see another example where the less than equal to relationship is the subset relation. And
relation is defined over the power set of 𝑆 = {𝑎, 𝑏, 𝑐} not over the set {a, b, c}, remember. So, how
many elements will be there in the power set of {a, b, c}? So, since set S has 3 elements the
cardinality of its power set will be 23 . There will be 8 subsets,
{𝜙, {1}, {2}, {3}, {1,2}, {2,3}, {1,3}, {1,2,3}}. So, these are the 8 subsets. Again, I have removed all
the self loops. I have removed all the transitively implied edges and I have removed the direction
of the edges.
So, for instance, I have not added the edge from the subset 𝜙 to the subset {𝑎, 𝑐}, because that is
transitively implied. Because 𝜙 is anyhow a subset of a which is represented by this undirected
edge and undirected edge always have an implicit direction associated with it. And a is a subset of
the subset {𝑎, 𝑐}. Again, the direction is not explicitly mentioned here, but as per my assumption
the directions are always upward. And as per my assumption the transitively edges are not
explicitly stated in the graph. That means I have an implicit edge from 𝜙 to the subset {𝑎, 𝑐}.
Because indeed the subset 𝜙 is a subset of the subset {𝑎, 𝑐}. But I do not need to explicitly add it
in the graph. I can remove it. So, this is the minimum piece of information which I need to have in
my graph to recover back the entire directed graph of the subset relationship over the power set of
{𝑎, 𝑏, 𝑐}. So, this will be the Hasse diagram of the subset relationship over the set {𝑎, 𝑏, 𝑐}. So, why
I am drawing all this Hasse diagram and all? Because that helps us to understand the next few
concepts, which we are going to describe next.
317
So, imagine you are given an arbitrary poset less than equal to relationship. This is not again a
numerical less than equal to, this is an arbitrary relation, R, which is reflexive, anti symmetric and
transitive. Then if I take a pair of elements 𝑥, 𝑦 then the element y is called the cover of element x
if the following two conditions hold. The element x should be related to the element y and of
course 𝑥 ≠ 𝑦, that is why the less than symbol. And there should not exist any intermediate element
∃𝑧, 𝑥 ≤ 𝑧 𝑎𝑛𝑑 𝑧 ≤ 𝑦.
So, pictorially, you can imagine that y is a cover of x if I view the Hasse Diagram then in the when
I go from bottom to up y is immediately occurring or y is occurring on top of x layer wise and
there is no intermediate element or no element z in the intermediate layer. So, for instance here in
this Hasse diagram the element 2 covers the element 1 because in between 2 and 1 there is no
intermediate element. You have the element 1 which is related to the element 2 and between 1
and 2 there is no intermediate elements. But the element 6 does not cover the element 1, even
though the element 1 is less than 6, because element 1 is indeed the related to 6 as per this Hasse
diagram. But in between 1 and 6 you have this element 3 such that 1 is related to 3 and 3 is related
to 6. So, that is why 6 will not be considered as a cover of 1, but 3 can be considered as a cover of
1 because in between 3 and 1 there is no intermediate element.
So, it turns out that in a partially partial order set every element need not have a cover. So, for
instance, if you take the Hasse diagram on your left-hand side the elements 8 the element 12, it
318
does not have any common. There is no element on top of 8, there is no element on top of 12.
Similarly, an element, we have more than one cover. So, as I said earlier both 2 and 3 covers 1.
And an element may cover multiple elements. So, for instance here, in this Hasse diagram or in
this poset 6 covers 2 as well as 3. So, these are the some of the properties of the cover of an element.
Let us next define what we call as the maximal and minimal element in a poset. So, if you are
given an arbitrary poset and an element a from the set 𝑆. Then the element a is called as the maximal
element if it is on the top most layer informally, or in a loose sense or if it has no cover. More
formally, 𝑎 is called maximal element, ∃𝑏 ∈ 𝑆, 𝑏 < 𝑎 i.e., is no element b on top of a that means
there is no element b such that a is related to b where a is different from b.
So, if I take this poset, 8 and 12 are both maximal elements. Because there is no element on top of
a or no element b such that 8 is related to that b. There is no element b such that 12 is related to
that element b or so. There is 4 will not be called a maximal element sorry and 6 cannot be called
a maximal element. Why 4 cannot be called a maximal element. Because 4 is related to 8, there is
something on top of 4 and so on.
Similarly, I can define what we call as a minimal element. So, an element is called as a minimal
element if it occurs at the lower level of your Hasse diagram or in other words, it has no if it covers
319
no element. Namely, 𝑎 is called minimal element, ∃𝑏 ∈ 𝑆, 𝑎 < 𝑏 i.e., there is no element b in your
set S which occurs below a or such that b is related to a. So, for instance the element 1 here is the
minimal element. Because there is no element b for the down 1 in your Hasse Diagram such that
b is related to 1.
That tells you that why when we constructed the Hasse diagram, we assume that arrows are pointed
from bottom to up. That helps us to understand these notions of maximal element and minimal
element in an easy fashion. Now, it is easy to prove that if you have a poset over a non-empty set.
I forgot to mention here over a non-empty set, then it has at least one maximal element and one
minimal element.
So, for instance if the poset is defined over a singleton element, then your Hasse diagram will be
just a node itself say the element is a only. That means this is a valid Hasse Diagram representing
the relation (a, a). And this relation is reflexive, anti symmetric and transitive and here the element
a is both maximal element as well as minimal element.
Whereas if your set S has multiple elements and you will have a structure like Hasse Diagram and
definitely there will be some element at the lowermost level and some element at the higher most
level. So, those elements will be the maximal elements and a minimal. We can prove this thing
formally but I am not going into that. Similarly, we can prove that an element of a poset can be
both maximal as well as minimal. It is not necessary that the maximal element and minimal
element should be different and there can be many maximal element many minimal element.
So, for instance if I consider the equal to relationship (=) over ℤ. Then I will have the elements of
the form (1,1), (2, 2), (3, 3) in my relation or the negative ordered pairs of the form
(−1, −1), (−2, −2) in my relation. What will be the Hasse diagram look like? The Hasse Diagram
will just look like each integer within itself. So, the Hasse diagram will have no edges first of all.
Why no edges? Because any element; is related to itself that is all. It is not a related to any other
element.
320
So, the actual directed graph for this equal to relationship will have only self loops. It will have no
other edges. And when we construct a Hasse diagram from that diagram, we will remove all the
self loops. And as a result we will have a Hasse diagram where no edges will be present in my
graph. So, in this graph all the elements are both maximal as well as minimal.
Now finally let us define what we call as the greatest element and the least element of a poset. So,
if you are given a poset S and with an arbitrary relation less than equal to and if you have an
element a then the element a of the set S is called as the greatest element if every element b is
related to the element a as per the relation R or the relation less than equal to. In the same way the
element a is called as the least element if it is related to every other element b as per your
relationship less than equal to.
So, let me demonstrate these two concepts with this example. Here the element 1 will be the least
element. Because you have 1 related to 2 you have 1 related to 3 you have 1 related to 4 even
though the edge from 1 to 4 is not explicitly there, but as per the notion of transitive as per the
definition of Hasse diagram all the transitively implied edges are there in my directed graph of the
relation. Then similarly 1 is related to 8 and 1 is related to 6 and 1 is related to 12.
I cannot say that the least element is 2 because 2 is not related to 1 because the implicit direction
of the edges are upwards. We do not have downward facing edges as in the Hasse diagram. So, the
321
least element will be 1, but there is no greatest element. The elements 8 and 12 they are the maximal
elements. But none of them is a greatest element, because there is no relationship between 8 and
12 and 12 and 8, they are incomparable elements here. So, I will have maximum elements, but that
is not it is not necessary that I should have the greatest element present in my poset.
If I take this poset then here the least element is phi because phi is related to all other subsets as
per the subset relationship. And the greatest element will be the subset {𝑎, 𝑏, 𝑐} because all other
elements in this poset are related to this element {𝑎, 𝑏, 𝑐}. So, if at all the greatest element exists in
my poset, it will be unique but this is not necessary that a greatest element does exist in my poset.
Similarly if at all these elements exist in my poset, it will be unique. But it is not necessary that
every poset should have a least element.
Using all the concepts that we have discussed in, now we will now do a very interesting exercise
here, which we call as topological sorting. So, in this topological sorting, you are given a set of
tasks which is denoted by S and you are also given a dependency relationship R defined over the
task in the set S and task a is related to task b provided b can start only after finishing the task a.
And what we want here is we want to get a schedule according to which we should finish the tasks
in given in the set S. That means we have to decide which task to finish first and then which start
to finish next and so on provided I have the dependency among the tasks given in the form of this
322
relationship R. So, it is easy to see that the dependency relationship here is a partial ordering which
can be described by a Hasse diagram. So, I am taking here a collection of task 1, 2, 3, 4, 5, 6 and
a dependency relationship is given like this and what I want here is a schedule for scheduling the
various task in the set S.
So, there can be multiples schedules possible. So, I have listed down four of them. I can finish the
task one first and then I can finish the task 2 and then the task 3 then the task 4 then the task 5 then
the task 6 that is one way of satisfying the requirement. Because once I am done with task 1 the
dependency is over one and now I can freely choose either to do task, 2 or task 3. So, if I decide
to task if I finish to, decide task 2, then next I can decide to either finish task 3 or I can decide to
finish task 4. So, depending upon in what sequence I follow I choose the next task to complete that
will give me for different possible schedules. So, it is not the case that there is only one possible
schedule here. There can be multiple possible shapes here.
Now each of the possible outputs that I have stated over can be viewed as a total ordering over the
set S compatible with my relation R. And why total ordering? You see in my original dependency
relation, there are incomparable elements. So, for example, neither 2 is less than 3 nor 3 less than
equal to 2 because they are not dependent at each other. Both of them depend on 1. As soon as I
finish one I can freely decide or I can freely choose either the task 2 or the task 3. There is no
dependency between the tasks 2 and 3 that is why it was only a partial ordering.
But if I say that my final schedule is this then in this final schedule is this then in this final
scheduling I am saying explicitly that 2 is related to 3. That means I should I have finished 2 and
then I have finished 3. So, in some sense this output sequence, which I have obtained here one of
the possible output sequence of that I have obtained here can be considered as a possible total
ordering on the task 1, 2, 3, 4, 5, 6 compatible with the relation R.
What do I mean by compatible with the relation R? By compatibility, I mean that if at all there
was any dependency between a and b, that means if a was dependent on, if the task a was related
to task b as per the dependency relationship. Then in the final sequence which I have obtained in
the final total ordering which I have obtained it still the case that a is related to b. That means if in
323
my Hasse diagram, if I was constrained to start task number 2 only after finishing task number 1,
then in the resultant output sequence that constraint should be satisfied, it should not be validated.
It is what I mean by compatible with my original relation R.
And you can see that each of the sequences which I have obtained here. In each of the sequences,
I am satisfying the constraints which were given with respect to the original relation. None of the
dependency which was maintained which was mentioned in my original relation R is violated in
any of the output sequences which I have listed down, on any of the schedule which I have listed
down.
So, the general goal of the topological sorting is the following. You will be given a relation over a
set S that relation may or may not be a total ordering. There may be incomparable elements present
as per the relation R. What you have to output you have to output now a total ordering over the set
S and the total ordering should respect the original relation R. It should be compatible with the
original relation R.
That means whichever pair of elements which were related as per the original relation R, they
should be still related as per your new ordering. It should not happen that a was related to b in the
old ordering but in the new ordering a is not at all related to b. That should not happen. For the
incomparable elements you are free to do whatever you want. But the elements which were
324
comparable as per the original relation R, you have to maintain those that comparability property
in the new ordering as well. That is the goal of Topological Sorting.
So, how do we do this? How do we output one such total ordering? So, the algorithm is as follows.
We start with k equal to 1 and I will iteratively do the following till my set 𝑆 ≠ 𝜙. As soon 𝑆 = 𝜙,
I will stop my algorithm. So, what I will do is I will start with the minimal element that is there in
my set S. And I will list it down; that means I have taken care of that element 𝑎𝑘 in my total
ordering and I remove that element 𝑎𝑘 from the set S.
So, using my set S keeps on getting updated and that is why I start with my original set S here and
every time in each iteration, I will be removing the current minimal element of the current set S
and I will update the set S. And I will increment k to the next value of k and I do this till 𝑆 = 𝜙.
And once my set is becomes empty, I will list down the elements in the order in which I have
removed them in this value. So, let me demonstrate this algorithm with this example here.
So, you have 𝑆 = {1, 2, 3, 4, 5, 6}. Your 𝑘 = 1. I start with the original S and find out the current
minimal element. And in this case, I have only one minimal element namely the element 1. So, I
will write down the element 1 to be the first task which should be taken care in my schedule and
then I am removing the element 1 from my set S. That means the task 1 is taken care.
325
So, you can imagine that since task 1 is taken care there is no dependency of other tasks on the
task number 1 and hence these two edges vanish from my Hasse diagram. Now, I have to find out
the minimal element of the updated S. And I have two possibilities here. Both 2 as well as 3 are
the minimal elements for the updated S and it is up to me. I can either choose 2 in my schedule to
be the next task or I can choose 3 to be the next task in my schedule.
It is up to be the algorithm does not say that you have to if you have multiple minimal elements
which one to choose. So, suppose I decide to take care of task 2. So, 𝑘 = 𝑘 + 1. So, I am not
writing down the values of k here. Since I have taken care of the task 2, that means this task has
vanished now. And now I have to choose the next minimal element and my minimal element are
4 as well as 3.
So, in my sequence, I can either put 4 or I can put or I can put 3, it is up to me. So, suppose I
choose 4. So, sorry, so I am following the order 1, 2, 4 because I am taking care of 4 here. So, if I
take care of 4, I am left with this set S. And now what is the minimal element? I can choose task 6
as well as task 3, because since 4 is also taken care, this edge also vanish. So, my minimal elements
are element 6 and element 3. 5 is not minimal because 3 is related to 5. So, 5 is not minimal here.
So, it is up to me whether I put task number 6 or I can put task number 3. So, if I put task number
6 here, then 6 is taken care then I am left with only 2 task here and my minimal element is now
only 3. So, I have to take care of the task number 3 and then finally I have to take care of the task
number 5. So, that is essence of this algorithm.
In every step you are finding the minimal element, which is there in your updated set S put it in
the sequence and remove it from your Hasse diagram. A very simple algorithm. So now we have
to prove that the resultant output which we obtain from this topological sorting will be compatible
with the original relation R. That means if at all the element b was related to element c or the task
b was related to task c in the original relation, then even in the new sequence or the ordering that
you have output the element c or the task c will appear after the task b. And that is very simple to
prove.
326
The proof follows from the fact that when you were removing the task c from the Hasse diagram
and putting it in the sequence, at that time it was the minimal element. Because in each step you
only decide or you only choose to remove the minimal element from the updated set or updated
Hasse diagram.
So, when it was the turn to remove the element c, at that point of time the element c was the
minimal element in the Hasse diagram. That means at that point the element b would have been
already removed from your Hasse diagram. If element b is still present in the Hasse Diagram, you
have not removed it yet then you would have removed element b instead of element c because as
per your original relationship 𝑏 ≤ 𝑐. So, no where you would have retained b and removed c,
because in your original Hasse Diagram b was occurring on a lower level than c. And that is a very
simple fact based on which we can state or throw this term.
So, that brings me to the end of this lecture just to summarize in this lecture we introduce the
notion of partial ordering. A partial ordering is a relation, which is reflexive, anti symmetric and
transitive. We introduce the notion of total ordering, Hasse diagram and we also saw the algorithm
for topological sorting. Thank you!
327
Lecture -24
Functions
Hello everyone. Welcome to this lecture.

Just to begin with, just to recap the last lecture we discussed about the notion of partial ordering
and we discussed about topological sort. The plan for this lecture is as follows. In this lecture, we
will discuss about functions. We will see the various types of functions and we will also see
composition of functions.
328
So, what is a function? So, imagine you are given two sets. Set A and a set B and when I say I
have a function, say, 𝑓: 𝐴 → 𝐵. Then it is a special type of relation from the set A to the set B. And
pictorially you can imagine that we have the set A and set B. So, they may be the same set or they
might be different sets. It does not matter.
And what is the specialty about this relation? So, I am saying that the function is a special type of
relation from 𝐴 → 𝐵. So, of course f is a subset of the Cartesian product of A and B, 𝑓 ⊆
𝐴𝑋𝐵, because that is a definition of a relation from A to B. Now, this is a special type of relation.
The specialty here is that, each element of the set A is assigned exactly one element of the set B.
So, that means in terms of ordered pairs if I consider this function f, then each element a belonging
to the set A will appear exactly in one of the ordered pairs in the relation corresponding to this
function f. And this holds for every element a from the set A. So, none of the elements a will be
missing. Each such element a will appear as part of an ordered pair in the function f. So, we use
this notation 𝑓(𝑎) = 𝑏 to denote that b is the unique element which is assigned to the element a
as per this function f and the element b is called as the image of element a. And a will be called as
the pre image of the element b.
So, again to state the specialty of this relation each element from the set A will appear as part of
the as part of an ordered pair as an ordered pair in the function f. And it would not happen that you
329
have element a mapping element a getting map to say element b1 in the function as well as element
a getting mapped to the element b2 which is different from b1 also in the function f.
That is possible in a relation. In a relation the same element, 𝑎 ∈ 𝐴 can be mapped to multiple
elements. It can be related to multiple elements from the set B. But the specialty of the function is
that there will be only one element from the set B which will be related to the element. And now
element from the set A will be missing as part of this function and that means every element will
have a corresponding related element from the set B as part of that function.
We also use the term domain and the co domain and the context of a function. So, the set A will
be called as the domain of the function while B is called as the co-domain of the function. So,
notice that all these definitions are with respect to a function 𝑓: 𝐴 → 𝐵. The direction of the
function matters a lot. If I say my function is from the set B to the set A. Then it will be it should
be interpreted as a special type of relation from the set B to the set A where the ordered pairs will
have the first component from the set B. And the second components of those ordered pair will
be from set A. And then the domain will be the set B and the co-domain will be the set A and so
on.
So, now we will be interested to study some important class of functions. So, the first important
class of functions is the one to one or injective functions. So, imagine you are given a function f
330
from the set A to the set B. It will be called as an injective function, provided distinct elements
from the set A have distinct images. So, pictorially you can see here that each of the elements the
circles from the set A is assigned a distinct element from the set B as an image.
So, to put it formally we want ∀𝑎, 𝑏 𝑓(𝑎) = 𝑓(𝑏) => 𝑎 = 𝑏 should hold for an injective function.
So, what exactly this universal quantification means it says that if you have two elements a and b
whose images are same then that is possible only if the elements themselves are the same or the
pre images are the same. Or equivalently the contrapositive of the same thing is ∀𝑎, 𝑏, 𝑎 ≠ 𝑏 =>
𝑓(𝑎) ≠ 𝑓(𝑏).
So, both these definitions are equivalent. If you prove any of these two conditions to be true for
the function f and remember both of them are universal quantifications. If any of these two
universal quantifications hold for your function f then we will call the function f to be an injective
function. Even if there is one pair of elements here a and b for which these universal quantifications
are not true then the function will not be called as an injective function.
So, you can see why it is called one to one function. The mapping is unique for every element. So,
if I consider the function f(x) define to be 𝑓(𝑥) = 𝑥 2 over the set ℤ+ . So, this set is the set of
positive integers. Then what do you think is this function is a one-to-one function? Well, it is not
an injective or one to one function over the set of entire integers. If I consider the set of all integers,
definitely this is not a one-to-one function because I have both +𝑥 getting mapped to 𝑥 2 as well
as −𝑥 getting mapped to 𝑥 2 , if my domain is the set ℤ if my 𝑥 can take positive values as well as
negative values.
But if I restrict my function only over the set of positive integers that means my 𝑥 ∈ ℤ+ and clearly
this is an injective function. So, you can see the importance of domain. If you change the domain,
the interpretation or the meaning or the property of the function changes immediately.
331
The next important category of function is the onto or surjective functions. So, again imagine you
are given a function 𝑓: 𝐴 → 𝐵. If we call it will be called as a surjective function provided the
following universal quantification hold. You take any element b from the co-domain, it should
have at least one pre-image. That is the condition.
So, you can see here and the universal quantification the domain of b is the co-domain of the
function. And the domain of a is the domain of the function. So, pictorially you take any element
from the set B, there should be at least one pre-image for that element. It might have multiple pre
images as well. So, for instance, if you take this particular element, it is the pre-image of two
possible elements. It is the pre-image of this element as well as it is the pre-image of this element.
The definition simply says there should not be the case that you have some element in the co-
domain set which do not have any pre-image. That will be a counter example for this universal
quantification. So, even if you have any such counter example, the function will not be called as a
surjective function. Whereas if this universal quantification is true for all b, a where the domain of
b is the B set and the domain of a is the A set, the function will be called as a surjective function.
So, here are two functions, the first function is over the set ℤ as 𝑓(𝑥) = 𝑥 + 1. Now, what can
you say whether it is a surjective function or not? Well, it is a surjective function. So, what you
can prove is, you take any integer 𝑦 ∈ ℤ. Its pre-image will be 𝑦 − 1. That means the element y
332
- 1 would have been mapped to the element y and this is true for any y from the co-domain set. So,
that is why the first function is a surjective function or onto function. But the second function is
not onto because 𝑥 2 ∈ ℤ+ . So, it is only the positive integers from the set of integers which will
have a pre-image. The negative integers will have no pre-image. You cannot have 𝑥 2 ∈ ℤ− .
So, now the question is how do we prove whether a given arbitrary function is an onto function or
not? So, we have to check whether this universal quantification is true for that given function. But
if the domain and the co-domain of the function f is an infinite set, we cannot check whether this
universal quantification is true for every b, and with respect to that b for some a.
So, what we will do is, instead we will follow the principle of universal generalization. Since we
have to prove that the condition is true for all the elements b from the co-domain set, we will
instead pick an arbitrary element b from the co-domain set. And for that arbitrarily chosen element
b, we will show we will prove the existence of at least one pre-image element a from the domain
such that 𝑓(𝑎) = 𝑏.
And since we are going to we will show this for an arbitrary b from the co-domain set that will
show that the condition holds for any element b from the co-domain set. That is all we will prove
whether a given function is an onto function or not. Whereas if you want to prove that a function
given function is not a surjective function, then we have to find a counterexample. Namely a
counterexample for which the universal quantification does not hold. Then we have to find out
element from the co-domain set say b which do not have any pre-image.
333
Now the third category of important functions is the one – one, onto function. They are also called
as bijective functions. Sometimes they have also used a term one to one correspondence. There
are many terms for the same concept. So, again you are given a function 𝑓: 𝐴 → 𝐵. It will be called
as a bijective function or bijection provided f is an injective function. That means this condition
should hold.
That means different element or distinct element should have different images and the function
has to be a surjective function. So, if I take the function 𝑓(𝑥) = 𝑥. This is also called as the identity
function. Why it is called an identity function? Because every function, every element is just
mapped to itself and it does not matter what is the domain and co-domain of this function. This
will be always a bijective function.
Clearly it is an injective function because if you have 𝑥1 ≠ 𝑥2 , then 𝑓(𝑥1 ) = 𝑥1 , and 𝑓(𝑥2 ) = 𝑥2 .
And since 𝑥1 ≠ 𝑥2 , 𝑓(𝑥1 ) ≠ 𝑓(𝑥2 ) that shows that the function is injective whereas if you take
any given random or arbitrarily chosen element y from the co-domain set, the corresponding pre-
image for that y is the element y itself. It shows that the function is a surjective function.
334
Now we will define what we call as the inverse of a function. So, imagine a function 𝑓: 𝐴 → 𝐵.
So, since f is a special type of relation, remember it is a special type of relation we can find the
inverse of that relation as well. So, basically we are trying to find out the inverse relation. If I
denote as 𝑓 −1 but it will be a function it will be defined as a function, that means it should satisfy
the requirements of a function.
That means f inverse will be a subset of B cross A and it will be a function. That means each
element 𝑏 ∈ 𝐵 will appear as part of an ordered pair in this collection of ordered pairs of 𝑓 −1 .
And what will be the mapping of any element b as part of this inverse function? The mapping will
be or the image of that element b will be an element a provided b was the image for that element
the a as per the function f.
So, it is easy to see that a function f will be invertible if and only if the function is a bijection. If
the function f is not a bijection then, we cannot define the inverse of that function. So, let us prove
this formally. We can show that if your function f is not an injective function then clearly f is not
invertible. So, since the function f is not invertible that means you have two elements in the domain
say, 𝑎1 ≠ 𝑎2 , but getting mapped to the same 𝑏.
As far as the function definition of function is concerned this is allowed. I am not saying my
function f is an injective function here. I am assuming my f is not an injective function. So, f is not
335
an injective function that means this is possible. But if this is the case, what can you say about the
inverse of b? Will you say 𝑓 −1 (𝑏) = 𝑎1 or will you say 𝑓 −1 (𝑏) = 𝑎2 ? it is ambiguous here.
We cannot find out the image of the element b as per the 𝑓 −1 function. So, 𝑓 −1 function will not
be defined. I stress that if this would have been a relation then it is absolutely fine. It would have
been a relation then in 𝑓 −1 you could have (𝑏, 𝑎1 ) and you could have (𝑏, 𝑎2 ) as well. Because in
relation there is no restriction, but 𝑓 −1 has to be a function, and you cannot have two images for
the same element if it is a function.
So, clearly this shows that if your function f is not an injective function then it is not invertible.
We can also prove here that if the function f is not a surjective function then also it is not invertible.
So, if it is not a surjective function that means you have some element at least one b from the co-
domain set which do not have any pre-mage. If that is the case, then what can you say about the
image of the element b as per the 𝑓 −1 function? It is not defined here. But that is a violation of the
definition of inverse of a function. So, that shows that a function is invertible if and only if f is a
bijection.
So, the last thing that we want to now define is the composition of functions. So, remember
336
functions are special relations. So, since we can compose relations, we can compose functions as
well provided certain conditions are satisfied. So, imagine you are given two functions, a function
𝑔 and the function 𝑓 with appropriate domain and co-domain. Then the composition of the f
function and g function is denoted by 𝑓𝑜𝑔.
So, let me make it more clear here. So, you imagine that 𝑔: 𝐴 → 𝐵 where 𝐴 = 𝐵 or 𝐴 ≠ 𝐵. And
𝑓: 𝐵 → 𝐶. Then the composition of 𝑓𝑜𝑔 will be a function from the set 𝐴 → 𝐶. And what exactly
will be that function. So, we will apply the function g first and obtain the images or possible
elements a from the set A. And then we will apply the function f on those resultant elements and
that will be the mapping of the element A as per the composition of f and g function, 𝑓𝑜𝑔.
So, this composition of function has to be a function and it is easy to see that since it has to be a
function then the composition of f and g is defined only if 𝑟𝑎𝑛𝑔𝑒(𝑔) ⊆ 𝑑𝑜𝑚𝑎𝑖𝑛(𝑓). So, what is
exactly the 𝑟𝑎𝑛𝑔𝑒(𝑔) means?
So, since g is a mapping from I am assuming here that 𝑔: 𝐴 → 𝐵, then A is called as the domain
and B is called as the co-domain. Then what is this range set? It is the set of all images of various
elements from the A set. That means this range is a subset. It may be the entire set B or it might
be a proper subset of the set B. So, if your function 𝑔: 𝐴 → 𝐵, what I am saying is I am just focusing
on all the elements which is a subset of this set B which have pre-images.
That means they are the images of some element from the set A. Because your function g may not
be a surjective function there might be some elements in the set B who do not have any pre- images.
So, I ignore those elements when I am considering the range set. Those elements will be considered
this s star elements will be considered as part of the co-domain set. But they would not be
considered as part of the range set.
And when I say the range set, I am just focusing on the image set, the images that the function g
assigned to various elements here. So, coming back to the composition of function, it is easy to
see that 𝑓𝑜𝑔, only if you take the range set of the g function then it should be a subset of the
domain of the f function. If your 𝑓: 𝐵 → 𝐶, then if I focus on this circled thing, each of them should
337
have an image as per the f function.
That means that circled thing should be a part of the domain set of the f function. If this condition
is not satisfied and clearly the composition of 𝑓𝑜𝑔 is not defined. And again, the composition of
functions need not be commutative in the sense 𝑓𝑜𝑔 ≠ 𝑔𝑜𝑓. First of all the composition of 𝑔𝑜𝑓
need not be defined at all, if 𝑓𝑜𝑔 is defined because 𝐴, 𝐵, 𝐶 sets could be arbitrary. So, only under
special conditions the compositions 𝑓𝑜𝑔 = 𝑔𝑜𝑓. So, that brings me to the end of this lecture.
These are the references for today's lecture. Just to recap in this lecture we introduce the concept
of set. We discussed about various types of various special types of sets and we also discussed
about compositions of relations. Thank you!
338
Lecture -25
Tutorial 4: Part I
Hello everyone, welcome to the first part of tutorial 4. So, let us start with question number 1(a).
This question you are given two equivalence relations on a non-empty set 𝑋. You are asked to
prove or disprove whether 𝑅1 ∪ 𝑅2 is an equivalence relation or not? So, it turns out at a union of
two equivalence relations need not be an equivalence relation.
And this is demonstrated by this counter example. So, I consider my set 𝑋={a,b,c}. And let me
have equivalence relations 𝑅1 and 𝑅2 . So, 𝑅1 = { (𝑎, 𝑎), (𝑏, 𝑏), (𝑐, 𝑐), (𝑎, 𝑏), (𝑏, 𝑎)} and 𝑅2 =
{ (𝑎, 𝑎), (𝑏, 𝑏), (𝑐, 𝑐), (𝑏, 𝑐), (𝑐, 𝑏)}. It is easy to see that both of them are equivalence relations
over the set X they are reflexive. Each of them is a reflexive relations symmetric relation and a
transitive relation I am not going through that, it is easy to verify that.
Now, if you take the union of these two relations, 𝑅1 ∪ 𝑅2 , you will have these ordered pairs. And
it is easy to verify that 𝑅1 ∪ 𝑅2 is a reflexive relation over the set X, because you have
{(𝑎, 𝑎), (𝑏, 𝑏), (𝑐, 𝑐)} present in the relation. The union is also symmetric because if you have any
339
(𝑎, 𝑏) in the union present and ordered pair (𝑏, 𝑎) is also present in the union.
But it is easy to see that union here is not a transitive relation, specifically you have (𝑎, 𝑏) and an
ordered pair (𝑏, 𝑐) present in 𝑅1 ∪ 𝑅2 , but a, c is not present in the union and hence the transitivity
properties violated. So, in general the union of two equivalence relations need not be transitive.
We can prove that they will be always it will be always reflexive and symmetric. If I take the union
it will be always reflexive and symmetric, but the union need not be a transitive relation.
In part b of question 1, we are supposed to prove whether the obvious as to prove or disprove,
whether the intersection is an equivalence relation or not. And it turns out that intersection of two
equivalence relations over the same set X is always an equivalence relation. So, let us prove the
three required properties. If I take 𝑅1 and 𝑅2 to be equivalence relations over the set X, then since
𝑅1 and 𝑅2 are individually reflexive relations. You will have ordered pair of the form (𝑎, 𝑎) present
in both 𝑅1 as well as in 𝑅2 . And as a result, you will have ordered pairs of the form (a, a) present
in the intersection of 𝑅1 and 𝑅2 as well. That shows that 𝑅1 ∩ 𝑅2 is a reflexive relation. Well let
us try to prove the symmetric property for 𝑅1 ∩ 𝑅2 . So, for that I consider an arbitrary ordered pair
(𝑎, 𝑏) to be present in 𝑅1 ∩ 𝑅2 .
Since it is present in the intersection as per the definition of intersection, it will be present in both
𝑅1 as well as 𝑅2 . Now it is given that my relation 𝑅1 and relation 𝑅2 are equivalence relations, so
340
individually both of them are symmetric relations. And as a result of that the ordered pair (𝑏, 𝑎)
will be present in 𝑅1 as well as the ordered pair where (𝑏, 𝑎) also will be present in 𝑅2 . So, since
(b, a) is present in both 𝑅1 as well as in 𝑅2 , it will be present in their intersection as well. And
which shows that the intersection will be a symmetric relation.
Now consider the transitivity property for which I consider arbitrary ordered pairs (a, b) and (b, c)
to be present in 𝑅1 ∩ 𝑅2 . So, since it is present in 𝑅1 ∩ 𝑅2 , there two ordered pairs will be
individually present in both 𝑅1 as well as in 𝑅2 and since 𝑅1 and 𝑅2 are individually equivalence
relations. Each of them satisfies the transitivity property, due to which I get that ordered pairs (a,
c) is present in both 𝑅1 as well as in 𝑅2 . And hence the ordered pair (a, c) will be present in their
intersection as well.
So, now you can see that the argument that we have given here for 𝑅1 ∩ 𝑅2 to be a transitive
relation need not hold for the union. And this is precisely the reason due to which the union of two
equivalence relations need not be transitive relation and hence any equivalence relation you can
verify it.
Now on question number 2 again, you are given two equivalence relations over a non- empty set
and we want to prove that 𝑅1 ∪ 𝑅2 will be an equivalence relation if and only if the composition
𝑅1 𝑜 𝑅2 is equal to the 𝑅1 ∪ 𝑅2 . And this is, an if and only if statement. So, we have to give two
341
proofs. We have to prove the implication in both the directions.
So, let us first prove the implication in the direction where I assume 𝑅1 𝑜 𝑅2 = 𝑅1 ∪ 𝑅2 . Under that
assumption I will be showing that 𝑅1 ∪ 𝑅2 is an equivalence relation. And I will be just focusing
on proving that 𝑅1 ∪ 𝑅2 is a transitive relation. Because we can always show that 𝑅1 ∪ 𝑅2 will
satisfy the reflexive property and symmetric property.
It is only the transitive property which is missing. And what we will show is that if this premise
hold, then 𝑅1 ∪ 𝑅2 will be satisfying the transitivity property. So, let us consider arbitrary ordered
pairs (a, b) and (b, c) to be present in 𝑅1 ∪ 𝑅2 . Now because of the union there could be three
possible cases depending upon where exactly the ordered pair (a, b) and ordered pair (b, c) belongs.
So, case 1 could be that the ordered pairs (a, b) as well as (b, c) are present in one of these two
relations, at least in one of these two relations. Say in the relation 𝑅𝑖 , there 𝑅𝑖 could be either 𝑅1
or 𝑅2 . If that is the case, then since that relation 𝑅𝑖 where both (a, b) and (b, c) are present is also
an equivalence relation mind it. We are given that individually, we are given that both 𝑅1 as well
as 𝑅2 , are equivalence relations.
So, if both (a, b) and (b, c) are present in the relation 𝑅𝑖 and as for the transitivity property the
ordered pair (a, c) will be present in 𝑅𝑖 as well and hence it will be present in 𝑅1 ∪ 𝑅2 . Case 2
could be where the ordered pair (𝑎, 𝑏) ∈ 𝑅2 and ordered pair (𝑏, 𝑐) ∈ 𝑅1 . In that case, you will
have (a, b), (b, c) present in the union as well. Now that is the case, then as per the definition of
composition of two relations, the ordered pair (a, c) will be present in 𝑅1 𝑜 𝑅2 because you have an
intermediate b. So, (a, b) is in 𝑅2 and (b, c) is in 𝑅1 . So, hence (a, c) will be in 𝑅1 𝑜 𝑅2 . But our
premise says that 𝑅1 𝑜 𝑅2 is exactly the same as the union. So, (a, c) is present in 𝑅1 𝑜 𝑅2 , it will
present in the union as well and hence in this case also we proved a transitivity property.
So, tricky cases when you have the ordered pair (𝑎, 𝑏) ∈ 𝑅1 and ordered pair (𝑏, 𝑐) ∈ 𝑅2 . In that
case, I cannot apply the same argument as I applied in case 2, I have to do something extra here.
So, what I do here is I apply the symmetric property for the relation 𝑅1 . Mind it 𝑅1 is an
equivalence relation and hence it has to symmetric property. So, since (𝑎, 𝑏) ∈ 𝑅1 we will have
342
(𝑏, 𝑎) ∈ 𝑅2 .
Due to the same reason, since 𝑅2 it also satisfies the symmetric property. We will have (𝑐, 𝑏) ∈
𝑅2 . And as a result I can say that the ordered pair (𝑐, 𝑎) is present in the composition. But now
that is not my goal. My goal is to show that (a, c) is present in the composition and then I can use
the fact that 𝑅1 𝑜 𝑅2 is same as 𝑅1 ∪ 𝑅2 and conclude that (a, c) is in the union as well.
So, what I do here is since I know I have the premise says that 𝑅1 𝑜 𝑅2 is same as their union, I
can conclude that ordered pair (c, a) is present in the union as well. Now if it is present in 𝑅1 ∪ 𝑅2 .
It will be present in at least one of these two relations either in 𝑅1 or 𝑅2 . So, without loss of
generality let it be present in 𝑅1 . So, this w.l.o.g here means, without loss of generality.
So, whatever argument I am going to give assuming that (c, a) is present in 𝑅1 , can be applied in
also symmetrically for the case when (c, a) is present 𝑅2 . So, assuming (𝑐, 𝑎) ∈ 𝑅1 , since 𝑅1 is
also symmetric we get (a, c) also present 𝑅2 and hence (a, c) will be present in the union of 𝑅1 .
We have proved the implication in this direction.
Let us prove the implication in the other direction. So, I am going to prove that if 𝑅1 ∪ 𝑅2 is an
equivalence relation, then there and 𝑅1 𝑜 𝑅2 is equal to their union. So, I have to prove the equality
of two sets. I have to prove that 𝑅1 𝑜 𝑅2 ⊆ 𝑅1 ∪ 𝑅2 . And I have to prove that 𝑅1 ∪ 𝑅2 ⊆ 𝑅1 𝑜 𝑅2 .
343
Then only I can conclude that these two sets are equal.
So, let us prove that 𝑅1 𝑜 𝑅2 ⊆ 𝑅1 ∪ 𝑅2 . And how do I prove that a set X is a subset of set Y. I
have to show that you take any element from the set X it will be present in the set Y as well. So,
that is why I am taking an arbitrary (a, c) present in 𝑅1 𝑜 𝑅2 . So, since it is present in the 𝑅1 𝑜 𝑅2
what I can say here is that, as for the definition of composition.
There should, exist some intermediate element b. Such that (a, b) will be present in 𝑅2 and (b, c)
will present in 𝑅1 . That is the definition of composition. And that means that ordered pair (a, b) is
present in 𝑅1 ∪ 𝑅2 and (b, c) is also present in 𝑅1 ∪ 𝑅2 . And since I am assuming here, that 𝑅1 ∪
𝑅2 is an equivalence relation that is the premise. It will be transitive as well.
And if (a, b) and (b, c) are present in the union and it is transitive that means (a, c) is present in the
union as well. So, what we have shown here is now. You take any element any ordered pair (a, c)
in 𝑅1 𝑜 𝑅2 , it will present in 𝑅1 ∪ 𝑅2 as well. Let us prove, now Y is the subset of X. Namely we
will prove that 𝑅1 ∪ 𝑅2 ⊆ 𝑅1 𝑜 𝑅2.
So, again, let us take an arbitrary element namely ordered pair arbitrary ordered pair (a, b) present
in 𝑅1 ∪ 𝑅2 , we will show it is present in 𝑅1 𝑜 𝑅2 . Again there could be two cases depending upon
whether (a, b) is in 𝑅1 or whether (a, b) is in 𝑅2 . So, again without loss of generality assume that
it is present in the first relation. And I know that my relation 𝑅2 is reflexive and I am assuming
here that the relations are over a non-empty set.
That means my set X has at least one element a. So, since my relation 𝑅2 is reflexive, I will also
have element ordered pair (a, a) in my relation 𝑅2 and now I can use the definition of 𝑅1 𝑜 𝑅2 . I
have (a, a) present in 𝑅2 and I have (a, b) present in 𝑅1 . So I can say (a, b) is present in 𝑅1 𝑜 𝑅2 .
So, that shows Y is also a subset of X and hence the two sets are equal.
344
In question number 3, we are defining a function P(n) which denotes the number of equivalence
relations over set S consisting of n elements. So, P(1) means, number of possible equivalence
relations over the set consisting of one element. P(2) will give you the number of equivalence
relations over set consisting of two elements and so on. Now the question ask you to either prove
or disprove whether P(n) satisfies this condition.
So, here the C function is the combinatoric function, namely it denotes here. So, C(n-1,j) here the
notation denotes the number of ways of selecting j objects, j distinct objects or j objects we say
from a collection of n - 1 objects. That is a notation C(n-1, j). Basically this is a recurrence equation,
what exactly we mean by a recurrence equation here. We are trying to express the value of the
function P on input n in terms of the value of function P on previous input namely on inputs of
size less than n.
So, we are supposed, we are asked to either prove or disprove whether this condition holds or not.
In fact, we will prove that this is true, this equation is true. The first thing to observe here is that
the function P(n) also denotes the number of partitions of a set S consisting of n elements. Because
remember we have proved that every equivalence relation gives a partition. And every partition
corresponds to an equivalence relation.
So, the number of equivalence relations is nothing but the number of partitions over that is it. Now
345
we will focus our argument based on the fact that we open denotes the number of partitions of a
set S consisting of n elements. So, imagine your S set is has n elements, and what we are going to
do is, we are going to discuss what are the various ways in which we could partition this set S.
So, for that I consider the first element 𝑎1 . Now in order to partition the set S into various subsets
definitely the element 𝑎1 will be present in one of the subsets in that partition. And along with the
element 𝑎1 , there could be j other elements from the set S in the subset in which the element 𝑎1 is
present. Now the j ranges from 0 to n - 1, what does that mean? That is either the element 𝑎1 might
be the only element in the subset in its partition, in the partition that means, when you are
partitioning the set S into various {𝑎1 } is the solitary element in {𝑎1 } is present. That is one case
in which case my j will be 0 or my partition could be such that that, the 𝑎1 is present along with
all other elements of the set S in its subset. In which case j can take the value n -1. So, that is why
the range of this j here is from 0 to n -1.
That means what I am saying here is that irrespective of the way you partition the set S, the subset
in which a1 is present along with a1 you will have j other elements. So, in total that subset will
have j + 1 element. And the j elements will be chosen from the remaining elements 𝑎2 to 𝑎𝑛 . So,
how many ways you can pick those j elements from the remaining n- 1 elements, that is why the
notation c of n- 1, j is coming into picture here.
It is not the case that all only the first j elements outside a1 will be present along with an. You can
pick any j elements from the set 𝑎2 to 𝑎𝑛 . That is why this expression 𝐶( 𝑛 − 1, 𝑗) will be picturing
here. Now once we have decided which j elements are going to come together with a1 in its subset,
the remaining elements which are now n - j + 1 in number, have to be partitioned. And there are
these many numbers of ways of partitioning a smaller set consisting of n- j + 1 number of elements.
That means once you have decided that I am deciding, I am defining a partition where along with
a1 these other j elements are going to come, once you have decided which j elements are going to
take the position along with 𝑎1 , now your remaining elements are n - j + 1. And now you have to
worry about how you are going to partition that is smaller subset. And now as for the definition of
my P function there are, P(n) - j - 1 number of base of dividing that subsets.
346
So, once you have decided which j elements to occupy or which j elements to put along with a1.
These will be the total number of ways in which you can partition the set S. So, that gives you one
type of partition. Now since j ranges from 0 to n - 1, you have n number of such possible types of
partitions. That is why we get this overall formula. My first class of partition is there, the element
𝑎1 is the only element in this subset.
Remaining n - 1 elements are now partitioned into various subsets. So, that is one category of
partition there are these many number of partitions of that type. My second category of partitioning
of set S is where along the a1, I also put one additional element in that subset. That additional
element will be chosen in these many numbers of ways P(n) – 1, 1 and now the remaining n - 2
elements are partitioned into P(n) - 2 number of ways.
And continuing like that my last category of partitioning is the following, where I put n - 1 element
along with the element 𝑎1 in its subset. That means that whole set is the only partition in which
case I have to now partition the remaining elements which are now 0 in number. So, that is why
𝑃(0), I can define to be 1 that means if you are set as only if a set is an empty set and there is only
one way of partitioning it, namely knowing.
So, I can define 𝑃(0) = 1 and now you can check that P(n) satisfies this equation and all these
different types of partitioning are disjoint. There will not be any partitioning which will present in
which can be considered as two different types of partitions, because the value of j is different for
each category.
347
Now in question number 4, we are supposed to find out the number of partial orderings over a set
S consisting of three elements. So, instead of enumerating all possible partial ordering, so what
over the set consisting of three elements remember partial ordering means your relation is
reflexive, antisymmetric and transitive. So, instead of enumerating all such relations what we will
do is we will count the number of distinct Hasse diagrams, which we can draw using these three
sets. Because remember, each partial ordering can be represented by Hasse diagram.
So, it turns out that we can draw five different categories of Hasse diagram over the set. And let
us consider each of them and each category, we will count how many Hasse diagrams we can
draw. The first category of Hasse diagram is where I have no edges among the nodes.
So, I have the nodes 𝑎𝑖 , 𝑎𝑗 , 𝑎𝑘 , where a i can be any value in the set {1, 2, 3}, 𝑎𝑗 can be any value
in the set {1, 2, 3} and 𝑎𝑘 is any value in the set {1, 2, 3}. What exactly is the relation corresponding
to this Hasse diagram? The relation here will be 𝑎𝑖 is related to 𝑎𝑖 , 𝑎𝑗 is related to 𝑎𝑗 and 𝑎𝑘 is
related to 𝑎𝑘 . Remember in Hasse diagram, the directions are not there, self- loops are always
implicit, transitively implied edges are also there and so on.
So, the relation corresponding to this Hasse diagram is this relation, which is a partial order. Now,
the question is how many types of Hasse diagrams of this category I can draw? I can draw only
one Hasse diagram like this, because it does not matter whether 𝑎𝑖 is 1 or 2 or 3. The resultant
348
partial ordering will be the same. So, I can have only one partial ordering whose Hasse diagram
will be of category a.
My category b Hasse diagram will be like this, where I will have. So, this is a relation this
corresponds to the relation where I have 𝑎𝑖 is related to 𝑎𝑖 , 𝑎𝑗 is related to 𝑎𝑗 and 𝑎𝑘 is related to
𝑎𝑘 , remember self-loops are always implicitly there. And we have the ordered pair 𝑎𝑖 related to
𝑎𝑗 , because the directions are always assumed to be from bottom to up. So, the question is how
many partial ordering of this type we can have?
In terms of we can have six different partial ordering depending upon what is your value of 𝑎𝑖 and
what is your value of 𝑎𝑗 . Because your, 𝑎𝑖 could be either 1 or 2 or 3. If my, 𝑎𝑖 is 1, then that is
different from the case when my 𝑎𝑖 is 2 and so on. But you have three choices for the element 𝑎𝑖 .
And once you have decided what is your a i you have now two choices for 𝑎𝑗 because 𝑎𝑗 has to be
different from 𝑎𝑖 .
And once you have decided 𝑎𝑖 and 𝑎𝑗 you do not have any other choice remaining for 𝑎𝑘 the third
element which is now left has to be 𝑎𝑘 . That is why I can have only six possible partial orderings
in this category. My category three could be where I have a total ordering among 𝑎𝑖 , 𝑎𝑘 , namely
by a Hasse diagram is a chain. And it turns out that we can have six partial orderings of this
category depending upon whatever my values of 𝑎𝑖 and 𝑎𝑗 .
So, I have three choices for 𝑎𝑖 . By 𝑎𝑖 could be either 1, 2, 3. Once I have fixed 𝑎𝑖 , I have two
choices for 𝑎𝑗 and once I have fixed 𝑎𝑖 and 𝑎𝑗 the third element will be mine. So, that is why will
have three different Hasse diagrams in this category. Fourth category is where you have a Hasse
diagram where you have a least element and two maximal elements. In this category, we can have
three partial orderings depending upon the choice of your least element.
Your least element 𝑎𝑖 could be either 1 or 2 or 3. Once you have decided your, 𝑎𝑖 , it does not
matter whether what is your 𝑎𝑗 and 𝑎𝑘 you are going to be the remaining two elements. You can
have only three partial orderings of this category. And your last category is when you have only
349
the greatest element namely 𝑎𝑘 and two minimal elements 𝑎𝑖 , 𝑎𝑘 . In this category, you can have
three partial orderings depending upon what is your greatest element.
The choice of your greatest element, the greatest element would be either 1 or 2 or 3. Once you
have decided what is your greatest element, it does not matter whether 𝑎𝑖 is the what are your
remaining elements 𝑎𝑖 , they are going to be the remaining two elements. So, that is why if I now
count all the different partial orderings and the various categories I get 19 different relations over
the set {1, 2, 3} which will be reflexive, anti- symmetric and transitive.
In question 5, you are given the following, you are given an arbitrary poset. And for any subset T
of that set S an element x from that set T will be called as a minimum element, if that element x is
related to all other elements y of depth subset T. So, here I am defining minimum element with
respect to the subsets here. It is not a global minimum element it is defined with respect to a subset
of the set S.
Now and the question you are given, the condition that is your poset is such that every non- empty
subset T of S as a minimum element. That means it does not matter what the size of your subset
T. You take any subset T of the set S is a minimum element as per this definition is bound to exist.
So, your poset is like that. Under that condition you have to show that your poset is actually a total
ordering. It is not a partial ordering but it is actually a total ordering.
350
So, remember the definition of total ordering is you take any pair of elements they will be
comparable either the first element is related to the second or the second is related to the first. You
will not have incomparable elements. So, that is what we are going to do here. We will take an
arbitrary pair of elements a, b which are distinct and we will show they are comparable. That means
either 𝑎 ≤ 𝑏 or 𝑏 ≤ 𝑎, remember this less than notation does not mean the numerical less than
notation. It means that a is related to b or b is related to a as per your relation less than equal to,
where less than equal to is not the numeric less than equal to relation, it is an arbitrary relation
which is reflexive, anti- symmetric and transitive.
So, how I am going to show that 𝑎 ≤ 𝑏 or 𝑏 ≤ 𝑎. I will take the subset T consisting of the elements
a, b. And as per the given condition this subset T also will have a minimum element. Let us call
that, denote that minimum element by x. Now the definition also says that a minimum element
will be within that subset itself. And my subset here is the set {𝑎, 𝑏}, that means that minimum
element can be either a or b. If the minimum element x is a then we get that 𝑎 ≤ 𝑏as per the
definition of the given minimum element.
And 𝑎 ≠ 𝑏, that means 𝑎 ≤ 𝑏 and hence a and b are comparable. Case 2, when my minimum
element x is the element b. In this case, again since 𝑎 ≠ 𝑏 and as per the definition of the minimum
element there is set element 𝑏 ≤ 𝑎 and hence a and b are comparable. So, with that we finish the
first part of this tutorial. Thank you!
351
Lecture -26
Tutorial 4: Part II
Hello everyone welcome to the second part of tutorial 4.

So, we begin with question number 6. In question number 6 you are asked to either prove or
disprove the following. You are given a function 𝑓: 𝐴 →A and you are given that the function is a
surjective. Then the question is, is it necessary that the function is a bijective as well. It turns out
that the statement is true provided your set A is a finite set. Because indeed if the set is a finite set
and the function is from the same set to itself and surjective.
Then we can show it is a bijective function as well. So, we will touch upon this fact sometime later
in this course. But the statement need not be true if the set A is an infinite set. So, here is a counter
example. So, imagine the function f given from the set of set 0 to infinity to the set 0 to infinity, .
So, that is my set A and the function is defined as follows. The mapping 0 → 0 and the mapping
of 1 → 0.
352
So, clearly the function is not injective, not 1 to 1 and what about the mapping of the remaining
elements. The mapping of element 2 is 1 the mapping of element 𝑥 → (𝑥 − 1) and so on. So, it
is easy to verify that the function is indeed a surjective function because you pick any element y
from the set 0 to infinity, the pre-image for that element y will be y + 1. Because 𝑓(𝑦 + 1) = 𝑦
as per the function f we have defined here.
So, clearly my function is a surjective function. But it is not one to one and that is why it is not the
bijective function. The problem due to which we cannot say it is a bijection is because the set over
which the function is defined can be an infinite set.
In question number 7, you are given an equivalence relation over a set A, where the set A has 30
elements. And since it is an equivalence relation, the relation partitions the set A into three subsets
each of equal size. So, the question asks you how many ordered pairs are there in that equivalence
relation? So, since the subsets 𝐴1 , 𝐴2 , 𝐴3 constitute a partition of the set A and it is also given that
the size of each subset is same and since the number of elements in the set A is 30, we get that the
size of each subset in the partition is 10.
Recall, when we showed that for every equivalence relation there is a partition and for every
partition there is an equivalence relation, we showed that if you are given a partition how you get
the corresponding equivalence relation whose equivalence class will be giving you that partition.
353
So, in that construction our equivalence relation was consisting of all ordered pairs of the form (i,
j), where for every subset 𝐴𝑘 in your given partition if the elements (i, j) are present in that subset
you add the ordered pair (i, j). So, based upon this fact we get here that the elements of the subset 𝐴1
within the partition will contribute to ten square ordered pairs of the form (i, j) and they will be
added to the relation R.
Similarly, you have 10 elements within the subset 𝐴2 and they will contribute to 10 square ordered
pairs as per our construction in the relation R and in the same way you have 10 elements in the
subset 𝐴3 and they will contribute to 10 square number of (i, j) ordered pairs or order tuples in the
relation so, as a result the number of ordered pairs in our equivalence relation with 300.
In question 8 part (a), (b), (c) we are supposed to count certain things. So, you are given two sets
X and Y consisting of m and n number of elements. So, I am calling the elements of 𝑋 =
{𝑥1 , … , 𝑥𝑚 } and the elements of the 𝑌 = {𝑦1 , … , 𝑦𝑛 }. We are supposed to find out the number of
functions from the 𝑋 → 𝑌. It turns out that the number of functions will be 𝑛𝑚 , why so?
Because when we want to build a function from the set 𝑋 → 𝑌, each element 𝑥𝑖 from the set X has
to be assigned an image that is the definition of a function. Now how many ways I can assign an
image for the element 𝑥𝑖 ? Well I can assign 𝑦1 as the possible image for 𝑥𝑖 , I can pick 𝑦2 as the
354
possible image for 𝑥𝑖 and in the same way i can pick 𝑦𝑛 as a possible image for the element 𝑥𝑖 . So,
there are n possibilities when it comes to assigning image for an element 𝑥𝑖 .
And the image for 𝑥𝑖 and the image for 𝑥𝑗 they are independently picked, there is no dependency
between the images of 𝑥𝑖 and images of 𝑥𝑗 that is important here because we are just interested in
counting the number of functions. That means it might be possible that the image of 𝑥𝑖 is same as
the image of 𝑥𝑗 and so on. So, there is absolutely no restriction on the way we can pick the images
for 𝑥𝑖 and we can pick the images for 𝑥𝑗 .
So, based on all these observations we can say that I have n number of possibilities when it comes
to assigning image for 𝑥1 . And like that for each of the elements from the X set I have n possible
images which I can choose. And that is why the number of functions are, nothing but the number
of ways I can pick the images for each of the element from the X set that is 𝑛 ⋅ 𝑛 ⋅ … 𝑛 , 𝑚 times
which is nothing but 𝑛𝑚 .
Part b asked you to find out the number of injective functions from the X set to Y set. Well we will
be using more or less similar argument that we used for part A except that now we cannot say that
the images for every element 𝑥𝑖 is chosen independently. Because now we are counting or we are
interested in the injective functions and in injective functions for every distinct element 𝑥𝑖 from
the set X you have to assign a unique image.
355
You cannot have both 𝑥1 and 𝑥2 getting mapped to the same element in the Y set. So, that is why
when it comes to selecting image for 𝑥1 , I have n possibilities. But once I have decided the image
for the element 𝑥1 , I cannot assign that image to be a possible image for element 𝑥2 , that is why
for 𝑥2 I have n - 1 possible images and like that when once I have fixed the images for 𝑥1 , 𝑥2 and
𝑥𝑚−1.
When I am assigning the image for the mth element from the X set that image has to be different
from all the images which I have selected for the previous elements of the X sets. That means I
have only these many number of possible images, namely n - m - 1 possible images to assign for
the element 𝑥𝑚 . So, that is why the total number of injective functions will be now 𝑛 ⋅ (𝑛 − 1) ⋅
(𝑛 − 2) ⋅ … (𝑛 − 𝑚 − 1).
Part c asks, you to find out the number of bijective functions from X to Y. So, the first thing to
observe here is that for a bijection from X to Y we need |𝑋| = |𝑌|. It is very easy to verify that if
their cardinalities are different, then we cannot have a one to one and onto mapping from the X set
to the Y set. Now if the cardinality of the X and the Y set are same.
That means I am talking about the case where m =n then any bijection from the X set to Y set can
be considered as a permutation of the elements 𝑥1 to 𝑥𝑛 . Because I can imagine that I have n
356
number of elements here and I have also n number of elements here and each 𝑥𝑖 has to be assigned
a unique image. So, that can be interpreted if 𝑥1 is assigned as the image 𝑦𝑖 as per your bijection,
then I can imagine that 𝑥1 is getting shifted to the ith position, that way I can think of bijection
between the X set to the Y set.
Even though 𝑥1 → 𝑦1, I can interpret in my mind 𝑦𝑖 to be same as 𝑥𝑖 and as a result if I do this
logical mapping, I can interpret a bijection from the X set to the Y set as nothing but a permutation
of 𝑥1 to 𝑥𝑛 . And how many permutations I can have for n elements, for us I can have 𝑛! number
of permutations. So, that will be the number of bijective functions from the set X to the set Y.
In part d of question 8 we are introducing a function, the S function this function is also called as
Stirling function of type 2. And this is a very important function when it comes to combinatorics
we will encounter it later again. So, what exactly is this function this is a two input function it
takes an input r and an input s and it denotes basically the number of ways of partitioning an r
element set into s non-empty disjoint subsets. Of course, 𝑠 ≤ 𝑟.
You have a bigger set call it A, |𝐴| = 𝑟 and basically we want to find out how many ways I can
split this bigger set into a pairwise disjoint subsets, basically s number of pairwise disjoint subsets
such that their union gives back you the original set. So, there might be several ways of dividing
357
this bigger set A into s number of pairwise non-empty disjoint subsets. The number of divisions is
nothing but the value of the stirling function of type 2.
Now using this stirling function we have to count the number of surjective functions possible from
the set X to the set Y. So, since we are interested to find out the number of surjective function,
remember in a surjective function each element from the codomain set should have at least one
pre-image. Well it can have more than one pre-image as well. So, let me define the set 𝐶𝑖 to be the
pre-image set of any element 𝑦𝑖 from the co domain set namely whichever elements could be the
possible pre-images for the element 𝑦𝑖 the collection of those pre-images I am calling it to be the
𝐶𝑖 set. Now it is easy to see that if I take any surjective function with respect to that surjective
function, if I focus on this pre-image set of element 𝑦1 , the pre-image set of the element 𝑦2 and
like that the pre-image set of the element 𝑦𝑛 . Then each of the collection of those subsets the
collection of those pre-image sets will constitute a partition of your set X namely the domain set,
why so?
So, it is easy to see first of all that the intersection of these pre-image subsets will be empty set.
You cannot have an x present in both 𝐶1 and say 𝐶𝑖 . That means you have two possible images for
the element x which is a violation of the definition of any function. And it is also easy to see that
if I take the union of these pre-image sets I will get back the domain X, Y.
Because as per the definition of a function each element x from the set X will have an image. So,
that is the way I can interpret any surjective function you give me any surjective function and if I
focus on the collection of pre-image sets of various elements from the co-domain that will
constitute a partition of the domain set. So, now what we will do is how many such partitions can
we have for the set X into n non empty disjoint subsets?
I can have 𝑆(𝑚, 𝑛) number of such partitions. So, now we are interested to find out how many
surjective functions we can have. So, what we can say is, if you want to construct a surjection you
first divide your set X into n pairwise non-empty disjoint subsets, call them as 𝐶1 to 𝐶𝑛 . How many
such partitions you can have? 𝑆(𝑚, 𝑛) number of such partitions. Now once you have divided your
358
set X into n pairwise non empty disjoint subsets, each permutation of those subsets leads to a
surjection.
So, what I am trying to say is you have divided your set X into 𝐶1 , 𝐶2 , … , 𝐶 𝑛. And now 𝐶1 could
be the pre-image set of either 𝑦1 or 𝑦2 or it could be assigned as a pre-image set of 𝑦𝑛 . Now once
we have decided that 𝐶1 is going to be the pre-image set of which element from the co domain.
We next have to assign the subset𝐶2 to be the possible pre-image set of any element from the Y
set except for the element which has been assigned to this subset 𝐶1 . And like that I can continue
for the remaining subsets in my partition. So, that is why I am saying here once you have decided
the subsets within your partition you take any permutation of that corresponds to a surjection. So,
that is why the total number of surjective function will be 𝑆(𝑚, 𝑛) ⋅ 𝑛!.
In question 9 we are continuing with the notion of our stirling numbers and you are supposed to
prove that the stirling function satisfies this recurrence condition. So, to prove this statement
consider a set X which has m + 1 number of elements and we want to divide this set X into n
pairwise non empty disjoint subsets. We want to find out how many ways we can do the division.
So, my claim is whatever way you divide this set X into n number of pairwise non empty disjoint
subsets, the division can be of one of the following two categories. Category 1 division where the
first m elements in the set X are divided into n - 1 number of pairwise non empty disjoint subsets
359
and the last element 𝑥𝑚+1 is occupying a solitary position in a single subset. So, like that you have
now total n number of subsets n - 1 number of subsets. Their union will give you 𝑥1 to 𝑥𝑚 and you
have an additional subset which has only element 𝑥𝑚+1.
That is one category of partition of the set X. How many partitions in this category we can have?
The number of partitions in this category is nothing but, 𝑆(𝑚, 𝑛 – 1) because basically the number
of ways in which you can partition the first m elements into n - 1 number of pairwise disjoint
subsets. In each such partition you just add one additional subset consisting of the solitary element
𝑥𝑚+1.
That will give you a valid partition for the bigger set X and the number of such partitions we can
have here is nothing but 𝑆(𝑚, 𝑛 − 1). That is one category of partition. The second category of
partition that we can have for the set X will be as follows, I divide the first m elements into now n
pairwise non-empty disjoints subsets. I can have 𝑆(𝑚, 𝑛 ) number of such subsets now what about
the element 𝑥𝑚+1 .
Well we can either include it in the first subset or in the second subset or in the third subset or in
the last subset and that will give you an overall valid partition for the bigger set X. And clearly the
partition in this category is disjoint from the partitions in the first category. Because in the, first
category of partition the element 𝑥𝑚+1 is present alone in a single subset. Whereas in the, second
category of partition the element 𝑥𝑚+1 is not the only element within its subset, it is present along
with some other elements as well. And you cannot have any other third category of partition for
the subset x.
You can have either partition of type 1 or partition of type 2. Now how many partitions of type 2,
I can have? I can have 𝑛 ⋅ 𝑆(𝑚, 𝑛) number of partitions. This is because the element 𝑥𝑚+1 can
occupy any of the n subsets and plus because the two categories of partitions are disjoint. And if I
sum them I will get all possible ways of partitioning the set X.
360
In question 10a, you are asked to either prove what is proof that every non-empty symmetric and
transitive relation is also reflexive. Well we can give a very simple counter example to prove that
the statement is false. Imagine you are given a relation R over this set X, the relation is clearly
symmetric. It is also transitive. If you are wondering why it is transitive you have (1, 1) and (1, 1)
present and also (1, 1) present and you have (1, 2), (2, 1) present.
So, you should have (1, 1) in the relation which is present in the relation but the relation is not
reflexive because (2, 2) not present. In part 2 you are given two functions 𝑓: 𝐴−> 𝐵 and 𝑔: 𝐵 → 𝐶
respectively. And you are also given that 𝑔𝑜𝑓 injective. Then the question is, is it necessary that f
is also injective. The statement is true and we can prove it by contradiction.
So, imagine that g o f is injective but f is not injective. Since f is not injective that means I have a
pair of distinct elements from the A set, say 𝑎1 and 𝑎2 getting mapped to the same image, say b
and say the image of b as for the g function is c. Then I get a contradiction that 𝑔𝑜𝑓(𝑎1 ) and
𝑔𝑜𝑓(𝑎2 ) are same namely c, but 𝑎1 ≠ 𝑎2 showing that 𝑔𝑜𝑓 is not injective which is a contradiction
to my premise here.
361
In question 10c you are given that 𝑔𝑜𝑓 is injective, then the question is, is it necessary that g is
also injective? Well we can give us counterexample to disprove this statement. So, take this f
function and g function clearly 𝑔𝑜𝑓(𝑎1 ) = 𝑐1 and 𝑔𝑜𝑓(𝑎2 ) = 𝑐2 . So, clearly my premise is
satisfied here 𝑔𝑜𝑓 function is injective, but what about the g function it is not injective you have
𝑔(𝑏3 ) and 𝑔(𝑏4 ) both mapping to 𝑐3 , g function is not injective.
So, this statement is not necessarily true. Part d you are given the f and g functions and your 𝑔𝑜𝑓
function is surjective none is it necessary that the function f is also surjective again this is not
necessary here is a very simple counterexample this is your f function this is your g function. Your
𝑔𝑜𝑓 is surjective because indeed there is only one element in your set C, namely 𝑐1 and the pre-
image for that 𝑐1 is 𝑎1 because you have 𝑔𝑜𝑓(𝑎1 ) = 𝑐1 .
So, the function 𝑔𝑜𝑓 is indeed surjective, but the function f is not surjective because if you take
the element 𝑏2 it has no pre-image. So, that shows this statement is not necessarily true. So, that
brings me to the end of part two of tutorial 4. Thank you!
362
Prof. Ashish Choudury
Indian Institute of Technology, Bangalore
Module No # 06
Lecture No # 27
Countable and Uncountable Sets
Hello everyone welcome to this lecture on countable and uncountable sets.

The plan for this lecture is as follows: in this lecture we will discuss about the cardinality of
finite sets. We will discuss about the cardinality of infinite sets and we will conclude with
countable and uncountable sets.
363
So let us begin with the cardinality of finite sets first. So if I ask you what is the cardinality of
this set X with consist of the elements Ram, Sham, Gita and Sita. You will say its cardinality is 4
because it has 4 elements. Another way to put it is as follows: We can say that the cardinality of
the set X is 4 because there is a bijection between the set X and the set consisting of the elements
1, 2, 3, 4. So there can be many bijections possible between the set X and the set 1, 2, 3, 4.
One of the bijection could be where the element Ram is mapped to 1, Sham is mapped to 2, Gita
is mapped to 3 and Sita is mapped to 4. You can have a bijection where Ram is mapped to 2
Sham is mapped to 1 and so on that is also a possible bijection. But since there is a bijection
between Ram, Sham, Gita and Sita and the set 1, 2, 3, 4 we can say that the cardinality of the set
X is 4.
Because I can say that whichever element from the set X is mapped to element 1 that is the first
element of set X whichever element from the set X is mapped to element 2. That is the second
element of set X and so on, that is why the cardinality of the set X is 4. Now due to the same
reason if I consider another set Y consisting of the elements Delhi, Kolkata, Mumbai and
Chennai, its cardinality is also 4.
Because it has 4 elements or another way to put it around is there is a bijection between the set Y
and the set 1, 2, 3, 4. So based on this example we can formulate the following definition we can
say that: two sets X and Y have same cardinality and for that we use this notation |X|. So
364
remember this notation denotes the cardinality of X : |X| and this notation denotes the cardinality
of Y : |Y|. So if the sets X and sets Y have the same cardinality we use this notation that their
cardinalities are equal. And when can we say that their cardinality are the same we can say that
provided there is a bijection or 1 to 1 correspondence between the set X and set Y.
So if I take this example here set X and set Y I have the same cardinality because they have the
same number of elements namely 4. But if I see it in terms of a function then I can say that the
set X and set Y have the same cardinality. Because there is a bijection between the set X and the
set Y.
Now how do we compare the cardinality of finite sets; say if I am given the set X and the set Y.
It is easy to see that the set X its cardinality is less than cardinality of set Y. Namely the number
of elements in the set X is less than the number of elements in the set Y. That is why the
cardinality of X is less than the cardinality of Y. But if I view the same thing in terms of a
function I can say that the set X is smaller than the set Y in terms of size because there is an
injective function from the set X to the set Y.
So there can be many injective functions one possible objective mapping I can define as Ram
getting mapped to Delhi and Shyam getting mapped to Kolkata right. That means you take any
element from the set X it will have an image a distinct image from the set Y. And distinct
365
elements of the set X will have distinct images so that automatically ensures that the number of
elements in the set Y as to be at least as large as the number of elements in the set X.
So based on this example I can give the following definition I can say that the cardinality of the
set A is less than or the same as the cardinality of set B. And we use this notation the cardinality
of A less than equal to cardinality of B provided there is an injective mapping from the set A to
B. You might be wondering why I am using less than equal to notation here because there might
be more elements in the set B compared to the set A.
That is why the less than equal to notation. Now if the set A cardinality is less than equal to
cardinality of set B and the cardinality of the set A and the cardinality of set B are different, then
clearly it implies that the cardinality of set A is strictly less than the cardinality of the set B right.
So now what are the countable sets so before going into the definition of countable sets let us see
some motivation. Why we want to study countable set: the whole motivation behind countable
sets is that we want to split the study of infinite sets into 2 categories. What are infinite sets: on a
very high level they are sets which have infinite number of elements. So what we want to
basically do is there might be several sets possible which are infinite, set of integers, sets of real
numbers, set of irrational numbers and so on.
366
So what we want to do is we want to categorize these infinite sets into 2 categories. Sets which
have the same cardinality as the set of the positive integers. And a set of positive integers are
denoted by this notation ℤ+. And we want to categorize the infinite sets into other category which
have different cardinality than the cardinality of the set of positive integers. That is the whole
motivation of defining this notion of countable sets.
So when exactly we say a set A is countable? We say a set A is countable if it satisfies one of the
following two conditions either it has to be finite namely it has finite number of elements or it
has the same cardinality as the set of non-negative integers, namely the set of positive integers to
be more precise. It has to be the same cardinality as the set of positive integers. If one of these 2
conditions are satisfied then we say that the set A is countable.
Whereas if a set A satisfies neither of the condition then we will say that the set is not countable.
So if you are given an infinite set say S which is countable so since the set is infinite that means
definitely we cannot say how many elements the set S has. But if its cardinality is same as these
set of positive integers then we will call the set S to be countably infinite.
So we have countably finite sets and we have countably infinite sets so countable sets can be
categorized into 2 categories either they can be countably finite that means its cardinality is
finite. Or there are infinite number of elements in the set but they are countable in the sense that
367
their cardinality is same as the cardinality of set of positive integers. Whereas if the set is not
countable then we will call it uncountable.
So if we are considering the second category of countable sets namely infinite sets whose
cardinality is same as the set of positive integers then we use this notation aleph null (‫א‬0) to
denote the cardinality of such sets. So this is a Hebrew character the aleph character and 0
denotes the null feature here. So the cardinality or the size of the set of the positive integers is
called as aleph null and if your set if countably infinite then its cardinality will be ‫א‬0.
So now we can quickly prove this theorem which will be very useful later on. If you are given a
set S which is countable that means your set S is countably infinite. Then it is countable if and
only; if it is possible to list the elements of the set S in the sequence indexed by positive integers.
So what it basically says is that even if you have infinite number of elements in the set S it is
countable in the sense I can give you the sequence, I can give you a method a well-defined
method according to which you can list down the elements of the set S.
And the method is well-defined in the sense that no elements of the set S will be missing as per
the definition of that sequencing. That means the way I will give the definition of the sequencing
or the listing of the element; each element of the set S will appear somewhere in that sequence.
In that way it is a well-defined sequence and no element will be repeated in that sequence. So in
that sense only my set is countable even though it has an infinite number of elements.
368
So the proof of the theorem is very simple since your set S is countably infinite that means its
cardinality is same as set of positive integers and the set of positive integers are 1, 2 upto infinity
and there has to be a bijection; call the bijection as f between the set of positive integers and
your given set S. Now if in that bijection the element a1 is mapped to integer 1 and element a2 is
mapped to integer 2 and so on. Then what I can say is that a1, a2, ai, an is a valid sequencing or
valid listing of the elements of the set S.
Why it is valid because first of all no element in this sequence is repeated and if you take any
element in the set S it will appear somewhere in this sequence; it is not the case that no element
will be missed in this sequence. Even though there are in finite number of elements. So what this
theorem basically says is this: That if you want to show a given set to be countable and there are
two ways to do that if it is finite just find out the cardinality of that set.
But if it is infinite then the only way you can show it is countable is either show a bijection or
you give me a well-defined sequence or well-defined way of sequencing in the elements of that.
That is equivalent to saying that you are giving me a bijection between the set of positive
integers and the given set.
So now based on these definitions let us see some examples of countably finite sets. So I start
with the set of odd positive integers and let me denote the set of odd positive integers by this
369
notation ℤ+
𝑜𝑑𝑑 . And my claim is that the set of what positive integers is the countable set. So the
statement might look very non-intuitive because definitely you have more integers than the set of
odd positive integers right.
The set of integers; if I consider the set of positive integers to be more precise then it has both
odd integers as well as even integers. And both these sets; the set of odd positive integers and the
set of even positive integers each of them is an infinite set. What I am going to show here is that
cardinality wise the set of positive integers and the set of odd positive integers - their
cardinalities are same by showing a bijection.
So the bijection is as follows. So consider the sequence 1, 3, 5, 7 like that where the nth term is
the sequence is 2n – 1 and so on. So clearly this sequence is sequence of infinite positive integers
and each number in this sequence is odd. My claim is that this is a valid sequence for the set of
odd positive integers it is valid in the sense no element in this sequence is repeated. And you take
any integer from the set of positive integers it will appear somewhere in this list.
It will not be the case that you keep on traversing the sequence infinitely but still you never come
to that element which you are considering. It give me any element X belonging to the set of all
positive integer it will appear somewhere in this sequence. To be more precise if you want to see
the exact bijection between the set of off positive integers and the set of positive integers. You
consider the mapping f(n) where f(n) is 2n-1.
370
And it is very easy to see that the function is injective because if you take the integers 2n-1 and
2m-1 and if they are same then that is possible only if your n is equal to m and clearly the
function is surjective as well. How? you take any element y belonging to the set of odd positive
integers corresponding to that element y, since that element y is an odd positive integer it can be
written in the form of some 2k- 1 for some k in the set of positive integers.
So the pre-image of y will be the element k so that is a bijection and that shows that cardinality
wise the set of odd positive integers is same as the set of positive integers. Even though,
intuitively you have more elements in the set of positive integers than the set of odd positive
integers.
Now what we are going to show next is a very interesting fact we are going to show here that the
set of integers is a countable set. Mind it the set of integers has both positive integers as well as
negative integers. But what we are now going to show is that cardinality wise the set of integers
is same as the set of odd positive integer, they have the same number of elements. So again I will
do it by giving you a valid sequence and also by giving you an explicit bijection.
So let me first how you a sequence. My sequence here is the following I start with 0 and then I
alternatively list down positive numbers, negative numbers. And then I gradually go along the
number line both in the positive direction as well as in the negative direction. My claim is that
371
this is a valid sequence in the sense you take any integer either it will be positive or it will be
negative and it will appear somewhere in the way I am listing down the elements here.
Let me ask you a question here, you can think in your mind. If I consider a sequence where I first
try to write down all the positive integers. And then followed by the negative integers can I
consider this to be a valid sequence for listing down the set of integers. The answer is no this is
not a valid sequence why this is not a valid sequence? Because if you take a negative number say
-1 you do not know where exactly -1 is going to appear in this list.
Because you are starting with the set of positive integers and when you are actually traversing
the set of positive integers you are going to an infinite limit. There is no guarantee that you will
come back and then start enumerating your negative integers. In that sense this enumeration is an
invalid from the enumeration or invalid listing. Whereas the way I have listed down the elements
where I am alternately listing down the positive negative integers.
That means along the number line I started with a 0 and then in each step I am listing down the
next positive number and then the next negative number. Then the next positive number then the
next negative number then the next positive number then the next negative number. Now if I
consider this sequencing, even though your given integer which you want to find out where
exactly it is appearing in the sequence is very large.
It might be appearing somewhere; you do now know where exactly it is appearing. It will
eventually be enumerated in the sequence and you will never miss it. That is why it is a valid
sequencing, it is a valid listing. Whereas the listing that I have considered here where I am first
trying to enumerate all the positive integers followed by all the negative integers it is not a valid
sequence.
372
Now if you want to see an explicit bijection between the set of integers and the set of positive
integers consider the following mapping. My f(n) will be n / 2 provided n is even whereas if n is
odd that means (n – 1) is divisible by 2 then my mapping will be –(n – 1) / 2. And it is the
straight forward exercise for you to verify that indeed is mapping f is a bijection which shows
that the cardinality of the set of positive integers and the cardinality of integers are the same.
Next let us consider the set of prime numbers and for those who do not know what is a prime
number? A number p is prime provided it is divisible by only 1 and p; that means there are no
other divisors for the number p apart from the number 1 and the number p. Of course 1 divides
373
any number. And the number divides itself by default these are the 2 valid divisors of any
number.
If your number p is such that you do not have any other divisors other than the number itself and
1 then the number p will be called as the prime number. So if you take say 2, 3, 5, 7 they are all
prime numbers. Whereas if you take the numbers like 4, 9, 15 they are not prime numbers
because divisors of 4 are 2, divisors of 9 are 3, divisors of 15 are 3 and 5 and so on. So my claim
is that a set of prime numbers is a countable set.
So again those who are not familiar with number theory they might be knowing that there are
infinite numbers of prime numbers. But what we are starting in this theorem is that cardinality
wise the number of prime numbers, the cardinality of the set of prime numbers is same as the
cardinality of the set of positive integers. So this I can either prove by giving you a valid
sequence and the sequence is very straight forward.
You just enumerate the prime numbers in increasing order that is all and it is a valid sequence
because this is an infinite sequence; that is fine. But it is valid in the sense that every number in
this sequence is a prime number, no number is going to be repeated and you take any element y
belonging to the set of prime numbers it will eventually appear in this sequence. It will not be the
case that you do not know where exactly; it is not the case that this number y will never appear in
this sequence.
374
Even if you are traversing towards infinity, if you are interested in the explicit bijection between
the set of positive integers and the set of prime numbers, the bijection is very straight forward.
You just output the nth prime number as the value of your function f on input n. And the function
is well defined and clearly it is bijection. So now what we have proved till now? We have proved
will now that the set of prime number, it has the same number of elements as the set of odd
positive integers which has the same number of elements as the set of positive integers. That
means even though element wise they are different but cardinality wise they are same.
So that brings me to the end of this lecture. These are the references for today’s lecture. We are
of course following the relevant chapters from Rosan book. But this is another interesting
document which you can refer where you have a very nice explanation regarding the whole
cardinality theory, the theory of countable sets and just theory of uncountable set. So that brings
me to the end of this lecture, thank you.
375
Prof. Ashish Choudhry
IIIT, Bangalore
Module No # 05
Lecture No # 28
Examples of Countably Infinite Sets
Hello everyone welcome to this lecture on examples of countably infinite sets.
(Refer Slide Time 00:27)
So just to recap in the last lecture we introduced the notion of countable and uncountable sets.
Countable sets are those sets whose cardinality is either finite or whose cardinality is same as the
set of positive integers. So the plan for this lecture is as follows. We will see several examples of
countably infinite sets and we will also discuss some properties of countable sets specifically in
the context of infinite sets.
376
So we first prove that the Cartesian product of the set of integers is a countable set. So again this
might look non-intuitive, you have many elements in the Cartesian product of the set of integers
compared to the set of integers itself because when I say that Cartesian product it is going to
consist of all ordered pairs of the form (i, j) where i can be any integer, j can be any integer. But
what this theorem says is that the number of elements in the set ℤ x ℤ is same as the number of
elements in the set of positive integers.
So we are going to prove that. So remember in the last lecture we proved that whenever you
want to prove that an infinite set is countable either you gave an explicit bijection between that
set and the set of positive integers. Or you give a well-defined sequence or a rule according to
which you specify or list down the elements of the given set which you want to prove to be
countably infinite. And argue that every element in that set will appear in the sequence that you
are specifying.
So what we will do is to prove this theorem we are going to show a sequence or a way to
enumerate all the elements of the set ℤx ℤ. But the question is how exactly we find out one such
sequence? So that we do not miss any element of set ℤ x ℤ. So the idea is very clever here what
we do is, So since we are considering the Set ℤ x ℤ it is nothing but the collection of all points in
your 2 dimensional plane.
377
So imagine that you have that infinite 2 dimensional plane where you have all the points
belonging to the ℤ x ℤ. And our goal is basically to give an enumeration of all the points in that
infinite plane such that the enumeration should be well defined and we do not miss any point in
the enumeration process. So here is the enumeration process I start with the center namely
coordinate (0, 0) which is the element of ℤ x ℤ. So imagine this is your (0 ,0) this point.
Then my next point is which I am going to enumerate in my sequence; which I am going to list
down in my enumeration is the point (1, 0). That means I move from my current point 1 unit to
the right hand side then from my current point I move 1 unit in the positive direction and get the
point (1, 1) and list it in down. And then I traverse or go 1 unit to the left hand side from the
current point so I will get the point (0, 1).
And now I cannot come down because if I come down then I will be coming to the element (0, 0)
which I have already listed down which I do not want to do. So what I am going to do is instead
of going down from (0, 1) I will continue left further 1 unit. And due to that I will get the point
(-1, 1) and I will list it down. And now I will come down because if I come down from my
current point the point which I am going to get I have not enumerated it already.
So I will get a new point will be (-1, 0) and then I continue this process I go down further 1 unit
and obtain the point (-1, -1). And then I will make this whole trip again. So what was the trip? I
started with (0, 0) go right go up go left left down down and then I will again make this circular
rotation. So what I will do is from my current point I will go right 1 unit again right 1 unit again
right 1 unit. And then go up up. and then continue this process.
So I will be next enumerating this… and the next point and then I will go up….. and then
continue left. So this is the process which I will follow and the idea here is that if I enumerate the
various points in this infinite 2 dimensional plane according to the procedure that I have
demonstrated here, any point in this infinite 2 dimensional plane will eventually appear along the
spiral. That’s the idea here, you will not miss any point in the infinite 2 dimensional plane.
So that is why this is a valid enumeration of all the elements in the Cartesian product of the set of
integers, which shows now; that the set of the Cartesian product of the integers of all points in
infinite 2 dimensional plane is a countable set.
378
Now, we will see next whether the set of rational numbers which I denote by this ℚ notation is
countable or not. Now intuitively it might look the answer is no because definitely rational
numbers is a super set of the set of the integers. And looks like there is no way of sequencing
because the fundamental fact about rational numbers is that you take any 2 rational numbers
there are infinitely many more rational numbers between the same 2 rational numbers.
That means if I consider 2 rational number x and y between x and y there are infinitely many
rational numbers. So how exactly we are going to list down or sequence all possible rational
numbers. So looks like that is not possible. But what we can do is we can show a very clever
enumeration of the set of rational numbers which will prove that the set of rational numbers is a
countable set.
And the sequencing that we are going to see here will be based on the sequencing of the elements
of the point in the 2 dimensional integer plane based on enumerating all the points along the
spiral that we had been seen in the last slide. So just to recall, this was the enumeration of the set
of all elements or points in the set ℤ x ℤ. And based on this enumeration we will get an
enumeration of the set of all rational numbers.
So the idea is if we consider any rational number and if it is a rational number it will be of the
form p / q, where p is some integer and q is some integer and q will not be 0. So the idea is you
379
traverse or you follow the enumeration of all the elements in the set ℤ x ℤ namely this
enumeration here. And based on this enumeration you come up with an enumeration of the
elements in the set of rational numbers as follows.
If you are at a point (p, q), then you list down the rational number (p / q) in your enumeration
provided q is not 0 because if q is 0 definitely that is not a rational number. And the rational
number (p / q) is not listed earlier as per your enumeration. Else you go to the next element (p, q)
in the listing of ℤ2 that is the idea. So what I am saying is demonstrated as follow. So if I apply
the rule on (0, 0) so if I start with (0, 0) so my p is 0 and q is 0.
So my rule says that if q is 0 do not do anything go to the next element. And my next (p, q) is (1,
0) and again q is 0. So my rule says do not do anything. Then I go to my (p, q) which is (1, 1)
and I will be applying the first rule here because here q is not 0 namely q is 1 and my (p / q) is (1
/ 1) which is the rational number 1 and which is not yet listed. So that is why I will list down the
element 1 then I will go to the next (p, q). q is not 0.
So again will be applying the first rule and (p / q) will be 0 in this case. Then my (p / q) will be (-
1 / 1) which has not been listed earlier. So I will list it down. Then my next (p / q) is not defined
because q is 0 so ignore this. Then if I go next my (p / q) is (-1 / -1) which is nothing but 1 and
which has been already listed. So that is why I will apply the rule in the else part. So that is why I
will miss this element as well and if I continue then when I go to the element (2, -1) it will be (2 /
-1) which is the rational number -2 which has been not listed earlier.
So now you can see even though there are infinitely many rational numbers if I follow these 2
rules of enumerating the rational numbers I will not be missing any rational number because you
take any rational number it will be of the form (p / q). And if will be eventually listed down in
the sequencing that I have specified here. So that means we now have a method of listing down
all the elements of the set ℚ in a well-defined fashion and that is why this set of rational numbers
will be a countable set.
It has infinitely many element but we can count it in the sense we can sequence down we can
write down all the elements in that set. So this will be the sequencing of the elements in the set of
rational numbers.
380
Now let us consider the set of binary strings of finite length. What exactly that means? So,
imagine a set Π consisting of 2 elements namely the element 0 and 1 and why 0 and 1? Because I
am considering binary strings so, binary strings will be just a string of 0’s and 1’s. And I used
this notation Π* to denote the set of all binary strings of finite length. What do I mean by that?
So more formally Π* is defined to be the union of the sets Π(i) where i is within parenthesis.
And i belongs to the set of natural numbers namely i ranges from 0 to infinity. And what is this
set Π(i) within parenthesis it is set of all possible binary strings. So I should specify here it is the
set of all possible strings of length exactly i over the alphabet Π. And since Π consists of only
symbol 0 and 1 what does Π(i) denote? It denotes the set of all possible binary strings of length
exactly i.
So if I consider the set Π(1) it will have only the binary strings of length 1. So it will have only 2
elements. If I consider the set Π(2) it will have all binary strings of length 2 and so on. So what is
this set Π*? It is the set which is obtained by taking the union of Π(1) Π(2) and so on including
Π(0) and where Π(0) denotes all possible binary strings of length 0.
So we use this special notation ε to denote the set of to denote an empty binary string. So based
on this fact it should be now clear that each subset Π(i) is finite. Why it is finite? Because it has
381
exactly 2i elements because Π(i) denote a set of all possible binary strings of length exactly i and I
can have 2i such binary strings.
And if I take the union of all such sets I get the set Π*. So it is easy to see that the set Π* is an
infinite set because the number of element is infinite. But it is the union of several subsets where
each subset is finite in the sense it has finite number of elements. So now the question is, is this
the Π* countable even though it has infinitely many elements it has infinite number of binary
strings can we numerate down all such strings in a well defined fashion.
So the answer is yes we can prove that the set Π* is indeed countably infinite. And what we will
do is to prove this theorem we will show a possible valid listing of the elements of Π*. And the
idea is to arrange or list down all the elements of Π* according to their length. So we start with
the length 0 strings and length 0 string will be the empty string denoted by the special notation ε.
Then we will go and enumerate or list down all valid string, binary strings of length 1. And there
are multiple strings of a particular length. We arrange them according to the binary order. So for
example here we have 2 possible binary string of length 1 : 0 and 1. But since numerical is 0 is
less than 1 we will list down 0 followed by 1. Then so basically what I am saying here is that you
go to the set Π(1) and list down the elements of the set Π(1) in binary order.
Next go to the set Π(2) and it will have 4 elements, list down those elements in binary order. So
we have 0 listed first followed by 1, followed by 2, followed by 3 and continue this process.
Next go the set Π(3) which will have 8 strings list down those strings in binary order and so on.
So, why this is a valid listing? The idea is you take any binary string x belonging to Π*.
It will have finite length because as per the definition of Π*, x will be belonging to some set Π(i).
We do not know what exactly is that index i it depends upon the number of bits or number of
characters in your string x but it is a well defined value that means x belongs to some Π(i). And
eventually after listing down all the elements in the set Π(0) to Π(i-1) when we will be listing down
the elements of the set Π(i) x will appear somewhere in our listing.
So we will not be missing the element x. And we know that after some step eventually the
chance for x will come as per this listing to be listed down in our enumeration. So that is why
382
this is the valid enumeration it shows that the set Π* even though it has infinite number of
elements it is possible to list down those elements in a well-defined way and hence proving that
the set Π* is countable.
So till now we had been seen several infinite sets and magically we have proved that they have
same cardinality. Now we will prove some general results about the cardinality of sets both with
respect to finite sets and infinite sets. So the first theorem is that if you have 2 sets A and B and
if they are countable then their union is also countable. So I am not saying anything about the
number of elements in the union A and B.
Of course, what I am saying is that it is always possible to list down the elements of A U B. So
how we are going to prove it? First of all there might be a possibility that A and B are not
disjoint but to keep our proof simple without loss of generality we assume that A and B are
disjoint. The proof can be simply adapted for the case when A and B are not disjoint. Now we
can have various cases depending upon whether A, and B are countably finite or countably
infinite.
So the theorem statement was that A and B are countable and the definition of countable set is
that either its cardinality is finite or its cardinality is same as ‫א‬0. So we can have 3 possible cases
here. Case 1 when both A and B are finite that means say if the cardinality of A is m and the
383
cardinality of B is n then it is easy to see that the cardinality of union of A and B will be m + n
which is a finite number and hence A U B is also countable.
Case 2 is when exactly 1 of the set A and B is finite whereas the other set is countably infinite.
Now again we can have 2 possible cases depending upon which of the 2 sets is countably
infinite. So what we can do is we assume without loss of generality that it is A set which is
countably infinite that means the cardinality of A is ‫א‬0. And the set B is finite that means it has
exactly m number of elements where m is some natural number.
So what we are now going to show is that even in this case the union of A and B is countable. Of
course the union of A and B will have infinite number of elements because A is infinite here. But
what we are going to do is we are going to show here a valid sequencing for the elements in the,
set A U B. So the idea here is that since A is countably infinite, it will have some valid
sequencing of its elements. So let that valid sequencing be a1, a2, an and so on. And of course we
know that set B has m number of elements.
So let the elements be, b1 to bm. So what we do is we list down the elements of; we can say that
we can list down the elements of A U B as follows. First list down the elements of B set which
are finite in number, m in number followed by the elements of the set A. Now you might be
wondering why we cannot do the following. Why we cannot we enumerate the elements of the
set A first and then followed by the elements of the set B. My claim is that this is not a valid
sequencing of the elements in the set A U B.
Why it is not valid is because since you are first listing down the elements of the set A you do
not know when you are going to return and come back and list down the elements of the set B
because that sequencing of the elements of the set A is an infinite process and you can get stuck
there forever. So you do not know when exactly you will finish the process and will come and
start listing down the elements of the set B.
So now what I mean here is that if I ask you, can you tell me where exactly b1 is going to appear
in this sequence? You cannot tell me because we do not know when exactly we will finish listing
down the elements of the set A and then we will come to and list down element B. But if I
384
consider this sequencing which I have listed here I know where exactly when exactly the element
will appear irrespective of whether it belongs to the set A or the set B in the sequencing.
If you are asking me to specify where exactly an element from the set B belongs to I can give
you that position. Whereas if you ask me where exactly is the position of an element from the A
set in this sequencing again I can tell you that it will appear somewhere because as per my
assumption that element has some position in the sequencing of the elements of the setting. So
that is why it is this sequencing which is valid and not this sequence.
The third case is when both A and B set are infinite and countable, because I am assuming my A
and B sets are countable and if A; and B are both infinite that means both the cardinality of A as
well as the cardinality of B is ‫א‬0. And I want to show that A U B is also countable by giving you
a valid sequencing for the elements in the union of A and B. So since A and B are countably
infinite they will have individual valid sequencing of the elements of the respective sets.
So, image that this is the sequencing of the elements of the set A and this is the sequencing of the
elements in the set B. We want to find out a valid sequencing of the elements in A U B so that
we do not miss any element in the union of A and B. And we know when exactly an element in
the union of A and B is going to appear in the sequence. So a valid sequencing of the elements in
union of A and B is as follows.
First list down first element in the A sequence followed by the first element in the B sequence.
Then go and list down the second elements of A sequence and B sequence and like that continue
and write down or list down the nth element in the A sequence and B sequence and so on. So
now you can see that you ask me any element belonging to the union of A and B it will
eventually appear in this sequence. It would not be the case that we get stuck infinitely for listing
down the element.
Whereas if I would have listed down first elements of the A set and then list down the element of
B set then this is not a valid sequencing for the elements of the A U B why? Because if you now
ask me, when exactly I am go to list or when exactly I am going to see b1 and the sequencing. I
do not know because the process of listing down all the elements of A set is a never ending
385
process. So we do not know when exactly we will finish that process and come and write down
or list or find the element b1.
So that is why this is not valid sequencing but the same problem would not happen with the
sequencing that I have specified here namely listing down the elements of A and B sets
alternatively because it does not matter what is the element in the A U B that will appear
somewhere in the a sequence or in the b sequence depending upon whether it belongs to the, A
set or the B set. Accordingly since I am listing down the elements of A set and B set alternately it
will appear somewhere in this sequence.
Now second interesting result about the cardinality theory is what we called as the Schroder
Bernstein theorem which says the following. If you have 2 sets such that the cardinality of A is
less than equal to cardinality of B and simultaneously the cardinality of B is less than equal to the
cardinality of A. Then we can conclude that both set A and B have the same cardinality. In terms
of function what we are saying here is that if |A| is less than equal to |B| then as per the definition
we have an injective mapping say the mapping f from the set A to B.
And since the cardinality of B set is less than equal to the cardinality of A set we also have a
injective function say g from the set B to set A. Now if we have these 2 individual injective
mappings, what this theorem basically tells you is that, using the injective mappings f and g you
386
can come up with the bijective mapping between the set A and B. That is the idea behind the
proof of this theorem.
However the proof is slightly involved and due to the interest of time I will not be going through
the proof of this theorem. But this is a very important theorem which we should keep in our
mind. What this theorem basically says is, if you want to show that the cardinalities of 2 sets are
same then one way of doing that is you show one injective mapping from the first set to the
second set and another injective mapping from the second set to the first set.
That automatically will conclude that you have; you can have a bijection also between the 2 sets.
And if you have a bijection between the 2 sets then as per the definition of (cardinality) equality
of 2 sets they have the same cardinality not equal not sorry equality of the cardinality of the 2
sets, they have the same cardinality. The sets A and B might different. They may have different
elements. But cardinality wise they will be the same.
Now the third result about the cardinality is the following. If I take any subset of a countable set
then it should be also countable. So, there are 2 cases the above statement is obviously true if the
set A is a countably finite set. That means if the set A has say n number of elements and if I take
subset B of the set A of course the cardinality of B will be upper bounded by n. So this statement
is obviously true statement is obvious also true even if the set A is infinite but countable.
387
So I can prove that even if the set A is infinite but countable that means its cardinality is ‫א‬0. Then
the cardinality of any subset B of that set A is also ‫א‬0, we can prove that. The idea behind the
proof is as follows. We can prove the theorem by contrapositive and the simple way to
understand the proof is that if the set B is not countable. That means if it is not countable that
means it is not possible at all to list down the elements of the set B.
So if you do not know any method of listing down the elements of the subset B how come it is
possible to list down the elements of the superset A. And that goes again the assumption that my
set A is countably infinite. If I assume that my set A is countably infinite that means I know how
to list down the elements of set A in a well defined fashion. So that is the proof for this fact. So
as a consequence of this statement I can also state that if you have any set which has an
uncountable subset, then the set is also uncountable.
So what I am saying is that if you have a scenario where B is the subset of A and you do not
know how to list down the elements of the set B that means the cardinality of B is not ‫א‬0. Then I
can conclude that the cardinality of A is also not ‫א‬0. This is because if I do not know how to list
down the elements of set B I do not know how to list out the elements of the set superset A as
well.
Because while listing down the elements of the superset A I need to list down the elements of the
subset B as well. But I do not know how to list down the elements of the subset.
388
Now what we now going to prove is that the set of all strings over a finite alphabet is also
countable. So what do I mean by that is just few slides back I took a binary alphabet which has
only 2 symbols 0 and 1. And I proved that the set Π* which is the set of all possible strings of
finite length which are binary is countable. Now I am generalizing this result to a bigger alphabet
which may have more than 2 symbols or 2 characters.
So I am assuming that I have an alphabet Π consisting of m number of characters s1 to sm or m

number of symbols. And Π* denote the set of all possible strings finite length strings over this
alphabet. So my claim is that is that Π* is countable. So again what is Π*, the way we have
defined Π* for the case of the binary alphabet we are going to follow the definition: Π* will be
the union of the various subsets Π(i).
Where Π(i) denote the subset of all strings of length exactly i over the alphabet Π. So, for
instance if my Π is consisting of alphabets a, b and c, 3 characters. Then Π(0) of course will be
the empty string, Π(1) will have all the strings of length 1. So I will have 3 strings. Π(2) will have
all possible strings of length 2. So I can have strings like this and so on. So it is easy to see that
each subset Π(i) is finite because each subset will have mi number strings.
And the set Π* is the union of all such subsets. So it will have infinite number of elements. But
now we want to show a valid sequencing of the elements in the set Π* . So here is how we can
list down all the elements of the set Π* without missing any of them. So since the set Π(1) is
389
finite it will have an enumeration of the elements of its set. So let that enumeration be this. So
the first string in Π(1) is denoted as str11, the second string is denoted by str12 and so on.
So in the subscript I have 2 variables. The first index here denotes the subset in which the string
belong. And the second subscript denotes the ordering of that element within that subset. In the
same way I will have a sequencing for the elements in the subset Π(2). So you can see here each
string the first index is 2 2 2 denoting that each such thing belonging to the second subset and
then we have the second level of indexing.
And the second level of indexing is from 1 to m2 because this because the subset Π(2) will have
m2 number of elements. And in the same way if I consider the subset Π(n) it will have mn number
of strings and like that. So now what we have to do is we have to come up with a valid
mechanism or valid sequencing for listing down the elements of set Π*. And that I can do by
following the sequencing by following this ordering what exactly is this ordering.
The idea is that you first list down all strings of the form strij where the sum of the indices i and j
is 2. Why we are starting with the summation of indices being 2, because you can see that my
first string here the least indexing I can have here is str11 and the summation of the indices will
be 1 + 1 namely 2. So I will start with str11. Then I will list out all the elements; all the strings
where the summation of the indices will be 3.
So that is why str12 and str21 because the summation of these 2 indices will be 3 and the
summation of these 2 indices also will be 3. Now if you have many strings where the summation
of their indices are the same value then you will follow the ordering among the subsets itself. So
since str12 appears in the subset Π(1) and the str21 follows comes in as the subset Π(2) and Π(1) is
appearing before Π(2) that is why I have listed down str12.
And then I have listed on str21. Then I will list down all strings such that the summation of the
indices is 4. And again you can see here there are 3 strings. So what I have done is I have first
taken the string from the set Π(1) and then I have taken the string from the set Π(2) and then I have
taken the string from the set Π(3) and so on. So you can see here if I follow this ordering this is a
well-defined ordering.
390
Why it is well defined ordering? Because you take any string x belonging to Π* it will belong to
some Π(i). That means it will be appearing somewhere in the listing of the elements of Π(i) and it
will have a form strα,β. So x will be of the form say strα,β. And α+β will be some integer. So say α
+ β is say γ. So once I have listed down all the strings where the summation of its indices is γ-1.
Next I will be listing down all the elements all the strings with such that the summation of the
indices is γ and during that process x will be appearing in my sequence. So I will not be missing
x and I know definitely we will not be waiting infinitely for listing down the element x. That
means we will never get stuck in this process of listing down or enumerating down the elements
of the set Π* and that is why this is the valid sequence.
So now based on the previous theory what we can prove here is that the set of programs or set of
valid programs in any programming language is also countable. So what do I mean by that, you
take Π to be the set of all keyboard characters. It is a finite alphabet because you have only finite
number of keyboard characters; even if you take various combination of keyboard characters that
will give you a new character.
But even if you take all such combinations the set of all the characters which you can type using
the keyboard in a finite alphabet, I am calling it Π. We already proved that Π* is countable if Π
is a finite alphabet. We just proved that because Π(i) will be the set of all possible strings of
length exactly i and we know how to enumerate out all the elements, all the strings of the set Π*.
391
Now imagine you have a programming language L, it can be C C+ +, java, python any
programming language. And let this calligraphic P denote the set of all valid programs in your
programming language. What do I mean by a valid program? I mean to say it has a start
instruction or a begin instruction and it has an end instruction. And in between the begin and the
end instruction or the start end instruction and you have arbitrary number of syntactically correct
instructions in that programming language.
Valid instructions in the sense when you compiled the program you do not get any error you get
some output. How many instructions you can have between the begin and end well that can be
arbitrary large but it will be finite. It would not be the case that you have infinite number of steps
between the begin and the end instruction. Why that is the case because if you have infinite
number of instruction between the begin and end instruction how can your program be valid.
How can your program will give you some output because to get the output from your program
you need to reach that end instruction you compiler need to reach the end instruction. That
means after parsing all the steps between your begin and end instruction the program has
compiled and given you an output. And that is possible only if your number of instructions
between the begin and end instruction is a finite quantity.
That means the number of steps is some natural number positive number. So this is my set P you
can imagine it as many programs but the claim is that set P is countable even though the number
of programs is infinite. Because you can keep on inserting, you can keep on taking existing
programs and keep on increasing the size of the program by inserting a new valid instructions in
the existing valid programs. That way you can keep on creating new programs, this process will
never stop.
You cannot say that after this program I cannot find a new program or new valid program. There
is no end point here you can always keep on coming up with new programs based on existing
program. The simple thing will be just take any existing valid program and just before the end
instruction insert a new valid instruction, that will give you a new program which is different
from the previous program.
392
And that is why this set P which is the set of all valid programs in your programming language is
an infinite set it is not a collection of finite number of programs. But the claim is that even
though if you have infinite number of programs in your programing language that set is
countable. We can list down or we can come up with an enumeration of all valid programs in
your programming language. And why that is the case because we just proved that any subset of
a countable set is countable.
And what exactly is the set of all valid programs in your programming language well? that is a
strict subset or a proper subset of the set Π*. Why? Because I am just considering only valid
programs I am not considering invalid program. My set P has only those programs which will
compile and will give me some output. I am not considering programs of the form which has
only a begin instruction that is all.
That is also string over the set Π* the string belongs to Π*. But this is not a valid program
because it has no end instruction. In the same way the set in the string end also belongs to Π* but
it cannot be considered as a valid program. But if you consider the string begin followed by end
then that is also string belonging to Π*. But that is a valid program because you have a begin
instruction and the instruction and in between you do not have anything but that is fine, this is a
valid program.
So that is why the set P will have only a subset of strings from the Π* because Π* will have all
the things that you have in the set P plus invalid programs as well because Π* just talks about
strings over the set Π whether the string is a valid program in your programming language or not
that is not necessary here. That is why the set P is the strict subset of Π*. And since we know
that Π* is countable that means we know how to list down the elements of the set Π*.
Using that process we can also come up with the process of listing down all the valid programs
in your programming language as well. So that proves a very interesting result. What we have
proved is that even though the number of programs the number of valid programs in any
programming language is infinite, we can always list down those valid programs so that we are
never going to miss any program of any valid program in your programming language in that
sequencing.
393
And it will not be an infinite process in the sense you would not be stuck for ever to find out the
position of any valid program in the programming language in that sequencing.
So that brings me to the end of this lecture. These are the reference for today’s lecture and again
I followed some of the examples from this article in the current lecture thank you.
394
IIIT, Bangalore
Module No # 05
Lecture No # 29
Cantor’s Diagonalization Argument
Hello everyone welcome to this lecture just a quick recap. In the last lecture we saw various
examples of countably finite sets. So we will continue the discussion on countably infinite sets
and the plan for this lecture is as follows. In this lecture we will see several other examples of
uncountable sets and we will discuss about Cantor’s diagonalization argument and Cantor’s
theorem.
395
So as I said earlier in the last lecture we saw examples of several countably infinite sets. And the
nice thing about those set is that their cardinality is same as set of positive integers. So we saw
several such sets it may be the set of integers, the 2 dimensional plane, integer plane, set of
rational numbers, set of prime numbers, set of all binary strings of finite length, and for any finite
alphabet the set of all possible strings of finite length over the alphabet.
So it might look like that for every infinite set, somehow we can show that its cardinality is same
as set of positive integers. But the interesting part here is that is not the case and the focus of this
lecture is to argue about the existence of infinite sets whose cardinality is different from that of
set of positive integers.
396
So we begin with our first set namely set of all binary strings but of infinite length. And this set
∞
is denoted by this notation {0, 1} . So; some examples of binary strings of infinite length if I
consider the string x equal to 0 0 0 0 and the sequence of 0’s which never ends then that is a
binary string whose length is infinite. Its length is infinite because the characters in the string
will never end.
Similarly if I consider this binary string where I have alternate 0’s and 1’s and the sequence goes
forever then that is again it is an example of a binary string which has a infinite length. Similarly
if I consider this binary string consisting of 0’s and 1’s where at the nth position the bit is 1
provided n is a prime number otherwise the bit is 0. So for example 1 is not prime so that is why
the first position I have bit 0, 2 is prime the integer 2 is prime.
So that is why at the second position I have bit 1, the integer 3 is prime that is why at the third
position I had the bit 1, the integer 4 is not prime that is why I have at fourth position the bit is 0
and so on. And again this is an example of an infinite length binary string. So before proceeding
further you might be wondering is there any difference between the set of all binary strings of
finite length, namely the set {0, 1}* and the set that we are considering right now, namely the set
of binary strings of infinite length. It turns out that indeed these two sets are completely different.
The difference is in the terms of the length of the strings in the individual sets. So when it comes
to the set of; first of all both the sets has the infinite number of sets remember that. Even if I
consider the set {0, 1}* , the number of the number of strings in that set is infinite.
397
However the length of each string in that set will be finite. So the difference, the primary
∞
difference between the 2 sets is the following. If I consider the set {0, 1} then the property of
the set is that the length of any string in this set cannot be bounded by a natural number. Because
you take any string in this set we can never say what will be the end digit or end bit of the string
because the sequence of characters in each string in the set will be an infinite sequence.
Whereas if I consider the set {0, 1}* then the property of this set is that each string in the set is of
finite length. That length might be arbitrary large, it might be enormously large positive number
but it is a bounded quantity. That means we will know that it starts with certain bit and it ends
with a certain bit. It is not the case that its end bit is not known. So that is the primary difference
between these 2 sets.
That is why you can see here in the examples that I have listed down. We do not know what
exactly is the end bit of these strings. That is why we have dot written down here.
So now what we are going to discuss is a very beautiful result, very fundamental result attributed
to Cantor is called a Cantor’s Diagonalization argument and using this diagonalization argument
is we are going to prove is that the set of all binary strings of infinite length is an uncountable
set. That means we cannot enumerate out or we cannot list down the elements of this set. So the
proof will be by contradiction; we will use a proof by contradiction mechanism here.
398
So we are supposed to prove that this set is an uncountable set. But we believe the contrary, we
assume the contrary and we assume that the set is countable and if it is countable then it must be
having a sequencing of elements of the set. Imagine that the sequence is this r1, r2, rn and so on
that means we know that what is the first element in this set, the second element in the set and so
on.
∞
So remember each element in the set {0, 1} is a binary string of infinite length. So imagine r1 is
of this form : it has first bit d11 second bit d12 third bit is d13 forth bit is d14 and so on. And it is an
∞
infinite length string because it is the number of {0, 1} . Similarly imagine that the bits of the
strings r2 are d21, d22, d23, d24 and so on. Similarly the bits of the string r3 are d31, d32 and so on.
And since the sequence is an infinite sequence because there are infinitely many elements in this
∞
set, this list of elements of the set {0, 1} will go on and that is why I will have dot here. Now
what I am going to do here is I have to arrive at a contradiction the way I am going to arrive the
contradiction is that I will show that there exist at least 1 string which is of infinite length and
which is binary and which is not there in the sequence of binary strings that we are assuming
exists.
We are assuming that we have a sequence r1, r2, rn and so on and that sequence is the
∞
enumeration of all the elements in the set {0, 1} . But what I am going to show is, I am going to
show the existence of one string which is going to the missed in the sequencing which will show
that the sequencing which we are assuming to exist does not exist actually. So what, exactly that
string, So you consider the binary string r which is obtained by focusing on the bits along the
diagonal here.
So remember this diagonal is an infinite diagonal because I have more elements to follow and the
sequencing I am just for r4 I am consider d44 and so on. So I am considering the diagonal binary
string, its is an infinitely long binary string. And what I consider here is now I consider a new
string which I denote as r and r is obtained by complementing each of the bits in this diagonal
binary string.
399
So d11 is 0, I take its complement. So this is denoted by d11 and so on. Now my claim is that the
string r which I have constructed here which I am considering here definitely is a member of the
∞
set {0, 1} why? First of all it is a binary string and its length is infinity. Because the diagonal
here which I am considering here; the diagonal binary string is of infinite length. So the
complement of that string is also will be infinite length.
∞
So definitely r is the member of the set {0, 1} . But the interesting thing here is that the string r I
am considering here will be different from all the string r1, r2, r3, r4 and so on in the sequencing
which I am assuming to exist. So you can verify that. So if I consider the first string r1 in my list,
r is definitely different from r1 because the first bit of r and first bit of r1 are different.
They are complement to each other. Similarly if I consider the second string r2 in my sequencing
it will be different from r because the second bit of r will be different from r2. And this process
will continue; you take any string in the sequencing which you are assuming there will be at least
1 bit in that string in the sequencing which will be different from the corresponding bit in r
compliment or r.
So that shows that the sequencing that we are assuming is not the complete sequencing of the set
∞
or the complete sequencing of the elements of the set, {0, 1} . We are definitely missing some
∞ ∞
elements from the set {0, 1} which we are not writing out. And that shows that your set {0, 1}
is an uncountable set because as per the definition countably infinite set if the set is countably
infinite there must be some valid sequencing some sequencing of the element of that set.
∞
So it does not matter what is the sequencing, if you show me any sequencing for the set {0, 1} I
will show you the existence of 1 string which will be missed in that sequencing. It will show that
∞
no sequencing of the elements of the set {0, 1} is possible.
400
So there are few subtleties involved here when running the Cantor’s diagonalization argument. It
might look to you that why I cannot Cantor’s diagonalization argument to even prove that the set
{0, 1}* is also uncountable. So {0, 1}* remember is the set of all binary strings, there will be
infinitely many elements in the set {0, 1}* but the length of each string in this set will be finite.
So let us see where exactly the Cantor’s diagonalization argument fails when we try to run it for
{0, 1}*.
So we will start assuming that we have a sequencing for enumerating out the elements for the set
{0,1}* where the elements where the listing is r1, r2, rn and so on. So again I will focus on the
individual bits of r1, r2, r3, and so on. The important point here to note is that since each of the
strings r1, r2, r3 are the members of the set {0, 1}* their length will be finite. I will know that d1i
is a bit. I will know that i is, actually some number, some natural number, some positive number.
It is not the case that the bits of r1 will keep on going forever. In the same way if I consider the
string r2 I know j is a natural number. Sorry for the typo here this should be d2j and similarly this
should be d3k and so on. So when I consider the bits of the string r2, I know that there are finitely
many bits in r2 that means j is a natural number. Similarly for r3 I know there are finitely many
bits in r3 that means k is a natural number and so on.
Although the number of elements in the set {0, 1}* is infinite that is why the sequencing will
keep on going forever. So as per the Cantor’s diagonalization argument I will consider the binary
401
string are compliment. It will be the compliment of the diagonal string here. So the diagonal
string will be like this and it will continue forever. Definitely r that I have constructed here will
be different from each of the strings r1, r2, rn in the sequencing, provided the bits of r complement
or r continue forever.
That means only when r goes forever that means the bits of the r keeps on going and going never
ends then only I can claim that the r is different from each of the strings r1, r2, rn in my
sequencing. But notice that is the case when string r which I am constructing here which is not
an element of {0, 1}*. It is because if at all r belongs to {0, 1}* then its length has to be finite.
I cannot say that the bits of the string r continue forever. That is the characteristic of binary
string which has infinite length. But if at all r is of finite length that means it stops some where
and if it is of finite length definitely it will be appearing somewhere in my sequencing and that is
where the Cantor’s diagonalization argument fails. Whereas when if we consider the argument
∞
for the set {0, 1} there was no restriction on the bits of r it was allowed to go forever.
So, that is the point where the Cantor’s diagonalization argument fails when we try to run it for
the set {0, 1}*. Let us see another subtlety to make my point more clear.
So let us try to see whether we can run the Cantor’s diagonalization argument to prove that the
set of integers is an uncountable set. Remember it is a countable set and we had shown
402
sequencing, we know how to sequence, list down the elements of the set of integers. So as per
the Cantor’s diagonalization argument the proof will be contradiction and so I will assume that I
have sequencing for listing down all the integers.
So r1, r2, r3, rn will be the elements in the sequencing and then each integer can be considered in
terms of its decimal representation, namely the digits that we have in that integer. So imagine
that digit of r1 are d11, d12, d13, d14, d15 namely r1 has i number of decimal digits. Similarly say r2
the integer r2 has j number of decimal digits. Again sorry for the typo error this should be d2j.
And similarly assume that r3 has k number of decimal digits and so on. Again remember each
integer has a magnitude. When I say it has a magnitude that means its possible only when the
number of digits in an integer is finite. You cannot have an integer which has infinitely many
digits in its decimal representation. That is not a valid integer at all because you do not know
what exactly will be the magnitude of that integer.
So each integer will have finitely many digits that might be arbitrary large that is the different
thing. And the number of elements in the set of integer it might be infinite that also fine. But the
property is that the length or the number of digits in each integer will be finite. So again if I
consider the diagonal digits here and flip them and obtain new string of decimal digits say r.
Then I can say that r is different from all the integers in my sequencing provided that the digits
of r continue forever.
Then only I can say that the r is different from r1 and r2 and r3 and r4 and every integer in my
sequencing. But if that is the case that means if I allow the digits of r to continue forever then
that is not a valid integer because every integer has to stop after certain number of digits. That
might be enormously large quantity but it has to stop somewhere. I cannot have an integer whose
digits continue forever. So the resultant sequence of the digits r which I will construct as per the
Cantor’s diagonalization argument will not be a member of the integer set.
403
So i am not getting a contradiction that I now have a new integer which is not there in my
sequence. In the same way I cannot run Cantor’s diagonalization argument to prove that the set
of rational numbers is uncountable. Remember the set of rational number are countable even
though it is an infinite set and we know how to enumerate or list down the element of the set of
rational number. Again let see where exactly the Cantor’s diagonalization argument will fail for
the set of rational numbers.
So we will assume that we will have sequencing or listing of the set of rational numbers and let
r1, r2, r3, be the rational numbers, listing of the rational numbers as per my sequencing. So again
what I have done here is I am not focusing here in my sequencing. I am not focusing on the
decimal point, that is just for simplicity. I am focusing on the remaining part whatever is
appearing after the decimal point in the decimal representation of your rational numbers and
focusing only the digits here and not focusing on the decimal points here.
That is just for simplicity; so again the digits of the first rational number I am assuming them to
be d11, d12, d13, d14 and so on. So now in this case what is happening is, the number of digits in
the decimal representation of any rational number might be finite or it might be infinite. It might
be finite in the sense it might terminate after some point.
Say for instance if I consider the rational number say 1 over 2. Then 1 over 2 is 0.5 so it
terminates somewhere. But 0.5, I can imagine as 0.500000 followed by infinite number of 0. So
404
even if the decimal representation terminates after certain positions or the given rational number
I can imagine that I append it by infinite number of 0 and hence the, number of digits continue
forever. Whereas I may have rational number where in the decimal representation the number
digits never terminates.
But it will have a never ending periodic recurring digit sequence. So for example if I consider the
rational number 1 over 3 then 1 over 3 is 0.333333333 never terminates. So that means when I
am writing down the decimal representation of the rational numbers in my sequencing I can
imagine that for every rational number in its decimal representation the digits continue forever.
But if it terminates it continues forever or it does not terminate in that case it will be a never
ending recurring sequence. It would not be a case that it does not terminate and simultaneously it
does not recur that will not be the case for a rational number. If at all, the decimal digits does not
terminate then it will be definitely a recurring somewhere.
So now, again it will focus on the digits along the diagonal entries here and we will flip them.
We will flip them in the sense that what you can imagine is that the d11 here represents any digit
different from d11. So for the notion of the compliment make sense in the context of binary
strings here but we can generalize it here and assume that d11 represents any digit different from
d11.
405
Similarly d22 represents any digit different from d22 and so on. So that will be new sequence, a
new string of decimal digit which we have constructed here. Definitely this new string of
decimal digits which we have constructed will be different from all the sequence of decimal
digits in your sequencing provided the digits of this r continue forever. Now the question is can I
say that this r definitely is a rational number.
Only in that case I can arrive at a contradiction. But the point here is that there is no guarantee
that the sequence of digits in this string r has a periodic recurring digit sequence. So what I am
saying is that it may be possible that in r you have a non-recurring sequence of digits. That
means even though the sequence of digits does not stop we do not get any recurrence or any
periodic recurrence or you do not get any recurrence of periodic recurring sequence in r. That
means, this r need not be a rational number because we do not know whether all these digit d11 ,
d22 , d33 , dnn they are distinct or they are going to be repeated. We do not have any guarantee
what so ever. So that is why we do not get the contradiction, we do not get the guarantee that this
string of digits r represents a rational number.
It may represent an irrational number so for instance if you consider √2 is a rational number then
you know that if I consider the decimal representation of √2. And if I focus on the sequence of
the digits in the decimal representation of √2. Then it is a never ending sequence and you do not
have any recurring or any periodic digit recurring sequence in this sequence of decimal digits.
So your r may be a sequence like this, you do not know. So you cannot say that the r that you are
obtaining here is definitely a rational number. So again Cantor’s diagonalization argument fails if
we try to apply it to the set of rational numbers.
406
∞
So now we know at least one set may be the set {0, 1} which is not countable. Now we will see
some other sets as well which are not countable. So what we are going to show here is first the
set of real numbers between 0 and 1 but excluding 0 and 1 is uncountable. So the set is denoted
by (0, 1) so this is the representation of the set of all real numbers between 0 and 1 excluding 0
and 1.
∞
So how I am going to show it is uncountable? Well I have already shown that the set {0 , 1} is
∞
an uncountable set. I will show you now a bijection between the set {0, 1} and the set of all real
number between 0 and 1. That will automatically show that; conclude that the set of all real
∞
numbers between 0 and 1 has the same cardinality as the set {0, 1} .
So what is the bijection? Bijection is very simple. So if I take any x, any real number between 0
and 1 excluding 0 and 1 that will have a binary representation. So let the binary representation of
that real number be 0.y where y is a binary string. So what will be the function f? The function
f(x) will be y, that means I will just chop off the 0 here and the point here and just I will focus
binary representation that means the bits in the representation y and that will be the mapping of x
as per the function.
So how exactly this function will look like. So if you consider say for instance x = 0.5. So since
the mapping of 0.5 will be the binary string 1 followed by infinite number of 0’s why? Because 1
over 2 in binary; can represented as 0.1 and I can always put infinitely many 0 even though 1
407
over 2 is 0.1. So this will be my y and that is why f(x) will be mapped to I chop off the 0 at this
point.
In the same way if I consider x = 1 over 3 then 1 over 3 can be represented as this infinite sum 1
over 4 + 1 over 6 + 1 over 64 and so on and this an infinite sum and it is in a geometric
progression, if you apply the rule of summation of infinite geometric series you will get the sum
that is nothing but 1 over 3. But now if I focus on binary representation of 1 over 4, 1 over 16
and so on, this will be a binary representation.
So anything after the binary point here it will my y and I will chop off this and that is why my x
will be now mapped to this y and so on. So it is easy to see that this function f is indeed a
bijection and I am leaving that as an exercise for you. Because the simple fact is you take any x,
any real number it will have unique binary representation that is all that is a simple observation
here.
Now if I consider the set of real numbers, this set R denotes the set of real numbers then it
contains the subset (0, 1) it also includes all the real number between 0 and 1 also. And since (0,
1) the set of all real numbers between 0 and 1 is uncountable and remember we had argued that,
if we had a set with a subset which is uncountable then the whole super set will also be
uncountable. That automatically shows that the set of real numbers is also uncountable.
And intuitively the main reason that the set of real numbers is uncountable is that it has irrational
numbers as well which we cannot enumerate out.
408
So Cantor proved a very interesting result as well. He showed that you take any set A then the
cardinality of that set is strictly less than the cardinality of its power set. So remember the
notation P(A) denotes the power set of A. Where the power set is the set of all subsets of that set.
So of course this statement is true if your set of A is finite namely if your set A has n number of
elements then its power set will have 2n elements.
And we can always prove that n is always strictly less than 2n. What if A is an infinite set can we
conclude that this theorem is true that is even for infinite set and Cantor showed yes, so the proof
is again is contradiction and we will run the diagonalization argument here as well. So we will
assume that: let the cardinality of the set A be greater than equal to the cardinality of its power
set.
Now before proceeding with fact which we will be using in this proof is the following. If you
have the sets X and Y and if the cardinality of X is greater than equal to the cardinality of Y then
there always exist a surjection from X to Y. This is a very simple fact which you can prove very
easily. So I am not going to the proof of that; we are going to utilize this fact in this proof. So I
am assuming here that the cardinality of A is greater than equal to the cardinality of its power
set.
That means there will be some surjective function from the set A to the power of set of A. I do
not know what exactly is the structure of the surjective function but I denote the surjective
409
function by f. So now what I have done here is let the elements of A be x1, x2, x3 and xn and so
on. It is an infinite set, so it has infinitely many elements, so I am assuming that elements of set
A can be listed down as x1, x2, x3 and so on. And I have listed down f(x1), f(x2), f(x3) and so on.
So each of the f value is nothing but a subset of A set that means it will be the element of the
power set. So depending upon which elements from the set A are present in f(x1) accordingly I
have put the entry 0’s and 1’s. So for example here I mean to say that f(x1) it is a set which does
not have x1, it does not have x2 but it has x3 and so on. Similarly f(x2) is a subset which has x1 , it
has x2, it has x3and I have listed down f(x1), f(x2), f(x3) and so on. That is the interpretation of
0’s and 1’s in the table,
Similarly f(x3) is the subset which has x1, which has x2 but it does not have x3 and so on. Now
what I am going to show is since I am assuming that the function f is surjective function and I
have to arrive at a contradiction. I will show that this f actually does not exist and how do I show
that f is does not exist? I have to show that f is actually not a valid surjective function. To do that
I will show you a subset which belongs to the powerset namely I will show you a subset of A set
which do not have any pre image.
That means it will be different from, f(x1), it will be different from f(x2), it will be different from
any f(xi) which shows that the set S, the subset S do not have any pre image, hence contradicting
that function f is a subjective function. So how do I construct that subset S; again I run the
410
diagonalization argument, so I focus on the diagonal entries here and the elements of my set S
will be constructed depending upon the diagonal entries.
So in this example so the first diagonal entry is 0 so I will include x1 because here the entry is 0
so I will flip it and I will make it 1 that means I have to include x1. The second entry is 1 along
the diagonal 1 so I will flip it and make it 0 that means I have to exclude x2. The third entry
along the diagonal is 0 so I will flip it and will make it 1 that means I have to include x3 and so
on. So that is the way I have constructed set S here.
So now you can check here that indeed the set S I have constructed will be an element of the
power set because it is a subset, it will have some of the element from the A set. I am not taking
the element in the S set some from outside. So that is why it will be the element of the power set.
But you can check here that the S will be not equal to f(x1) that means the set S will have at least
1 element which is not there in f(x1).
So for example x1 is present in S but x1 was not present in f(x1). Similarly the set S will be
different from f(x2) why? Because f(x2) has x2 but I have not included x2 in S. Similarly f(x3) will
be different from set S why? Because f(x3), does not have x3 but I have included x3 in S. So the
way I have constructed the set S it will be different from the image of f(x1), f(x2), f(x3) and so on.
That means S will not be the image of any xi and hence S is not a valid surjective function. That
means whatever I assumed here namely I have assumed existence of the surjective function from
the set A to its power set which is not a valid assumption. This is not a valid assumption because
I made here a wrong assumption that the cardinality of the set A is greater than equal to the
cardinality of its power set. So that means indeed the statement in this theorem is the correct
statement.
411
So what is the implication of the Cantor’s theorem it has a very beautiful implication. So this is
the statement of the cantor’s theorem. So if I apply it over the set A being the set of positive
integers or the set of natural number then I obtain the fact that the cardinality of the set of natural
number is strictly less than the cardinality of its power set. That means the cardinality of the set
of natural number is ‫א‬0.
But what I am showing here is that its ‫א‬0 is strictly less than the cardinality of the power set of
the natural number. That means the power set of the set of natural number is uncountable. Now if
I treat the power set of natural number as the set A then the power set of this power set will have
more cardinality and this process will keep on going forever. So what basically Cantor showed is
that there are infinite number of infinities.
You do not have only one infinity so ‫א‬0 is one of the infinity it is one of the infinite quantities.
But you can have now an infinite number of infinities because now you have a hierarchy of
infinite quantities. So that is the very interesting fact about the cardinality of infinite sets. So that
brings me to the end of this lecture. Just to summarize this lecture we saw some uncountable sets
and we proved that those sets are uncountable by using Cantor’s diagonalization argument.
Thank you.
412
IIIT, Bangalore
Module No # 06
Lecture No # 30
Uncomputable Functions
Hello everyone, welcome to this lecture so just a quick recap; in the last lecture we discussed
about Cantor’s diagonalization argument. And we saw examples of uncountable sets; the plan for
this lecture is as follows. We will discuss about computable and uncomputable functions and we
will discuss about the existence of uncomputable functions.
413
So what exactly are uncomputable functions; so they are some special type of functions. So say I
have function f defined from a set X to a set Y then I will call the function f to be computable if
there exists some computer program in a programming language which can compute or give you
the value of this function for every possible input from the domain of that function. So mind it I
am not focusing here on the running time of the computer program or the resources utilized for
the program to give you the value or the output of that function.
I am interested whether there exists a program or not which can give you can give you the output
of that function for every input from the domain. If you can write a program in the programming
language for such a function I will call that function to be a computable function. Otherwise I
will call it uncomputable function. So as per the above definition a function which is not
computable will be called an uncomputable function.
414
So what we now want to prove is that there indeed exists uncomputable function that means it
does not matter how much resources time and memory space you provide. And write down a
program there always exist some function such that you cannot write down a program respective
of resources allowed to compute the output of that function for every possible input. And what
will be proof strategy that we will follow to prove this theorem?
So we will begin with some known fact; so just recall that in one of our earlier lectures we
proved that the set of all valid programs in any programming language is countable. That means
we can enumerate them even though they are infinitely many valid programs. When I say
programs I mean to say the valid programs; which complies and give you an output. That means
it has a begin instruction and an end instruction and a sequence of arbitrary number of
instructions in between the begin and end instructions and it complies and it gives you an output.
So the collection of such programs is denoted by the set P calligraphic P, so in one our earlier
lectures we proved that even though we can have infinitely many programs we can always
enumerate them. Namely the cardinality of the set of all valid programs in a programming
language is ‫א‬0. So this is a known fact. What we will prove is we will prove that the set of all
possible functions from the set of positive integers to the set of integers {0,…,9} call that set to
be calligraphic set F.
415
We will prove that the cardinality of this collection of all possible functions is not ‫א‬0 that is what
we are going to prove. That means what we are going to show is that we have more functions
than the number of possible programs. Because the number of possible programs is ‫א‬0 but we
will show that we will have more number of functions from the set of positive integers to the
integers {0,..,9}.
Now any program from your collection of valid programs can compute a single function from
this collection F. We cannot have the same program which gives you the value of both, function
f1 as well as function f2. Because function f1 and function f2 will have different characteristics
how can it be possible that you have a common program P1 which simultaneously gives you the
output of function f1 as well as it gives you the function output for f2.
You cannot have such special programs; that means if we prove this claim then based on the
known fact we come to the conclusion that you have some function in this collection of all
possible functions for which you cannot find a matching program in the list of all valid programs
in your programming language. That means, there is no program in programing language which
can help you to compute the output of that specific function and that is the specific function will
be an uncomputable function.
So what is the proof strategy we are using here we are actually arguing about; we are giving here
a non-constructive proof. Just to recall what is a non-constructive proof? Non-constructive
proofs are used for proving existentially quantified statements. So this statement is an
existentially quantified statement because it says that there exists at least one uncomputable
function.
And we are logically arguing that indeed one such function exist we are not giving a concerte
function for which you can never write a function. We are logically arguing the existence of such
a function so that is why this is a non-constructive proof here.
416
So the set calligraphic F denotes as I said the set of all possible functions from the set of positive
integers to the set {0,…,9}. And my claim is that this set is not countable that means its
cardinality is not ‫א‬0 or in other words you cannot enumerate out, list down all the functions in
this collection. How we are going to prove this? Well we already know a set which is not
countable. What is the set?
This is the set of all real numbers between [0,1) including 0 that is why you have the square
bracket within 0 but excluding one. So this set is already known to be uncountable what we will
show is, we will show an injective mapping from this set to the collection calligraphic F which
will prove that the cardinality of this set of all real numbers between 0 and 1 is strictly less than
equal to the cardinality of the set of all possible functions.
And since this collection is uncountable any set which has more cardinality than that collection
also will be uncountable. So how do we show the existence of the injective mapping? The
mapping is very simple you take any real number x between 0 and 1 possibly including 0. So it
will have a decimal representation; let the digits in decimal representation d1, d2, d3, dn and so on.
And again I am assuming here that the number of digits in the decimal representation is infinite
why is that? Because even if x as finite number of digits in its decimal representation say for
instance x is equal to 0.25 I can always interpret that 0.25 to be 0.2500; I can always plugin
417
infinite number of zero’s at the end. So that way I am assuming here that the number of digits in
the decimal representation of every element x of the domain here is infinite.
Now I have to tell you the corresponding image for this element x, the image is computed as
follows and remember the image will be a function. Because this set calligraphic F denotes the
set of all possible functions from the set of positive integers to the set {0,…,9}. So it will be this
set is basically a collection of functions so I have to show you one function from this collection
which will be the mapping of this element x.
So what will be that function? The corresponding function f which is the mapping of this element
x is as follows. The function f(1) will take the value d1 the function f(2) will take the value d2
and like that the function f(n) will take the value dn and so on. So what basically I am doing here
is that I am focusing on the function which gives basically the nth digit in the decimal
representation of your real number x.
That is the function f here, and since I am assuming that I have infinite number of digits in
decimal representation of x this function will take the values and input 1, 2, n up to infinity. And
possible values or outputs of these functions can be only between 0 to 9 because each of the
decimal digits in the decimal representation of x belongs to {0,..,9}. So that is the function which
will be the image for this element x and it is very easy to verify that this mapping is an injective
mapping.
Because if you take 2 elements x and y here which are different then their corresponding or
decimal representation also will be different. So x will be mapped to f and y will be mapped to a
different function f’. You cannot have 2 different numbers x and y getting mapped to the same
function f. So that is why clearly this function is an injective function. So we have shown here
that indeed the set of all possible functions from the set of positive integers to the set {0,…,9} is
an uncountable set.
That means if I go back to the previous slide in the proof strategy I have proved my claim that
means you have more number of functions than the number of programs which you can write in
a programing language. And since each program can give you the output of only a single
418
function from the set of all possible functions you have more functions that the number of
programs.
And hence here are some functions for which you do not have a corresponding matching
program and that is why you do have uncomputable functions that exist. So that is a very
interesting theorem because generally we believe that using computers we can compute
anything. You can always write down programs which can give you the output of any function
and any computational task in the real world you can think it you can abstract it out in terms of
function.
So the remarkable thing about this theorem is that it tells you that its computers are not kind of a
god I cannot compute everything using computers or writing programs. There always exist tasks
which you cannot compute or cannot compute or cannot find out their values you cannot solve
those tasks using computers irrespective of how much time or how much memory you are
allowed while writing down the program.
So and all these things we can prove that the theorem proved using the theory that we have
developed extensively regarding the cardinality theory. So till now you might be wondering
where exactly the concepts that we learnt till now regarding the cardinality theory will be useful.
So it is useful while proving the existence of uncomputable functions which is a very
fundamental fact in computer science.
419
So that brings me to the end of this lecture these are the reference for today’s lecture. Just to
summarize in this lecture we have introduced the notion of computable and uncomputable
functions and we showed non-constructively that indeed there exist uncomputable functions
which you cannot compute by writing down computer programs thank you.
420
IIIT, Bangalore
Module No # 07
Lecture No # 31
Tutorial 5
Hello everyone welcome to tutorial number 5 so let us start with question number 1 we have to
show in question number 1 that these two sets have the same cardinality. So the first set here is
the set of all real numbers between (0,1) but excluding 0 as well as 1. Whereas the second
collection here it is also the set of all real numbers between (0, 1] but 1 is inclusive that means 1
is allowed. That is why the square bracket here and 0 is not allowed.
So that is the interpretation of these two sets we have to prove that these two sets have the same
cardinality. How we can do that so recall the Schroder-Bernstein theorem which says that, if you
want to prove that two sets have same cardinality show injective mappings from the first set to
the second and from the second set to the first. So we are going to do the same thing here so here
are our two sets.
Let us consider the injective mapping f which is the identity mapping so clearly this mapping is
an injective mapping from this set to this set. Because you take any two different real numbers x
and y the corresponding image will be x and y and they will be different and they will be in the
421
range which is allowed as per function f. So your domain is not allowed to have the numbers 0
and 1 and so is the images.
Now if I want to take the injective mapping in the reverse direction then consider the injective
mapping g defined to be x / 2 that means. If you want to find out the value of g(x) the output is (x
/ 2). So that means if your x here which we are considering is different from 1 then clearly that
will fall in the range (0, 1). And excluding 0 and 1 but if your x = 1 as well then the image of 1
will be 0.5 which is well within the allowed limit.
And it is again easy to verify that your mapping g is an injective mapping in the sense we have
shown here two injective mappings so we can conclude that the cardinality of these two sets are
the same.
In question number 2 we have to prove that there is no infinite set A whose cardinality is strictly
less than ‫א‬0. That means its cardinality is strictly less than the cardinality of set of positive
integers. So in some sense we want to prove here that is ‫א‬0 is the smallest infinity here. So to
prove this statement we will use 2 claims and we will assume for the moment that these two
claims are correct and later on we will focus our attention on proving these two claims.
.
The first claim is that if you have any set A whose cardinality is less than equal to the cardinality
of set of positive integers then you can always find a subset of the set of positive integers which
422
has the same cardinality as your set A. So pictorially what I am saying here is that you may have
a set A whose cardinality is less than equal to the cardinality of this set Z+. What I am saying
here is that in this claim the claim says that you can always find a subset B within the set Z+
whose size is exactly the same as the size of A.
And claim 2 is that any subset of the set of positive integers is either finite or has the same
cardinality as ‫א‬0, you cannot have any other third category of subset of the set of positive
integers. So for the moment assume that these 2 claims are true let us come back and prove the
statement that we are interested to prove and the proof will be by contradiction. So we want to
prove that is no infinite set A satisfying this condition.
So we will assume that on contrary suppose you have an infinite set A whose cardinality is
strictly less than ‫א‬0. Now by applying claim 1 on that set A we also know that there exist some
subset B of the set of positive integers whose cardinality is same as the cardinality of your A set.
And now if I apply the claim 2 on that subset B I know that the subset B has to be countably
infinite.
Because the subset B is a subset of the set of positive integers; and it will be either finite or
countably infinite. So B is the definitely not finite because the cardinality of B is same as the
cardinality of A and we are assuming here that A is an infinite set. So we are left with the second
category here that means the cardinality of the set B is countably infinite. But we also know that
the cardinality of B is same as cardinality of A that means the cardinality of A is also ‫א‬0 because
B is countably infinite.
So the cardinality of B is ‫א‬0 that means the cardinality of A is also ‫א‬0 which is a contradiction. So
now the proof boils down to how exactly we prove these two claims; they are very simple.
423
So let us first prove claim number 1; since the cardinality of A is less than equal to the
cardinality of Z+ then as per the definition of cardinalities we know that there exist an injective
mapping from the set A to the set of positive integers. Then only the cardinality of A can be less
than equal to the cardinality of Z+ so based on that injective mapping f I am going to show you
the existence of the required set B.
What do I do basically is, I just focus on the range of the function that means I pick up the set of
images of this function f that means all the valid images as per this function f and since my
function f is an injective mapping each element of A will have a unique image. So how many
images I will pick? I will pick same number of the images as I have the number of elements in
the domain that means the range set of f namely the set B will have the same cardinality as the
cardinality of your A set.
So that is the very simple proof for claim number 1. For claim number 2 we have to prove any
subset of Z+ is either finite or countably infinite. So if we pick a finite subset of Z+ then the proof
is trivial. So let us focus on the case when we have chosen an infinite subset of Z+. Then the idea
here is that we know that Z+ is countably infinite that means it is possible to list down all the
elements of Z+ and B is a subset of Z+.
That means what I can do is, I can always say that it is possible to list down the elements of B
why? You just list down the elements of B in sorted order, that is all, because B is the subset of
424
Z+ and it is always possible to list down all the elements of Z+. So depending upon which
elements are there in B or which elements are not there in B, in the sequencing of Z+ remove
first all the elements which are not there in B. And if you arrange all those remaining elements in
a sorted order then that gives you a valid listing of the elements of the set B.
And if we have the valid listing of the elements of an infinite set then that is a countably infinite
set. So that shows that your subset B is countably infinite which proves your claim.
In question number 3 we are asked to show that if you are given an infinite set then it does not
matter whether A is countable or uncountable,you can always find the subset of A which is
countably infinite. So of course if my set A itself is countably infinite then the subset would be
the set itself but in the statement I am asking you to prove this even if the set A is uncountable.
So even if my set A is the set of real numbers which is an infinite set and not countable I am
asking you to prove the existence of a subset of real numbers which is countable and infinite.
So we will prove this general statement how we are going to prove this? So since A is an infinite
set it will have at least one element arbitrarily I pick that element I call it as a1. Now my claim is
that if I remove that element a1 from the set A, the remaining set is still an infinite set why?
Because if this remaining set A – a1 if this is finite and if it is cardinality is say m then what can I
say about the cardinality of A.
425
I can say the cardinality of A is m+1 which is a finite quantity but that goes against my
assumption that A is an infinite set. So that means if I remove the elements a1 from the set A I
will still left with an infinite set.
Now I again apply the same argument. I focus on the left over set namely the set which I
obtained by removing element a1. And my claim is that since it is an infinite set it will have at
least 1 element a2 I will pick it arbitrarily. And my claim is that if I now remove the element a2
also I have already removed a1 from A and now what I am saying is, even if I now remove a2 the
left over set will be an infinite set.
Again the proof follows using similar argument because if this left over set is not infinite
suppose its cardinality is finite number say n then what I can say about the cardinality of A? I
can say the cardinality of A will be n+2 which is a natural number, positive number. And which
goes against the assumption that A is an infinite set.
426
So I can keep on running this argument and what I can say now is that, in each step in each
iteration the arbitrarily element which I am picking from the left over set if I list it out, if I list
down all those elements, then that sequence of the elements will be or the set of those elements
which I am picking in each iteration from the left over set will of course be a subset of my
original set. Because in each iteration I am picking an element from the left over set and in each
iteration the left over set is a subset of the original set.
And what I can say about the set of numbers which I am removing from each iteration. I can say
that it is cardinality is ‫א‬0 because I now have a valid sequencing for the elements in that subset.
Namely I can arrange the elements in that set T in the order in which I have removed those
elements in each iteration. So that shows that I can always find out an infinite subset from the set
A whose cardinality is ‫א‬0.
427
In question number 4 we want to show that if I take several countable sets and take their union
namely if I perform union of countable number of countable sets then the resultant set is again
countable. So imagine you are given several countable sets namely say you might be given
infinite number of countable sets and without loss of generality let they be disjoint. So I call the
bigger set 𝑆 ≝ 𝑖∈ℕ 𝑆𝑖 to be the union of all these sets Si and I want to show that this bigger set
S is also countable, if each of these individual sets were countable. So what we have to do
basically is we have to show how to list down the elements of this bigger set S. So since each of
the sets S1 to Sn and Si each of the sets Si is countable it will have a listing of its own. So imagine
that the listing of the elements of the set Si is this. It is an infinite list and the guarantee is that
each element of the set Si will eventually occur somewhere.
So I have written down the listing of all the elements of various sets here; so this is the listing of
S1 , this is the listing of S2 , this is the listing of Si and so on. Now based on all these listings I
have to come up with the listing of all the elements in the set S. So that I will not be missing any
element and we will never get stuck infinitely while finding any element in the resultant listing.
So here is the way we can list down the elements of the set S.
So if you see here within each set Si I have used 2 indices to denote an element in the listing.
First index denotes the subset number or the set number that i am focusing now. So if it is i then
the first index will be i and the second index basically denotes the position of the element.
Whether, it is the first element or the second element or the nth element and so on. So what I am
428
going to do is when I want to list out the elements of the set S I will list down according to the
pair of indices i, j.
And I will start by listing down all the elements of the form Si,j such that the summation of the 2
indices is 2; remember the minimum value of the summation of the 2 indices will be 2. Because I
cannot have i, j such that i + j is 1 for that to happen at least one of the, i or j has to be 0. But I do
not have any index taking the value 0 here. So the least value of the summation of the 2 indices
can be 2 only.
So I will be first listing down all the elements where the summation of the indices will be 2
following by listing down all the elements whose summation of indices will be 3 and so on. Now
when I am listing down all the elements whose summation of indices is n then you can have
several numbers of that form whose summation of indices is n. So within that collection I will be
following a sorted order, that means I will first write down according to the index i and then
according to the index j.
So to demonstrate here if I focus on all indices all pair of indices i j whose summation is 2 then I
have only one entry namely S1 1. Next I will focus on all Si j’s where the summation of i and j is
3. So I have S1 2 and S2 1 and you can see that I am listing S1 2 first and then S2 1. Because i gets
preference first so I will list down all the Si j’s where i is equal to 1 and then followed by all Si j’s
where j is bigger than i.
I have 2 elements here I have S1 2 and S2 1 to list out so I will give reference to lower value of i
first. So that why I have written down S1 2 first and then I have written down S2 1 then I will
focus on all Si j’s where the summation of i and j will be 4 and so on. So now you can see that
this is a valid listing of the elements of the bigger set S you will never miss any element and you
will not get stuck forever to find out an element in this listing.
Because you take any element x belonging to the bigger set S, x will belong to some Si. You do
not know which set it belongs to; that means x will be of the form Si j for some j and i + j will
take some value in the set of positive integers call it n. So when you are listing down all S i j’s
such that the summation of i and j is n, you will be listing down the element x; that is why this is
a valid listing of the elements of the set S.
429
In question number 5 you are asked to give examples of uncountable sets A and B that satisfy
certain conditions. So we want to find out uncountable sets such that their intersection is finite.
So there can be several examples: let us take these 2 sets the set A= [0,1], the set of all the real
numbers between 0 and 1 including 0 and 1. And the set B= [1,2], the set of all the real numbers
between 1 and 2 including 1 and 2.
So remember we proved that a set of all real between 0 and 1, its cardinality is not ‫א‬0. So it is an
uncountable set we can follow the same argument to even prove that a set of all real numbers
between 1 and 2 is also uncountable. So I am not going into the proof of that I leave it as an
exercise for you.
430
Now what can you say about intersection of this A set and B set well there is only 1 element
which is there in the intersection of these 2 sets namely the number 1 and this is the singleton set
and hence it is a finite set. We now want to find out examples of 2 uncountable sets such that
their intersection is countably infinite. So what we can say here is, let us take the set A which is
the union of the set of integers along with the set of all real numbers between 0 and 1.
And we can prove that this set is uncountable and in the same way I take the set B to be the set of
all integers along with all real numbers in the range 2 and 3 including 2 and 3 also. And we can
prove that both these 2 sets are uncountable. Their cardinalities are not ‫א‬0; A is not countable; its
cardinality is not ‫א‬0 even though Z is part of that.
So Z is countable but the set of all real numbers between 0 and 1 including 0 and 1 is not
countable. So that is why the overall set A becomes uncountable and similarly the overall set B
also becomes uncountable. Now what can you say about their intersection? Their intersection is
nothing but the set of all integers and we know that the set of integers is countable because its
cardinality is ‫א‬0.
In part C you are supposed to give uncountable sets A and B whose intersection is also
uncountable and a very simple example could be take the set A and B to be an uncountable set
and the same uncountable set. So if I take the set A to be the set of the real numbers and the set B
431
also to be the set of real numbers both of them are uncountable. And clearly A ∩B will be set A
itself which is the set of real numbers and which is also uncountable.
So what does this question demonstrates is that; you can have various properties among 2
uncountable sets you cannot say anything with guarantee regarding the union and intersection.
Their intersection may be finite, it may be uncountable it might be countably infinite and so on.
In question 6, part a, you want to find out whether the set of integers divisible by 5 but not by 7
is countable or not. So first of all what exactly is this set so let me call, denote this set by S. So
the set S will have the number 0 it will have +5 – 5 it will have +10 – 10 and so on. It would not
have 35 it would not have + 35 it would not have -35 because 35 is divisible by 5. But it is also
divisible by 7 so are not supposed to include multiples of 7.
We are not supposed to include multiples of both 5 and 7; so now we want to show whether the
set S is countable or not. So definitely S is an infinite set but the question is can we enumerate
the elements of the set S. And it is easy see that we can always list down the elements of the set
according to their absolute values. And if I focus on the absolute values then both +x and –x will
take the same absolute value x.
So what I will do is in the listing I will write down +x and –x so basically what I am saying is
that my listing here is 0 followed by +5 followed by -5 then +10 followed by -10 and so on. You
432
cannot do the following: you cannot say that first list down all the positive things up to infinity
and then followed by the minus things. This is not a valid listing because if we do this then we
do not know whether we will come back ever and start listing down the negative numbers here.
Because when we start going towards the positive multiples of 5 in infinity we will never stop
and there is no coming back. So that is why it is only the first listing which is a valid listing not
the second one.
Part B we want to find out whether the set of all real numbers whose decimal representation
consist of only 1s is countable or not. So remember the set of real numbers is uncountable we
proved that but here we are not focusing on the set of all real numbers. We are focusing only on
the real numbers whose decimal representation have only 1s and no other digit. And it turns out
that this set is countable because I can view it as union of several countable sets okay.
So let S1 denotes the real numbers consisting of only 1s where you do not have anything before
the decimal point. And after the decimal point you list down all the numbers starting with real
number where you have recurring 1s that means you have a series of infinite ones and occurring
again and again. Followed, by the real number 0.1 followed by real number 0.11 followed by real
number 0.111 and so on.
433
Remember 0.1 is not same as the real number where 1 is recurring. 0.1 you do not have anything
after the first 1 here. But 0.1 with the recurring one denotes this real number which is clearly
different from 0.1. So these are the elements in my set S1 and I have a valid listing here. Then in
S2 what I do is, I will list down all the real number with decimal representation has only 1s and
where before the decimal point I have only 1 digit.
So these are the elements in the set S2 : 1.1, 1.11, 1.111, … In S3 what I am going to do is I will
list down all the real numbers whose decimal representation consist of only 1s , and where before
the decimal point I have 2 1s, so you can imagine that when I am considering the set Si I will be
focusing on all the real numbers whose decimal representation has 1s provided there are i – 1
number of occurrences of 1 before the decimal point.
And after the decimal point I will give preference first to the recurring of occurrence of 1 then 0
occurrence of 1 and 1 occurrence of 1 and 2 occurrence of 1 and 3 occurrence of 1 and so on.
And each of this listing is a valid listing and if I take the union of all this sets S 1, S2, S3 that will
give me the bigger set S.
That means now I now have a valid listing here and bigger set S is the union of several countable
sets and hence the bigger set S will be countably infinite. So that brings me to the end of this
tutorial thank you.
434
Module No # 07
Lecture No # 32
Basic Rules of Counting
Hello everyone, welcome to this lecture. The plan for this lecture is as follows.
In this lecture we will introduce the basic rules of counting namely the sum rule and product rule.
And we will discuss about the Pigeon-hole principle.
435
So we will start with the problem of counting and counting is a very fundamental problem in
discrete mathematics. The reason is that in discrete mathematics we are dealing with discrete
objects and since the objects that we are dealing with are discrete we can count them. So very often
we will encounter questions like how many; and our main aim is to come up with methodologies
to address those questions.
So we will introduce some basic counting rules in this lecture. So the first basic rule is the product
rule. And I am sure all of you are aware of this product rule so let me demonstrate the product rule
first with an example. So here the problem description is the following. You have 2 employees say
436
employee number 1 and employee number 2 and they are going to join our office and there are 3
office spaces available.
So I call it office 1, office 2 and office 3; so 3 rooms are available and our goal is to identify in
how many ways we can allocate disjoint offices to these 2 employees. So pictorially these are the
various ways in which I can assign disjoint office to employee number 1 and employee number 2.
So I can assign office number 1 to the employee 1 and given that I have assigned office number 1
to the employee 1, I cannot assign the same office to the second employee.
Because they need to be allocated disjoint offices, so I can either allocate office number 2 or office
number 3 to the employee number 2. Or I can assign office number 2 to the employee 1 but in that
case I cannot assign office number 2 to the employee number 2 in which case I can only assign
office number 1 and office number 3 to the second employee. And similarly, I have an option of
assigning the third office to the employee 1 in which case I have the options of either assigning
office number 1 or office number 2 to the second employee.
So in total we have 6 ways but if you see here closely what's happening is we have a task T, a
bigger task. In this example the task T was that of allocating disjoint offices to the 2 employees.
And we can break that task into a sequence of 2 subtasks: subtask 1 and subtask 2. Subtask number
1 basically requires allocating office space to the first employee and subtask 2 is the problem of
allocating office space to the second employee.
Suppose 𝑛! is the total number of ways in which we can solve the subtask 1. So in this example
there are 3 ways; either I can assign office number 1 to the first employee or office number 2 to
the second employee or office number 3 to the second employee. So there are 3 ways of solving
the first subtask, so 𝑛! = 3 in this case and for each of these ways of solving the first subtask I
have 𝑛" ways of solving the subtask 2.
So for instance in this example, once I have assigned office number 1 to the employee 1, I have
the option of either assigning office 2 or office 3 to the second employee. So corresponding to this
method of solving subtask 1; namely that of assigning office number 1 to the first employee I have
437
2 ways of solving subtask 2. So 𝑛" = 2 here. In the same way if I consider the method of assigning
office number 2 to the second employee.
Then corresponding to this way I have 2 ways of solving subtask 2 and in the same way
corresponding to the method of assigning office number 3 to the first employee. I have 2 ways of
solving the subtask 2. So if this is the case then I can say that the total number of ways of solving
the overall task or the bigger task is 𝑛! ∗ 𝑛" . And that is why in this case the answer, namely, the
total number of ways of assigning the disjoint office space to the 2 employees is 6.
𝑛! = 3 in this case, because I can either assign office number 1 to the first employee or office
number 2 to the first employee or office number 3 to the first employee. So there are 3 ways and
for each of these 3 ways I have 2 ways of solving the subtask 2. And that is why the total number
of ways of solving the bigger task is 𝑛! ∗ 𝑛" . So that is the product rule. So in this case I have
considered the scenario where the task T was divided into 2 subtasks.
But in general, the product rule can be applied even for cases where your task 𝑇 can be divided
into subtask 𝑇! , 𝑇" , … , 𝑇# . So, if you have 𝑛! ways of solving subtask 𝑇! and for each of this 𝑛!
ways you have 𝑛" ways of solving 𝑇" , and for each of these ways of solving subtask 𝑇! , 𝑇" , … , 𝑇$%!
you have 𝑛$ ways of solving subtask 𝑇$ till 𝑛 ways of solving subtask 𝑇# . Then the total number of
ways of solving task 𝑇 will be 𝑛! ∗ 𝑛" ∗ 𝑛& ∗ … ∗ 𝑛$ ∗ … ∗ 𝑛' . That is the generalized product rule.
438
So now let us see some examples of product rule. So suppose we want to count the number of
possible functions from a set 𝐴 to a set 𝐵. My set 𝐴 has 𝑚 number of elements which I am denoting
as 𝑎! to 𝑎' and my set 𝐵 as 𝑛 number of elements namely 𝑏! to 𝑏' . So we have already answered
this question when we discussed functions. But now let us see how exactly product rule is
applicable to solve this problem. So your bigger task is to find out the number of functions here.
And the bigger task is basically to assign images to each element from the set 𝐴. But now I can
divide that bigger task into subtask; namely I can identify the subtask 𝑇$ which is that of assigning
an image to the element 𝑎$ . And it is easy to see that the subtask 𝑇$ can be solved in 𝑛 ways because
if I consider the element 𝑎$ then its image can be either 𝑏! or its image can be 𝑏" or its image can
be 𝑏( or its image can be 𝑏# .
So they are n ways of solving the subtask 𝑇$ and each of this sub task are independent so that is
why the total number of ways of solving the bigger task namely that of assigning image to each of
the elements from the set A is 𝑛 ∗ 𝑛 ∗ … ∗ 𝑛, m number of times. And that is why the total
number of functions will be 𝑛' .
Now let's see another example. Namely we are interested to find out the total number of bit strings
of length 𝑛. And there are plenty of ways to come up with an answer for this question but let us
see how we can apply the product rule here. And what we will do here is instead of counting the
number of bit strings of length 𝑛 let's see a related problem. Namely, finding the number of binary
functions; namely the number of functions from a set 𝐴 consisting of 𝑛 elements to a set 𝐵
consisting of only 2 elements namely 0 and 1.
And from the previous exercise, here we know that the number of possible binary functions will
be 𝑛' . So the notations are actually swapped here. So we have 𝑛 elements here and |𝐵| = 2 so 𝑎!
can have 2 possible images either 0 or 1, 𝑎" can have 2 possible images either 0 or 1, and similarly
𝑎# can have 2 possible images either 0 or 1. So that is why we have 2 ∗ 2 ∗ … ∗ 2, 𝑛 number of
times namely 2# possible functions.
But our goal is to find out a number of bit strings of length 𝑛 but what we have counted here is the
number of binary functions.
439
So what we are now going to do is we will show here that the problem of finding the number of
bit strings of length 𝑛 is equivalent to finding the number of binary functions. Namely we can
show that there exists an injective function from the set of bit strings of length 𝑛 to the set of binary
functions. And it is also easy to see that we can establish an injective function from the set of all
possible binary function to the set of binary strings of length 𝑛.
And since we have established injective functions in both the directions that shows that the number
of bit strings of length 𝑛 is exactly the same as the number of binary functions. And the number of
binary functions is 2# . So if you are wondering what are the injective functions here, so consider
you are given binary string of length 𝑛. Some arbitrary binary string of length 𝑛 say 0, 1 0, 1 … like
that.
Then the corresponding binary function is the following: the mapping of 𝑎! is 0, the mapping of
𝑎" is 1, the mapping of 𝑎& will be 0, the mapping of 𝑎) will be 1 and so on. That is the
corresponding binary function. Whereas if you want to go from a binary function to a binary string
just we do the reverse thing. So imagine you are given a binary function say where 𝑎! is mapped
to a bit 𝑏! , 𝑎" is mapped to a bit 𝑏" and like that 𝑎# is mapped to a bit 𝑏# .
440
Then the corresponding binary string will be 𝑏! to 𝑏# . That is the injective mapping in this
direction. So that shows that the number of binary strings of length n is same as the number of
binary functions.
Now let us consider another fundamental counting rule which is the sum rule and again let me
demonstrate it first with an example. So imagine you have a set of students in a university and a
set of faculty members. Of course they are disjoint because you can't have a student who is also a
faculty member. And our goal is to find out the number of ways in which we can form a committee
of just 1 member.
That 1 member can be either a student or a faculty. There is no restriction. We are just interested
to find out how many distinct committees consisting of 1 member we can form. And it is easy to
see that there are 12 ways. Why 12 ways? Because I can have a committee which consists of only
a student and it could be either this student or the third student or the fourth student or the fifth
student or the sixth student each of them is a distinct committee.
Or I can have a committee which has this faculty member, or this faculty member, or this faculty
member, or this faculty member, or this faculty member, or this faculty member, each of which
will be a distinct committee. So there are 12 different committees which we can form here. So now
how we can view this as a counting rule? So the rule is the following: you have a task 𝑇 which can
441
done either in one of the 𝑛! ways or in one of the 𝑛" ways. Of course, so there is another restriction
and the case here is that none of the 𝑛! ways is the same as the 𝑛" ways.
So for instance if you take this example, 𝑛! ways is correspond to the case when the committee
consists of a student and 𝑛" ways correspond to the case and the committee consists of a faculty
member. And both these cases are disjoint. You cannot have committee member which is
simultaneously a student as well as a faculty member. So if both these 2 conditions are satisfied
then I can say that the total number of ways of solving the task 𝑇 is 𝑛! + 𝑛" .
Of course in this case I have considered the scenario where the task 𝑇 can be divided into 2 disjoint
cases. If you have multiple disjoint cases then I can have a generalized sum rule.
So we have now seen 2 basic counting rules but it turns out that we encounter scenarios where we
have to combine both these 2 rules that, means we can encounter problems which will require us
to apply both the sum rule as well as the product rule. So let me demonstrate an example. So
suppose we are interested to find out the number of passwords of length either 6 or 7 or 8
characters. That means the password can be either of length 6 or of length 7 or of length 8.
And the restriction is that each character can belong to the set A to Z or the numeric 0 to 9. That
means the characters could be your English alphabets or digits and we also want passwords to have
at least 1 digit. So these are the various requirements on the password. So it should be of length
442
either 6 or 7 or 8. The character should be either English characters or digits and the password
should have at least 1 digit.
And we are interested to find out how many such passwords we can have. So again this is a very
common problem we encounter. So for instance if you consider net banking password then we
have certain restrictions on the net banking password. It should be of at least this much length, it
can be at most of this much length, it should have some special character etc.,
So in that case one can often ask how many such valid passwords we can form? So let us see how
we can apply the sum and the product rule in this particular example. So our password, the set of
all valid passwords I am denoting it as the set 𝑃 and this set 𝑃 actually can be divided into 3
disjoints subsets. The subset 𝑃* which is the set of all valid passwords of length 6; by the way by
valid I mean that it has at least 1 digit and all the characters belongs to this set. That is what I mean
by valid in this explanation.
So my 𝑃* is the set of all valid passwords of length 6, 𝑃+ is the set of all valid passwords of length
7 and 𝑃, is the length set of all valid passwords of length 8 and it is easy see that these 3 sets are
disjoint and by the sum rule I can say that the set of valid password, its cardinality is same as the
cardinality of the set 𝑃* and 𝑃+ and 𝑃, . And there is no overlap; you can't have a password which
is simultaneously of length 6 as well as length 7 as well as of length 8.
So that is why we can apply the sum rule here. Now how do we find the cardinality of the set 𝑃* ,
𝑃+ and 𝑃, . So let's see the logic of counting or finding the cardinality of the set 𝑃* , the same logic
is applicable to find the cardinality of the set 𝑃+ as well as cardinality of 𝑃, . So what exactly is the
set 𝑃* ? The set 𝑃* is the set of all valid passwords of length 6. That means it should have exactly
6 characters, which could be either English characters or the digits, and it should have at least 1
digit.
So it can have 1 digit or it could have 2 digits or it could have 3 digits or it could have 4 digits it
could have 5 digits or it could consist of all 6 digits. All these are valid passwords. So you might
be attempting to apply the sum rule here but it turns out that if I apply the sum rule to find the
cardinality of the set 𝑃* then there might be some overlaps which I have to take care off. So instead
443
what I can do here is, I can apply the following logic. The cardinality of the set 𝑃* is nothing but
the following.
It is the difference of the following two sets. You take the set of all strings of length 6. When I say
all strings of length 6 that means they have 6 characters. But those 6 characters may or may not
constitute a valid password. So for instance I may have a string of the form 𝐴𝐴𝐴𝐴𝐴𝐴; 6 As
belonging to the set of all strings of length 6 but this is not a valid password because it does not
have a digit which is a requirement for a valid password.
So that is why the set of all strings of length 6 have both valid passwords of length 6 as well as
invalid passwords of length 6. Now from this set if I subtract the set of all invalid passwords of
length 6 and by invalid passwords of length 6 I mean strings of length 6 which do not have any
occurrence of a digit. Those will be the invalid password. So if I subtract those strings from this
set then it is easy see that I will get the cardinality of the set 𝑃* .
So now what is the cardinality of the set of all strings of length 6? Well it is 36* and this I get by
applying the product rule. Why 36* ? Because I have 6 positions to fill. That means I can identify
6 sub tasks and at each position I have 36 options. I can either fill a character, English character,
so 26 possibilities or I can fill any of the 10 digits. So imagine you have 6 slots here; at the first
slot I have 36 options to fill, at the second slot I have 36 options, and like that at each of the slots
I have 36 options.
So that is why 36* and what is the cardinality of invalid passwords of length 6? It is 26^6. Because
here I am interested to find out in how many ways I can fill 6 slots such that none of those 6 slots
is occupied with a digit. Because then only that strings of 6 characters can be considered as an
invalid password. So I have 26 options now for each slot because I cannot fill any slot with a digit.
So that is why I have 26* options. So again here I am applying the product rule. And now if I
subtract 26* from 36* that will give me the cardinality of 𝑃* . The same logic you can apply to find
out 𝑃+ and similar logic you can apply to find out 𝑃, . And if you sum those 3 quantities that will
give you the required answer.
444
Now let's see another interesting counting rule which we encounter very often in discrete
mathematics and this is called pigeon-hole principle. So what is the scenario here? So in this
example you have 13 pigeons and you have 12 holes and suppose the pigeons are going to
randomly occupy these 12 holes. We don't know in what order they will be going and occupying
these holes. But irrespective of the way they are going to occupy these 12 holes we can say that
there always exists at least 1 hole which will have 2 or more pigeons.
A very simple common sense. And how we can prove that? We can simply prove it by
contradiction. The contradiction will be, if each hole is occupied by exactly 1 pigeon then since
we have 12 holes we get 12 pigeons. But we have 13 pigeons; so that automatically implies
definitely there will be 1 hole which has more than 1 pigeon. So very simple common sense here.
So now how do we apply; how do we generalize this rule as a counting principle?
So the generalized pigeon-hole principle is the following. So imagine you have 𝑁 objects, in this
case you had 𝑁 pigeons, and suppose those 𝑁 objects are assigned to 𝐾 boxes in a random fashion,
then the pigeons-hole principle states that there will be at least 1 box which will have ⌈N/K⌉ many
objects.
So this notation is called as the ceil notation. We have ⌈2.3⌉ = 3. Basically you take the integer
which is higher than the integer 2 here, that will be the ceiling of 2.2. ⌈2.1⌉ = 3; basically you
take the next integer which is a complete integer and larger than the current number. Whereas the
445
⌈2.0⌉ = 2 only. So in this example if I apply the generalized pigeon hole principle it basically says
that there will be at least 1 box with 13/12 pigeons and 13/12 will be a real number.
And if I take the ceil of that I will be take the next higher integer which is 2. So the proof is by
contradiction.
So let's see an application of pigeon-hole principle. So what we are now going to show is a very
interesting result. So imagine you have 6 people present in a party and it is guaranteed to you that
you take any pair of individuals then they will be either friends or enemies. You don't know what
exactly is the situation because the party consists of 6 random people but whichever 6 random
people are there in the party it is guaranteed that you take any 2 people in that party they will be
either mutually friends or enemies.
Then our claim is the following: our claim is that irrespective of the way the people are mutually
friends or enemies there always exist either 3 mutual friends in the party or 3 mutual enemies. One
of these 2 will definitely be the case. So how we are going to prove this? We are going to prove it
by applying the pigeon-hole principle and various other proof mechanisms.
And remember, and I am making this claim; the claim is irrespective of the way of those 6 people
are friends or enemies with each other. It might be the case that all of them are mutually friends
then automatically the claim is true. It might be the case that none of them are friends with each
446
other then again the claim is true. The claim is if you have 6 people definitely one of these 2 cases
will always hold.
So how we are going to prove this? So we will consider an arbitrary party consisting of 6 people
and out of those 6 people let's randomly choose 1 person. So we are now left with 5 people. So
what can I say about those remaining 5 people. By pigeon-hole principle I can say that out of those
remaining 5 people at least 3 people will be mutually friends with this person that I have chosen
or there will be 3 people who are enemies, mutually enemies, with this chosen person.
I do not know what exactly is the case because that depends upon the exact way in which the
persons or the people are mutually friends or enemies in the party but irrespective of the case one
of these 2 will always hold. Because I have 5 people; so even if out of those 5 people say 2 are
friends with this person and 2 are enemies with this person I'm left with 1 person who has to be
either a friend or has to be a enemy with this person. That is a simple logic. So that is what I am
saying here. So you have 2 possible cases.
So what I can say is the following: without loss of generality imagine that out of those remaining
5 people there are 3 people who are friends with this person. And since I am applying this argument
without loss of generality, the same argument can be applied for the second case as well when
there are 3 people who are enemies with this person. So again since there are 3 persons who are
friends with this person I am taking any 3 person here who are friends with this fixed person.
447
Now my claim is not yet proved here because individually these 3 people are friends with this
person that does not mean that I have the existence of 3 people who are mutually friends with each
other that means they all have to be friends with each other that is not guaranteed as of now. As of
now I have just guaranteed that this person is a friend with this fixed person, this second person is
a friend with a fixed person and the third person is a friend with a fixed person.
So this notation basically denotes friendship. Now I can say that the following 2 cases hold. The 3
people who are friends with this fixed person, they can be mutually enemies. That means, these 2
are enemies and these 2 people are enemies and these 2 people are also enemies. So let me; these
2 are enemies and this too. So if this is the case then I got 3 people who are mutually enemies with
each other and my claim is true.
Whereas I can have a second case where those 3 people they are all new not mutually enemies but
there exist a pair among those 3 people who are friends. Say the first 2 people are friends with each
other. Then I got 3 people who are all friends with each other. This proves my claim. So now you
can see I have proved my claim irrespective of the way that 6 people would have been friend or
enemies.
Now the question is what is the specialty of the number 6 here. I took, I proved my claim for the
case when there are 6 people in the party.
448
What if there would have been 5 people in the party? Then can I say that irrespective of the way
those 5 people are mutually friends or enemies I will always have either 3 mutual friends or 3
mutual enemies. And answer is no. The claim is not true for the case when there are 5 people in
the party. So consider the case when I have these 5 people and there is a fixed person, who is
friends with this person, this person. But he is not friend with this person, who is not friend with
this person and these 2 people are friend with each other and these 2 people are friend with each
other so on, and these 2 people are friend with each other and so on. So in this case you can see
that among these 5 people I neither have the presence of 3 mutual friends nor I have the presence
of 3 mutual enemies. So for instance, if I take these 3 people then this girl is a friend with this
person but that girl is not a friend with this person.
Whereas I require for my claim all the 3 people to be mutually friend or mutually enemies with
each other. So when there are 5 people in the party my claim is not true.
449
So now let us generalize this example to a beautiful theory of Ramsey numbers. So I defined this
function 𝑅(𝑚, 𝑛) so this function 𝑅 is attributed to Ramsey who invented these numbers and here
𝑚, 𝑛 ≥ 2. So what exactly is the value of Ramsey function 𝑅(𝑚, 𝑛)? It is the minimum number
of people required in a party such that you either have 𝑚 mutual friends or 𝑛 mutual enemies
irrespective of the way the people are friends or enemies with each other in that party.
Assuming that every pair of people are either friends or enemies. So for instance what we have
demonstrated is that 𝑅(3,3) = 6. Why 6? Because only when you have 6 people in the party then
you can claim that you will either have the presence of 3 people who are all friends with each other
or you will have the existence of 3 people none of them are friends with each other.
𝑅(3,3) ≠ 5. It is not 5 because we have given a counter example namely we can have a scenario
where we have 5 people in a party such that we might have the presence of 3 mutual friends or 3
mutual enemies. So it turns out that even though this function is well defined we do not have any
generic formula to find out the value of the Ramsey number or the output of this Ramsey function
𝑅(𝑚, 𝑛) for any given value of 𝑚 and 𝑛. It is only for certain values of 𝑚 and 𝑛 that we can
compute the value but there is no pattern or relationship or any observation which is there in the
output of the Ramsey function due to which we do not have any generic formula.
450
So that brings to the end of today's lecture. These are references used for today's lecture. Just to
summarize, in this lecture we started our discussion on counting. We introduced 2 fundamental
counting rules namely the sum rule and the product rule and we also discussed the pigeon-hole
principle. Thank you.
451
Module No # 07
Lecture No # 33
Permutation and Combination
Hello everyone, welcome to this lecture on permutations and combination.

Just to quickly recap, in the last lecture we started our discussion on combinatorics and we
discussed the basic counting rules like the sum rule and the product rule. So in this lecture, we will
recall the concepts related to your permutation and combination that you might have studied during
your high school. And we will also discuss about the combinatorial proofs.
452
So to being with, what is a permutation of a set of objects? As the name suggests, it is an ordered
arrangements of objects and when I say the ordered arrangement of the objects, that means the
ordering of the objects matter here. So for instance if I consider 2 persons, person number 1 and
person 2 then these two orders are different. If I consider person number 1 followed by person
number 2 then this order is different than the ordering where the person number 2 is appearing
before the person number 1.
So, we define what we call as 𝑟-permutation and 𝑟-permutation is nothing but an ordered selection
of 𝑟 elements from a set. So you are given a set which has certain number of elements, of course
it should have 𝑟 or more number of elements. If you select 𝑟 elements in an ordered fashion then
that is called an 𝑟-permutation and the number of such 𝑟-permutations from a set consisting of n
distinct elements is denoted by this quantity or this permutation function 𝑃(𝑛, 𝑟).
So you are given a set with 𝑛 distinct objects and we want to find out how many 𝑟-permutations I
can have from this set. So for instance if I consider 𝑛 = 3 and 𝑟 = 2 then 𝑃(𝑛, 2) = 6. Why?
Say you have 3 persons; person 1, person 2 and person 3. So you have a set of 3 objects or 3
persons here and I want to find out how many 2 permutations I can have; how many ordered
selection of 2 elements I can have from this collection.
453
So pictorially these are the 6 possible ordering. I can choose person number 1 followed by person
number 2, that's one ordering. I can choose person number 1 followed by person number 3, that's
another ordering. I can choose person number 2 followed by person number 1 as one of the
orderings.
I can have person number 2 followed by person number 2 as another ordering. In the same way I
can have person number 3 followed by person number 2 as one of the orderings and I can have
person number 3 followed by person number 1 as another order. So these are the different possible
2 permutations that you can have from this collection of 3 people. So now it's easy to see that if I
apply the product rule then I can derive the formula 𝑃(𝑛, 𝑟) = 𝑛 ∗ (𝑛 − 1) ∗ (𝑛 − 2) … (𝑛 − 𝑟 +
1).
Of course, for this formula to make sense you require your 𝑟 ∈ {1, … , 𝑛}; otherwise you get into
the issues of negative quantities. So how exactly we get from product rule the output of 𝑃(𝑛, 𝑟)
function to be this? So, you can imagine that I have 𝑟 slots to be occupied. And I have 𝑛 objects
to choose from. I have object number 𝑥! to 𝑥" . Now when it comes to the first slot, here I can put
either object number 𝑥! or object number 𝑥# or object number 𝑥" .
So that is why I have 𝑛 choice for the first object or first slot here. Now once I have decided which
of the 𝑛 objects to put in the first slot corresponding to that I have now 𝑛 − 1 options or 𝑛 − 1
objects to choose from to put in the second slot and so on.
By the way here repetitions are not allowed because right now I am considering the case of
selecting or forming 𝑟-permutations where in the permutations repetitions are not allowed. So that
is why when I am considering the second slot here. I can't consider the object which I have already
assigned to the first slot. So that is why I have only 𝑛 − 1 options instead of 𝑛 options to choose
from when it comes to the second slot. The formula becomes different if in the permutation that I
am forming repetitions are allowed. So now it is easy that by applying the product rule I get the
output of this 𝑃(𝑛, 𝑟) function to be this value.
454
Now, we can define 𝑃(𝑛, 0) = 1, namely, no way of permuting 0 objects. So if you are given 𝑛
objects and you don't want to select any objects or you don't want to permute any object then that
can be considered as 1 way of doing that. Because there is no way; so no way is considered as the
only way of permuting 0 objects. So that is why we define 𝑃(𝑛, 0) = 1. That is defined; it is not
coming from the product rule, that is coming as part of our definition.
So now we have the value of 𝑃(𝑛, 𝑟) where 𝑟 is non-zero and in the range 1 to 𝑛 and we have the
value of 𝑃(𝑛, 𝑟) when 𝑟 = 0.
So if I unify these 2 values I get that 𝑃(𝑛, 𝑟) = 𝑛!/(𝑛 − 𝑟)!. You can easily verify that.
455
Now you consider the case when we are selecting objects but while selecting the objects the order
does not matter. That means we are now interested in unordered selection of 𝑟 elements from a set
and each such unordered selection is called as an 𝑟-combination. So that means if I am now
selecting 2 objects out of 3 objects then it does not matter whether I pick 𝑥! before 𝑥# or whether
I pick 𝑥# before 𝑥! . So the arrangement (𝑥# , 𝑥! ) will be considered the same irrespective of
whether 𝑥! comes before 𝑥# or whether 𝑥# comes before 𝑥! .
So we use this notation 𝐶(𝑛, 𝑟). This is often treated as choosing 𝑟 objects from a set of indistinct
objects. And this function basically denotes a number of 𝑟 combinations that we can have for a set
which has 𝑛 distinct elements.
456
So again you must have studied it during your high school that the output of this function or the
$!
value of this function is nothing but &!("(&)!. So there are several ways of deriving that. The simplest
way could be to find out a relationship between the permutation function and the combination
function. So my claim is that 𝑃(𝑛, 𝑟) = 𝐶(𝑛, 𝑟) ⋅ 𝑃(𝑟, 𝑟). Why so?
Because if you see, the left-hand side it is nothing but the number of ordered selection of 𝑟
elements. So you are given a set with 𝑛 objects and you are interested to find out how many ordered
𝑟 permutations you can have over a set consisting of 𝑛 elements that's nothing but your function
𝑃(𝑛, 𝑟). My claim is that the number of ordered selection of 𝑟 elements is nothing but the
following.
You first find out the number of unordered selection of 𝑟 objects or equivalently find out how
many ways you can first select those 𝑟 elements which you want to order in your permutation. And
that you can do in 𝐶(𝑛, 𝑟) ways. Now once you have decided which 𝑟 elements you are going to
put in your permutation; right now you are considering the unordered selection of those 𝑟 elements,
if you have decided the 𝑟 objects then the number of ordered arrangements of those selected 𝑟
elements, if you multiply that with the number of unordered selection of 𝑟 elements that will give
you the total number of ordered selection of 𝑟 elements.
457
So it's like saying the following. You have {𝑥! , … , 𝑥" }. You first decide the 𝑟 objects for
permutation. Where the order does not matter as of now. This can be done in 𝐶(𝑛, 𝑟) ways. Now
once you have decided that I have selected object number 𝑥*! , 𝑥*" … 𝑥*# .
We have selected this 𝑟 objects. Every arrangement, every possible ordered arrangement of these
𝑟 elements will give you 1 possible permutation. And how many ordered arrangement of this 𝑟
objects you can have? You can have 𝑃(𝑟, 𝑟) such ordered arrangements. So that will give you the
total number of 𝑟 permutations that you can form from this subset of 𝑟 elements 𝑥*! , 𝑥*" … 𝑥*# .
So that's a relationship between the 𝑃 function and 𝐶 function. So now your goal is to find out the
value of the 𝐶(𝑛, 𝑟) function so you just take this 𝑃(𝑟, 𝑟) in the denominator. And 𝑃(𝑟, 𝑟) is
nothing but 𝑟!. That's how we get the value of the 𝐶 function.
Fine, so till now we considered the case of permutations and combinations where repetitions were
not allowed. Now we will consider the case where even in the selection the repetitions are allowed
as well. So we are now interested to first find out the number of 𝑟 permutations of a set of objects
where I am allowed to have repetitions. So for instance if I consider a set with 3 persons; person
1, person 2, and person 3.
458
Now if I ask you how many 2-permutations I can have over this set where, I can repeat the person
when I am forming the permutation. It now turns out that instead of 6 possible permutations I will
now 9 possible permutations. The 6 possible permutations which we had earlier where repetitions
were not allowed they will be still present. So those permutations are still present here.
So these were the 6 permutations which were there earlier when the repetitions were not allowed
but now since I am allowing you repetition I can have a permutation where I have 𝑃! followed by
𝑃! . I can have a permutation where I have 𝑃# followed by 𝑃# and I can now have a new permutation
where I have 𝑃+ followed by 𝑃+ . These are all allowed now because repetitions are allowed in this
case. So now again if I want to find out the number of 𝑟-permutations of a set of 𝑛 distinct elements
where repetitions are allowed then it turns out to be the product of 𝑛, 𝑟 number of times. Because
I have to fill 𝑟 slots and the first slot can be occupied in 𝑛 ways.
And for each of those 𝑛 ways in which I can occupy the first slot I can fill the second slot also in
𝑛 ways. Because repetitions are allowed. And now corresponding to each of the ways in which I
would have filled the first 2 slots, I have 𝑛 ways to fill the third slot and so on. So that is why the
total number of 𝑟-permutations of 𝑛 distinct elements that I can form is 𝑛& .
Now let's try to find out number of 𝑟 combinations where repetitions are allowed and this is slightly
tricky. So before going into the derivation of this formula let me give you a motivating example
459
and then we will try to relate this example with the problem of coming up with the number of 𝑟-
combinations where repetitions are allowed. So consider the following, you must have seen cash
box when you go to super market and do the billing.
So in cash box you have various slots available each slot is occupied with the currency of some
specific denomination. So imagine you have cash box which has 4 slots; the first slot has bills of
1 dollar right, many bills of 1 dollar, the second slot has many bills of 2 dollars, the third slot is
for 5 dollar bills and fourth slot is for 10 dollar bills. By the way in this problem when we are
finding the number of 𝑟-combinations with repetition, so I consider that the objects in the set from
which you want to find out the r-combinations each object has many copies available and each of
those copies are indistinguishable.
So for instance in this example I assuming that in the 1 dollar bill slot you might have several 1
dollar bills and all those 1 dollar bills are indistinguishable that means you can't say that if I choose
the 1 dollar bill which is on the topmost position then that will consider different from the 1 dollar
bill which is present at the bottom right.
So I won't consider those things here because I will be making the assumption here that the bills
of each denomination are indistinguishable. That is important while deriving the formula. Now
suppose my problem is the following, I want to select total 3 bills from the bills which are available
in the cash box and here order does not matter because I am interested to find out the number of
𝑟-combinations where 𝑟 = 3 here and 𝑛 = 4 here.
You have 4 types of objects namely object of type 1 dollar bill, bills of type 2 dollars, bills of type
5 dollars, and bills of type 10 dollars. And you have many copies of them. At least 3 copies of bills
of each denomination is available and your goal is to find out how many ways you can pick 3 bills
in total where the order does not matter. So for instance one way of picking 3 bills is you pick all
1 dollar bill that means you pick 3, 1 dollar bills or you may decide to pick 1 bill of 1 dollar and 1
bill of 2 dollar.
460
Or you may decide to pick one 5 dollar bill, one 2 dollar bill and one 1 dollar bill. Or you may
decide to pick all the 3 bills of 10 dollar type and so on. So these are the various ways. Our goal is
to find out how many such selections are possible? So for finding that consider the following
mapping. So what I do is I think in my mind that in your cash box the bills of various denominations
are separated by a boundary.
So you can imagine that boundary is nothing but a vertical line so you have a vertical line or a
boundary between the 1 dollar bills and the 2 dollar bills in the cash box. Similarly you have a
boundary here, you have a boundary here. Now suppose I pick bills of 1 dollar in 3 numbers. So
remember my goal is to pick 3 bills in total. So one way of doing that is I pick three 1 dollar bills.
So I represent this selection by saying that I have picked three 1 dollar bills.
So I put 3 cross under the heading 1 and I don't put any cross under 2 dollar denomination, 5 dollar
denomination, and 10 dollar denomination. So I can say that there is mapping here between a
possible way of selecting 3 bills. So here is a way of selecting 3 bills where I have picked three 1
dollar bills and corresponding to that I am giving you a string consisting of 3 crosses or 3 cross
and 3 vertical lines.
Now consider this way of selecting 3 bills; I pick two 1 dollar bills and I pick one 2 dollar bill. So
my claim is corresponding to that I can associate this string where I put 2 cross under 1 dollar
461
because I have chosen 2 bills of type 1 dollar and I put 1 cross under the 2 dollar denomination
because I have chosen 1 bill of 2 dollar. Under, 5 dollar and 10 dollar I do not put any cross. So I
can say that corresponding to this way of selecting 3 bills I can associate this string of 3 cross and
3 vertical lines.
In the same way if my way of picking 3 bills is the following namely picking one 1 dollar bill, one
2 dollar bill, and one 10 dollar bill then the corresponding string will be, you put 1 cross under 1
dollar 1 cross under 2 dollar and 1 cross under 10 dollar.
462
And now I can say that I have a bijective mapping, namely one-one and onto mapping. So whatever
mapping I have discussed till now my claim is that, that is a bijective mapping; namely a one-to-
one onto mapping between the set of all possible ways of selecting 3 bills out of bills of 4
denominations in an unordered fashion. And a set of all strings of length 6 where there are 3 crosses
and 3 vertical lines. My claim is that the mapping that I have discussed is a bijection.
Of course it is an injective mapping; from this direction to this direction. Why so? Because, you
take 2 different ways of selecting 3 bills the corresponding string of 3 cross and 3 vertical lines
will be different. And my claim is that this mapping is surjective as well. You give me any string
of 3 cross and 3 vertical lines; I can tell you a corresponding way of picking 3 bills. So for instance,
if you give me a string like say, “|xxx|”, then what is the corresponding way of picking 3 bills here?
So here no bill of type 1 dollar will be picked and all the 3 bills of 2 dollar type will be picked and
no bill of 5 dollar will be picked and no bill for dollar for 10 dollar will be picked. So this mapping
is surjective as well. So since I have 2 sets here and I have defined the mapping one-to-one namely
injective and surjective as well, this mapping is bijective function. And that means the total number
of way of selecting 3 bills is equivalent to finding the total number of string of length 6 which has
3 cross and 3 vertical lines.
And it's easy to see the number of such strings is this. Why? Because your string is of length 6 and
you have to choose 3 positions where the cross has to appear. Because once you decide the 3
positions where the cross is going to appear, automatically at the remaining 3 positions from the
problem definition the vertical line will be present. So you don't have to worry about the positions
or the 3 positions where the vertical lines are going to appear because once you have chosen the 3
positions where the cross are going to appear the remaining things are automatically frozen for
vertical line to appear.
And why I am writing it in this form “4 - 1 + 3” because in general, the general formula for general
𝑛 and 𝑟 will be 𝐶(𝑛 − 1 + 𝑟, 𝑟). Because how many vertical lines will be there if the number of
objects of various types is 𝑛? So you will have 𝑛 slots because you have objects of 𝑛 types.
463
So you will have object of type 1 and then there will be a vertical line and then you will have object
of type 2, you will have 1 vertical line, and like that you have an object of type 𝑛. So how many
vertical lines will be there 𝑛 − 1 and how many cross positions you have to fill? You have to fill
𝑟 number of cross positions. So that is why the length of the string will be 𝑛 − 1 + 𝑟 because it
will have 𝑛 − 1 number of vertical lines and 𝑟 number of crossings.
So that is why the general formula for 𝑟-combinations where repetitions are allowed is
𝐶(𝑛 − 𝑟 + 1, 𝑟).
So now let's see some examples of combination with repetitions. This is a very interesting formula
which is used in lots of counting problems. So suppose I want to find out the number of integer
solutions for the equation 𝑥! + 𝑥# + 𝑥+ = 11 where𝑥! , 𝑥# , 𝑥+ are allowed to be greater than equal
to 0. So my claim is that this, the number of solutions is nothing but the number of all strings of
11 crosses and 2 vertical lines.
This is because you can interpret this formula; you can interpret this problem as the following.
You have bills of type 𝑥! , bills of type 𝑥# and bills of type 𝑥+ . Your goal to pick total 11 number
of bills. You can pick all the 11 bills of type 𝑥! . That is one possible solution which corresponds
to 𝑥! = 11 and 𝑥# = 0 and 𝑥+ = 0. Or you can pick 10 bills of type 𝑥! and 1 bill of type 𝑥#
which corresponds to 𝑥! in 10 and 𝑥# being 1.
464
Or you could have 𝑥! = 6, 𝑥# = 4 and 𝑥+ = 1 which corresponds to picking 6 bills of type 𝑥! , 4
bills of type 𝑥# and 1 bill of types 𝑥+ . And here order does not matter. So that's why your problem
now reduces to picking total 11 number of bills from bills of 3 possible denominations where order
does not matter, and repetitions are allowed and we have derived just now that the formula for that
is nothing but this quantity.
Now suppose I change the problem slightly. I am now interested to find out the number of solutions
where 𝑥! ≥ 1, 𝑥# ≥ 2 and 𝑥+ ≥ 3. Earlier it was allowed that 𝑥! = 0 but now 𝑥! is not allowed
to be 0. 𝑥! has to be at least 1, 𝑥# has to be at 2, 𝑥+ has to be at least 3. So my claim is that this is
equivalent to finding the number of solutions for a new equation, 𝑥! + 𝑥# + 𝑥+ = 5 where there is
no restriction on 𝑥! , 𝑥# , 𝑥+ .
That means any of them can be 0. Why so? Because again if I consider the bill analogy you have
bills of denomination 𝑥! , 𝑥! type bills, you have 𝑥# type bills and you have x_3 type bills. Now
𝑥! ≥ 1 means definitely I have to choose 1 bill of type 𝑥! . My goal is to pick 11 bills right out of
those 11 bills 1 bill has to be of type 𝑥! . 2 bills have to be of type 𝑥# and 3 bills have to be of type
𝑥+ .
465
That means the problem already states that I have already chosen 6 bills definitely I have chosen
1 bill of type 𝑥! and 2 bills of type𝑥# . So total 3 bills I have already chosen. And 3 bills of type 𝑥+
that means 3 more bills are already chosen. So 3 + 3 = 6 so 6 bills are already chosen. My goal
was to pick 11 bills. So now I am left with the problem of choosing 5 bills and now for choosing
these 5 remaining bills I have no restriction.
I can pick all of them of type 𝑥! or all of them of type 𝑥# or all of them of type 𝑥+ . Or 1 bill of type
𝑥! and 4 bills of type 𝑥# or 1 bill of type 𝑥! , 2 bills of type 𝑥# and 2 bills of type 𝑥+ and so on.
And now we know how many ways I can satisfy this equation where there are no restriction on
𝑥! , 𝑥# , 𝑥+ . That will be nothing but 𝐶(3 − 1 + 5, 𝑟).
466
Now let us go to the last topic for today's lecture namely combinatorial proofs and again I am sure
that you have studied it during your high school. So what exactly are combinatorial proofs. This
are some common proof strategy which we often use in combinatorics. Namely it's a counting
argument to prove identities where you have something on the left-hand side and something on
your right-hand side and you want to prove mathematically that your expression in the left-hand
side and the right-hand side are same.
But to do that we use a counting argument and we prove that the expression in the left-hand side
and the expression on the right-hand side count the same number of objects but in different ways.
But nowhere in the proof we actually expand our expressions on the left-hand side or right-hand
side and show by simplification that left-hand side is same as right-hand side. We do not do that.
That is not the goal of a combinatorial proof. So let us see a very simple combinatorial proof which
you must have definitely studied. We will want to prove that the value of 𝐶(𝑛, 𝑟) is the same as
the value of 𝐶(𝑛, 𝑛 − 𝑟). Of course one way of doing that is I expand 𝐶(𝑛, 𝑟) and rewrite it as
$!
. And I expand my right-hand side. And in this case actually both the expressions are same.
&!("(&)!
So I could have simply said that they are same. But that is not the goal of combinatorial proof. In
general when we are giving a combinatorial proof for proving LHS and RHS are same we do not
expand or simplify the expressions in the left-hand side and right-hand side. If you do that, that's
467
not a combinatorial proof. You will get 0 marks if you are asked to prove something by
combinatorial proof and you end up simplifying expressions.
The way we are going to prove this equality using combinatorial proof, is the following. Suppose
you are given 𝑛 objects then your left-hand side is nothing but the number of ways in which you
can pick 𝑟 objects out of those 𝑛 objects. That's the interpretation of 𝐶(𝑛, 𝑟) function. Now it turns
out for each of the ways in which you can select 𝑟 objects there is a way of excluding 𝑛 − 𝑟 objects.
So I can reinterpret my problem and say that instead of worrying about how many ways I can pick
𝑟 objects out of 𝑛 objects, I instead count the number of ways I will decide or the number of ways
I will choose the 𝑛 − 𝑟 objects which I want to leave. Because once I decide that these are the 𝑛 −
𝑟 objects which I am going to leave that automatically gives me the r objects which I will be taking
or considering.
So that's why it is easy to see that the LHS expression and RHS expression are same. And we are
counting 2 different things here. The left-hand side expression basically counts the number of ways
you would have selected the objects. Whereas the right-hand side expression counts the number
of ways in which you would have left objects. And there is a mapping. Whatever you left
corresponding to that you are left with object which you are picking.
468
Now let's prove an interesting combinatorial identity using combinatorial proof. This is often
called as Pascal's identity. So to prove this consider a collection of 𝑛 + 1 objects. So I am calling
those 𝑛 + 1 objects as say 𝑥! to 𝑥",! they are distinct. Now what is my left-hand side expression?
That denotes the total number of 𝑘-combinations I can have out of those 𝑛 + 1 objects. I have to
pick 𝑘 objects. I can do that in 𝐶(𝑛 + 1, 𝑘) ways. That is the left-hand side.
Now I have to show that the same thing can be counted by adding these 2 quantities that are there
in the right-hand side. How do I do that? So my claim is that different 𝑘-combinations that I can
have out of 𝑥! to 𝑥",! can be divided into 2 groups. That will take care of the addition that we
have on your right-hand side expression. I have to somehow show that the total number of different
𝑘-combinations that I can have can be divided into 2 categories, 2 disjoint categories to be more
specific and those 2 disjoint categories are the following.
You consider all k-combinations that you can form out of those 𝑛 + 1 objects where a specific
object is always present. Say the object 𝑥! is always present. And the number of such 𝑘-
combinations is nothing but 𝐶(𝑛, 𝑘 − 1). Because if the object 𝑥! is always going to be included
in the 𝑘 objects which you are finally choosing, then you have to worry about how many ways you
can pick the remaining 𝑘 − 1 objects out of 𝑥! to 𝑥" .
469
So you had 𝑥! to 𝑥",! so you are always going to choose 𝑥! that is the category we are right now
considering. So now you are left with 𝑛 objects namely 𝑥# to 𝑥",! and you have to choose 𝑘 − 1
objects out of this remaining 𝑛 objects which you can do in these many ways. In each such 𝑘-
combination you include the object 𝑥! that will give you category 1 of 𝑘-combinations.
Whereas category 2 𝑘-combinations are the one where the object 𝑥! is never included. And it is
easy to see that the number of different 𝑘-combinations of this category is 𝐶(𝑛, 𝑘). Because if you
are not going to include𝑥! then your problem is still to choose 𝑘 objects and now you are left with
only 𝑛 objects to choose for those 𝑘 objects. You can choose your 𝑘 objects only from the
collection 𝑥# to 𝑥",! .
So you are left with only with 𝑛 possibilities and the number of 𝑘-combinations that you can now
have in the second category is this. And now if I focus on the total or the different 𝑘-combinations
that I can have out of this 𝑥! to 𝑥",! , I can have either a 𝑘-combination of category 1 or a 𝑘-
combination of category 2. Namely in the 𝑘 combination either, 𝑥! is there or 𝑥! is not there. I
cannot have any third possible category.
And these 2 categories are disjoint. There is no 𝑘-combination where 𝑥! is present as well as 𝑥! is
absent. So if I sum the total number of 𝑘-combinations that I have in category 1 and the number
of 𝑘-combinations that I have in category 2 that will give me the total number of 𝑘-combinations
that I can have for a set consisting of 𝑛 + 1 objects. And that's precisely your right-hand side. And
this is a combinatorial proof because now I have not expanded my left-hand side expression, I have
not expanded my right-hand side expression and simplified them. I am just giving a counting
argument and proving that LHS and RHS are counting the same things.
470
So that brings me to the end of today's lecture. These are the references used for the today's lecture.
Just to summarize, in this lecture we introduced permutations, combinations, we saw the formula
for permutations and combinations both with repetitions and without repetition. And we also
discussed about combinatorial proofs.
471
Module No # 07
Lecture No # 34
Counting Using Recurrence Equations
Hello everyone, welcome to this lecture on counting using recurrence equations.

Just to recap, in the last lecture we discussed about the rules of permutations and combinations
used for counting. We also discussed about permutations with reputations and combinations with
repetitions. And we also discussed about combinatorial proofs. So in this lecture we will introduce
a new counting technique which is extensively used in discrete mathematics and in computer
science and this is basically counting using recurrence equations.
472
So it turns out that there are plenty of instances or counting problems which gets significantly
simplified by recurrence equations. And to recap, what exactly is a recurrence equation? So you
must have encountered different equations of the following form: 𝘧𝘢𝘤𝘵(𝑛)! = 𝑛 ∗ 𝘧𝘢𝘤𝘵(𝘯 − 𝟣). So
this is a recursive function in the sense that the value of the factorial function on input 𝑛 is
expressed in terms of the value of the factorial function on smaller inputs.
Similarly we are familiar with the famous Fibonacci function where we know that 𝘧𝘪𝘣(𝑛) =
𝘧𝘪𝘣(𝑛 − 1) + 𝘧𝘪𝘣(𝑛 − 2). So again this is an example of recurrence equation. So, our idea here
will be that now we would like to count the number of things by formulating recurrence equations.
And later we will discuss how to solve those recurrence equations.
473
There are many counting problems which gets significantly simplified by formulating the
recurrence equations. So let us see an example here to clarify my point. So imagine I want to find
out the number of 𝑛 bit strings that do not have an occurrence of 2 consecutive 0’s. That means
the substrings 00 is not allowed to appear in such a string and we want to find out how many such
strings can be there.
And we want the general answer namely we want to find the number of such strings for any 𝑛. So
the way we are going to count this is as follows. We say that let 𝐴(𝑛) be a function which on input
𝑛 gives you the number of 𝑛 bit strings which do not have two consecutive 0’s. That is the
definition of my 𝐴(𝑛) function. So if I say 𝐴(1) that will give me the number of bit strings of
length 1 that will not have the occurrence of 2 consecutive 0’s.
𝐴(2) will give me the value of number of strings of length 2 that do not have occurrences of 2
consecutive 0’s and so on. That is a definition of my function 𝐴(𝑛). Now we want to find out what
exactly will the 𝐴(𝑛) function look like or what will be the output of the function 𝐴 on input 𝑛.
And I want to set up a recurrence equation for that. That means I would like to express the value
𝐴(𝑛) in terms of the output of 𝐴 function on small size inputs.
474
So for that what we are going to see here is the following. If we consider any 𝑛 bit string which do
not have 2 consecutive 0’s then there are only 2 possibilities for the starting bit of such a string.
The starting bit of such a string could be 1 in that case the remaining 𝑛 − 1 length substring should
not have any occurrence of 2 consecutive 0’s. Because if 2 consecutive 0’s occur anywhere in the
remaining substring of length 𝑛 − 1 then the overall string cannot be a valid string of length 𝑛 bits.
And without having any occurrence of 2 consecutive 0’s that is not going to happen. So that is one
possibility.
The second possibility could be that the string of length 𝑛 starts with 0. If the string starts with 0
and if we want that overall string should not have any occurrence of 2 consecutive 0’s. Then
definitely the second position of that string should be 1. Because if the second position of such a
string is also 0 and anyhow the first position is 0 then that is a violation. Violation of the property
that the string has no occurrence of 2 consecutive 0’s so definitely the second bit position has to
be one.
And now what can I say about the remaining 𝑛 − 2 bits in this string. I can definitely say that the
remaining 𝑛 − 2 bits in the string should not have any occurrence of 2 consecutive 0’s. Because if
00 occurs or appears anywhere in the remaining substring of length 𝑛 – 2 then the overall string
cannot be considered as a valid string of length 𝑛.
475
And now you can see that these are the only 2 categories of string of length 𝑛: either the string can
start with 1 or the string can start with 0. So now let us try to count the number of strings in both
the categories. If I consider category 1 then the number of strains of this category is 𝐴(𝑛 − 1).
Why so?
Because I am interested to count the number of strings of length 𝑛 − 1 which do not have an
occurrence of two consecutives 0’s and as per the definition of my 𝐴 function the number of such
string is nothing but 𝐴(𝑛 − 1). You take any string of length 𝑛 – 1 which do not have an
occurrence of two consecutive 0’s and you put a 1 at the beginning of such a string that will give
a valid string of length 𝑛 that do not have an occurrence of 2 consecutive 0’s.
So that is why I will have 𝐴(𝑛 − 1) number of strings in this category. And how many strings in
the second category? My claim is that the number of strings in this category is nothing but
𝐴(𝑛 − 2). Because as, per the definition of my 𝐴 function the number of strings of length 𝑛 − 2
which do not have an occurrence of two consecutive 0’s is 𝐴(𝑛 − 2). And you take any string of
length 𝑛 − 2 bits which do not have an occurrence of two consecutive 0’s and you put 0 and 1 at
the beginning of such a string that will give you a valid string of length 𝑛 bits which do not have
an occurrence of two consecutive 0’s. And as I said these are the only 2 categories of strings of
length 𝑛 which do not have an occurrence of two consecutive 0’s in terms of that the number of
476
strings of length 𝑛 will be the output of A function on input 𝑛 − 1 and the output of A function on
the input 𝑛 − 2 i.e., 𝐴(𝑛) = 𝐴(𝑛 − 1) + 𝐴(𝑛 − 2).
However the argument or the discussion that we had holds only if 𝑛 ≥ 3. Because if I take 𝑛 = 2,
then this argument would not hold because we 𝐴(0) upon substitution. A(0) does not make any
sense since, as per the definition of my 𝐴 function, it is the number of strings of length 0 which do
not have occurrence of two consecutive 0’s.
So that is why this definition of this recursive function holds for all 𝑛 ≥ 3. So now you might be
wondering what about the output of the 𝐴 function on inputs which are less than 3. So we will be
giving some initial conditions. They are called initial conditions because the recursive functions
are not applicable for the case when 𝑛 = 1 and 𝑛 = 2.
Because if 𝑛 = 1 then the number of bit strings of length 1 which do not have an occurrence of 2
consecutive 0 is 2. Because both the strings, 0 as well as the string 1, will be considered as valid
string. There is no occurrence of consecutive 0’s in this string, there is no occurrence of
consecutive 0’s in this string. Similarly, if I consider 𝑛 = 2 then the possible strings of length 2
which do not have occurrences of 2 consecutive 0’s are 11, 10 and 01.
These are the valid string and how many strings you have? 3. That is why 𝐴(2) = 3. But for any
𝑛 ≥ 3 I can find out the value of 𝐴 of that particular 𝑛 by using this recurrence equation.
477
So now we know how to count using recurrence equations. The next thing that we would like to
do now is how to solve those recurrence equations? And when I say I want to solve a recurrence
equation I mean finding a closed-form formula for that recurrence equation. So what exactly that
means? So, suppose someone gives me this recurrence equation. The recurrence equation is
𝐻(𝑛) = 2𝐻(𝑛 − 1) + 1 for all 𝑛 ≥ 2 and initial condition is that 𝐻(1) = 1.
Now, if suppose someone asked me can you find the value of this H function on the input 100. I
can say well that is not difficult. As per my relation 𝐻(100) will be 2𝐻(99) + 1 and 𝐻(99) will
be 2𝐻(98) + 1 and so on. But then this is a time consuming affair. I would not be able to do this
or solve this very quickly using paper-pen. So that is a difficult thing. So in fact 𝐻(100) you still
might be able to do but what if I ask to compute say 𝐻(10000).
Then you will be stuck; you cannot find out the value of 𝐻(10000) very quickly. Now suppose I
tell you that 𝐻(𝑛) = 2! – 1. We will verify if this is indeed the case or not. Now, as per this formula
𝐻(1) = 2" – 1 = 1 and it is also easy to verify that 2! – 1 = 2(2!#" - 1) + 1. That means I can
say that this H(n) function satisfies this recurrence condition. Satisfies in the sense it has all the
properties or characteristic that are specified by the recurrence equation.
If I substitute 𝑛 = 1 here I get the same initial condition as given in the recurrence function and
indeed this 2! – 1 is a function where if I substitute 𝑛 being 𝑛 − 1 then I get this recurrence
condition. That means this 𝐻(𝑛) function will be considered as a solution for this recurrence
478
equation. And why it is called as solution for this recurrence? Because it is a closed-form formula.
Closed-form formula means it is just a function of 𝑛. Here 𝐻(𝑛) does not depend on the value of
𝐻 function on previous inputs.
It is a function of variable 𝑛. You just substitute the value of 𝑛 you will get the answer. That means,
now, if I ask you what will be the value of 𝐻(100) you do not have to follow the complicated
process. That means for computing 𝐻(100) you need not have to compute 𝐻(99), you need not
have to compute 𝐻(98), you need not have to compute 𝐻(97) and so on. You will just go and
substitute 𝑛 = 100 in this function and get the answer and that is all.
That is what we mean by solving a recurrence equation. So when I say I want to solve a recurrence
equation by that I mean I am interested to find out a closed-form formula for that recurrence
equation. Namely a function; in 𝑛 which satisfies that the recurrence condition.
So now we will discuss about the general methods of solving recurrence equations. It depends
upon the type of the recurrence equation. It is not the case that the same method of solving the
recurrence equation will be applicable for every category of recurrence equation. So first of all you
have to understand here that a recurrence equation basically specify a sequence of values.
So the terms recurrence condition, recurrence equation, recurrence function etc., will be used
interchangeably. All these are equivalent terms. So whenever I am specifying a recurrence
479
equation basically I am talking about an infinite sequence of values. And the recurrence equation
basically specifies the characteristic of the n-th term of that sequence.
That n-th term you might either represent as 𝐻(𝑛) where 𝐻 is a function or you might omit the
parentheses and you can instead write down 𝑛 in the subscript. Both these notations are equivalent.
So, for instance we know that Fibonacci sequence is defined as follows: the first term is 0, the
second term is 1, and then after that the next term is the summation of the previous 2 terms.
If I somehow want to specify the n-th term of this sequence, I can use 2 notations. I can either use
the notation 𝑓! to denote the n-th term of this infinite sequence. So remember this Fibonacci
sequence is an infinite sequence. Or I can use the function 𝑓(𝑛); both means the same thing. If I
am using this notation 𝑓! then I will use this notation: 𝑓! = 𝑓!–" + 𝑓!–% to denote that the n-th term
of the sequence is related to the previous 2 terms of the sequence.
Whereas if I am using the function notation for denoting the n-th term of this infinite sequence
then I will characterize this property by saying that 𝑓(𝑛) = 𝑓(𝑛 − 1) + 𝑓(𝑛 − 2). Both these
interpretations are equivalent.
And that is why for the rest of our discussion on recurrence equation I will be interchangeably
using the terms recurrence function or the n-th term of a sequence and so on. So whenever we are
talking about a recurrence function we are talking about an infinite sequence and we are interested
to specify the property of the characteristic of the n-th term of that sequence by saying how exactly
it is related to the previous terms of the same sequence.
So somehow someone gives me this infinite sequence; now I want to characterize the property of
this sequence.
480
That means what exactly is the characteristic of the n-th term of this sequence. So it is easy to see
that the n-th term of this sequence satisfies the following characteristics. The first term of the
sequence is 1 and if you take the n-th term for any 𝑛 ≥ 2, it is twice the (𝑛 − 1)-th term, that
means twice the previous term plus 1. So for instance if you take 7 where 𝑛 = 3, 7 = 2 ∗ 3 +
1 i.e., it is one more than twice the previous term. Similarly, the fourth term 15 = 2 ∗ 7 + 1
i.e., it is one more than twice the previous term.
So that is what is the interpretations of a recurrence equation. If someone gives me a recurrence

equation and does not give me the infinite sequence; solving a recurrence equation basically means
finding the sequence which satisfies that recurrence condition. So, someone tells me “solve this
recurrence equation” without telling me the infinite sequence then my goal will be to find out that
infinite sequence.
But I would not be finding the whole sequence because it is infinite, instead I will just find out the
closed-form formula for the n-th term of that sequence as a function of 𝑛. That is what I mean by
solving a recurrence equation. It turns out that a recurrence equation may have more than 1
solution. So for instance if I take this recurrence equation namely I am talking about an infinite
sequence where the n-th is the difference of twice the previous term and the previous-to-previous
term.
481
For all 𝑛 ≥ 2, this sequence has this recurrence condition being satisfied. Now my goal is to find
out a possible infinite sequence whose n-th term has this characteristic for every 𝑛 ≥ 2. So I will
now show you 2 possible sequences. So consider this infinite sequence whose n-th term is 𝑎! =
3𝑛. You can easily verify that this infinite sequence satisfies this recurrence condition. You take
any term, say 6, it is indeed twice the previous term minus the previous-to-previous term.
So 6 = 2 ∗ 3 – 0. The term 9 = 2 ∗ 6 – 3 and so on. So, this is one of the infinite sequences
which satisfies this recurrence condition. Now you take another infinite sequence namely the
sequence where all the terms are 5 i.e., 𝑎! = 5. This is also a sequence where the values in the
sequence are completely different from the values in the upper sequence.
But the second sequence also satisfies the recurrence condition. You take any term of the sequence
it will be 5. And 5 is indeed twice the previous term which is also 5 minus previous-to-previous
term which is also 5. This shows that if someone gives me a recurrence condition then there can
be multiple solutions or multiple sequences satisfying that recurrence condition.
So we will kick start our discussion with some simple methods of solving recurrence equations.
We will discuss advanced methods in our later lectures. So we will discuss the simplest method
called as the iterative method. What exactly we do in the iterative method? So consider this
recurrence equation. I am supposed to find out an infinite sequence whose zeroth term or the
starting term is 2 and from the next term onwards the term is the pervious term plus 3.
482
So I have to find one such sequence satisfying this condition. And when I say I want to find out
the sequence as I said I would not find the entire sequence but rather I will find a closed-form
formula characterizing the n-th term of that infinite sequence. So this is how we can do that using
iterative method. So we starting with 𝑎& which is the starting term. It is given to be 2. Then we
will say that as per our recurrence condition the second term of the sequence will be the first term
plus 3. And now we have already obtained the value of 𝑎" = 2 + 3, so I am substituting that. And
if I substitute overall then I get 𝑎% = 2 + 2 ∗ 3. And now if I want to find out 𝑎' , again I will apply
the recurrence condition. 𝑎' will be summation of a_2 and 3. 𝑎% I have already obtained so I can
substitute here and now you see that I start getting a pattern here that 𝑎! = 2 + 3𝑛. That is the
general pattern that I am getting here and that is why if I solve and I keep on solving then I will
get the fact that 𝑎! = 2 + 3𝑛. And this is now a closed-form formula. Why it is a closed-form
formula? Because, here the value of 𝑎! is just a function of 𝑛.
You substitute 𝑛 = 2, 𝑛 = 3; you get whatever we have derived till now. So this will be the
solution of this recurrence equation. I can get the same closed-form formula as follows. So here
what I did I start with the initial condition and derived the n-th condition. I can do it in the reverse
direction as well. I will say that as per the recurrence condition 𝑎! is this, then I apply the
recurrence can be condition again on 𝑎!#" .
I will say that as per the recurrence condition 𝑎!#" = 𝑎!#% + 3 + 3 and 𝑎! = 𝑎!#% + 2 ∗ 3. Now
I will again solve 𝑎!#% by applying the recurrence condition and I will get the value of 𝑎! in terms
of 𝑎!#' . And now if I continue like this I’ll obtain the initial condition which is 𝑎& and if I stop at
𝑎& then I will get a closed-form formula for 𝑎! .
So now you can see that it does not matter whether I go from 𝑎& to 𝑎! or whether I go from 𝑎! to
𝑎& . In both the cases I obtain the same closed-form formula. The first method here is called the
forward substitution iterative method. The second process here is called as the backward
substitution iterative method. So this is one of the simplest methods of solving recurrence
equations.
483
Now the recurrence equation that we considered here can be categorized as a linear homogenous
recurrence equation of degree 𝑘 with constant coefficient. And why we are interested in this class
of recurrence equations is that we will next see a generic method of solving this category of
recurrence equations. So what exactly is this category of linear homogenous recurrence equations
of degree 𝑘 with constants coefficients.
So there are multiple terms here so let us, decode each of them one by one. The general form of
recurrence equations in this category is the following. You will have 𝑎! being expressed as the
linear function of previous terms. So that is why the name linear. So you can see that here 𝑎! here
is a linear function of previous terms. What is the linear function?
Because the combiners namely 𝑐" , 𝑐% , 𝑐( are constants here and 𝑎! depends linearly on previous
term. Namely, it depends only the first power of 𝑎!#% , only the first power of 𝑎!#% , first power of
𝑎!#( and so on. Why it is called homogeneous because the dependency of 𝑎! is only on the
previous terms and nothing else. We will see what exactly non-homogenous means very soon.
Why it is called degree 𝑘? Because 𝑐( is not allowed to be 0 here. If 𝑐( = 0 then 𝑎! does not
depend on 𝑎!#( . In that case the degree will not be called 𝑘. 𝑐" , 𝑐% … 𝑐(#" are allowed to be 0. But
𝑐( cannot be 0 if the degree is 𝑘. That is what the degree means here. It is similar to what we say
in the context of polynomials. If I say I have a polynomial in 𝑥 of degree 𝑡 then the coefficient of
𝑥 ) should be non-zero. The coefficients of other powers of 𝑥 may be 0. But since the degree is 𝑡
484
the coefficient of 𝑥 ) cannot be 0. In the same way when I say degree 𝑘 this 𝑐( is not allowed to be
0. And the constants coefficients basically say is that my combiners are constants here; 𝑐" to 𝑐( .
So let us see some examples of this category of recurrence equations. Your well-known Fibonacci
function or Fibonacci recurrence equation falls in this category. Here the degree is 2. Because 𝑓(𝑛)
depends on the previous 2 terms. Now the equation 𝑎! = 𝑎!#" + 𝑎!#% % is not falling in this
category. The problem is that 𝑎! does not depends linearly on 𝑎!#% . It depends quadratically on
𝑎!#% that is why it is a non-linear function. Whereas this third function or third equation is not
falling in this category because we have a non-homogenous component here.
Now the next thing that we want to show here is that, I said few slides back that if you are given a
recurrence equation then there might be multiple solutions for that recurrence equation and we saw
an example as well. Now the question is, is it possible that a given recurrence equation has a unique
solution. Well it is possible provided you are given initial conditions as well. More specifically,
imagine you are given a linear homogenous equation of degree 𝑘. That means 𝑐( is not 0.
Then my claim is that if you are also given 𝑘 initial conditions, that means you are given the value
of say 𝑎& , 𝑎" and 𝑎(–" , if these values are explicitly given to you which we call as initial conditions
then there always exists a unique solution satisfying the recurrence condition as well as the initial
conditions. See, the example that we considered, there the initial conditions were not given. You
just were given the recurrence equation and that is why we found 2 different sequences satisfying
485
the same recurrence conditions. But what I am saying here is that apart from the recurrence
equation you are also given the initial conditions and my claim is that only when you are given
those many initial conditions which are same as the degree of your recurrence equation then you
always have a unique solution.
And this can be proved very easily using strong induction. So my claim is that there is only a
unique solution satisfying the initial conditions as well as the recurrence condition. This is because
since you are given the initial conditions namely the 𝑘 initial conditions the first 𝑘 terms are
explicitly given to you. You cannot substitute the first 𝑘 terms arbitrarily. Now since the first 𝑘
term are given to you, I can say that by applying the recurrence condition on the first 𝑘 terms I get
the next term.
The next term cannot be an arbitrary term because my reference condition explicitly says that what
will be the term V( once I have frozen 𝑉& , V" … V(–" .
In the same way, once I have frozen or decided that what are 𝑉" , 𝑉% … 𝑉(–" and 𝑉( . That means
these are the 𝑘 terms which I am considering. As per the recurrence condition by applying the
linear function or the recurrence function here, that automatically freezes the next term which is
𝑉(*" . And now this process keep on going. You take the next 𝑘 terms apply the linear homogenous
function, that will give you the unique value of 𝑉(*% and so on.
486
That means since my initial conditions are fixed and the number of initial conditions that are given
to me are same as the degree of your equation, I will be getting a unique sequence satisfying the
initial conditions as well as the recurrence condition. If you take the example where I showed you
2 sequences satisfying the same recurrence condition that was because you were not given any
initial conditions.
And that is why I can fill any initial conditions or any initial terms in those sequences. So in the
previous example you were just given the recurrence condition, you were not given the initial term.
So that is why I was freezing the initial terms here in 2 different ways. In my first sequence my
initial terms were 0 and 3.
And as soon as I froze the initial sequence or initial terms to be 0 and 3 that automatically froze
my entire sequence. Whereas if I fit the first 2 initial terms with 5 and 5 and then apply the
recurrence equation that will give me another sequence. So it is due to the absence of the initial
conditions that I was getting 2 different solutions or sequences for the same recurrence condition.
But if you are given the initial conditions as well and the number of initial conditions if they are
same as the degree then there will be a unique sequence or solution satisfy your recurrence
condition.
487
So that brings me to the end of today’s lecture. These are the references used for today’s lecture.
Just to summarize, in this lecture we introduced a new counting method called counting by
recurrence equations. And we discussed a very simple method of solving linear equations (as well
as non-linear equations depending upon the structure of the equation) called as the iterative method
and we saw the forward substitution as well as the backward substitution method under the iterative
method. We also discussed about an important category of recurrence equations namely linear
homogenous recurrence equations of degree 𝑘 with constant coefficients. Thank you.
488
Module No # 07
Lecture No # 35
Solving Linear Homogenous Recurrence Equations – Part I
Hello everyone, welcome to this lecture on solving linear homogenous recurrence equations part
1.
So just to quickly recap, in the last lecture we discussed how we can solve counting problems by
formulating recurrence equations and we also started discussing about how to solve the recurrence
equations. Because when you want to count certain number of things using recurrence equations
then there are two parts. First thing is formulating the recurrence equation and the second thing
will be finding the closed-form formula or the solution for the recurrence equation.
Because, until and unless you do not have a closed-form formula you may not be able to come up
with a solution. You have to solve the recurrence equation. So we already discussed the iterative
method in the last lecture. In this lecture, we will continue our discussion on solving linear
homogenous recurrence equations. And we will discuss one category, namely when we have non-
repeated characteristics roots.
489
So just to quickly recap, what exactly are linear homogenous reference equations of degree 𝑘? The
general form is 𝑎! = 𝑐" 𝑎!#" + 𝑐$ 𝑎!#$ + ⋯ + 𝑐% 𝑎!#% . You have an infinite sequence where the
n-th term of the sequence depends upon the previous 𝑘 terms i.e., 𝑎! is always dependent on 𝑎!#% ,
or in other words 𝑐% ≠ 0. The recurrence equation for the Fibonacci sequence is an example of
linear homogenous equation.
$
𝑎! = 𝑎!#" + 𝑎!#$ is a non-linear equation and ℎ! = 2ℎ!#" + 1 is a non-homogenous equation.
So we are interested to come up with the general method for solving recurrence equations of this
type.
490
So, I will first demonstrate the process assuming that we have a linear homogenous recurrence
equation of degree 2. When I say degree 2 that means the n-th term of that infinite sequence which
I am interested to find out depends upon the previous two terms, namely, it depends on 𝑎!#" and
𝑎!#$ where 𝑐$ ≠ 0. 𝑐" can be 0 but 𝑐$ definitely cannot be 0.
And you may or not may not be given initial conditions. So again, recall in the last lecture we
discuss that if you are not given initial conditions then there could be multiple number of sequences
or solutions satisfying the recurrence condition. Because since the initial conditions are not given
you are free to put any value as the initial condition; any term as the initial condition. And if once
you freeze that initial conditions, that determine what will be the remaining terms of the sequence?
So in this case I am assuming that you are given the initial conditions. Say the initial conditions
are 𝑎& = 𝑉& and 𝑎" = 𝑉" . So the first step here will be to construct what we call as characteristic
equation and the characteristic equation will be an equation in an unknown 𝑟.
So 𝑟 is an unknown variable here whose value is not known. This characteristic equation will be a
quadratic equation in 𝑟. Why quadratic? Because right now we are considering degree 2 recurrence
equations. And the form of the characteristic equation will be 𝑟 $ – 𝑐" 𝑟– 𝑐$ = 0. So that is why it is
important that your recurrence condition should be of this form.
Now since this is a quadratic equation, we will have 2 roots for this equations. So I call those roots
as 𝑟" and 𝑟$ and those roots are called as characteristic roots because they are the roots of this
characteristic equation. Now there could be 2 possibilities: the roots 𝑟" 𝑟$ and are distinct or 𝑟" =
𝑟$ and they could be the same. When I say non-repeated I mean the former case where the roots 𝑟"
and 𝑟$ are different.
So once you have solved the characteristic equation you will have the value of the characteristic
roots and you can check whether you are in this case or not. Now if you are in this case then we
can prove that any sequence which is the solution of the recurrence equation that is given to you
will be of the form α" 𝑟"! + α$ 𝑟$! .
491
So for the moment imagine that you are not given the initial conditions. You are just interested to
find out one possible sequence satisfying the given recurrence condition. Then what this theorem
says is, any infinite sequence whose n-th term satisfies this recurrence condition will have its n-th
term of the form α" ∗ 𝑟"! + α$ ∗ 𝑟$! for some arbitrary constants α" and α$ .
That is what the theorem says and now if you are given this initial conditions that means you are
interested to find out the sequence whose initial terms are 𝑉& and 𝑉" as well. Then the exact values
of this constants α" and α$ can be obtained by utilizing the initial conditions.
So, let us first prove this theorem statement here. So what are the things which are given to you?
Your goal is to find out an arbitrary sequence whose n-th term satisfies this recurrence condition.
You have found the characteristic equation, you solved the characteristic equation and the roots
are distinct. Our goal is to prove that you any arbitrary sequence whose n-th term is of the form
𝑎! = α" 𝑟"! + α$ 𝑟$! where α" and α$ are constants, satisfies the given recurrence condition.
So let us prove that. And this is true irrespective of the initial conditions i.e., irrespective of the
initial conditions the n-th term of that sequence will be of this form. So let us prove that and the
proof is very simple.
So, what is the n-th term of the arbitrary sequence that we are considering? Or to put it another
way, our goal is to show that if the n-th term is of this form then this recurrence condition is
492
satisfied. So let us prove that whether indeed it satisfies this recurrence condition or not. So what
is the recurrence condition? The recurrence condition says that 𝑐" 𝑎!#" + 𝑐$ 𝑎!#$ (where 𝑎!#" and
𝑎!#$ are the (𝑛 − 1)-th and (𝑛 − 2)-th terms of the arbitrary sequence respectively) should be
equal to the n-th term of this arbitrary sequence. That is what we have to prove.
By substituting 𝑛 – 1 in this formula we obtain that the (n - 1)-th term will be α" 𝑟"!–" + α$ 𝑟$!–" .
And its (𝑛– 2)-th term will be α" 𝑟"!–$ + α$ 𝑟$!–$ ; again, just obtained by substituting 𝑛 = 𝑛 – 2
in the formula for the n-th term of the sequence.
Now we will solve it. So we will rearrange the terms and after rearranging the terms we get
α" 𝑟"!#$ [𝑐" 𝑟" + 𝑐$ ] + α$ 𝑟$!#$ [𝑐" 𝑟$ + 𝑐$ ]. And now what we are going to do is, we are going to
utilize the fact that 𝑟" and 𝑟$ are the roots for this characteristic equation. That means both 𝑟" as
well as 𝑟$ satisfies the condition 𝑟 $ − 𝑐" 𝑟 − 𝑐$ = 0. That means 𝑟"$ – 𝑐" 𝑟" – 𝑐$ = 0 or in other words
𝑟"$ = 𝑐" 𝑟" + 𝑐$ . So that is why I can substitute this part by 𝑟"$ and similarly your 𝑟$ also satisfies
the characteristic equation.
So we also have 𝑟$$ – 𝑐" 𝑟$ – 𝑐$ = 0 which in other words implies 𝑐" 𝑟$ + 𝑐$ is 𝑟$$ . So by substituting
this we get α" 𝑟"! + α$ 𝑟$! . And what is this? This is nothing but the n-th term of the arbitrary
sequence. Thus we have proved part 1. So we have shown that you take any arbitrary sequence, if
its n-th term is of this form then definitely that satisfies the recurrence condition.
Now, any value of the constants α" and α$ will give you a sequence which satisfies the given
recurrence conditions. So I can have (α" , α$ ) = (1,1) and that will give me one arbitrary sequence
satisfying the given recurrence condition. I can put (α" , α$ ) = (0,0) and that also will be satisfying
the recurrence condition and so on.
In fact the arbitrary sequence where all the terms are 0, trivially satisfies the recurrence condition.
But we are not interested in such trivial solutions. So this proves the theorem in one direction. That
means you know how to find out at least one sequence satisfying the recurrence condition. But the
theorem statement is an if and only if statement. It basically says that if at all there is a solution,
then it has to be of this form where the n-th term is some constant times 𝑟"! plus another constant
times 𝑟$! .
493
We had shown right now that you give me a sequence whose n-th term is of this form, namely
α" 𝑟"! + α$ 𝑟$! , it will satisfy the recurrence condition. But now I want to prove the other way
around, that if at all there would have been a solution, the structure of the n-th term of that solution
will be constant times 𝑟"! plus another constant times 𝑟$! .
So that will be the part 2 of the proof. So now, here we are assuming that suppose there is some
solution satisfying this recurrence condition and the initial conditions that are given to you then
we want to prove that this n-th term of the sequence is of this form for some constants α" and α$ .
And the proof strategy here will be as follows.
We will first prove that you take another sequence different from the sequence that we are
considering right now or the solution that you are considering right now. So I am taking some
(! #(" )# (" )! #(!
another sequence whose n-th term is of the form 𝐴! = : ; 𝑟"! += : ; 𝑟$! . Then I will
)! #)# )! #)#
show that this satisfies not only the recurrence condition but also the initial conditions. The above
claim automatically implies that the n-th term of the arbitrary solution we are considering is of the
(! #(" )# (" )! #(!
form α" 𝑟"! + α$ 𝑟$! where the constants α" = and α$ = . And why so?
)! #)# )! #)#
This is because I cannot have two different arbitrary sequences satisfying the same recurrence
condition and having the same initial conditions. As per my claim, I have one sequence satisfying
494
the recurrence condition as well as the initial conditions. And I am also given another sequence
whose structure I do not know; whose 𝑎! I do not know, which also satisfies the recurrence
condition as well as the same two initial conditions.
Then as we discussed in the last lecture, if I want to satisfy simultaneously the initial conditions
as well as the recurrence conditions then there can be only one possible sequence, you cannot have
multiple possible sequences whose terms are different but they are satisfying the initial conditions
as well as the recurrence conditions. That cannot happen. Because if the initial conditions of the
two sequences are same then that automatically implies that all the following terms of the
sequences are also going to be the same, because both of them satisfies the recurrence condition.
So assuming my claim is true, I end up showing that the arbitrary solution whose n-th term I do
not know is of this form: constant times 𝑟"! plus constant times 𝑟$! . Because as per my claim there
(! #(" )# (" )! #(!
is another sequence whose n-th term is : ; 𝑟"! += : ; 𝑟$! .
)! #)# )! #)#
By the way the reason I am highlighting this (𝑟" – 𝑟$ ) here in the denominator is that these constants
α" and α$ are well defined even though in the denominator I have (𝑟" – 𝑟$ ). This is because I am
considering the case where 𝑟" and 𝑟$ are distinct and if 𝑟" and 𝑟$ are distinct their difference will
not be 0. That is why constant α" and α$ that we are considering in this proof are well defined.
So what is left now? We have to now show this claim. We have to prove that this claim is true.
495
So let us prove this claim. So here we are given some arbitrary solution for the recurrence condition
as well as satisfying the initial condition. We have to prove that if that is the case then the n-th
term of that arbitrary solution is of this form. For that, we first observe from the proof of the part
1 of this theorem that any sequence, irrespective of the initial conditions, whose n-th term is of the
form A* = β" 𝑟"! + β$ 𝑟$! where β" and β$ are constants always satisfies the recurrence condition.
This is what we proved in the proof of part 1. We use 𝐴 to differentiate from the sequence whose
n-th term is 𝑎! i.e., the sequence whose n-th term is 𝐴! is different from the arbitrary solution
whose n-th term is represented as 𝑎! .
We know that this satisfies the recurrence condition. That means we know that 𝐴! = 𝑐" 𝐴!#" +
𝑐$ 𝐴!#$ . The proof is similar to that of part where we utilized that 𝑟" is a characteristic root, 𝑟$ is a
characteristic root and substituted 𝑟"$ with 𝑐" ∗ 𝑟" + 𝑐$ and so on.
496
Now, if I want to find out the initial terms of that 𝐴 series we would like that 𝐴& = 𝑉& and 𝐴" =
𝑉" . If that is the case, then I get 2 equations in β" and β$ . How? If I substitute 𝑛 = 0 here, I get 1
equation, and if I substitute 𝑛 = 1 here, I get another equation. So now what are the things known
to me? 𝑉& and 𝑉" are already given to me because they are the initial conditions.
β" and β$ are the unknowns. And 𝑟" and𝑟$ are known to you. So you have now two equations in
2 unknowns and you can solve them and get the value of β" and β$ . That means I have now formed
a concrete sequence, namely the A sequence, whose n-th term is 𝐴! = β" 𝑟"! + β$ 𝑟$! . I know it
satisfies the recurrence condition.
And I also know the values of β" and β$ for which this A series will satisfy the given initial
conditions, namely I know the values of β" and β$ for which 𝐴& would have given me 𝑉& and 𝐴"
could have given me 𝑉" .
497
So our goal was to prove this claim; and we proved it. We proved that indeed any 𝐴 sequence, a
sequence whose n-th term is this, satisfies the recurrence condition as well as the initial conditions
𝑉& and 𝑉" . What this means? This means that now I have 2 different sequences, 𝐴 sequence
satisfying the recurrence condition as well as the initial condition. And arbitrary solution which I
assumed satisfying the given recurrence condition as well as the initial conditions.
And now both these 2 sequences are same because as I said earlier I cannot have 2 different
sequences with the same initial condition but different n-th term simultaneously satisfying the
recurrence condition. It is possible to have two different sequences 𝐴 and 𝐵 if we do not put the
restriction that their initial conditions are the same. But I cannot have 2 difference sequences 𝐴
sequence and 𝐵 sequence whose n-th terms are different while satisfying the same recurrence
conditions with identical initial terms.
We have two sequences satisfying the recurrence condition and both of them are satisfying the
initial conditions. That is possible only when 𝐴! = 𝑎! . That means the arbitrary solution that you
considered here, it is of the form some constant times 𝑟"! plus some constant times 𝑟$! . Because
that is the term of this 𝐴 sequence. So that completes the proof for the part 2 here.
498
Till now, we focused on the case of degree 2. Now we will try to extend or generalize this theory
for the case of degree 𝑘 linear homogenous recurrence equations. So remember the degree 𝑘
equation, the general formula is this where 𝑐% is not allowed to be 0. And you may or may not be
given the initial conditions. If you are not given the initial conditions then you will stop with
showing the closed-form formula for the n-th term in terms of some arbitrary constant.
Those constants you can put as any constant. But if you are given the initial conditions as well
then you can solve and find out those concrete constants. So, what will be the process in the general
case? We will first form a characteristic equation; this will be a degree 𝑘 equation. Next we will
solve it and find out characteristic roots. Let us denote the characteristic roots by 𝑟" , 𝑟$ … 𝑟% .
Now there could be multiple cases. The case that we considered when 𝑘 was 2 and that we are
going to consider in this case, is the case of distinct roots. Namely, when all your 𝑘 characteristic
roots are different. Then extending the theorem that we proved for the case of 𝑘 = 2, we can
show that any sequence which satisfies this recurrence condition will have its n-th term of the
following form.
Some constant times 𝑟"! plus another constant times 𝑟$! and continuing like that some constant
times 𝑟%! . This will be the general form of the solution satisfying the recurrence condition. The
exact values of these constants α" , α$ … α% can be obtained from the initial conditions.
499
So if you are given the initial conditions then by substituting 𝑛 = 0, 𝑛 = 1 … 𝑛 = 𝑘 − 1; we will
get 𝑘 equations. In this, 𝑘 unknowns α" to α% and then we can get the concrete values of the
constants α" to α% . But if you are not given the initial condition then the only thing that we can do
is we can just find out the general form of the solution. It is up to us what constants α" , α$ … α%
we substitute. That will determine the sequence satisfying the given reference condition. But if
you want a unique condition then you also need to have the initial conditions available.
So now let us see an example where we will apply the method that we had discussed. We now
want to find out an explicit formula for the n-th term of the Fibonacci sequence. So just to recall,
the n-th term of the Fibonacci sequence is the following. Then n-th term depends on the previous
2 terms and the initial conditions are this. So the first step will be to find the characteristic equation
so here 𝑐" = 1 and 𝑐$ = 1.
So that is why our characteristic equation will be 𝑟 $ − 𝑟 − 1 = 0. Now if I solve the characteristic
equation and find the characteristic roots then I see that I obtain 2 distinct roots. That means I can
apply the theorem that we have discussed in this lecture. What I will say is that the n-th term of
! !
"+√- "#√-
any sequence whose n-th term is of the form 𝑓! = α" : ; + α$ : ; will satisfy the
$ $
recurrence condition.
500
Now whatever value for α" and α$ I substitute that will determine a different Fibonacci sequence.
If someone just gives me the recurrence condition and not the initial conditions, and asks me to
find out a sequence satisfying this recurrence condition, I can say that any sequence whose n-th
! !
"+√- "#√-
term is 𝑓! = 𝛼" : ; + 𝛼$ : ; for any value of the constants α" and α$ will satisfy the
$ $
recurrence condition without worrying about what are the first 2 terms. But in this case, I am given
the initial conditions. So if I am given the initial conditions, I will utilize them to find out exact
value of α" and α$ which is consistent with the initial conditions of actual Fibonacci sequence.
So I am interested to find out the sequence whose zeroth term is 0 and the next term is 1. That
means I have to substitute 𝑛 = 0 and 𝑛 = 1 in this general formula. And then I will get 2
equations in α" andα$ and by solving them I can get the exact values of my constants α" and α$ .
" #"
They will be and . And then I will say that here is the exact solution of the actual Fibonacci
√- √-
sequence which satisfies not only the initial conditions but also the recurrence condition.
So that brings me to the end of this lecture. These are the references for today’s lecture. Just to
summarize in this lecture we started discussing about how to solve linear homogeneous recurrence
equations of degree 𝑘. And we saw one of the cases, namely we saw the case where the
characteristics roots are distinct. In this case we saw how to find the characteristic root and if all
the characteristic roots are different then we know what will be the general solution and depending
501
upon whether the initial conditions are available or not we can find out the exact solution. Thank
you.
502
IIIT, Bangalore
Module No # 08
Lecture No # 36
Solving Linear Homogeneous Recurrence Equations – Part II
Hello everyone, welcome to this lecture. In this lecture we will continue our discussion regarding
how to solve linear homogeneous recurrence equations.
So just to recap, in the last lecture we discussed how to solve linear homogeneous recurrence
equations for the case when the characteristic roots were all distinct.
503
So this was the summary of the discussion that we had in the last lecture. We had a linear
homogeneous recurrence equation of degree k, which is imposed by saying that 𝑐! is not allowed
to be 0. And you may or may not be given the initial conditions. The first step will be to form the
characteristic equation which will be an equation of the degree k and then we find the characteristic
roots.
They may be real roots, complex roots, they may be all distinct, some of them may be distinct,
some may be repeated and so on. So in total we will have k roots which we denote as 𝑟" to 𝑟! . And
the case that we discussed in the last lecture was the following. If it turns out that all the k roots
are different, then any sequence will be satisfying the given recurrence condition provided the n-
th term of that sequence is of the form 𝑎# = α" 𝑟"# + α$ 𝑟$# + ⋯ + α! 𝑟!# . Namely, it is some
constant time the first characteristic root power n plus constant times second characteristic root
power n and so on. For some arbitrary constant α" to α! .
Depending upon what are the constant you fit, you get different sequences satisfying the recurrence
condition. So this general form of the solution is irrespective of whether it is satisfies the initial
condition or not. If you want to satisfy the initial conditions as well, then you can find out the exact
constants or the unique solution.
So remember, if we are not bothered about initial conditions then there can be plenty of infinite
sequences satisfying the same recurrence condition and all of them can be obtained by this general
504
form. Now if you want to find out the unique sequence satisfying the initial conditions as well then
you have to find out the exact values of this constantα" , α$ … α! and which can be obtained by
utilizing the initial conditions if they are given to you. That is the idea.
Now, in this lecture we will discuss the case when the characteristic roots are repeated. And again
for simplifying the discussion, we start with the case when the degree is 2. That means the general
form of the recurrence equation is this, where c$ will not be 0. And you may or may not be given
the initial conditions. So if you are not given the initial conditions then we will end up with the
general form of the solution. That means any sequence which will satisfy this recurrence condition
what will be the general form for the n-th term of the sequence? We will end up with that.
But if you want to find out the exact sequence then we have to utilize the initial condition as well.
So this was the theorem for the case where the roots where distinct. If the roots are distinct then
any sequence whose n-th term is of this form for some arbitrary constants α" and α$ will be
satisfying the recurrence condition. But it turns out that the theorem no longer holds if the roots
are equal.
We are considering the case when we have degree 2 characteristic equation and hence we have 2
roots 𝑟" and 𝑟$ and if 𝑟" and 𝑟$ are same then the theorem no longer holds. This is because if you
recall the proof of the last theorem then there, if you want find out the exact sequence satisfying
505
the given recurrence condition as well as the initial conditions; so for that our constants α" and α$
%! &%" '# %! &%" '#
turned out to be α" = - . and α$ = - ..
'! &'# '! &'#
And they were well defined because for the previous case your 𝑟" and 𝑟$ were different. In which
case the denominator will be non-zero. But if your 𝑟" and 𝑟$ are same then theseα" and α$ are not
well-defined and in which case we cannot find out the exact sequence satisfying the given
recurrence condition as well as the initial condition. And if we cannot find out the exact sequence
from the general form, that means that general form which was applicable for the previous case is
not applicable for this case.
So that means the general form of the solution, or the general form of any sequence satisfying the
recurrence condition for the case of distinct characteristic roots will be different. So, the theorem
statement for this case is as follows. We can prove that if the characteristic roots are distinct, and
say the characteristic roots are 𝑟" and 𝑟$ are same so I denote the characteristic root as 𝑟( . It is the
common characteristic root.
Then this theorem statement basically says that any sequence whose n-th term is of this form will
satisfy the recurrence condition for some arbitrary constants α" and α$ . Now again, this is the
general form of any sequence satisfying the recurrence condition and there could be infinite
number of such sequences. So right now I am not utilizing the initial conditions. If you want to
506
find out the exact sequence satisfying the recurrence condition as well as the initial conditions then
you have to solve the, or you have to find the exact constants which you can obtain by utilizing
the initial conditions, if they are given to you. But if they are not given to you then we end up with
the generic form or the general term of any sequence satisfying the recurrence condition and this
will be 𝑎# = α" 𝑟(# + α$ 𝑛𝑟(# where α" and α$ are constants.
So that is the difference here. In the previous case if 𝑟" and 𝑟$ are the two roots then the general
form was α" 𝑟"# + α$ 𝑟$# . My claim is that this general form is not applicable for the distinct root
case. The general form will be now 𝛼" 𝑟(# + 𝛼$ 𝑛𝑟(# . And the proof can be shown similar to the way
we proved the theorem for the case of distinct roots.
There we utilize the fact that 𝑟" and 𝑟$ are 2 distinct root of the characteristic equation and then we
showed that whatever general form was there, it satisfies the recurrence condition. The same we
have to do in this case as well and we have to utilize the fact that we have equal roots for the
quadratic equation. So the part 1 of the proof will be, we have to show that any sequence of the n-
th term is this, irrespective of what is the value of the constants α" and α$ will satisfies the
recurrence condition that 𝑎# = 𝑐" 𝑎#&" + 𝑐$ 𝑎#&" . And we can prove that very easily.
And the second part of the proof will be that if at all this sequence whose n-th term is this satisfies
the recurrence condition as well as the initial conditions then we can find out the constants α" and
α$ . They are well defined. Now the form of the constants α" and α$ will be different from the form
of the constants α" and α$ which were there for the case of distinct roots.
Now the α" and α$ will not be the same. Specifically, in the denominator, in the earlier case we
had 𝑟" – 𝑟$ which was non-zero. But now the form of α" and α$ will be different for this case. And
the proof is very similar to the proof for the case of distinct root case. So I am not going into the
details you can easily do that.
507
So lets see an example for this case. So suppose I want to solve this recurrence condition and for
the moment ignore the initial conditions. So the first step will be forming the characteristic
equation, the characteristic equation will be of degree 2. Here 𝑐" = 6 and 𝑐$ = 9, so accordingly
I get the characteristic equation as 𝑟 $ − 6𝑟 + 9 = 0. Next step will be finding the characteristic
roots. And in this case the characteristics roots are same, namely, 3 and 3.
That means the general form of the solution will be 𝑎# = α" 3# + α$ 𝑛3# where α" and α$ are
constants. This will be the general form of the solution. Now substituting the different values of
α" and α$ you get various sequences satisfying the recurrence condition. In fact, if you substitute
α" = α$ = 0 that will give you one sequence. Namely a sequence where all the terms are 0s. That
is a sequence satisfying the given recurrence conditions.
If you substitute α" = 1 and α$ = 0 then that gives you a sequence where the n-th term is 3# which
can be easily verified to satisfy the same recurrence condition. So, by substituting the different
values of α" and α$ you will get different sequence satisfying the recurrence condition.
But now since we are also given the initial conditions we will be interested to find out the exact
sequence satisfying the recurrence condition as well as having the initial terms 1 and 6. So for that,
I will be substituting 𝑛 = 0 here and 𝑛 = 1 here and get 𝑎( = 1, 𝑎" = 6 which will give me
now 2 equations in α" and α$ where α" and α$ are my unknown constants and by solving those
508
two equations I will be find out the exact α" and α$ and hence the exact sequence satisfying my
recurrence condition as well as the initial conditions.
So that was the case of degree 2 characteristic equations with repeated roots. Now we will consider
the general case where we have a linear homogeneous recurrence equation of degree 𝑘 with
repeated roots. So, this will be the form of the recurrence equation. You may or may not be given
the initial conditions if you are not given the initial conditions, we will just end up writing down
the general term of the sequence satisfying the recurrence condition.
But if you are also given the initial conditions you can find out the exact sequence satisfying the
initial conditions as well as the recurrence condition. So, the first step here will be forming the
characteristic equation which will be an equation of degree 𝑘 and hence it will have 𝑘 characteristic
roots. Some of them might be distinct, some of them might be same, and so on. So what I do here
is, I denote the distinct roots by 𝑟" to 𝑟) and they occurred with multiplicities 𝑚" to 𝑚) respectively.
That means the root 𝑟" occurs 𝑚" times, the root 𝑟$ occurs 𝑚$ times and like that the root 𝑟) occurs
𝑚) times. What is the relationship here? Each of these values 𝑚" , 𝑚$ and 𝑚) are greater than equal
to 1, because these are the distinct roots. So 𝑟" will be occurring as a root definitely at least once.
So that is why 𝑚" will be at least l. Of course, it might be possible I have 2 roots, namely, 𝑟" and
𝑟" in which case 𝑚" will be 2; or I might have a case when the only root is 𝑟" . That means in that
case 𝑚" will be 𝑘 and so on.
509
That means depending upon number of times 𝑟" is occurring as a root determines the value of 𝑚" .
Since 𝑟" is occurring as a root definitely at least once, 𝑚" will be at least once. Similarly 𝑚$ will
be at least 1 and similarly𝑚) also will also be at least 1. So, that justifies why each 𝑚* is greater
than equal to 1. And the second condition here is that the sum of all the multiplicities here will be
𝑘 because you have total 𝑘 number of roots.
So 𝑟" will be occurring certain number of times as root, 𝑟$ is occurring certain number of times as
a root, and 𝑟) is also occurring as a root certain number of times, namely 𝑚) number of times. If I
sum up all these multiplicities, that should give me the total number of roots, namely, 𝑘. That is
the relationship between these multiplicities and the various roots. Now what the theorem basically
says is the following.
The theorem says that any sequence whose n-th term is of this form will satisfy the recurrence
condition.
So let us decode this complicated looking general term here. So, what is the first term here? The
first term is basically a polynomial of degree (𝑚" – 1) multiplied by the first characteristic root
raised to power 𝑛. Why (𝑚" – 1)? Because the first characteristic root namely 𝑟" occurs 𝑚" number
of times. So that is why a polynomial degree (𝑚" – 1). So that means if 𝑟" occurs as a root only
once, in that case 𝑚" would have been 1.
510
In which case, this polynomial of degree (𝑚" – 1) is nothing but a constant. So, in that case it will
be constant times 𝑟"# . But if 𝑚" is say 3, that means 𝑟" occurs as a root 3 number of times, then
this polynomial, the first polynomial here will be a polynomial of a degree 2 multiplied by the first
characteristic root power n. The constant, the coefficients in this polynomial of degree (𝑚" – 1)
they are all unknown constant.
We can substitute any value for them and that will determine the sequence satisfying the recurrence
condition. The next term in this general term will be an unknown polynomial of degree (𝑚$ – 1)
multiplied by the second characteristic root power n. And like that the last term will be a
polynomial of degree (𝑚) – 1) because the last characteristic root namely 𝑟) has multiplicity 𝑚) .
So it will be raised to power n and it will be proceeded by a polynomial of degree (𝑚) − 1). So
all these alpha values, they are constants here, unknown constants. By substituting various values
for this alphas you get various sequences satisfying the recurrence condition. Now if you want to
satisfy the initial conditions as well, then you can substitute the values of 𝑛 = 0, 𝑛 = 1 and all
the way 𝑛 = 𝑘 − 1.
You get various equations in this unknowns alphas. And you can get the exact alphas satisfying
the initial conditions as well as the recurrence condition as well. Specifically, how many alphas
will be there? In the first polynomial you have 𝑚" alphas. Second polynomial you had 𝑚$ alphas
and like that the last polynomial you had 𝑚) alphas. So total how many unknowns are there? 𝑚" +
𝑚$ + ⋯ + 𝑚) which is nothing but 𝑘.
So you will be having 𝑘 unknown constants here and if you are given k initial conditions namely
𝑉( , 𝑉" up to 𝑉!&" by substituting 𝑛 = 0, 𝑛 = 1, … , 𝑛 = 𝑘 − 1 in this general formula you get
𝑘 equations in 𝑘 unknowns, by solving them you get the exact constants. Now you can tell that
this general formula captures all the cases that we had discussed still now.
So if you are in the case when all the roots are distinct, that means you have 𝑘 roots, each of the
roots has multiplicity 1. That means 𝑚" is 1, that means 𝑟" occurs as a root exactly once. 𝑚$ is 1,
that means 𝑟$ occurs as a root exactly 1 and like that 𝑚! is 1 that means 𝑟) occurs as a root exactly
511
once. Then each of these polynomials will be a constant polynomial followed by that characteristic
root raised to the power 𝑛. And that is precisely was the general form of a sequence or the n-th
term of the sequence satisfying the recurrence condition for the case when all the characteristic
roots where distinct.
So you can easily verify that. The proof for this theorem will be again similar to the case where
we took the degree 2 equation and the roots where distinct. I am leaving the proof for you.
So now let see an example for applying the general formula. So imagine you are given the
recurrence condition 𝑎# = −3𝑎#&" − 3𝑎#&$ − 𝑎#&+ and for the moment assume you are not given
the initial conditions. So the first step will be forming the characteristic equation. The characteristic
equation will be 𝑟 + + 3𝑟 $ + 3𝑟 + 1 = 0. It is a cubic equation so it will have three characteristic
roots 𝑟" , 𝑟$ , 𝑟+ . It turns out in this case that 𝑟" and 𝑟$ and 𝑟+ are all same. And hence the number of
roots is 1 and its multiplicity is 3.
So the general solution will be a known polynomial of degree 𝑚" – 1 = 2, multiplied by the
characteristic root raised to the power 𝑛 and the characteristic root here is -1. Thus, the general
solution is 𝑎# = ?α",( + α"," 𝑛 + α",$ 𝑛$ @(−1)# . By substituting different values of
theα",( , α"," and α",$ you get various sequences satisfying the recurrence condition.
Now since you are also given the initial condition you can find out the exact sequence satisfying
this initial condition. That means you start with the terms 1, -2, -1 and n-th term satisfies the
512
recurrence condition. That you can do by substituting 𝑛 = 0, 𝑛 = 1, and 𝑛 = 2 in general form
and equating the resulting expression with 1, - 2, and -1 respectively. This results in 3 equations
which can in turn be used to solve for α",( , α"," and α",$ and you get the exact value of this
constants and hence the exact sequence satisfying the recurrence condition.
So that brings to me to the end of today’s lecture. So these are the references used for today’s
lecture. Just to summarize, in this lecture we continued our discussion regarding how to solve
linear homogeneous recurrence equations of degree 𝑘 and we consider the case when the
characteristic roots are repeated. Thank you.
513
IIIT, Bangalore
Module No # 05
Lecture No # 37
Tutorial 6: Part 1
Hello everyone, welcome to the first part of tutorial 6.

In question number 1, we are supposed to find the required relation for the number of strictly
increasing sequences of positive integers that has 1 as its first term and 𝑛 as its last term where 𝑛
is some positive integer. So for instance, if 𝑛 = 1 then there is only 1 sequence which starts with
1 and ends with 1 and which is a strictly increasing sequence. Whereas as if 𝑛 = 2 then also I
have only 1 sequence starting with 1 and ending with 2 and which is strictly increasing.
By strictly increasing sequence, I do not mean that I need all the numbers between 1 to 𝑛. Similarly,
if 𝑛 = 3 then a possible sequence could be starting with 1 ending with 3. This is a strictly
increasing sequence or I can have a sequence where I have (1, 2, 3) starting with 1 and ending with
3 and having 2 in between. So these are the various sequences possible for different values of 𝑛.
Now our goal is to find out a recurrence equation for this problem.
Basically, what we are asking here is the following. Imagine you have an arbitrary sequence
starting with 𝑎! ending with 𝑎" . The restrictions are here that the first term should be 1, namely
514
𝑎! should be 1 and the last term 𝑎" should be 𝑛 and in between you have the numbers 𝑎# … 𝑎"$!
where they appear in the strictly increasing order. They can be any numbers from {2 … 𝑛 – 1}. So
let 𝑠% denotes my functions which is the number of valid sequences ending with 𝑛.
So a trivial recurrence condition for 𝑠% is the following. I can say that 𝑠% = 𝑠! + 𝑠# … 𝑠%$! . This
is because, the second last value in the sequence, namely 𝑎"–! , can be 1. If that is the case then
basically what I am saying is find out all possible sequences starting and ending with 1 and append
it with the value 𝑛. That will give you one category of valid sequences starting with 1 and ending
with 𝑛.
How many such sequences you can have? You can have 𝑠! such sequences. Or 𝑎"–! can be 2. In
that case what I am saying is you find out all valid sequences starting with 1, ending with 2, and
which is a strictly increasing sequence and in each such sequence you put an 𝑛 at the end and that
will give you another category of strictly increasing sequences starting with 1 and ending with 𝑛.
How many such sequences you can have? Can have 𝑠# number of such sequences and the same
way your second last value in the sequence 𝑎"–! could be 𝑛 − 1. So basically I am saying take
any valid or strictly increasing sequences starting and starting with 1 ending with 𝑛 − 1 and put
an 𝑛 at the end and that will give you another category of strictly increasing sequences starting
with 1 and ending with 𝑛.
And you can have 𝑠%–! such sequences. And all this category of sequences that we have discussed,
they are disjoint. So this is one of the recurrence conditions for our function 𝑠% . But we want a
more compact recurrence condition. Compact in the sense, the degree of this equation that we had
just formulated, what is its degree? Its degree is 𝑛 − 1 because the n-th term of the sequence that
we had formulated here depends on 𝑛 − 1 previous values in the sequence.
We want a more compact recurrence condition. So let us derive that alternate recurrence equation
and this alternate recurrence equation is derived by using the following argument. So imagine you
have, considering a valid or strictly increasing sequence starting with 1, ending with 𝑛, and the
second last value in the sequence is 𝑎"–! . Now there can be 2 categories. Category 1 that your,
515
𝑎"–! is 𝑛 − 1. That means you are considering all such strictly increasing sequences where the
second last term is 𝑛 − 1.
How many such sequences you can have? You can have 𝑠%–! sequences and you take any such
sequence and append it by a value of 𝑎" = 𝑛, you get a strictly increasing sequence starting with
1 and ending with 𝑛. And there will be 𝑠%–! such sequences. Category 2 is the following. The
second last value namely 𝑎"–! in your sequence can be either 1 or 2 or 𝑛 − 2. And you end this
𝑎"–! ; you append the 𝑛 at the end of this 𝑎"–! you get a valid or strictly increasing sequence starting
with 1 ending with 𝑛.
Now what we can do is? We can interpret the 𝑛 which is there at the last position, last position
means after 𝑎"–! in your actual sequence ending with 𝑛. We can interrupt this last 𝑛 as 𝑛 – 1. It is
just some mental interpretation. So for instance what I am saying is here is one of the strictly
increasing sequences in category 2. Why? Because my second last value namely 𝑎"–! here, is 𝑛 −
3. It is not 𝑛 − 1 it is 𝑛 − 3 and then there is an 𝑛 following this 𝑛 − 3.
So I can just view this sequence as another sequence where instead of n being appearing at the last
position I have 𝑛 − 1 appearing at the last position. Mind it, I stress here that these second
category of sequences are different from the first category of sequence. Because in the first
category of sequence I have only 𝑛 − 1 allowed in the second last position. But in this category
2 of sequences I am not allowing 𝑛 − 1 in my second last position.
My second last position could be anything from the set 1, 2 or 𝑛 – 2 and then the last position is
occupied by 𝑛. What I am thus basically saying is that just read that instead of being 𝑛 having at
the last position, you have 𝑛 − 1 at the last position. And it is easy to see that how many such
sequences you will have? By doing this logical interpretation, that means, by substituting the last
𝑛 with an 𝑛 − 1 in your mind you can easily see that there will be 𝑠%–! such sequences.
And also this category 1 of sequences and category 2 of sequences are disjoint. And you cannot
have any third category of strictly increasing sequences because your second last position namely
𝑎"–! could be either 𝑛 − 1 or different from 𝑛 − 1. And these are the only 2 possible cases which
516
we have considered and hence we can say that our alternate recurrence equation will be 𝑠% = 2 ∗
𝑠%$! .
Now this is an equation of degree 1 because now the dependency of the n-th term is only on the
previous term. So definitely this is a better recurrence equation in terms of solving it and in terms
of the number of initial conditions that we need here. It turns out that even though this equation;
this alternate recurrence equation is of degree 1, we need 2 initial conditions. So 𝑠! = 1 this is
because if 𝑛 = 1 there is only 1 sequence which is strictly increasing starting with 1 and ending
with 1; namely with sequence 1 i.e., the sequence which is having only the value 1 in it.
And if I take 𝑛 = 2 then also I have only 1 sequence which is strictly increasing; starting with 1
and ending with 2. The sequence will be 1 followed by 2. That is why 𝑠# = 1. Now you might be
wondering that why 𝑠# is explicitly specified as an initial condition. Why cannot I just have an
initial condition 𝑠! = 1 because this is an equation of degree 1 here.
This is because if I do not explicitly specify the value of 𝑠# = 1 and then if you try to derive 𝑠# by
substituting 𝑛 = 2 in this recurrence equation you should get 𝑠# = 2𝑠! = 2 ∗ 1 = 2, which is not
the case because you do not have 2 strictly increasing sequences starting with 1 and ending with
2. You have only one strictly increasing sequence. Or does that mean that our recurrence condition
that we have derived is wrong?
Well that is not the case because if you see here the problem that is happening here is that if I take
𝑛 = 2 then this case 2 or category 2 of strictly increasing sequences would not be there at the first
place. Because for 𝑛 = 2 this case 2 boils down to saying that at the second last position you have
one at the beginning and you’re 𝑎"–! , namely the second last string, could be from the set 1 to 0
which is not the case.
Because 𝑛 − 2 = 0 for 𝑛 = 2, you cannot have a second last position occupied by something,
say 0. That is not allowed. So that is why case 2 or category 2 is not going to occur for the case of
𝑛 = 2. For 𝑛 = 2 it is only the case 1 which can occur namely your second last position can be
occupied only with 1. And then you append it with 1 that is why you have only 1 valid sequence
for the case of 𝑛 = 2 as well.
517
It is only from 𝑛 = 3 onwards that your category 2 of strictly increasing sequences appear. And
that is why we have 2 initial conditions needed.
In question 2 you have to find out the recurrence relation for the number of bit strings of length 𝑛
that contain the substring “000”. It might have more than 1 occurrence of “000” as well. In fact, a
string of length 𝑛 consisting of all 0’s is a valid string. So let us try to find out this by formulating
a recurrence equation. So let 𝑠% denote the number of bit strings of length 𝑛 containing the
substring “000”.
So what we will do is instead of counting the number of strings containing “000” we will rather
count the number of strings of length 𝑛 not containing “000” because it will be relatively easier to
formulate a recurrence equation for the number of bad string. Here, the term bad string denotes
strings that do not have an occurrence of “000”. So, I call or denote the sequence of all 𝑛 length
bit strings not containing “000” by this 𝑏 sequence and 𝑏% is the n-th term of such sequence.
Now it is easy to see that as per the definition of 𝑠% and 𝑏% that 𝑠% = 2% − 𝑏% . This is because 2%
is all possible 𝑛 bit strings. 2% denotes all possible 𝑛 bit strings which has an occurrence of “000”
and which do not have an occurrence of “000”. And you subtract from such strings all such strings
which do not have an occurrence of “000” we will get the number of strings of length 𝑛 which has
an occurrence of “000”.
518
So our goal now will be boiling down to find out the recurrence condition for the 𝑏 sequence. And
it turns out that there are 3 disjoint categories of bit strings of length 𝑛 not containing “000”. Let
us look into those 3 categories. Category 1 consists of those strings of length 𝑛 that start with 1. If
it starts with 1 then in order that the overall string does not have any occurrence of “000”, it should
be the case that the remaining portion, namely the remaining portion of length 𝑛 − 1 bit, should
not have any occurrence of “000”.
How many such strings we can have? As per our definition of 𝑏 function, there will be 𝑏%–! such
strings. So this is category 1 of bad strings. Category 2 of bad strings are those that start with “01”
and if that is the case then the remaining 𝑛 − 2 bit positions should not have any occurrence of
“000”. How many such bit strings we can have? We can have 𝑏%–# such strings.
And category 3 of bad strings where the first 2 positions are 0, and in that case, the third position
cannot be 0 because if the third position has a 0 then it is not a bad string. The definition of bad
string is that it should not have any occurrence of “000”. So in category 3 we are considering the
case where the first 2 positions are 0 the third position is 1. And after that in order that the overall
string do not have any occurrence of triple 0 the remaining substring of length 𝑛 − 3 should not
have any occurrence of “000”.
How many such bad strings I can have? I can have 𝑏%–' such strings. And these are the only 3
categories of bad strings. I cannot have any other fourth category and it is easy to see that they are
disjoint. So how many bad strings I can have? The number of bad strings of length 𝑛 will be the
summation of the number of bad strings of length 𝑛 − 1, 𝑛 − 2 and 𝑛 − 3 and hence we get this
recurrence equation for the 𝑏 series.
And since this is an equation of degree 3, we need 3 initial conditions. 𝑏! = 2 because both the
string 0 as well as string 1 do not have any occurrence of “000”. Similarly, 𝑏# = 4 and 𝑏' = 7.
519
So you can see in this question that why we did not formulated directly the recurrence condition
for 𝑠% . Because that might be slightly involved but formulating a recurrence of 𝑏% is very easy and
that is why we formulated our recurrence condition or expressed our recurrence condition for 𝑠%
in terms of 𝑏% .
In question 3, we have to find out the recurrence relation for the number of bit strings of length 𝑛
that contain “01”. So let 𝑠% denote the number of bit strings of length 𝑛 containing “01”. Again,
we need at least 1 occurrence of “01” there could be more occurrences of “01” allowed as well.
My claim is that 𝑠% satisfies the recurrence condition 𝑠% = 𝑠%$! + 2%$! − 1. Let’s see why. So it
turns out that we can have 2 disjoint categories of 𝑛 length bit string containing “01”. Category 1
where the string starts with 1. If it starts with 1 and in order that the overall string has an occurrence
of “01” we need that remaining 𝑛 − 1 length bit string should have an occurrence of “01”.
As per our definition of s function I will be having 𝑠%–! such bit strings. And category 2 where the
string starts with 0. Now the strings starts with 0; I may have only 1 zero, or I may have 2 zeros,
or I may have a sequence of 𝑘 zeros followed by a 1 and then I do not care what is the remaining
𝑛 − 𝑘 − 1 substring. And here 𝑘 ranges from 1 to 𝑛 − 1. Why from 1 to 𝑛 − 1? Because we
definitely need 1 occurrence of 0 followed by a 1.
So that is why k = 1. So 𝑘 denotes the number of zeros which are there at the beginning of the
string. So I can have either 1 number of 0, or 2 number of 0, or 𝑛 − 1 number of zeros because I
520
might be having a string of the form “000…0” 𝑛 − 1 number of time and the last bit is 1. That is
also a valid string in this category 2. So that is why my 𝑘 ranges from 1 to 𝑛 − 1.
And then I do not put any restriction. Once I have occupied 𝑘 + 1 bits here, the remaining 𝑛 −
𝑘 − 1 positions can be occupied by any bit, either 0 or 1. So how many strings in this second
category I can have? It depends upon how many slots I am left here. Because the first 𝑘 + 1 slots
are reserved for 𝑘 number of zeros and one single 1.
So the remaining 𝑛 − 𝑘 − 1 positions can be occupied by either 0 or 1 and hence I can have 2%$"$!
many strings in the second category. So if I sum these 2 things I will get the total number of strings
but it turns out that in category 2 my 𝑘 ranges from 1 to 𝑛 − 1. So that is why I have to put 𝑘 =
1, 𝑘 = 2, 𝑘 = 3 … 𝑘 = 𝑛 − 1. And hence I get total 2%$# + 2%$' + ⋯ + 2( many strings in
category 2 and this is nothing but a geometric progression. If I sum them up I get 2%$! – 1 value
and that is how we get 𝑠% = 𝑠%$! + 2%$! − 1.
Now let us go to question number 4. We want to find out a recurrence relation for the number of
ternary strings of length 𝑛 that has an occurrence of “00” or “11” or “22”. It may have occurrence
of all of them or it may have an occurrence of “00” as well as “11”. I need either an occurrence of
“00” or an occurrence of “11” or an occurrence of “22”. And by ternary strings I mean that the
only characters which are allowed in your string are 0, 1 and 2.
521
So let 𝑠% denote the number of such ternary strings. The claim is that 𝑠% is satisfying the recurrence
condition 𝑠% = 2𝑠%$! + 3%$! . So let us derive that. It turns out that we can have 2 disjoint
categories of ternary strings of length 𝑛. Let’s see them. Category 1 where I take a ternary string
of length 𝑛 − 1 which satisfies my condition, namely it has an occurrence of “00” or “11” or “22”.
If I take any such 𝑛 − 1 length ternary string and there will be 𝑠%–! such strings and if I put any
character at the beginning of such a string; that character 𝑥 could be either 1 or 2 or 3 it does not
matter because I have already encountered an occurrence of “00” or “11” or “22” in the remaining
portion of the string. So it does not matter what is my 𝑥 or it will give me overall a ternary string
of length 𝑛 satisfying my conditions. So how many such strings I can have? So since my 𝑥 can
take 3 possible values; I have 3𝑠%$! strings in this category.
Category 2 where I take an 𝑛 − 1 length ternary string which has no occurrence of “00” or “11”
or “22”. How many such strings I can have? The number of such strings will be all possible ternary
strings of length 𝑛 − 1 minus the number of ternary string length 𝑛 − 1 which do have an
occurrence of “00” or “11” or “22”. So that will be the total number of strings of this form. And
let 𝑥 be the first position of this 𝑛 − 1 length ternary string. Right now there is no occurrence of
“00” or “11” or “22”.
I want to convert the string into a string of length 𝑛 which has an occurrence of “00” or “11” or
“22” and the only way I can do that is I have to put the same character 𝑥 at the beginning of this
string as well. So that will ensure that the overall length of the string becomes 𝑛 and then will have
an occurrence of either “00” or “11” or “22”. So how many strings of this category I will have? It
will be exactly 3%$! − 𝑠%$! because I do not have an option of 𝑥 here.
Once this 𝑥 is frozen I have to repeat the same 𝑥 at the first position. So that is why the total
number of strength in this category will be 3%$! − 𝑠%$! and if I sum them, these two things, I get
𝑠% = 2𝑠%$! + 3%$! .
522
Let us go to question 5. In question 5 I have defined the function 𝑇(𝑚, 𝑛) which is as follows. So
you have 2 sets. A set 𝐴 whose cardinality is 𝑚 and set 𝐵 whose cardinality is 𝑛 and 𝑇(𝑚, 𝑛)
denotes here the number of onto functions from the set 𝐴 to the set 𝐵. I have to show that 𝑇(𝑚, 𝑛)
is satisfying the recurrence condition 𝑇(𝑚, 𝑛) = 𝑛) − ∑%$!
"*! 𝐶(𝑛, 𝑘)𝑇(𝑚, 𝑘). So let us prove that.
The idea behind the derivation of this recurrence condition is that since a subtraction is involved
here and you can see and identify the structure of this 𝑛) .
𝑛) denotes the number of all possible functions from 𝐴 to 𝐵. It could be either onto or it could be
non-onto. So if I subtract all non-onto functions from the number of all possible functions from 𝐴
to 𝐵 that will give me the number of onto functions. So it is easy to see that what this recurrence
condition is saying is to derive 𝑇(𝑚, 𝑛) subtract the number of non-onto functions.
That means I have to show that this latter part of this recurrence condition denotes the number of
non-onto functions from 𝐴 to 𝐵 and indeed that is the case. So why that is the case? So you are
given the sets 𝐴 and 𝐵 and we want to find out the number of non-onto functions. What will be
the characteristic of a non-onto functions? A function will be a non-onto function from 𝐴 to 𝐵 if
there is at least 1 element from the set 𝐵 which do not have any pre-image.
Or in other words I can view any non-onto function as follows. So let 𝑓’ be an arbitrary non-onto
function from 𝐴 to 𝐵. If I focus on the set of possible images which are assigned as per this function
523
𝑓’ and call that set as 𝐷, the image set i.e., 𝐷 is the set of all the elements from 𝐵 which are the
images as per the function 𝑓’.
Then since my function 𝑓’ is a non-onto function clearly this image set 𝐷 will be a proper subset
of 𝐵. So more specifically if the cardinality of image set 𝐷 is 𝑘; 𝑘 < 𝑛. 𝑘 could be either 1, 𝑘
could be either 2, 𝑘 could be either 3 or 𝑘 could be either 𝑛 − 1. If you ensure that if I take these
values of 𝐷 then it ensures that there is at least 1 value from the set of B which is left which is not
going to have any pre-image.
So now in how many ways I can select my set 𝐷? Since its cardinality is 𝑘, I can select it in 𝐶(𝑛, 𝑘)
ways and as I said 𝑘 ranges from 1 to 𝑛 − 1. So that takes care of the fact that I have to consider
a summation from 𝑘 = 1 to 𝑛 − 1. Now once I have decided the value of 𝑘, I have to find out
which subset or what should be my subset 𝐷 of size 𝑘. So I have 𝑛 choose 𝑘 or 𝐶(𝑛, 𝑘) ways of
selecting my set 𝐷 of size 𝑘.
And once I have chosen by set 𝐷, the set or the function 𝑓’ will be an onto function from the set 𝐴
to 𝐷. That means I if I just forget about the set 𝐵 and just imagine that I have a set 𝐷 of size 𝑘 and
consider an onto function from the set 𝐴 to the set 𝐷 that can be interpreted as a non-onto function
from the set 𝐴 to the set 𝐵 because my set 𝐷 is a proper subset of the set 𝐵. So now how many
such onto functions I can have from 𝐴 to 𝐷?
So as per my definition of the 𝑇 function, there are 𝑇(𝑚, 𝑘) such functions. So once I have decided
the cardinality of the set 𝐷, I have 𝐶(𝑛, 𝑘) ways of selecting that as per the product rule. For each
way of selecting a subset D of size 𝑘, I have 𝑇(𝑚, 𝑘) number of onto functions from the set 𝐴 and
the chosen 𝐷 set and hence if I multiply that, that will give me the number of non-onto functions
where the image set of those non-onto functions is the set 𝐷.
And then as I said if I take the summation from 𝑘 = 1 to 𝑛 − 1 that will give me the overall
number of non-onto functions.
524
Let us go the question 6. Here we recall our sterling functions of type 2, namely the 𝑆(𝑛, 𝑘)
function which is the number of partitions of an 𝑛 element set into 𝑘 non-empty disjoint subsets
and we have to prove that it satisfies this recurrence condition. So, since the plus is involved here
definitely I have to show that I have 2 different categories or disjoint, categories of partitions.
Indeed that, is the case.
So consider an arbitrary set 𝑋 consisting of 𝑛 elements which I denote as 𝑎! to 𝑎% . And I have to

find out in how many ways I can partition this collection into 𝑘 disjoint subsets. Such that if I take
the union of those subset I get the whole set 𝑋 and the subset should be pairwise disjoint. So I can
have 2 categories of partitions. Category 1 where I divide or I partition the first 𝑛 − 1 elements
into 𝑘 − 1 subsets; that means I am not considering 𝑎% right now.
I am imagining that I have only 𝑛 − 1 elements and I want to partition them into 𝑘 − 1 subsets.
And then create an extra subset which only has the element 𝑎% . That will now result in total 𝑘
number of disjoint subsets which will whose union will cover the entire set 𝑋. So how many such
partitions I can have? The number of such partitions is nothing but the number of partitions
possible for partitioning the 𝑛 − 1 elements into 𝑘 − 1 disjoint subsets and as per the definition of
𝑆 function 𝑆(𝑛 − 1, 𝑘 − 1) is the number of such partitions.
Category 2 where I take the 𝑛– 1 elements and partition it into 𝑘 disjoint subsets not 𝑘 − 1. Now,
I have to still put 𝑎% somewhere. In this category, what I am going to do is I am going to consider
525
the case that 𝑎% is going to occupy any of these 𝑘 subsets. So this is clearly different from the case
where we have explicitly reserved a whole partition 𝑃" for the n-th element. And the remaining
𝑘 − 1 subsets where covering the remaining 𝑛 − 1 elements.
Here the entire 𝑘 subsets are going to cover the 𝑛 − 1 elements and 𝑎% also is supposed to occupy
1 of those subsets. So it is easy to see that the number of partitions in this category will be first we
have to find out how many ways I can partition 𝑛 − 1 elements into 𝑘 − 1 subsets. And once I
have decided the subset 𝑃! to 𝑃" , I have the option of either putting 𝑎% namely the n-th element in
𝑃! that is 1 option, or I have the option of putting 𝑎% into the second subset and so on.
And that is why 𝑘 is multiplied here because of the product rule. And if I sum these 2 categories
of partitions that will give me the overall number of partitions. So what will be the value of
𝑆(𝑛 + 1, 𝑛)? The number of partition of an 𝑛 + 1 element set into 𝑛 non-empty disjoint subsets.
It is nothing but 𝐶(𝑛 + 1,2) because if I have 𝑛 + 1 elements namely 𝑎! to 𝑎%+! .
And if I want to divide it into 𝑛 subsets then that means that there will be only 1 subset with 2
elements and remaining all other subsets will have 1 element each. So now I have to just find out
the 2 special elements which are going to occupy that subset which is allowed to have 2 elements
and that will automatically ensure that the remaining 𝑛 − 1 elements which are left each of them
go to a single distinct subset. So that is why the answer here is 𝐶(𝑛 + 1,2) because there are
𝐶(𝑛 + 1,2) ways of picking 2 elements from 𝑛 + 1 elements.
So it could be either 𝑎! , 𝑎 2 which go together in one subset. And remaining all elements go to
single distinct subset each or it could be 𝑎! , 𝑎' which go together in 1 subset and remaining
elements go to single distinct subsets each and so on. That is why the answer is 𝐶(𝑛 + 1,2).
526
In question 7, we have to give a proof for combinatorial proof for this identity. So let’s see the
following setting. You are given 𝑛 mathematics professor and 𝑛 computer science professor; you
can imagine like that and say that your goal is to select the committee consisting of total 𝑛
professors where the head of the committee is a mathematics professor. That is my goal. Now I
will show that the number of committees satisfying this goal is the expression both in your LHS
as well as in the RHS.
So why LHS expression satisfies or gives a number of committees satisfying my goal? Because
suppose the committee has 𝑘 mathematics professors and 𝑘 will range from 1 to 𝑛. Definitely I
have to include 1 mathematics professor because the head will be a mathematics professor and in
the worst case I can have all the committee members being the mathematics professors. In that
case the remaining 𝑛 − 𝑘 committee members will be from computer science.
So that tells you that how many ways you can pick the committee. You first pick the 𝑘 mathematics
professors; that automatically ensures that the remaining 𝑛 − 𝑘 committee members are from
computer science. So they can be selected in 𝐶(𝑛, 𝑛 − 𝑘) = 𝐶(𝑛, 𝑘) many ways. And once you
had decided which 𝑘 mathematics professors are going to be there and which 𝑛 − 𝑘 computer
science professor are there the committee head itself can be selected out of those 𝑘 chosen
mathematics professors in 𝑘 ways.
527
It could be either the first chosen mathematics professor or the second chosen mathematics
professor or the k-th chosen mathematics professor. So that is why you multiply it with 𝑘 because
of the product rule and that tells you that why the LHS expression gives you the number of
committee members satisfying my goal. Let us see the RHS expression. I can say that in order to
satisfy my goal I have to do the following. I have to first select the committee head and this can be
done in 𝑛 ways because there are 𝑛 mathematics professors.
So that is why this 𝑛 here. Now once I have decided the committee head I have to still choose the
remaining 𝑛 − 1 committee members. And they can be chosen from 𝑛 computer science professors
and the remaining 𝑛 − 1 mathematics professors. Namely I have still 2𝑛 − 1 professors left. I have
to choose 𝑛 – 1 members out of them which can be done in 𝐶(2𝑛 − 1, 𝑛 − 1) many ways. And
now by the product rule, if I multiply the number of ways in which I can select the committee head
and the number of ways I can pick the remaining committee members I get the number of
committees satisfying them. So with that we end our part 1 of tutorial 6. Thank you.
528
IIIT, Bangalore
Module No # 08
Lecture No # 38
Tutorial 6: Part II
Hello everyone, welcome to the second part of tutorial 6.

Let us start with question number 8. You are given here arbitrary distinct points in 2 dimensional
planes. Each point will have an x-coordinate, y-coordinate and the points are having integer
coordinates. So they are arbitrary points except that they are distinct. So, I am denoting the points,
their respective coordinates as 𝑥! , 𝑦! , 𝑥" , 𝑦" , 𝑥# , 𝑦# , 𝑥$ , 𝑦$ and 𝑥% , 𝑦% . And our goal is to show that
irrespective of the way these 5 points are chosen arbitrarily they are always exist a pair of points
such that if you consider the midpoint of the line joining those 2 points it has integer coordinates.
So just to recap if you have 2 points, a point with coordinates (𝑎! , 𝑏! ) and another point with
coordinates (𝑎" , 𝑏" ) then the midpoint of the line joining these 2 points is given by the formula
&! '&" (! '("
) , *. And we want to apply here pigeonhole principle. So remember for pigeonhole
" "
principle we have to identify the set of pigeons and the set of holes here and then the mapping
which relates the pigeon and the holes.
529
So let us do that. So consider the set of 5 arbitrary points which are all distinct and have integer
coordinates. We are trying to map this point depending upon what is the nature of their x-
coordinate and y-coordinate. So depending upon whether the x-coordinate is even, or x-coordinate
are odd, or whether the y-coordinate is odd, or the y coordinate is even, I have 4 possible
combinations.
And my function 𝑓 maps these 5 points to the corresponding pair; say if 𝑥! is odd and 𝑦! is even
then I will say that 𝑓(𝑥! , 𝑦! ) is (odd, even) and so on. That is the mapping here. So now, it follows
from pigeonhole principle that we have now 5 items here in the set 𝐴 and 4 items in the set 𝐵 then
there always exist a pair of points among these 5 points say (𝑥) , 𝑦) ) and .𝑥* , 𝑦* / such that both of
them are mapped to the same ordered pair.
So it could be any 2 out of those 5 points; it could be the first 2 points, it could be the last 2 points,
it could be the third point or the fourth point and so on; we do not know. It depends upon the exact
5 points that we chose. So, without loss of generality assume that out of those 2 points which are
guaranteed to be mapped to the same ordered pair are the first 2 points.
So say (𝑥! , 𝑦! ) and (𝑥" , 𝑦" ) be the 2 points such that the corresponding 𝑓 output of the 𝑓 function
for these points are the same. Now we want to inspect what happens to the midpoint of the line
joining these 2 points 𝐴 and 𝐵. So as per the formula the midpoint of the lines joining these 2
+! '+" ,! ',"
points 𝐴 and 𝐵 will be ) "
, "
*.
And since both the points 𝐴 and 𝐵 are mapped to the same ordered pair; so for instance it could be
the case that both 𝑥! as well as 𝑥" are odd or it could be the case that both 𝑥! as well as 𝑥" are
even. So irrespective of the case 𝑥! + 𝑥" will always be divisible by 2. If both of them are even
definitely sum of 2 even quantities is divisible by 2. Whereas if both of them are odd then also the
sum of 2 odd quantities is divisible by 2. And as per our assumption it is not the case that 𝑥! is odd
and 𝑥" is even that is not the case because we are considering the case when the output of the
𝑓 function on these 2 points 𝐴 and 𝐵 are the same.
530
In the same way we cannot have the case where 𝑥! is even and 𝑥" is odd because that is not the
property of the point 𝐴 and 𝐵. Due to the exactly the same reason, the type of 𝑦! and 𝑦" coordinates
are the same. Either they are both odd or both of them are even right. And again in this case it is
easy to see that 𝑦! + 𝑦" will be divisible by 2. And that shows that this statement is a correct
statement.
So let us go to question number 9. Here you are given the following. You are choosing 5 integers
from the set 1 to 8 arbitrarily. Our goal is to show irrespective of the way you choose those 5 points
there always exists at least one pair of integers among those chosen 5 integers whose sum is 9. So
say you pick 1, 2 and 5 and then if you pick 3 then you still do not have any pair of integers whose
sum is 9. But as soon as you pick the fourth point, so if you pick 4 that is the fifth number then
you have 5 and 4 which is summing up to 9.
If you pick 6 as the fifth number, then you have 3 and 6 summing up to 9. If you have if you pick
7 as the fifth number, then you have 7 and 2 summing up to 9. If you pick 8 as the fifth number,
then you have 1 and 8 summing up to 9 and so on. So you can verify this by an example but we
want to prove it irrespective of the 5 numbers that we are going to pick.
So one way of proving this is that you take all possible 8 choose 5 ways of picking 5 numbers and
for each of those combinations you show that the statement is true but that will be an overkill
because this is a relatively large value. Instead we will apply the pigeonhole principle and again
531
for applying the pigeonhole principle we have to identify the pigeons and the holes and the
mapping. So my pigeons here are the 5 integers among the numbers 1 to 8 that I am picking
arbitrarily and my holes are the ordered pairs of distinct integers in the set 1 to 8 whose sum will
give you 9.
So you have either the ordered pair (1, 8) or the ordered pair (2, 7) or ordered pair (3, 6) or the
ordered pair (4, 5). And you do not have any other ordered pair from the set 1 to 8 summing up to
9. And my function 𝑓 basically maps these 𝑥) values to the corresponding ordered pair depending
upon whether 𝑥! is 1 or 8 I will say that 𝑓(𝑥! ) is either (1,8). Or if 𝑥! takes either the value of 2
or the value of 7 then I will say 𝑓(𝑥! ) is (2, 7).
Or if my 𝑥! is either 3 or 6 then I will say that 𝑓(𝑥! ) is (3, 6) or if my 𝑥! is either 4 or 𝑥! is either

5 then I will say that 𝑓(𝑥! ) is (4, 5). That is the interpretation for my mapping 𝑓.
Now it follows simply from pigeon-hole principle that there always exists a pair or two values out
of the 5 numbers say .𝑥) , 𝑥* / such that 𝑓(𝑥) ) and 𝑓.𝑥* / are the same. It could be say the first 2
values, the last 2 values, the second or the third value, the third or the fourth value, or the first
value or the fifth value; it could be any 2 values out of those 5 numbers.
532
We do not know which one. So again without loss of generality, suppose both of them got mapped
to (1, 8); we do not know what is the identity of 𝑥) or 𝑥* and we do not know the corresponding
mapping as well. It could be either (1, 8), (2, 7), (3, 6) or (4, 5). So, without loss of generality; that
means whatever reasoning we are giving here for the case where 𝑓(𝑥) ) = 𝑓.𝑥* / = (1,8) hold, the
same argument will hold even if 𝑓(𝑥) ) is same as 𝑓.𝑥* / is equal to say (2, 7); the same reasoning
will hold symmetrically for that case as well.
Symmetrically for the case when it is (4, 5), symmetrically for the case when it is (3, 6) and so on.
So that is why we do not consider the remaining 3 cases. We just consider the case when 𝑓(𝑥) )
and 𝑓.𝑥* / is (1, 8).
533
If that is the case then since your 𝑥) and 𝑥* are distinct and they got mapped to (1, 8) that means
either 𝑥) is 1 and 𝑥* is 8 or 𝑥) is 8 and 𝑥* is 1. Irrespective of the case, the sum of 𝑥) and 𝑥* is 9. So
now you can see that even without enumerating all possible 𝐶(8,5) arrangements or combinations
of picking 5 numbers out of these 8 numbers we ended up arguing in a very simple fashion that
our statement is true using pigeonhole principle. It shows the power of this proof strategy or
counting mechanism basically.
So question 10 we want to prove a universally quantified statement. Namely, we want to prove

that you take any integer 𝑛, there is always a multiple of 𝑛 which has only the digits 0’s and 1’s in
534
its decimal expansion. So before going into the proof if you want to take few examples say 𝑛 = 1
then I always have the number 1 which is a multiple of 1 and which has only the digit 1 in its
decimal expansion.
Remember it is not mandatory that you have both 0’s as well as 1 in the decimal expansion. The
only restriction is we have to show that in the decimal expansion you only have either the digits
0’s or 1’s. If you take 𝑛 = 2 then I can take the number 10 which is a multiple of 2 and which
has only 1’s and 0’s and in its decimal expansion. If I take 𝑛 = 3 then I can take the number 111
which has only the digit 1 in its decimal expansion and which is divisible by 3.
So at least by taking few examples we found that the statement is true. But this is a universally
quantified statement and we cannot prove a universally quantified statement just showing
examples for a few cases. So we have to give the proof for arbitrary 𝑛. Again, we are going to
apply here pigeonhole principle. So let me define a few decimal numbers here.
I define the first decimal number to be 1. I define second decimal number as 11, the i-th decimal
number as a decimal number consisting of 𝑖 number of 1’s and the 𝑛 + 1 decimal number which
has the digit 1, 𝑛 + 1 number of times. Let me define another set of values. So my value 𝑟! is the
remainder which I obtain by dividing 𝑎! by 𝑛. Similarly, I define 𝑟" to be the remainder obtained
by dividing 𝑎" by 𝑛. I define 𝑟) to be the remainder obtained by dividing 𝑎) by 𝑛.
And in the same way I define 𝑟-'! as the remainder obtained by dividing 𝑎-'! by 𝑛. Now what
can I say about this remainders? It is easy to see that these remainders belong to the set 0 to 𝑛 − 1
because of the simple fact that you divide any number by n the only possible remainders could be
0 if it is completely divisible by 𝑛 or the remainders could be 1, 2 … 𝑛 − 1. Now you have to
apply the pigeonhole principle.
So my pigeons are the numbers 𝑎! to 𝑎-'! that I have constructed here. And my holes are basically
the remainders which I can obtain by dividing these 𝑛 + 1 numbers by 𝑛. And I have 𝑛 possible
remainders and my function 𝑓 map the numbers to the corresponding remainder which I have
obtained by dividing that number by 𝑛. So you have more number of numbers and less number of
remainders.
535
So it follows from the pigeonhole principle that you always have a pair of numbers 𝑎) and 𝑎* out
of this 𝑛 + 1 numbers which gives you the same remainder if you divide 𝑎) and 𝑎* by 𝑛. I do not
know the remainder it could be either 0, or the remainder could be either 1, or the remainder could
be 𝑛 − 1.
I do not know what are the individual remainders that 𝑎) and 𝑎* are going to give on dividing by
𝑛. But what I know is that they are leaving the same remainder. And again without loss of
generality assume that 𝑎) is occurring before 𝑎* in my sequence here. Now what can I say about
this number 𝑎* – 𝑎) . So 𝑎* will be a number which has 𝑗 number of 1’s and 𝑎) is another number
which has 𝑖 number of 1’s. Both of them gives me the same reminder on dividing by 𝑛.
So if I take 𝑎* – 𝑎) then this will be a decimal number which will have trailing 0’s and then at the
leading positions you will have the 1’s. That means it is a decimal number which has only the
characters 1s and 0’s. But what can you say about its divisibility by 𝑛. This number will be divisible
completely by 𝑛 because 𝑎* gives you the same remainder, say 𝑟, so I can say 𝑎* is some 𝑞* ∗ 𝑛 +
𝑟 and 𝑎) also gives me the same remainder 𝑟, so I can write 𝑎) as some 𝑞) ∗ 𝑛 + 𝑟.
Then if I take 𝑎* – 𝑎) the effect of 𝑟 cancels out and I get that its completely now a multiple of 𝑛.
So, I showed you constructively here that irrespective of what is your 𝑛, I can always give you a
number which is divisible by 𝑛 and which has only 1’s and 0’s in its decimal expansion right.
536
So now let us go to question number 11 which is really a very interesting question. Here we want
to show the following that you take any sequence of 𝑛" + 1 distinct real numbers. They are
arbitrary real numbers; may be positive, negative in any order you take them. The only condition
is that they have to be distinct. Then the claim is that irrespective of the 𝑛" + 1 real numbers that
you have in your sequence you always have a subsequence of length 𝑛 + 1 which is either strictly
increasing or strictly decreasing.
First of all what what is a strictly increasing sequence? A sequence of the form (𝑎! , 𝑎" , … ) where
𝑎! < 𝑎" < 𝑎# < ⋯ < 𝑎).! < 𝑎) … . Whereas if I have a sequence of the form (𝑎! , 𝑎" , 𝑎# , … )
where 𝑎! > 𝑎" > 𝑎# … > 𝑎).! > 𝑎) > ⋯ then it is a strictly decreasing sequence.
Now what does a subsequence means? A subsequence mean here that the values may not be
consecutive. That means I am allowed to miss few numbers. In the sense, say I take a sequence 1,
3, 0, -5, 2, 8 and so on. Then I can choose to pick 1 and then exclude 3 and 0 and -5. This is a
subsequence. In the same way I can pick a subsequence saying 3, 2 and 8 that means I skip 0, I
skip -5.
So what this question basically says is that irrespective of the way your 𝑛" + 1 distinct real
numbers are chosen you always have a subsequence. By that I mean that you have a set of 𝑛 + 1
values going from left to right but need not be in consecutive locations; some of the locations
537
might be skipped. But the number of values are 𝑛 + 1 such that if you view those 𝑛 + 1 values
they are either strictly increasing or strictly decreasing. That is what we have to prove.
Again, if you want to convince yourself whether this is indeed a true statement or not you can take
some concrete values of 𝑛, try to draw any possible sequence of 𝑛" + 1 for that value of 𝑛 and you
can verify that this statement is true. But now we want to prove it for any arbitrary sequence. How
do we do that? So let the arbitrary sequence of 𝑛" + 1 distinct real numbers be denoted by 𝑎! to
𝑎-" '! .
Why I am taking arbitrary here? Because I want to prove this statement for every sequence. So
this is a universally quantified statement and to prove a universally quantified statement I can use
the universal generalization principle by proving that a statement is true for some arbitrary element
of the domain. My domain here is the set of all possible sequences of 𝑛" + 1 distinct real numbers.
I am just taking one candidate element from that domain arbitrarily.
I do not know the exact values of 𝑎! … 𝑎* … 𝑎-" '! . What I will do is to prove this statement, I will
use pigeonhole principle along with proof by contradiction. So let me first define two values. I
define 𝐼) as the length of the longest increasing subsequence starting at 𝑎) . So a_i will have some
value depending upon what is the arbitrary sequence and it will have some various possible
increasing subsequences starting at 𝑎) .
One might be of length 1, a trivial increasing subsequence starting at 𝑎) is the value𝑎) itself. That
is a subsequence of length 1. But I might be having a subsequence of say length 2 which is strictly
increasing and starting at 𝑎) . I might have a subsequence of length 3 starting at 𝑎) and so on. So
whatever is the length of the longest increasing subsequences starting at 𝑎) , I am denoting by 𝐼) .
In the same way, I define 𝐷) as the length of the longest decreasing subsequence starting at 𝑎) . I
might have several strictly decreasing sequences starting at 𝑎) . In fact the sequence 𝑎) itself is a
subsequence of length 1 which is strictly decreasing. But I might be having a subsequence of higher
length which is strictly decreasing and starting at 𝑎) . So the length of the longest decreasing
subsequence starting at 𝑎) I am denoting it as 𝐷) . So that means with 𝑎! , I have associated the
values 𝐼! and 𝐷! . With 𝑎" , I would have associated the value𝐼" and 𝐷" .
538
And similarly with 𝑎) , I would have associated the value 𝐼) and 𝐷) , with 𝑎* I would have associated
the value 𝐼* and 𝐷* ; and with 𝑎-" '! I would have associated the value 𝐼-" '! and 𝐷-" '! . It is easy
to see that 𝐼-" '! will be 1, 𝐷-" '! is 1. Because I have only one sub sequence starting at 𝑎-" '!
namely the value 𝑎-" '! itself.
It is both an increasing subsequence starting at 𝑎-" '! as well as it is a decreasing subsequence

starting at 𝑎-" '! because there is nothing after the number 𝑎-" '! . Now, what is my goal? The
question basically asks me to show that there always exist some 𝑖 or some value 𝑎) such that there
either exists an increasing subsequence of length 𝑛 + 1 that means 𝐼) is greater than equal to 𝑛 +
1 or there is a decreasing subsequence of length 𝑛 + 1. That means 𝐷) is 𝑛 + 1.
I have to show the existence of one such number 𝑎) in this subsequence. I prove that by assuming
a contradiction. So assume that the statement is false and that means for each 𝑎) in the sequence,
the value 𝐼) is at most 𝑛. That means you take any number in the sequence the maximum length
increasing subsequence of length 𝑛 and the maximum length decreasing subsequence is also of
length 𝑛.
What does that mean? That means if I try to pair all 𝐼) and 𝐷) pairs then they can take the values
in the range (1,1) to (𝑛, 𝑛) namely 𝑛" possible pairs. These are the possible values of (𝐼) , 𝐷) ) pairs.
But how many numbers I have in the sequence? I have 𝑛" + 1 values in the sequence that I have
chosen. That means I have more pigeons and less holes. What does that mean? So by PHP here I
mean pigeonhole principle.
So pigeonhole principle guarantees me that you definitely have a pair of values here say 𝑎) and 𝑎* .
Such that your 𝐼) and 𝐼* are same. That means the length of the longest increasing subsequence
starting at 𝑎) is the same as the length of the longest increasing subsequence starting at 𝑎* . And in
the same way the length of the longest decreasing subsequence starting at 𝐷) is same as the length
of the longest decreasing subsequence starting at 𝐷* .
539
And as per my assumption 𝐼) , 𝐼* , 𝐷) , 𝐷* are all upper bounded by 𝑛 because I assume the
contradiction. Now how do I arrive at a contradiction here? So there could be two possible cases
with respect to the magnitude of 𝑎) and 𝑎* .
The first case 𝑎) < 𝑎* . If that is the case, then what I can say is the following. I can say that you
take the increasing subsequence starting at 𝑎* ; what is its length? Its length is 𝐼* and if I put the
value of 𝑎) at the beginning of that subsequence then that gives me now a new increasing
subsequence starting at 𝑎) and of length 𝐼)'! . But that goes against the assumption that the length
of the longest subsequence starting at 𝑎) was𝐼) . So that is how I arrive at a contradiction.
On the other hand if I take the case when 𝑎) > 𝑎* then I have to just give a symmetric argument.
What I can say is the following. I know that there is a decreasing subsequence starting at 𝑎* and
its length is 𝐷* . My claim is if you take that subsequence and put an 𝑎) at the beginning then that
now give me a new decreasing subsequence starting at 𝑎) and the length of this new decreasing
subsequence is 𝐷_{𝑖 + 1}.
Which now goes against the assumption that the length of the longest decreasing subsequence
starting at 𝑎) was 𝐷) . That means in both the cases I arrived at a contradiction and that shows that
whatever I assumed here that means the value of each 𝐼) and each 𝐷) was upper bounded by 𝑛 is
incorrect. That means there is at least one 𝑎) where either 𝐼) is greater than 𝑛 or 𝐷) is greater than
𝑛. I do not know what exactly is that 𝑎) . So I gave you a non-constructive proof here. But I argued
that the existence of such 𝑎) is guaranteed.
540
Now let us go to question 12. In this question we want to show the following you are arbitrarily
picking 9 people in the age group of 18 to 58. That means the minimum age it is allowed is 18 the
maximum age is allowed 58. Now we want to prove that irrespective of what exactly are their
ages, as long as they are in the range 18 to 58 it is always possible to choose 2 disjoint groups of
people out of this 9 people whose sum of ages is the same.
Again, we will do this by pigeonhole principle. So the first thing is since we want to argue about
a non-empty set of people because when I want to consider the age of the people there have to be
people in the group. So I have to focus on non-empty subset. So if I have 9 people then the number
of non-empty groups that I can form out of those 9 people need not be disjoint is 511. And now
what I can say about the range of the sum of ages in these 511 subsets.
If I consider the minimum sum of ages possible in a group it could be 18. This is possible only
when I have a group of just consisting of one person and that person has age 18. That is a minimum
possible sum. Whereas the maximum possible sum can occur when in my group I have all the 9
people person 1, person 2 and up to person 9 and each of them has age 58.
That is a maximum possible value of sum of the ages in a group we picked from 9 people. That
means the range of possible sums here is 505. So now let us apply the pigeonhole principle. My
pigeons are the various possible non-empty set of people that I can form out of this 9 group of 9
people. So I have 511 possible subsets and my holes are the range of sum of ages. That means
541
what can be the sum of ages if I consider the various possible subsets given that the ages could be
in the range 18 to 58.
So I have more pigeons than holes so by pigeonhole principle I can say that they always exists a
pair of group 𝑆) and 𝑆* such that the sum of ages of the people in 𝑆) and 𝑆* are the same. But my
question wants me to show that the group should be disjoint. So how do I argue that? I can always
form disjoint groups of people out of this 𝑆) and 𝑆* . Well if they are already disjoint then I have
showed the existence of 2 groups having the same sum of ages.
But if the sets 𝑆) and 𝑆* are not same; if they have some common people just remove the common
people from both the set 𝑆) and as well as 𝑆* . The common people in the set 𝑆) and 𝑆* were
contributing the same amount to the sum of ages in the set 𝑆) as well as in the set 𝑆* . So if I remove
those common people the same amount will be removed from the sum of ages in 𝑆) and 𝑆* . And
now where I will get 2 disjoint groups of people having the same sum of ages.
In question 13 you are given the set of 𝐴 consisting of the numbers 1 to 2𝑛 and we want to show
that if I pick an arbitrary subset 𝐵 consisting of 𝑛 + 1 elements from the set A and irrespective of
the subset there always exist a pair of values such that one divides the other. And this is again a
very interesting question. So for applying the pigeonhole principle what I do is I divide this set 𝐴
542
into 2 disjoint subsets namely the subset consisting of the odd values and the subset consisting of
the even values.
Both of them will have the cardinality 𝑛. Now my claim is the following: you take any value in
the set 𝐴, it has a unique factorization of the form that you have some power of 2 multiplied by the
remaining value where the remaining value will be a number, specifically an odd number in the
subset 1 to 2𝑛 − 1. For example, if your number 𝑥 that you are taking in the set 𝐴 is already an
odd number then I can write it in the form 2/ ∗ 𝑥.
So in this case my a will be 0 and my 𝑥 will be 𝑥 itself. So my statement is true. Whereas if your
𝑥 would have been say 6 then 6 can be written as 2 into 3. So you have 2! times an odd value. If
your 𝑥 is say 10 then you can write it as 2 1 ∗ 5. If your 𝑥 is say 20, then you can write it as 2" ∗
5. So you can see that irrespective of the case, whether your 𝑥 is odd or even, this claim is always
true.
So for 𝑥 being odd this statement is always true. But the statement is true even for a general 𝑥
which is even because for such 𝑥 where 𝑥 is either 2, 4, or 2𝑛; I can express it in the form 2 power
sum positive exponent 𝑒! followed by the remaining values. And this is because of the fundamental
theorem of arithmetic that every integer has a unique prime factorization. The claim is that if I
consider the remaining prime factorization here then that will be an odd value.
And that odd value will be in the set 𝑂 here and it is easy to verify that. So that means this claim
is true. Now based on this claim I have to apply the pigeonhole principle. For applying the
pigeonhole principle I do the following. Let 𝐵 be the arbitrary set of 𝑛 + 1 values that I have
chosen. And I mapped those arbitrary chosen values to the leftover value in its unique factorization
that this claimed guarantees.
So 𝑎! will be written in the form of some 2+! ∗ 𝑏. So 𝑎! will be mapped to this 𝑏! ; 𝑎" will be
written in the form of some 2+" into leftover thing. That left over thing is an odd number in the set
𝑂. So a_2 will be mapped to 𝑏" and so on. That is a mapping 𝑓 here. Now what is the cardinality
of set 𝑂? That is 𝑛; that means my number of holes is 𝑛. But the number of pigeons is 𝑛 + 1. That
543
means by pigeonhole principle it is guaranteed that there exists a pair of values 𝑎) and 𝑎* out of
this n+1 values.
Where 𝑎) is sum 2+# into some left over thing which is an odd value. And 𝑎* is some 2+$ multiplied
by the same leftover value. I do not know the exact value of that left over odd value 𝑏. But that
left over odd value 𝑏 will be the same; that is a guarantee. And exponents 𝑥) ≠ 𝑥* because I am
considering the distinct vales 𝑎) and 𝑎* . But what is guaranteed is that the leftover odd value here
that is there as per this unique factorization claim will be the same. Now I have 2 possible cases if
𝑎) is greater than 𝑎* then clearly 𝑎* divides 𝑎) because if I divide 𝑎* by 𝑎) then the effect of 𝑏 goes
out and the exponent 𝑥) is greater than exponent 𝑥* . So, whatever is leftover that will be the quotient
and the remainder will be 0.
Whereas if 𝑎* is greater than 𝑎) then again the effect of 𝑏 vanishes and 2+$ /2+# that will give you
0 remember. So irrespective of the case my statement is correct.
Now let us go the last question. Here we want to find out how many solutions are there for the
equation 𝑥! + ⋯ 𝑥0 = 29 where there are various possible restrictions on 𝑥) . So in part a, we have
the restriction that each 𝑥) has to be greater than 1. So you can imagine that you are given here
bills of type 𝑥! , bills of type 𝑥" and bills of type 𝑥0 . We have to pick total 29 bills with the
restriction that you have to definitely pick more than one bill of each type.
544
That is the interpretation of this first restriction. That means I have to compulsorily pick 2 items
of type 𝑥! , 𝑥" … 𝑥0 . That means I had already picked 12 items compulsorily. That means now I am
left over with the problem of picking 17 bills in total, out of this 6 different bill types where there
are no restrictions. And remember as per the formula for the number of combinations with
repetitions the answer is 𝐶(6 − 1 + 17,17) = 𝐶(22,17).
In part b, the restriction is 𝑥) ≥ 𝑖. Again, if I interpret this restriction that means I have to definitely
include one bill of type 𝑥! , 2 bills of type 𝑥" , 3 bills of type 𝑥# , 4 pills of type 𝑥$ , 5 bills of type
𝑥% and 6 bills of 𝑥0 . That means I have already picked 22 bills of various types. That means now
my goal was to pick 29 bills; 22 definitely I have already picked. So, I am left over with the
problem of picking 7 bills where those 7 bills can be of type 𝑥! , 𝑥" to 𝑥0 in any possible order, no
restrictions. So again, from the formula for number of r-combinations with repetition the answer
will be 𝐶(6 − 1 + 7,7) = 𝐶(12,7).
In part c, the restriction is that 𝑥_1 \𝑙𝑒 5. So, what we do here is the following. We first find out
the number of solutions where there is no restriction. That means 𝑥_1 maybe 0 as well; those
solutions are also included in this quantity. And now I try to find out those solutions where this
condition namely 𝑥_1 less than equal to 5 is violated. That means find the number of solutions
where 𝑥_1 is greater than equal to 6.
That means definitely I have to pick 6 bills of type 𝑥_1 which further implies that now I am
interested to pick the remaining 23 bills without putting any restriction that how many bills of
different types I have to choose. The number of solutions for this case will be this. But this is not
what we want. We want to find out the number of solutions which do not have this condition. So
what I do? I subtract this value from the set of or from the number of solutions that I have without
any restrictions and that will give me the answer.
545
The last part here, my restrictions are 𝑥_1 < 8 and 𝑥_2 > 8. Let us first try to find out the number
of solutions where 𝑥_2 \𝑔𝑒 9. That means just try to satisfy the second restriction here. The
number of solutions will be this because if 𝑥_2 is greater than equal to 9 that means 9 bills of type
𝑥_2 definitely have to be chosen. That means now I am left with the problem of picking 20 bills
from bills of 6 types without any restrictions.
And now let us try to find out the number of solutions where this first restriction is violated, namely
𝑥_1 is greater than equal to 8 and 𝑥_2 is greater than equal to 9. So, what basically I am trying to
do is the set 𝐴 that I have defined here it has all those solutions where 𝑥_1 is less than 8 as well as
𝑥_1 is greater than 8. So, I am trying to take out those solutions where 𝑥_1 is greater than equal to
8 from this set A. I am denoting that set as B and the cardinality of the set B is this because if I am
supposed to satisfy 𝑥_1 greater than equal to 8 and 𝑥_2 greater than equal to 9 that means I have
already picked 17 bills. My goal will be now to pick 12 more bills from bills of 6 types without
any restrictions. This will be the number of ways the number of solutions. And as I have said from
the interpretation of the set 𝐴 and 𝐵 the required number of solutions is the difference of these 2
cardinalities which we can easily find out. So with that we finish our tutorial number 6. Thank
you.
546
Lecture -39
Solving Linear Non- Homogeneous Recurrence Equations
Hello everyone welcome to this lecture. So, in this lecture we will continue our discussion
regarding how to solve recurrence equations, linear recurrence equations. And in this lecture we
will focus on how to solve linear non-homogeneous recurrence equations.
547
So, let us first discuss the general form of any linear non-homogeneous recurrence equation of
degree k with constant coefficients. So the general form will be this, the nth term will depend on
previous terms plus some function of n, F(n). So, here your coefficients c1 to ck are real numbers
they could be 0 as well but the only restriction is that ck is not allowed to be 0 that means the nth
term definitely depends upon the (n – k)th term.
And that is why the degree of this equation is k. And F(n) will be a function of n that is why it is
called non-homogeneous recurrence equation. So, some examples of recurrence equation in this
category are as follows. So, in this equation your F(n) is 2n and this equation your F(n) is n2 + n
+ 1 and so on, it turns out that unlike linear homogeneous recurrence equations of degree k for
which we have a standard method of finding the general form of any solution, we do not have
any standard method for solving the non-homogeneous recurrence equation because we do not
know what could be the structure of this function F(n). But it turns out that for some specific
form of this function F(n) we have some well-known methods and in this lecture we will discuss
those methods.
548
So, the first thing that we do, while solving the linear non-homogeneous recurrence equation is
the following. We form what we call as the associated recurrence relation, associated
homogeneous recurrence relation to be more specific and this is obtained by chopping off this
F(n) function. So, if I chop off this F(n) function then whatever recurrence relation I am left over
with that is called as the associated homogeneous recurrence relation.
It will be of degree k and then we know how to solve this. We have seen extensively in the
earlier lecture how to solve a linear homogeneous recurrence equation of degree k, the general
form of it can be obtained by using those methods. So, let the solution be denoted by the
sequence whose nth term is an(h). So, this h denotes that the sequence is the solution of the
associated homogeneous sequence relation. It may not satisfy the entire recurrence equation.
So, remember we have to solve or we have to find out a sequence satisfying the entire recurrence
equation where F(n) is also a part of the equation, but the sequence an(h) is a solution only for the
associated homogeneous recurrence relation. And then what we do is the following, we try to
find out one of the solutions satisfying the whole recurrence equation. Namely a sequence
satisfying the entire recurrence equation and we call that solution as a particular solution.
So, the sequence satisfying the entire recurrence equation which is the particular solution the nth
term of it is denoted by an(p). Then the claim is that any sequence which satisfies the entire
549
recurrence equation; its nth term will be the summation of the nth term of the sequence satisfying
the associated homogeneous equation and the nth term of the particular solution; that is the
statement.
So, what basically we are trying to say here is that once you have obtained one of the solutions
satisfying the entire recurrence equation,then you can express any solution satisfying the entire
recurrence equation in terms of that particular solution or one of the solutions that you have
obtained and the solution for the associated homogeneous reference equation. By the way now if
you substitute an(h) to be 0 then you get automatically that bn = an(p) is also one of the solutions
satisfying the entire recurrence equation.
So, we can derive any solution satisfying the entire recurrence equation from this generic
solution.
So, let us prove this theorem, we want to prove that any solution satisfying the entire recurrence
equation is of this form. Namely its nth term is the summation of the nth term of the
homogeneous recurrence equation or associated homogeneous sequence equation plus nth term
of the particular solution that is what we want to prove. So, since an(p) is one of the solutions
satisfying the entire recurrence equation, we can say that its nth term is c1 times the n -1 term of
that sequence + c2 times the n - 2 term of that sequence.
550
And like that ck times the (n – k) term of that sequence + F(n), because that is what is the
implication of saying that this particular solution satisfies the entire recurrence equation. And
now if there is another sequence whose nth term is bn and that is also one of the sequences
satisfying the entire recurrence equation, then we get the implication that bn is equal to c1 times
b(n – 1) + c2 times b(n – 2) + … + ck times b(n-k) + F(n).
Then what we can say is the following, if I subtract the nth term of the particular solution and the
nth term of the b sequence then the effect of F(n) and F(n) cancels out and we get this property.
And now how can you interpret this property? You can interpret this property as if you have a
sequence whose nth term is the difference of the nth type of the b sequence and the particular
solution and this sequence is now satisfying the associated homogeneous recurrence relation.
Why so? Because if you take this property here, this property basically says that this bn - an(p) is
equal to c1 times (b(n – 1) – a(n – 1)(p)) + up to that like that ck times (b(n – k) - a(n-k) (p)). So, you can
imagine that you have now a sequence satisfying the associated homogeneous recurrence
equation because the associated homogeneous equation is that an the nth term should be equal to
c1 times (n – 1) th term + c2 times (n - 2) th term + … + ck times (n – k) th term.
And indeed you have a sequence satisfying this recurrence relationship and the nth term of that
sequence is basically bn - nth term of the particular solution. So, based on this what we can say is
the following, we can say that the nth term of the sequence which is basically the difference of
the nth term of the b sequence and the nth term of the particular solution is a solution for the
associated homogeneous equation.
And since we have used the notation an(h) for denoting the solution for the nth term of the
sequence satisfying the associated homogeneous equation, we get the implication that any
sequence satisfying the entire recurrence relation; its nth term is basically the summation of the
nth term of the sequence satisfying the associated homogeneous recurrence relation plus the the
nth term of the particular solution satisfying the entire equation, so that proves our theorem.
551
So, coming back to the method for how to solve linear non-homogeneous recurrence relation the
first step will be to solve the associated homogeneous equation which is easy. Because the
associated homogeneous equation will be of degree k and we will know how to solve it because
we have seen extensively how to solve linear homogeneous recurrence relation of degree k. So,
say the general form of the solution satisfying the associated homogeneous equation is this.
The step two is the difficult part, namely coming up with a particular solution satisfying the
whole equation, and this is the challenging part of, coming up with the general solution
satisfying the entire equation. So, remember we want to find out the general form of any solution
that satisfy the entire recurrence equation and for that we need one of the solutions satisfying the
entire equation.
You might be wondering that if we can find out one of the solutions satisfying the entire
equation why we are bothered to find other solutions satisfying the other equation. That this is
because we want to find the general formula which covers all possible solutions all the sequences
satisfying the given recurrence condition. Remember a recurrence equation can have infinite
number of solutions if I do not give you the initial conditions.
So, I would like to find out a general formula, a general form of the solution which covers all
possible sequences satisfying the given non-homogeneous recurrence equation. Just finding one
552
of the solutions is not sufficient, that is one of the sequences satisfying the entire equation but I
might be interested to find out other sequences as well and that is why we need the particular
solution.
So, for finding this particular solution we do not have any well known methods or rules. What
we do is to try to find out the particular solution by using what we call as trial and error and this
trial and error method becomes easy for some specific forms of this function F(n). We will see
those specific forms and assuming that we have obtained a particular solution then we can write
down the general solution as a summation of the associated homogeneous equation and the
particular solution. That is a method for solving linear non-homogeneous recurrence equations.
So, now let us see how we can find out the particular solution for some specific forms of F(n)
function using the trial and error method. So, let us take this example where my F(n) is 2n and
the associated homogeneous equation is of degree 1, So, I can solve it, the characteristic equation
will have only one root namely 3. So, the general form of the associated homogeneous equation
will be this where α is some constant, unknown constant.
And now we have to find out the particular solution satisfying the whole equation. So, for that I
will make some guess about the particular solution. So, in this case I observe that my F(n) is a
polynomial of degree 1, because my F(n) here is 2n which is a polynomial of degree 1. So, I
553
make a guess that let my particular solution be a polynomial of degree 1 for some constant c and
d. I do not know the exact values of c and d I am just guessing that let this be the particular
solution : cn + d.
And now I have to check whether my guess is correct or not about the particular solution. How
do I check whether my guess regarding the particular solution is correct or not? I have to check
whether there exist values of c and d, such that if I substitute those values of c and d in my
guessed particular solution then it satisfies the entire recurrence equation. So, let us do that, In
order that cn + d or a sequence whose nth term is cn + d satisfies the entire recurrence condition
this relationship should hold.
Namely the nth term is this, should be equal to 3 times the (n – 1) term and the (n – 1) term as
per my guess of the particular solution will be this plus the function of n, and this relationship
will hold if c is equal to - 1 and d is equal to – 3/2. How do I get these values of c and d? Well
basically I rearrange the terms here, and then compare the LHS part and RHS part and based on
the comparison I come up with these values.
That means it turns out that if indeed c is equal to - 1 and d is equal to – 3/2, then I have a
particular solution namely a sequence whose nth term is (– n – 3/2) and this sequence satisfies
the entire recurrence equation. So, I am successful in finding the particular solution; successful in
the sense my guess about the particular solution here is correct. And then I will say that my
general solution satisfying the entire recurrence equation namely any solution satisfying the
entire recurrence equation will be of this form: it will be the summation of the nth term of the
sequence satisfying the associated homogeneous equation which is some constant times 3n, plus
the nth term of the particular solution which I am able to find out using the trial and error
method. So, this will be the general form of any solution. Now if you are interested to find out
the unique solution satisfying the entire recurrence equation as well as the initial condition.
You have to substitute n equal to 1 here and by substituting n equal to 1 here you will be able to
find out the value of the exact constant α satisfying the initial condition as well as the entire
recurrence condition.
554
Now let us see another case or another structure of F(n) function. So, in this case my F(n)
function is some constant power n. So the step one will be solving the associated homogeneous
equation, so it will be an equation of degree 2. So, the characteristic equation will have two roots
and the two roots are distinct. So the general form of the solution for the associated
homogeneous equation will be some constant times 3n plus another constant times 2n.
Then my goal will be to find out the particular solution satisfying the entire recurrence equation.
So, what I do is in this case as I said my F(n) is some constant and the constant is 7n, I make a
guess that let my particular solution be some constant times 7n. So, α3 is now a constant here. So,
I am using different constants I am using α3 here as a different constant to distinguish it from the
constants α1 and α2 which are there as part of the solution of the associated homogeneous
equation.
Now I have to check whether indeed my guess about the particular solution is correct or not, how
do I check that? Well I have to see whether I can find out the value of α3 such that, that value of
α3 times 7n is a solution for the entire recurrence equation. So, I have to check whether there
exists a value of α3 such that a sequence whose nth term is α3 times 7n satisfies the condition that
it is equal to 5 times the (n – 1)th term.
555
So, the 5 times (n – 1)th term for that particular solution or the guessed particular solution will be
this minus 6 times the (n – 2)th term of the guessed particular solution plus 7n. And then again if
I rearrange the terms I see that indeed it is possible to have a value of α3 namely if α3 is 49/20
then I can say that a sequence whose nth term is 49/20 times 7n satisfies the entire recurrence
condition.
That means in this case again I am able to successfully find out a particular solution and now the
rest of the steps are simple. I will now say that the general solution of any sequence satisfying
the entire recurrence condition will be this; the nth term will be this. So, here the unknowns will
be now α1 and α2 if you are given two initial conditions for the given recurrence equation then by
substituting n = 1 and n = 2 you can find out the exact values of α1 and α2.
But if you are not given the initial conditions then you will say that any sequence satisfying the
entire recurrence condition will be of this form.
Now let us see another example. Here my F(n) is 3n, so I will first solve the associated
homogeneous equation. The solution will be this, this is because 3 will be the characteristic root
and it will be repeated two times. So that is why the general form the solution for associated
homogeneous equation will be an unknown polynomial of degree 2 followed by the
characteristic root raised to power n.
556
Step 2: I have to find out a particular solution. So as I did in the previous example my guess will
be that the particular solution is some constant times 3n. And now if I proceed to check whether
my guess about a particular solution is correct or not, I have to check whether I can find out the
value of α4 such that this condition holds, namely the nth term of this particular solution should
be equal to 6 times the (n – 1)th term of the particular solution - 9 times the (n – 2)th term of the
particular solution + 3n.
And now if I rearrange the terms and try to solve and come up with the value of α4 you will see
that I cannot find the value of α4. There exists no value of α4 such that this condition holds, I
cannot do that. Then where I am going wrong, it worked for the previous example where my
F(n) was 7n but then why it is not working here? Well the reason it is not working here is that
your F(n) is 3n and 3 is also a characteristic root.
Whereas in the previous example my F(n) was 7n but 7 was not a characteristic root, it turns out
that if I now make a guess that my particular solution is some constant times (n2 3n). And then
try to check whether this satisfies the recurrence condition or not, namely whether I can find out
the value of α4 such that it satisfies the given recurrence condition I will be able to find out the
value of the constant α4. Namely by rearranging the terms you will see that α4 being 1/2 is a valid
solution.
557
So, now let us unify all the examples that we have discussed till now and come up with the
general theorem statement. So, the claim is the following, imagine you are given a linear non
homogeneous equation of degree k and suppose your F(n) function is of the following form, it is
a polynomial of degree t and some constant power n. Suppose it is of this form then we have to
check the following.
We will check if this constant s is a root of the characteristic equation of the associated
homogeneous equation or not. So, remember the step 1 for solving the non-homogeneous
recurrence equation will be to solve the associated homogeneous equation. And when we try to
solve the associated linear homogeneous equation we will be forming a characteristic equation so
it will have characteristic roots.
So, we have to check whether the constant s which is occurring in the function F(n) is one of
those characteristic roots or not. So, it could be either a characteristic root or it may not be a
characteristic root. So, if it is not a characteristic root then the theorem states here that the
particular solution of the form: a polynomial of degree t followed by the same constant sn is a
valid particular solution.
Namely any sequence whose nth term is this value will satisfy the entire recurrence equation;
this is for the case where the constant s is not a characteristic. But if it is a characteristic root then
558
depending upon how many times that root is repeated in the characteristic equation; that suppose
if s is a root and that too with multiplicity m where m is greater than equal to 1 then the general
form of the particular solution will be the following.
You still have a polynomial of degree t and then you also have sn but you also now need nm. So,
that is the difference between case 1 and case 2. In case 1 you do not have this nm this is not
there, because in case 1, s was not occurring as a characteristic root but in case 2, s is occurring
as a characteristic root and that too m number of times if m is equal to 1 then you will have n1; if
m is 2 that means s is occurring as a root 2 times then it will be n2 and so on.
And we can prove this easily, I am not going into the exact proof you can check that easily; these
are the two cases.
And now let us see; let me demonstrate the application of this general form with some specific
examples. So suppose I want to find out the nth term of this summation. So, this an here basically
denotes the sum of first n natural numbers; basically an here denotes the sum of the numbers 1 to
n so easily it is easy to see that an is a sequence depending upon what is the value of n you get
different values.
559
So, we have to find out the formula for the nth term of the sequence and that too using recurrence
equation. So let us first formulate the recurrence equation. It is easy to see that an is nothing but
the summation of the (n – 1)th term of the sequence plus n, because your, an is 1 + 2 + … + n - 1
+ n. Now the summation 1 to n - 1 is nothing but a(n – 1) and the + n is carried over. So, now this
is a linear non-homogeneous recurrence equation of degree 1.
So, let us solve it. So we will first solve the characteristic equation which will be of degree one
it will have only one root and in this case the root is 1 and its multiplicity is 1. So, that is why the
general form of the solution for the associated homogeneous equation will be some constant
times 1n and now we have to come up with a particular solution. So, what is the general, what is
the function F(n) here?
You might be saying that F(n) is n but that is not the case, you have to be very careful here even
though explicitly you will see that F(n) is just n, but I can always say that there is an implicit 1n
which is there in the F(n) function and why I am taking this implicit 1n ? this is because I am in
the case where this constant 1 which is implicitly present in F(n) is also occurring as one of the
characteristic roots.
Otherwise I would not have considered this implicit 1n. I could have simply ignored it, but since
one is a characteristic root and the same constant 1n is occurring in the function F(n) I have to be
careful. So, now if I use the general form; if I use the result that I stated for the general form of
the particular solution my F(n) here in this case is a polynomial of degree 1 multiplied by 1n.
And this 1 is a characteristic root with multiplicity 1 so that is why there will be n1 outside in the
particular solution followed by a polynomial of degree 1, followed by 1n. Now this 1n I can
ignore. So, this will be the general form of the particular solution and now I have to find out the
values of α2 and α3 satisfying the particular solution that means I have to check whether my
guess regarding the particular solution is correct or not.
For that I have to check whether this guessed particular solution satisfies the entire recurrence
equation; for that I have to check whether an(p) is equal to an(p – 1) + (n * 1n), check this and try to
560
find out the values of α2 and α3 and it turns out that by solving and rearranging the terms I will
get the values of α2 and α3 that means my guessed particular solution is correct.
And now if I have the guessed particular solution I can say that the overall solution will be the
summation of the nth term of the associated homogeneous equation plus the solution for the nth
term of the particular solution. So, this will be the overall solution. If you want to find out the
exact value of this constant α1 you can use the fact that a1 is 1; you can have an initial term and
then substituting n equal to 1 you can find out the exact value of this constant α1.
So, that brings me to the end of this lecture so these are the references for today's lecture just to
summarize in this lecture we discussed how to solve linear non-homogeneous recurrence
equations of degree k. The general solution is obtained by solving the associated linear
homogeneous equation and getting a particular solution. And coming up with the particular
solution is done by a trial and error method but it becomes methodical if you have the F(n)
function in some specific form, thank you.
561
Lecture -40
Catalan Numbers
Hello everyone. Welcome to this lecture. So, we will continue our discussion regarding how to
count number of things for certain problems by formulating recurrence equations. And in today's
lecture we will discuss a very interesting class of problems whose solution is a common
recurrence equation and the resultant solution for that corresponding recurrence equation gives
rise to a sequence of interesting numbers which we call as Catalan numbers. So, we will also
discuss various problems in this lecture whose solution is the Catalan numbers.
562
So, let me formulate this problem. So, imagine the function C(n) denotes the number of ways of
parenthesizing the product of n + 1 numbers to specify the multiplication order. So, you are
given n + 1 numbers, I am denoting them as x0 to xn. And I can multiply only two numbers at a
time. So, I want to specify the order in which I can multiply them. However I do not want to
shuffle the positions of the numbers x0 to xn; they should be in the same order.
That means x0 should be there at the first position second position x1 should be there third
position x2 should be there and so on. I am not allowed to shuffle them, because even though the
multiplication is commutative and associative, I am not allowed to invoke those rules here. But I
am interested to keep x0 to xn at their respective positions and then want to find out the number
of ways in which I can parenthesize them to specify the multiplication order.
So, for instance C(3) is equal to 5, because C(3) means I have 4 numbers. So, I stress here if I am
considering the problem instance where there are n + 1 number then the resultant number of
ways of parenthesizing is C(n), not C(n + 1). So, I have the number x0, x1, x2 and x3 and these are
the five ways of parenthesizing them to specify the multiplication order. So, the first ordering
should be interpreted as follows.
I multiply first x0 with x1 then the resultant product is multiplied with x2 and then the resultant
product is multiplied with x3. As I said I am not allowed to swap positions that means I cannot
563
say that let me first multiply x0 and x3 and then multiply it with x1 and so on that is not allowed.
In this second way of parenthesizing the sequencing is as follows. I am first asking to multiply x1
and x2 and then the resultant product is multiplied with x0.
So, you see I am not changing the position of x0. And then the resultant product is multiplied
with x3 and so on. So, there are five ways if I had four numbers and that is why C(3) is equal to
5. So, now we want to formulate a recurrence equation for C(n) because I want to find out
finally a closed form solution for C(n). So, C(n) is an infinite sequence or it is a function which
gives you the outcome for different values of n.
So, I want to find out the general formula or the closed form formula for this sequence or this
function C(n). How do I do that? For that I first have to formulate a recurrence equation.
So, let us proceed toward formulating the recurrence equation and I am retaining here the case
where I had four numbers and the five ways of parenthesizing them. So, as a general principle
whenever we want to formulate a recurrence equation we have to think of how to break a
problem instance or the nth problem instance into problem instances of smaller size. And then try
to see the relationship.
564
That is the general philosophy of formulating recurrence equation. So, same thing we will do
here and to break a problem instance where we are given a task of sequencing n + 1 number into
task of sequencing a sequence of smaller numbers or less number of values, we have to do the
following. I focus here on the final dot, namely the last multiplication which needs to be done in
any sequencing for this sequence of n + 1 numbers.
So, for instance if my n is equal to 3 here, then in this sequencing it is the final dot is appearing
here. So, I will be focusing on this final dot, so I can interpret this sequencing as follows. I can
say that my problem instance is divided into two problem instances. Namely I have to figure out
how to sequence or how to specify the parenthesizing order for three values namely x0, x1, x2 and
another value consisting of x3.
Once I know the number of ways of parenthesizing x0, x1, x2 and individually I know how to
parenthesize or the number of ways of parenthesizing x3 or the remaining portion of the
sequence. If I put a final dot then that gives me a way of parenthesizing the overall sequence. In
the same way if I consider this second sequencing here, see I am doing the same thing. In the
same way, if I consider the third sequencing I am imagining as if I have two values here and
whatever is the number of ways of parenthesizing two values and whatever is the way or
whatever is the number of ways of parenthesizing the two values here namely x2 and x3, if I put a
final dot in between them that will give me the total number of ways of parenthesizing four
values and so on. So, the idea here is that in order to formulate the recurrence equation I should
focus on the final dot or the last multiplication.
So, in general I can interpret as if the final dot is appearing between xk and x(k + 1). So, remember
your n + 1 numbers are x0, x1, x2 and xn. So, you have n + 1 numbers so you can interpret that in
any parenthesizing order if the final dot is appearing between xk and x(k + 1). Then what you can
say is that you have the bigger sequence consisting of n + 1 numbers is now divided into two
individual sequences.
And they are disjoint; the sequence to the left hand side of the final dot and the sequence to the
right hand side of the final dot. The sequence to the left hand side of the final dot has k + 1
565
values. So, what can be the total number of ways of parenthesizing that sequence or specifying
the multiplication order for those k + 1 values; well as per our problem definition there are a total
C(k) ways of parenthesizing those k + 1 values.
And now we are left with remaining values namely after the final dot; they are the values from
x(k + 1) to xn and again as per the problem definition there are C(n-k-1) ways of parenthesizing
those remaining values which are going to appear after the final dot. Now if you find out the total
number of ways of parenthesizing the LHS part before or the part before the final dot and the
number of ways of parenthesizing the expression that is there after the final dot.
Then if you multiply them as per the product rule that will give you the total number of ways of
parenthesizing the numbers x0 to xn provided the final dot appears between xk and x(k + 1). But
now my k could be anything, k ranges from 0 to n.
More specifically, if my final dot is between x0 and x1, that means I am taking the case where x1,
x2, xn. So, whatever way I can parenthesize x0 and whatever way I can parenthesize the
remaining n values and then put the final dot that gives me one category of parenthesizing where
the final dot is between x0 and x1. Similarly I can have another category of parenthesizing where
the final dot is between x1 and x2. That means I parenthesize the subsequence x0, x1 and I
parenthesize the remaining portion namely x2, x3 up to xn.
566
So, there are C(1) ways of parenthesizing this part, there are C(n – 2) ways of parenthesizing this
part. And if I multiply C(1) with C(n – 2) that gives me the total number of ways of
parenthesizing where the final dot would have appeared between x1 and x2. And now if I
continue like that I can argue that if the final dot is between x (n-1) and xn, then the total ways of
parenthesizing will be the product of C(n – 1) and C(0).
So, that is why k ranges from 0 to n - 1 your k could be 0 or 1 or up to n - 1. So, that is why I get
the recurrence equation that C(n) is equal to summation of k equal to 0 to n - 1 product of C(k)
and C(n - k – 1). And why I am adding all of them, because all these are disjoint categories of
parenthesizing. There is no overlap, there is nothing which gets counted twice. I do not have to
exclude anything.
So, this C(n) function is called as the nth Catalan number and it turns out that there are plenty of
problems in combinatorics whose general solution is nth Catalan number. So, let us see another
related problem. From the problem description it might look similar to the problem of specifying
the parenthesizing or multiplication order for n + 1 values. But that is not the case but still the
answer is the nth Catalan number.
567
So, here we want to find out how many valid strings of n pairs of parenthesis we can have. And
what is my definition of valid strings of n pairs of parenthesis. Well if I parse that string from left
hand side to right hand side. Then it should be the case that each left parenthesis or opening
parenthesis should have a corresponding matching closed parenthesis. That is what is my
definition of a valid string of n pairs of parenthesis.
So, for instance this string is valid, because if you scan from left hand side to right hand side then
each instance of opening parenthesis has a corresponding matching closing parenthesis. It is not
the case that you have an occurrence of closing parenthesis but till that point you do not have an
occurrence of a corresponding opening parenthesis. In the same way this is a valid sequence ( ) (
), but this is invalid ( ))(.
Because you have an opening bracket and then a closing bracket and then followed by a closing
bracket. For that you do not have a corresponding opening bracket as of now. So, that is why this
is an invalid string. So, we want to find out how many such valid strings of n pairs of parenthesis
we can have. So, I have demonstrated here the total number of ways of formulating valid strings
of n pairs of parenthesis for different values of n.
If n is equal to 0 then you do not have any string. And so that no string is denoted by an empty
string, the star denotes empty string. And since empty string is one of the strings that is why for n
equal to 0; I consider that answer is 1. Namely there is only one way of coming up with a valid
string consisting of zero pairs of parentheses. That one way is nothing but writing down the null
string.
For n equal to 1 you have only one string namely an occurrence of namely a string where you
have a left parenthesis followed by a right parenthesis. Whereas a string where you have an
occurrence of right parenthesis followed by a left parenthesis this is invalid. Similarly for n equal
to 2 you have two strings for n equal to 3 you have five strings and so on. So, now we want to
find out or derive a formula for general value of n.
568
And it turns out that the general value of n or the number of strings for general n is nothing but
the value of the nth Catalan number. To do that, what we can show is the following. We already
know that the solution for specifying the multiplication order for n + 1 values is the nth Catalan
number. Because that was our starting point, and now we have a new problem. Namely that
problem of finding the number of valid strings with n pairs of parentheses.
Instead of trying to find out the solution from scratch for the new problem what we can show is
the following. If we show that there is a bijection between the set of each valid parenthesis, each
valid way or each valid mechanism of parenthesizing n + 1 numbers and a set of valid strings
that you can formulate with n pairs of parenthesis. Then we can say that the solution for both the
problems is the Catalan number.
So, how exactly we establish a bijection. So, what we have to show is that you take any sequence
of parenthesizing n + 1 values corresponding to that you can formulate or you can find out a
valid string with n pairs of parenthesis in an injective fashion and vice versa. If we do that then it
establishes a bijection. So, it might look on a very high level that the bijection establishment is
very simple.
What you may say is that well you take any sequence or any valid sequence specifying the
parenthesizing of n + 1 numbers and if corresponding to this sequence specifying the
569
parenthesizing order you want to find out the matching valid string with n pairs of parentheses
what you can do the following. You can say that you remove all the terms namely forget about
x0, x1, …, xn you simply remove them.
Because in our valid string; consisting of n pairs of parentheses we do not have any occurrence
of x0 to xn. So, you can say that erase x0 to xn from your sequencing which specifies the
parenthesizing order. And you just retain the parenthesis namely the left parenthesis and the right
parenthesis. And the claim is that you obtain a valid string consisting of n pairs of parentheses. If
you do that then resultant string will be a valid string of n pairs of parenthesis.
So, for instance what I am saying is you take the sequence, say this sequence (a.(b.c)). And what
I am saying is you retain the left parenthesis forget about a, forget about dot then retain the left
parenthesis forget about b, forget about dot, forget about c and then take the right parenthesis,
then take the right parenthesis. So, you will say that this is the valid string consisting of two pairs
of left and right parenthesis corresponding to the multiplication sequence where you multiply b
and c first and then multiply the product with a. That is what I am saying here basically.
But it turns out that even though you get a valid string consisting of n pairs of parenthesis. This is
not an injective mapping. Because as per this process the sequence where b and c are multiplied
first and then the product is multiplied by a will lead to this sequence. Because you will forget
about a you will forget about dot return the left parenthesis and then you again have a left
parenthesis.
Then you forget about b forget about dot forget about c then you have right parenthesis right
parenthesis. You will obtain the same sequence of two pairs of left and right parenthesis for
another multiplication order where a and b are multiplied first and then the product is multiplied
by c. So, this is not going to lead you to an injective mapping. But in order to claim that the
solutions for both the problems the mapping should be a bijective mapping and hence it has to be
an injective mapping as well.
570
So, how exactly we go from this set to this set. We have to do, we have to convert a valid
ordering of specifying the multiplication order into a string of n pairs of parentheses in a slightly
different way and this is done as follows. So, you take any multiplication order specified by your
parenthesizing and you erase all the terms x0 to xn and you erase all the left parenthesis as well.
But you retain the dots and the right parenthesis.
And then so for instance what you do here is if I take the same counter example for my earlier
mapping then I forget about this left parenthesis, I forget about this a but I retain the dot and then
I forget about this left parenthesis, I will forget about b, I retain the next dot, I remove c and then
retain the parenthesis. The same I do for the other string. And now I replace each dot by the left
parenthesis if I now do this mapping then I can show that this is an injective mapping.
In fact this is a bijective mapping from this set to this set. And that shows that the solution for the
new problem is the Catalan number as well.
So, as I said there are plenty of problems whose solution is the nth Catalan number; we have
seen two of them. And in tutorial we will see some other problems as well. But now the question
is how exactly we find a closed form formula for the nth Catalan number. We know the
recurrence equation, but that recurrence equation is not what we need. We need a closed form
formula namely a value of the nth Catalan number just as a function of n.
571
Because that will be useful if someone asked me tell me the value of say the 100th Catalan
number or the value of the 500th Catalan number and so. How do we do that, so for that what we
are going to do is the following. We will introduce a third problem whose solution also will be
the Catalan number. And then we will derive the closed form formula for our Catalan number by
solving or coming up with the number of solutions for this third problem, so what is this third
problem.
So, in this problem we are given 2n values a1, a2, a3, …, a2n. So, we are given 2n values and those
2n values consist of n 1s and n -1s. So, basically it is the string of n number of 1s and n number
of -1s. Now this n 1s and n -1s can occur in any order, but I am interested in only those
sequences of n 1s and n -1s where in that sequence if I parse from left hand side to right hand
side, then at every point the partial sum namely if I am at the kth position in that sequence then
the partial sum will be the summation of the k characters which I have encountered till the kth
position from the left hand side to the right hand side. So, basically my sequence has the values
the first value is a1 which could be either + 1 or - 1. I have the second value, again which could
be + 1, - 1 and like that I have the 2nth value which could be + 1 or - 1.
The first restriction is that in this sequence of 2n values +1 should be there at n positions -1
should be there at 1 position that is one of the restrictions. And the second restriction is that if I
go from LHS to RHS that means from the starting position to the end position. And if I stop at
any position k it could be the first position, I could stop at the second position, I could stop at the
third position; if I stop at any position k where k ranges from 1 to 2n, and if I take the summation
of the first k values which I have encountered till now while scanning then that partial sum I call
it as sk and that partial sum sk should be non negative. That means it should be greater than equal
to 0, it could be 0 or it should be more than 0. So, in some sense intuitively what it means is that
if I parse from left hand side to right hand side then at each and every position the number of
occurrences of + 1 should be either more than the number of occurrences of - 1 or it should be
same.
572
It should not happen that the number of occurrences of -1 is more than the number of
occurrences of +1 at any point of time when I parse the sequence from LHS to RHS. So, for
instance this automatically means that my string cannot start with - 1. Any sequence any valid
sequence satisfying these conditions cannot start with - 1. Why so, because if I take k equal to 1
then my partial sum s1 becomes -1.
And that automatically shows that this is an invalid sequence. So, it automatically means that my
sequence has to start with + 1. So, I can have a sequence where I have all the + 1s appearing first
and then all the minus 1 appearing later. That is a valid sequence or I can have a sequence where
+1 and -1 occurs alternatively. That is also a valid sequence and so on. But I cannot have a
sequence of the form where I say start with 1 and then I have a -1 and then I have a -1, that is not
allowed.
Because if I take this partial sum s3 in this sequence then s3 becomes -1, so that is not allowed.
So, I am interested to find out the number of sequences consisting of n 1s and n -1s satisfying
this condition. And it turns out that the number of sequences satisfying this condition is exactly
the same as the number of valid strings consisting of n opening parenthesis and n closing
parenthesis.
This is because you can interpret the new problem as the following. You take any valid string
consisting of n opening parenthesis and n closing parenthesis. And you replace the opening
parenthesis by 1 and you replace the closing parenthesis in those valid strings by - 1. So, you will
obtain now a sequence consisting of n numbers of + 1 and n numbers of - 1. And you will see
that in that string of n number of 1s and a number of -1s, the number of 1s is always greater than
or equal to the number of occurrences of - 1 at any point of time if you scan that string from left
hand side to right hand side. So, for instance if you take this string ( ) then this gets converted
into +1 followed by -1. If you take this n equal to 2 case then the corresponding string is ( )( )
you have one then you have a closing parenthesis, so you put - 1 then you have an opening
parenthesis, then you have a closing parenthesis. Whereas the second string (( )) gets mapped to
the following you have left parenthesis left parenthesis. So, you have 1, you have 1 and then you
have two closing parenthesis so - 1, - 1 and so on. So, that shows that the solution or the number
573
of valid sequences satisfying our condition is same as the number of valid strings with n pairs of
parenthesis.
And what we will show in our next lecture is that the number of valid sequences satisfying these
restrictions is nothing but this formula : C(2n, n) / (n+1). So, this C is not the Catalan function
this is now the combinatorics function of 2n choose n. Basically the number of ways of selecting
n values out of 2n values. So, the nth Catalan number: its value is 2n choose n over (n + 1). This
we will prove in our next lecture.
So, that brings me to the end of this lecture. These are the references for today's lecture. So, just
to summarize, in this lecture we introduced Catalan numbers, we saw three problems and the
solution for each of those three problems is the recurrence relation for the Catalan number. And
also, I have shown you the value of the Catalan number. In the next lecture I will explicitly solve
the recurrence relation for the Catalan number and we will derive that the value of the nth
Catalan number is 2n choose n over (n + 1), thank you.
574
Lecture -41
Catalan Numbers- Derivation of Closed Form Formula
So, hello everyone. Welcome to this lecture, just a quick recap.

In the last lecture we discussed various problems whose solutions constitute Catalan numbers.
So, in this lecture we will continue our discussion on Catalan numbers and we will derive a
closed form formula for the recurrence relation for Catalan numbers.
575
So, we recall from the last lecture that we discussed two problems whose solutions are the
Catalan numbers. The first problem was that of coming up with the number of strings with n
pairs of opening and closing parenthesis where a string is called valid, if whenever we scan the
string from left to right then at any point of time or at any position in the string the number of;
each instance of an opening parenthesis has a closing parenthesis.
And the second problem that we saw in the last lecture is that of coming up with a number of
sequences consisting of n number of 1s and n number of –1s such that if we scan the string from
the first position to the last position then each partial sum should be greater than equal to 0. And
we saw a bijection between the number of sequences of n 1s and n –1s where each partial sum
is greater than equal to 0.
And we saw a bijection between the set of sequences consisting of n 1s and n –1s where each
partial sum is greater than equal to 0 and the set of all strings all valid strings with n pairs of
opening and closing parenthesis. So, in this lecture, we will consider the set of sequences of n 1s
and n –1s where each partial sum is greater than equal to 0 and we will show that the number of
C(2𝑛 ,𝑛)
such sequences of length n of length 2n is this quantity namely . That will show that the
𝑛 +1
C(2𝑛,𝑛)
closed form formula for the Catalan number is .
𝑛 +1
576
So, this is the statement which we want to prove. We want to prove that the number of sequences
consisting of n 1s and n –1s, where in each sequence the partial sum at any position is greater
C(2𝑛,𝑛)
than equal to 0 is . So, for that the proof strategy will be the following. We will first find
𝑛 +1
out the cardinality or the number of sequences consisting of n 1s and n –1s without any
restriction.
So, that means let A denote the set of sequences of n 1s and n –1s with no restriction. Then the
set A has all the sequences where the partial sums are greater than equal to 0, as well as it has all
the sequences where the partial sum at every k may not be greater than equal to 0. And then we
will find out the set B of all bad sequences and by bad sequences I mean the sequences
consisting of n number of 1s and n number of –1s which violate the restrictions.
And of course then it is easy to see that the required value or value of the required number of
sequences of n 1s and n –1s where the partial sums are greater than equal to 0 will be the
difference of the cardinality of the A set and the B set. So, that is the proof strategy. So, it is easy
to see that the cardinality of the A set is C(2n, n). This is because, what is the set A? It is the set
of all sequences with n number of 1s and n number of –1s where we do not put any restriction
whatsoever over the partial sums in the sequences.
577
So, any sequence in this set will have n number of 1s and n number of –1s. So, it is easy to see
that the cardinality of A is nothing but the number of ways in which we can find out n locations
out of 2n locations where we can put the 1s. Because once we find out the n locations where we
can put the 1 the remaining locations are of course has to be occupied by –1.
So, that is why the cardinality of the set A is C(2n, n). Now what we will show is that the
cardinality of the set B is C(2n, n+1) and if we subtract the cardinality of B from the cardinality
of A then we will get our required answer.
So, now for the rest of our discussion our focus will be to find out the cardinality of the set of
bad sequences and what is a bad sequence? A sequence is a bad sequence if it consists of n
number of 1s, n number of –1s such that in such sequence there is at least one occurrence of a
partial negative sum. That means if I parse the string from a1 to a2n at least at some position k,
some index k the values are such that if I just take the sum of a1 to ak then the partial sum is
negative.
There might be multiple positions or multiple such indices k in that bad sequence but at least one
such bad index or the index case is there. So, that is the definition of an invalid sequence. So,
what we will do is we will introduce a very nice method called as reflection method or why it is
called reflection method will be clear very soon. So, we will find the cardinality of the set B
578
using the reflection method. So, for that let us consider an arbitrary bad sequence and we know
that this bad sequence has n number of 1s and n number of –1s and at least one partial negative
sum, where exactly the partial negative sum is appearing we do not know.
But we know that this is a bad sequence and I call this bad sequence as S. So, let r be the index
or let r be the index at which the first negative partial sum occurs in the sequence S. So, that
means if the values in the bad sequences are a1 to a2n then it is at the index r that the first instance
of partial negative sum occurs. So, that means pictorially you can imagine that if I take the sum sr
which denotes the summation of the values a1 to ar it is negative.
And if I consider all other partial sums up to the r – 1th positions namely the partial sum s1, the
partial sum s2 and partial sum sr–1, all of them are greater than equal to 0. This is because of our
assumption that the index r is the index where the first negative partial sum is occurring in the
bad sequence S. So, now we will do the following. We will derive or conclude some properties
regarding the values that are there in our bad sequence and based on that we will complete our
proof.
So, our first claim is that in this bad sequence S the value at the position r is definitely –1. And
this is because as per our assumption the partial sum namely the summation of the first r – 1
values is greater than equal to 0. So, that means that if ar; so remember by the way that at rth
position we can have either + 1 or –1. So, if sr –1 is greater than equal to 0 and if my rth value the
number at rth position is + 1 then definitely sr will also be positive.
But that goes against the assumption that sr namely the partial sum at rth position is negative.
This is an easy claim. The second claim is that if the index r is greater than one then the partial
sum till the r – 1th position is 0. And again this is because of the fact that r is the index, where
the first negative partial sum is occurring. So, if r is equal to 1 definitely this claim is not true
because r – 1 is not there if r is equal to 1.
579
But if r is greater than 1 then definitely I know that s r –1 is equal to 0. Because if r – 1 would
have been positive say + 1, + 2 or + 3 or + 4 even if you take the least positive value namely + 1.
That means the summation of the first r – 1 value is say + 1 or greater than + 1, then even if we
are putting –1 at the rth position that positive value added with –1 would have given the partial
sum sr to be 0 or more than 0.
But that goes against the assumption that sr or the partial sum at rth position is negative. This
shows that if r is greater than 1 then r – 1 is an even quantity. Because if the partial sum at r – 1th
position is 0 that means by the time we have reached r – 1th position we have encountered equal
number of 1s and –1s. So, that is why r – 1 will be an even quantity and it will have and in the
first r – 1 positions we would have encountered k number of 1s and k number of –1s where k is
greater than equal to 1.
So, I have retained the summary of the claims regarding the various values that we have in the
bad sequence S. Now what we are going to do is corresponding to the bad sequence S; so this is
our bad sequence S. We will find another sequence S’ which will have n + 1 number of 1s and n
–1 number of –1s. Namely the number of 1s will be two more than the number of –1s.
Remember in the bad sequence S we had an equal number of 1s and –1s.
580
But now we are going to define a sequence S’ corresponding to the sequence S which will have
two more 1s than the number of –1s. And this is done as follows; So, let me first demonstrate
how exactly we construct the sequence S’ corresponding to the sequence S for the case where n
is equal to 2 and then we will see the general method for any n.
So, for n equal to 2 we have 4 possible bad sequence S consisting of 2 1s and 2 –1s. And now
for each of this bad sequence S I have highlighted the first occurrence of partial negative sum in
that sequence. So, for instance if I consider the first sequence my r is equal to 1. Because at r
equal to 1 I have an occurrence of partial negative sum. For the second string also my r is equal
to 1, for my third bad sequence also r is equal to 1.
But for my fourth bad sequence r is equal to 3, because if I consider the partial sums s1 for this
bad sequence then it is positive. If I consider the partial sum at position 2 then it is 0, still it is not
negative. And only when I consider the partial sum at position 3 it becomes negative. So, that is
why r is equal to 3 for the fourth bad sequence. Now the corresponding string S’ for each of
these bad sequences S is as follows.
If you see here what I have done basically is for each of the bad sequence S’ the remaining
portion of that bad sequence which is occurring after the first instance of the partial negative sum
is retained as it is. So, we had this –1, 1, 1 they are retained as it is. And whatever partial
sequence we had here the first occurrence of partial negative sum is occurring, I am just
converting each –1 to + 1 and each + 1 to –1.
Well in this case there is only one value in the sequence. So, that –1 gets converted into + 1. In
the same way for the second bad sequence the remaining portion of the sequence after the first
occurrence of partial negative sum is retained as it is and then in the sequence which has the first
occurrence of partial negative sum we replace –1 to + 1 and so on. If you take the third sequence
then this unhighlighted portion remains as it is.
581
And now you see the highlighted portion, namely the sequence which has the first partial
negative sum; there we convert each 1 to –1 and vice versa. So, if this is my S, this is the S’. If
this is my S, this is my S’. This is my S and this is my S’ and for this fourth S this is my S’. And
now you can see in S’ we have the number of 1s exceeding the number of –1s by 2.
So, now let us see the general process. The process to obtain S’ from S is as follows. We reverse
the sign of a1 to ar. So, remember as per our assumption r is the first index such that the partial
sum sr is negative. So, what we do is we convert a1 to - a1, a2 to - a2 and so on and the remaining
portion of the bad sequence S is retained as it is, namely ar + 1 is retained as it is, a r + 2 is retained
as it is and so on in S’.
So, now let us count the number of 1s and –1s in the sequence S’. So, if I consider the portion
where the occurrence of partial negative sum is there namely if I focus on the portion of the
sequence till the rth position then I know that the number of –1s is more than the number of 1s
by one position. This is because as per our claim 2, till the r –1th position the sequence S has
equal number of 1s and –1s , namely k number of 1s and –1s.
And since at rth position the partial sum becomes negative. That is because at the rth position we
have a –1. So, that is why we have one more –1 compared to the number of 1s till the rth
position and since my overall sequence S has n number of 1s and n number of –1s that means
in the remaining portion of the sequence S the number of 1s will be n - k and a number of –1s
will be n - k + 1 which is this one.
So, now what we can say about the number of 1s and –1s in S’. So, the number of 1s and the
number of –1s in this half; in this portion of S’ is same as the number of 1s and number of –1s
in this portion of S. No change in the number of 1s and –1s in this portion of S’, the later
portion of S’, the remaining portion. Whereas if I consider the first portion of S’ namely which is
obtained by reversing the signs of a1 to ar then the number of 1s and –1s are as follows.
The number of 1s in this portion, it will be the same as the number of –1s. Because due to the
reversal of the signs all the –1s they will be converted into 1s. And due to the same reason; due
582
to the reversal of the 1s to –1s and –1s to 1s the number of –1s in this portion of S’ will be the
same as the number of 1s in the highlighted portion of S which is k.
So, that tells you that if I sum the total number of 1s in S’ then it will be k + 1 + n - k which is n
+ 1. And if I find the number of –1s it will be k + n - k –1 which is the same as n –1. So, that
means the number of 1s is 2 more than the number of –1s in S’.
So, that and now you might have understood why we are calling the method of finding the
cardinality of the set B as the reflection method. If you see closely this process of reversing the
sign of a1 to ar, it is like reflecting the 1s to –1s and –1s to +1s. So, that is why the method is
called as the reflection method. So, what we have done till now is we have converted, we have
found a new sequence S’ for each bad sequence S. My claim is this process of getting the
sequence S’ from the sequence S is an injective process.
That means the S’ that are obtained from S are obtained in an injective fashion. That means it
cannot be the case that there are two bad sequences S’ with equal number of 1s and –1s and each
of which has an occurrence of partial negative sum such that if we find the corresponding
sequences S’ for these two bad sequences they are the same. How do we prove that? It is very
simple. So, imagine S1 and S2 are two distinct bad sequences.
583
So, each of them has equal n number of 1s and –1s and each of them has an occurrence of
negative sum somewhere. So, what I am doing here is that I focus on the first index in the
sequence S1 and the first index in the sequence S2 where we have an occurrence of partial
negative sum. So, let for S1 the index r is the first index where we have an occurrence of partial
negative sum and in the same way for sequence S2, let the first partial negative sum occur at the
position t. We do not know whether our index r is the same as index t or not. So, there are two
possible cases. If the portion of S1 and S2 with respect to the rth index and the tth index they
could be either same or they could be either different. So, let us take the case one. If the
subportion of S1 till the rth position and the subportion of S2 till the tth position they are different
then I do not care what is the remaining portion of S1 and S2.
Because of the reflection method the corresponding sequences S1’ and S2’ which are obtained by
the reflection method they will be different. Because in the reflection method in S1’ the signs of
a1,1, a1,2, a1,r will be reversed and in S2‘ the signs of a1,1, a2,2, a2,t will be reversed. Now since in S
and S’ the portion till the rth position and the portion till the tth position in S1 and S2 were
different, if I reverse their signs then I know that till the rth position and the tth position the
portion till the rth position and the portion till the tth position in the reflected strings in S1’ and
S2’ also will be different. Whereas case 2 is when r is equal to t basically. So, if that is the case
then in S1’ and S2’ the values till rth position and the values till the tth position also will be the
same because of the reversal of the sign. But then what I can say is that in S1 and S2 the
remaining portions of the strings were different.
This is because as per our assumption S1 and S2 are two distinct strings. So, if till the rth position
and the tth position S1 and S2 were respectively the same, since S1 and S2 are overall different. It
means that the remaining portion of S1 that means the portion of S1 from (r + 1)th position to the
(2n)th position and the portion of S2 from the (t + 1)th position to the (2n)th position they are
different. Because if they are also the same that means S1 and S2 are the same string. But that
goes against the assumption that S1 and S2 are two distinct strings.
584
So, in this case even if the reflected portions in S1’ and S2’ are same remaining portion of S1’ and
S2’ which are copied as it is from S1 and S2 respectively they will be different. So, that shows
that the above process of a mapping from S to S’ is an injective mapping.
So, if I consider this red circle which is the set of all bad sequences; so this is our set B as per our
construction. Namely it has all sequences with equal number of 1s and –1s which have at least
one occurrence of partial negative sum. And we have another set which I call as say the set C
which is the set of all sequences where the number of 1s is two more than the number of –1s.
And we have established an injective mapping namely the reflection method from the set B to
the set C. Now what we will prove is that the above process of converting any bad sequence S to
a corresponding sequence S’, that mapping is also a surjective mapping. And that will show that
the cardinality of the set B and the cardinality of the set C are the same. (Refer Slide Time:
27:16)
585
So, for proving that our mapping f is surjective mapping what we have to do is, we have to take
any arbitrary sequence in the set C and we have to show corresponding to that there is a bad
sequence. So, let us do that. So, imagine I take an arbitrary bad sequence S’ where the number of
1s is two more than the number of –1s. Now corresponding to that, our goal will be to show the
existence of a bad sequence which has equal number of 1s and –1s and which has at least one
negative partial sum.
So, for that intuitively what we will do is we will just reverse the process that we followed for
getting the string S’ from the string S. So, what we can say about the string S’ is that since it has
more number of 1s than –1s, it has number of 1s is two more than number of –1s , definitely it
has one positive partial sum not negative partial sum. So, there could be multiple positions in S’
where we have positive partial sum.
But let us focus on the first occurrence of positive partial sum in S’ and suppose it occurs at the
rth position. So, again pictorially you can imagine that if I scan the sequence S’ then at rth
position if I take the sum of all the values till the rth position then the sum becomes 1. But till the
(r – 1)th position if I take the partial sums they were either 0 or negative. Then again we make
similar claims as we did when we converted the sequence S to S’.
586
So, we can say that the value at rth position in S’ will be definitely 1. It cannot be –1, because
your sr –1 namely the partial sum till the (r – 1)th position was either 0 or negative. And if at the
rth position also we put a –1 then we get that at rth position the partial sum is still 0 or negative.
But that goes against the assumption that the partial sum at rth position is positive namely 1.
Similarly we can claim here that if the index r is greater than 1 then the partial sum at (r – 1)th
position is exactly 0. It cannot be negative because if it would have been negative then if in that
negative sum if we add a 1, namely even if we put 1 at rth position then the partial sum at rth
position would have stayed 0 or negative it cannot become 1. That shows that if r is greater than
1 then till (r – 1)th position we have equal number of 1s and –1s, say k number of 1s and k
number of –1s.
So, these are the summary of whatever we have concluded till now about S’. Now what we do is
we will show a method we will follow the reflection method and what we will do is that
corresponding to S’ we will show a bad sequence consisting of equal number of 1s and –1s and
which will have one negative partial sum. I stress in S’ we do not have equal number of 1s and –
1s.
But we will convert S’ to another sequence S which will have equal number of 1s and –1s and at
least one negative partial sum. And the idea is just to do the reflection method here. We just
587
reverse the sign of the numbers b1 to br and retain the remaining portion of the sequence S’ as it
is. So, b1 gets converted into - b1. So, if it is + 1 it becomes –1, if it is –1 it becomes + 1.
Similarly b2 becomes –b2 and like that br becomes –br. The remaining portion of S‘ is left
untouched in S and then again we can use similar counting argument to find out the number of 1s
and –1s in S. So, we know that in S’ in the rth position the number of 1s is exactly one more
than the number of –1s. This is because the partial sum at rth position is exactly 1 and the partial
sum till the (r – 1)th position is 0.
And since the number of 1s overall in S’ is n + 1. That means the number of 1s in the remaining
portion will be n + 1 – (k + 1) which is same as this n - k and the number of –1s will be; overall
we will have n – 1 number of –1s in S’. We already had k of them till the rth position. So, in the
remaining portion this will be the number of –1s : (n – 1 – k). So, these statistics regarding the
number of 1s and –1s will be carried over in S as well.
And what we can say about the number of 1s and –1s in S? Well the number of –1s will now
become the same as the number of 1s. Because each + 1 has been converted into – 1, whereas the
number of –1s will now become the number of 1s. Because each –1 has been converted into + 1.
So, that shows that the number of 1s overall will be n - k - k which is n and the number of –1s
will be k + 1 + n –1 - k which is again n.
And it is easy to see that if we take the partial sum till the rth position it will be negative.
Because till the rth position in S’ the sum was positive and the sign of every + 1 and –1 has been
reversed. Because of that the partial sum till the rth position S will now become negative. So, we
have shown that the mapping is a surjective mapping as well.
588
So, going back to the proof of finding the cardinality of the set of bad sequences, we have the set
A which is the set of all sequences of equal number of 1s and –1s, without any restriction. We
know there are C(2n, n) such strings and we just established that the number of bad sequences
which violates the restriction will be C(2n, n + 1). Because we just established a bijection from
the set of all bad sequences with equal number of 1s and –1s, and violating the restrictions to the
set C of all sequences which has n + 1 number of 1s and n – 1 number of –1s. And the
cardinality of this set will be C(2n, n + 1) because it is equivalent to saying that out of 2n
positions we have to identify the n + 1 positions where +1s will be there. Automatically the
remaining positions will be occupied by –1s. And that shows that the number of actual
sequences the valid sequences which we are interested to find out is the difference between the
cardinality of the A set and B set.
And if we find the difference of the cardinality of the A set and B set we get the result of the nth
C(2𝑛,𝑛)
Catalan number which is . So, that brings me to the end of this lecture. Just to summarize
𝑛 +1
in this lecture we extensively derived the closed form formula for the Catalan number and for
that we introduced the reflection method, thank you.
589
Lecture -42
Counting Using Principle of Inclusion-Exclusion
So, hello everyone, welcome to this lecture the plan for this lecture is as follows. In this lecture
we will introduce the principle of inclusion-exclusion and we will see some of its applications to
various counting problems.
590
So, what exactly is the principle of inclusion-exclusion? Well it basically says that if you want to
find out the cardinality of the union of 2 sets then it is same as taking the summation of
cardinalities first of the individual sets and subtracting the cardinality of the intersection of the 2
sets. And why this is true because if we see pictorially or if we follow the Venn diagram method
then if we just add up the cardinalities of the A set and B set then the common portion or the
elements which are common to both sets A and B are counted twice.
So, to compensate or to avoid this over counting we basically subtract the cardinality of the
intersection of the A set and the B set. Now extending this principle to the case of 3 sets if we
want to find out the cardinality of the union of 3 sets then it is the summation of the cardinalities
of the individual sets. Then we have to take the difference of the cardinalities of 2 sets at a time
and then again we have to add the cardinality of the intersection of all the 3 sets.
Again we can prove it easily using the Venn diagram method. So, why we are subtracting the
cardinality of A ∩ B, A ∩ C and B ∩ C. Because the elements in A ∩ B are counted twice if we
add individually the cardinalities of the A set and B set. Similarly when we are adding up the
cardinalities of A set and C set the common portion or the elements common to A and C are
included twice and so on.
591
Whereas the elements which are common to all the 3 sets they are counted thrice. If we just add
up the cardinality of A B and C and if we do not add this cardinality of intersection of A B and C
set in this overall formula then because since we are subtracting out the common portion between
A and B, common portion between A and C and common portion between B and C their
cardinalities, if we do not do this extra addition then the contribution of the common elements in
the 3 sets is ignored. So, that is why this plus is there. Now we have seen the principle of
inclusion-exclusion for this case of 2 sets, for the case of 3 sets.
We can generalize it to the case of n sets. So, the general formula says the following. If you have
n sets they may be disjoint, there might be overlaps and so on. Then the formula says that if you
want to find out the cardinality of the union of n sets then this is this formula. That means it says
that it is same as you have to first individually take the summation of the cardinalities of the
individual sets.
Then you have to subtract the cardinality of pairwise intersection of sets. Then you have to add
the cardinality of intersection of triplets of sets and so on. That means alternately, first we add
then we subtract and we add and we subtract and so on. So, we have to prove that this formula is
correct. You can use proof by induction but even without using proof by induction over n we can
prove it.
592
So, the idea here will be the following, so consider an element a which is present in the union of
the n sets. It might be present in just one of the sets, it might be present in 2 of the sets we do not
know in how many sets the element is present. We have to show that if at all an element a is
present in the union of n states then by this complex looking argument, complex looking formula
in the right hand side expression the element a is counted exactly once.
And that is what we want to prove here basically. So, imagine that the element a is present in r
number of sets out of the n sets where r is at least 1. Because we are considering the case where
element is present in the union of all the n sets. So, if it is present in the union of all in n sets it is
possible only when it is present in at least 1 of the n sets. And it might be the case that it is
present in all the n sets. So, that is the value of r is in the range 1 to n.
Now what we have to show is our goal is to show that by the RHS expression the element A is
counted exactly once overall. So, for that, we observe that the first part of the formula on the
right hand side is where we are taking the summation of the cardinalities of the individual sets A1
to An. Now because of this, the element a will be counted C(r, 1) number of times. So, for
instance if the element a is present, say in the first r sets, then because when we are taking the
cardinality of A1 the element a was counted once. When we are taking the cardinality of A2 the
element a is again counted once. When we are taking the cardinality of A3 the element a is again
counted once and so on. When we are taking the cardinality of ar + 1 we are not increasing the
count of the element a. So, that means through the first part of the formula in my right hand side
expression the element a is counted these many number of times : C(r,1).
Now let us take the second part of the formula where we are taking the cardinality of intersection
of 2 sets at a time. So, basically it is like saying the following we are taking the cardinality of A1
∩ A2, A1 ∩A3 and like that A1 ∩An and then we are taking the cardinality of A2 ∩ A3, A2 ∩ A4
and so on by the second formula. So, we have to find out, and when we are taking the
cardinalities of this pairwise intersection of sets, how many times the element a is getting
counted. For the moment we are forgetting about this minus here, we are just trying to find out
what will be the effect or how many times the element a will be counted when we are taking the
cardinality of intersection of 2 sets at a time. So, as per our assumption the element is a is present
593
in r number of sets. When we are taking the intersection of 2 sets at a time, element a will be
counted only when both the sets where we are taking whose intersection we are taking element a
is present.
If we are taking intersection of 2 sets where in one of the sets element is a is not present then the
overall contribution for the cardinality of intersection of those 2 sets for the count of a will be 0.
So, because of this overall, I can say that because of the second part of the expression the
element a is counted C(r, 2) number of times. In the similar way, I can say that when we are
taking the cardinality of intersection of 3 sets at a time, then element a will be counted only when
the 3 sets whose intersection we are taking all of them has the small element a. If element a is not
present in even in one of them then it will not be present in the intersection of 3 sets. Due to that
because of considering the cardinality of intersection of 3 sets at a time the element a will be
counted C(r, 3) number of times. And then if we continue this argument, we can say that when
we are taking the cardinality of intersection of r sets at a time the element a will be counted
exactly once namely C(r, r) number of times. And after that when we are taking the cardinality of
intersection of r + 1 sets at a time, r + 2 sets at a time through those parts of my RHS formula the
element a will not be counted at all.
Because as per our assumption the element is a is present only in r number of sets. So, now what
we can say is that the total count of this element a by the RHS expression is the following C(r, 1)
– C(r, 2) + C(r, 3) - … (-1)r+1C(r, r). This is because through the first part of the expression the
element A is counted C(r, 1) number of times, through the second part of my expression the
count of element A gets decremented by C(r, 2) number of times.
Why decremented? Because of this minus. Then due to the third part of the expression, the count
of element a gets incremented by C(r, 3) number of times and so on. And we know that this
expression is nothing but C(r, 0). This follows from the properties of the combinatorics function
or the binomial coefficients. So, it turns out that this summation is same as C(r, 0). And C(r, 0)
is nothing, but 1.
594
So, that shows that by the expression in your right hand side the element a which is present in the
union of n sets is counted exactly once. If there is an element which is not at all there in the
union of n sets it will not be counted at all by the formula.
Now what we will do is, we will look into an alternate form of inclusion-exclusion which is very
powerful and it is this alternate form of inclusion-exclusion which we use in varieties of
problems. So, what happens in this alternate form is that we will be facing scenarios where we
want to count the number of elements from a set A which has n number of elements and we will
be interested to count the number of elements in this set which has none of the properties say P 1,
P2, Pn.
So, P1, P2, Pn will be some abstract properties and we will be interested to find out the number of
elements in the set A which neither have the property P1 and nor the property P2 , nor the
property P3 and so on. So, for that what we will do is we will first identify the subset A i
consisting of elements which has property Pi. I stress Ai is the set of elements in the set A which
has the property Pi.
And if I take the union of the subsets A1 to An, I get all the elements in the set A which has either
the property P1 or the property P2 or the property P3 or so on. That means it will have at least one
of the properties P1 to Pn, but that is not what we want. We are interested in the elements which
595
do not have any of these properties. So, it is easy to see that now the desired answer or the
number of elements which do not have any of the properties P1 to Pn will be the difference of the
cardinality of A which is n and the unique cardinality of the union of n sets.
And now we will apply the rule of inclusion-exclusion to find out the cardinality of the union of
the n sets because we know how to find that. So, this is the alternate form of inclusion-exclusion.
So, let me demonstrate this alternate form of inclusion-exclusion through some examples.
Suppose we want to find out the number of solutions for this equation : x1 + x2 + x3 = 11; number
of integer solutions. And my restrictions are x1 should be in the range 0 to 3, x2 should be in the
range 0 to 4 and x3 should be in the range 0 to 6. Now using the; methods that we have seen till
now we cannot find it directly.
But what we will do is the following : we will use the alternate form of inclusion-exclusion. So,
we will first find out the cardinality of the universal set. Universal set in this context will be the
set of all solutions where I do not put any restrictions on x1, x2, x3. So, they could be anything, In
fact I can have a solution where x1 is 11 as well. And I know the cardinality of this universal set
because that comes from the formula for combinatorics with repetition.
596
So, it is equivalent to saying that we have to select 11 bills and we are given bills of either
denominations x1, x2 and x3. And I have no restriction because that is what is the interpretation of
the set A. Now I let me define a set A1 to be the set of solutions for this equation where x1 is
greater than 3. So, now you see your property P1 was that your solution should not have x1 more
than 3. But I am now finding solutions which violate the property P1 namely solutions where x1
is allowed to be more than 3.
In fact x1 has to be more than 3 and we know how to find the cardinality of A1 set. If x1 is more
than 3 that means I have to definitely pick 4 bills of x1 type. So, I will be left with now selecting
7 more bills without any restriction and the number of ways of doing that is 36. In the same way
my property P2 is that my solution x2 should be strictly less than equal to 4. But now I try to find
out solutions which violate the property P2 namely solutions where x2 is definitely greater than 4.
And let A2 be the set of such solutions the cardinality of A2 will be this : 28. And similarly my
property P3 could be that I am interested in solutions where x3 should be less than equal to 6. But
then I try to find out solutions which violate this property and define the set of such solutions to
be A3 and the cardinality of the set A3 will be this: 15. And now it is easy to see that my overall
solution the number of overall solutions will be the difference of the cardinality of the universal
set which is 78 and the union of, and the cardinality of the union of the set A1, A2 and A3.
Now to find out the cardinality of the sets A1, A2, A3, I have to use the principle of rule of
inclusion-exclusion. For that I need the cardinality of a pairwise intersection of sets. So, let us
find out cardinality of the set A1 ∩ A2. So, A1 ∩ A2 means the solutions where both x1 is greater
than 3 as well as x2 is greater than 4. And it is easy to see that the number of solutions of such
type will be this : 6.
Because if x1 is greater than 3 that means x1 is at least 4. That means I have picked 4 bills of type
x1 and if x2 is greater than 4 that means x2 is at least 5 that means I have picked 5 bills of type x2.
So, that means overall 9 bills have been already picked and I am now left with the problem of
choosing 2 bills of either denominations x1 x2 x3 without any restrictions. Similarly the
cardinality of A1 ∩ A3 will be this: 1.
597
This is because if x1 is greater than 3 that mean x1 is at least 4 and x3 greater than 6 means x3 is
greater than equal to 7. That means I have already chosen 11 bills that means there is only one
solution possible where x1 is greater than 3 and x3 is greater than 6. Namely we take x1 to be 4
and x3 to be 7. That is why the number of solutions or the cardinality of A1 ∩ A3 is 1. Similarly if
I take A2 ∩ A3 it turns out that there are no solutions.
Because I cannot have any solutions where x2 is greater than 4 and x3 is greater than 6 because
overall the sum of x1, x2, x3 should be 11. If I take the intersection of A1, A2, A3 again there are
no solutions. And now I have all the things to find out the cardinality of union of A1, A2 A3 and
as I said if I subtract it from my universal set cardinality then that will give me the number of
required solutions.
So, now you can see that how I use the alternate form of inclusion-exclusion for solving this
problem.
Let us use alternate form of inclusion-exclusion to solve another problem. So, you are given 2
sets a set A consisting of 6 elements and another set B consisting of 3 elements and I am
interested to find out how many onto functions are there. We already have found the answer for
598
this in one of our earlier exercises using Sterling numbers of second type. But now I will be
using the alternate form of inclusion-exclusion to solve this problem.
So, for finding this and using the alternate form of inclusion-exclusion what we will do is, we
will find out the total number of functions and from that we will subtract the total number of
non-onto functions. So, let us define the set A to be the set of all functions and it is easy to see
that the cardinality of the set A is 36. Because I have 6 elements; for each of the elements I have
3 possible images to pick from.
Mind it the set A has all the functions which are onto as well as all the functions which are non
onto. Now I will be subtracting the number of non-onto functions. For that I define a subset Ai to
be the set of all functions where the element bi is not an image. So, it is like saying the following:
An onto function will have the property that b1 has a pre image b2 has an pre image as well as b3
has a pre image.
So, these are my 3 properties. If any of these 3 properties is violated I get a non-onto functions.
So, that is what I am now trying to find out how many ways I can violate this properties P1, P2,
P3. So, I try to analyze how many ways I can violate the property Pi, so property Pi is the element
bi is an image whenever I am picking a function. So, violation of that property will be that
element bi is not chosen as an image when a function is from the A set to the B set.
And let Ai denote the set of all such functions where the element bi do not have any pre image.
So, for instance the cardinality of the set A1 will be 26 because if b1 is not allowed to be a
possible image then for each of the 6 elements I have 2 possible images to assign: either b2 or b3.
In the same way the cardinality of A2 also will be 26 because A2 means the element b2 is not
allowed to be a possible image.
That means the only images could be b1 or b3 and so on. So, I can say that if I take the
cardinality; if I add the cardinalities of A1, A2, A3 then that is same as the following. You first
select the element bi which should be ignored that means which should not have a pre image.
And then form a function from the A set to the B set with respect to the remaining 2 elements.
599
So, the element bi which has to be ignored it can be chosen in C(3, 1) ways and now you are left
with only 2 elements.
So, each of the elements in the A set can be assigned those 2 images. So, this will be the overall
count if I take the summation of the cardinalities of the sets A1, A2, A3. Now if I take the
pairwise intersections of Ai and Aj sets, then basically they denote the functions where neither
the element bi nor the element bj can be the possible images. So, for instance the cardinality of
the set A1 ∩ A2 will be 16.
Because A1 ∩ A2; means the following. You cannot have element b1 as a possible image, you
cannot have the element b2 as a possible image. That means we are considering the function
where all the elements are mapped to b3. So, there is only 16 such functions basically 1. Same
way the cardinality of A1 ∩ A3 will be 16 and so on. So, again we can say that if we take the
summation of the cardinalities of the pairwise intersection of sets then that will be same as the
following.
You find out the 2 elements which have to be ignored that means which are not allowed to be the
possible images. So, you can do that in C(3, 2) ways and then you will be left with only one
element which is the only allowed image. And then if I take the cardinality of intersection of 3
sets at a time then the interpretation of that will be: I am considering functions where neither the
element bi, bj and bk are allowed to be possible images.
In this example since I have only 3 possible images to choose from, I am saying that I want to
design a function where none of those 3 elements can be the images. And I cannot have any such
function, so that is why the cardinality of the intersection of triplet subset will be 0 in this case.
So, as I said now the union of A1, A2, A3 will give you all the non-onto functions. Because the
union of these 3 sets will have those functions where either b1 is not allowed to be an image or b2
is not allowed to be an image or b3 is not allowed to be an image.
600
And as per the rule of inclusion-exclusion the number of non-onto functions will be this and if
you subtract this quantity from this value the size of the universal set that will give you the
number of onto functions.
So, now let us generalize this formula, so in the previous case we had the case where the first set
has 6 elements and the set of possible images has 3 elements, namely my m was 6 and my n was
3. So, now we want to generalize it, we have a set A having m elements and a set B having n
elements we want to find out the number of onto functions. So, for that I define a set Si to be the
set of all functions where the element bi is not allowed to be an image.
So, I am assuming that my elements of the set B are b1 to bn and I am trying to find out; I am
defining the set Si to be the set of all functions where all the elements in the set B are allowed to
be images except the element bi. And as we did in the demonstration in the earlier slide the total
number of onto functions will be the difference of the number of all possible functions which
include both onto and non-onto functions and the number of non onto functions.
And the number of non-onto functions is given by the cardinality of union of the sets S1 to Sn. So,
this is the number of non-onto functions. And if we expand this cardinality of the union of S1 to
Sn then it will turn out to be this. So, the first term here denotes basically that the cardinality of S1
601
+ S2 +…+ Sn is nothing but you decide the single element which is not allowed to be image that
can be done in C(n, 1) ways.
And then the remaining n – 1 elements in the set B could be the images for the elements in the set
A. So overall contribution of the cardinality of S1 to Sn will be this. Similarly if I want to take the
pairwise intersection and their cardinalities and sum them up then it is equivalent to saying that Si
∩ Sj basically denotes all those functions where neither bi nor bj can be the images.
That means the remaining n - 2 elements can be the images; now this i and j can be; any i and j
from the set 1 to n the indexes. So, that is why it is equivalent to saying that you decide, you
choose the 2 elements i and j or bi or bj which are not allowed to be the images and then your
function will have the remaining n - 2 elements as the possible images and so on.
Now our last case study for the alternate form of inclusion-exclusion is the number of
derangements. So, what exactly is a derangement, so imagine you have 3 persons, person 1 (P1),
person 2 (P2) and person 3 (P3) and they have their respective caps, cap 1 (C1). cap 2 (C2) and
cap 3 (C3). Now the derangement of caps is an arrangement of the 3 caps, so that no cap go back
to its original owner.
602
So, this is one of the derangements. Now P1 gets the cap number C3 P2 gets cap number C1 and
P3 gets cap number C2. So, none of the person get backs its original cap. So, that is in the
derangement and the derangement of n objects basically denotes a permutation of those objects
such that it leaves no object in its original position.
And the number of derangements of n objects is denoted by this quantity D n. So, D1 is 0 because
if there is only one object and you cannot derange it. It will be at its position, D2 is one because if
you have cap number 1 at position 1 and cap number 2 at position 1 and the only way to derange
it is take cap number 1 to position 2 and take cap number 2 to position 1 and so on.
So, we want to find out the number of derangements namely the value of Dn. And there are
several ways of doing that in the tutorial we will derive a recurrence relation. In this lecture we
will see how to find out the value of Dn using the alternate form of inclusion-exclusion. So, again
remember in the alternate form of the inclusion-exclusion you have to identify the property Pi
and then you have to see in how many ways you can violate the property Pi.
So, property Pi with respect to the derangement is that you do not want the ith element to be at
the position i. Because if ith element is still there at position i; then that is not a derangement. We
want to violate that property, so let Si denote the set of all possible permutations of n elements
603
where the ith element is still at its position namely it is still at the position i. So, it is easy to see
that the cardinality of the set Si is (n – 1)!, because think of it as follows.
You have position 1, position i, position n you are finding all possible permutations where the
element i is definitely at its position namely i it is still at position i. I do not know what is the
status of the remaining n - 1 values. They may be at their position some of them may be at their
position, none of them may be there at their position and so on. So, it is equivalent to saying that
I am interested to permute the remaining n - 1 elements which can be done in (n – 1)! ways.
So, that is the cardinality of the set Si and as per our definition if I take the union of the sets S1 to
Sn it gives me all the permutations where at least one of the elements is still at its original
position. It could be either the first element or the second element or the third element or the nth
element, that means the arrangements or the permutations in the union of the sets S1 to Sn are not
derangements.
Because as per derangements; none of the n elements should be at its position. So, that means as
per the alternate form of inclusion-exclusion the value of Dn will be the cardinality of the
universal set. The universal set in this case will be the number of all possible arrangements or
permutations of n elements. They could be derangement or they need not be derangements. So,
this is n! and from that I have to subtract the cardinality of the union of the n sets.
Now, if I expand the cardinality of the union of n sets this will turn out to be this. So, if I take the
effect of the summations of all Si 's the cardinality of the summations of Si this will be same as
you pick the element i which is still at its position namely at position i. That could be done in
C(n, 1) ways and then the cardinality of each set Si is (n – 1)!. Similarly if I take the cardinality
of intersection of Si and Sj then the elements i and j can be chosen in C(n, 2) ways.
That means Si ∩ Sj denotes all those permutations where the ith element is at position i and the
jth element is still at position j. That means the remaining n - 2 elements can be permuted in any
604
order and so on. So, that will give you the value of 𝐷𝑛 = 𝑛! − [𝐶 𝑛, 1 𝑛 – 1 ! −
𝐶(𝑛, 2)(𝑛 – 2)! + … + (−1)𝑛 𝐶(𝑛, 𝑛)0! ] .
So, these are the references for today's lecture and with that I end this lecture. Just to summarize
in this lecture we discussed the principle of inclusion-exclusion, we derived the formula for that
and we saw an alternate form of inclusion-exclusion and some case studies for the alternate form
of inclusion and exclusion, thank you.
605
Lecture -43
Tutorial 7
Hello everyone welcome to tutorial number 7, so let us start with question number 1.
So, we define first what we call as full binary tree and it is a binary tree where every internal
node and by internal node I mean the nodes which are non-leaf nodes, so every internal node has
either 0 child or 2 child. And Hn is defined to be the number of full binary trees which has n + 1
leaves. So, we have to derive a recurrence relation for this Hn, so let me first demonstrate the
value of Hn for some small values of n.
So, if I consider H1 that means I want to find out the number of full binary trees which has 2
leaves and there is only one full binary tree which has 2 leaves namely this binary tree. By the
way here I am not interested in the label of the nodes I am interested in only the structure of the
nodes, so there is only one possible structure of a full binary tree possible which has 2 leaves.
What will be H2?
606
So, there are 2 structurally different full binary trees which as 3 leaves this is binary tree number
1 this is binary tree number 2. If I consider H3, H3 basically denotes a number of structurally
different full binary trees with 4 leaves. So, this is one of the trees this is second tree, this is the
third tree and this is the fourth tree. As I said earlier I am not focusing on the label of the node.
So, you cannot say that a tree where this node root has label a1 and the leaves have label a2, a3 is
different from a tree where the label of the root is a2 and the leaves are a1 and a3. No I am not
focusing on the label of the nodes and just focusing on the structure of the tree that is all. So, my
claim here is that, the number of full binary trees with n + 1 leaves is same as the value of nth
Catalan number and for that what we can do is the following.
Either you can try to derive a recurrence relation explicitly for Hn but we will not do that instead
what we will do is we will say that; we will establish a bijection between the set of full binary
trees with n + 1 leaves nodes and a set of all ways of parenthesizing n + 1 values to specify their
multiplication order and we know already that the number of ways of parenthesizing or the
cardinality of the number of ways of parenthesizing n + 1 values is nothing but the nth Catalan
number, we already know that.
So, if we establish a bijection between the set of solutions for this new problem and the existing
problem whose solution we know is Catalan number then we are done. The bijection is easy to
formulate. I am not giving the exact details but just giving you a high level idea. So, I have to do
the following: I have to take each and every full binary tree with n + 1 leaves. And
corresponding to that I have to specify what exactly is the parenthesizing or multiplication order
that I can formulate for n + 1 leaves.
So, the n + 1 leaves, I call them as x0 to xn, so in this case my n is equal to 1 so that is why I am
taking the case of 2 leaves, my 2 leaves are x0 and x1 and the only possible parenthesizing here is
that I want to multiply them because I have a kind of a balanced tree here. Whereas if I consider
this tree for the case where n is equal to 2. So, this is one possible multiplication ordering and
this is another possible multiplication ordering.
607
The multiplication ordering corresponding to these 2 trees are respectively this. So, you can
interpret here that this tree is left indented that I am going down first left, so that is why x0 is
multiplied with x1 first and then whatever is the result that is multiplied with x2, so treat it as
follows x0, x1, x2. Whereas the next tree that you have you have x0, x1 and x2. So, it is equivalent
to saying that x0 is going to be multiplied with the product of x1 and x2.
If I take the case of n equal to 3 that means 4 leaves, then this ordering corresponds to x0 . x1 . x2
and then multiplied with x3. So, that will be the order: x0 getting multiplied with x1 then that
result getting multiplied with x2 and that result getting multiplied with x3. Whereas if I consider
this tree this will be equivalent to x0 multiplied with x1 that is getting multiplied with x2 and x3
here. So, it is x2 and x3 that gets multiplied then their product is getting multiplied with x1 and
their product is getting multiplied with x0. So, that is the bijection here.
In question 2 you are given the following you are given a square grid where you have the
coordinate (0, 0) and you want to go to the coordinate (n, n) and you have cells here and the only
movements which are allowed to you is that you can at a time you can either go one cell either to
the right from the current cell or to the top of the current cell and you have to find out the
number of valid paths.
608
So, for instance: one valid path could be that I go top from the current cell and then again I go
top and then again I go top then I go top and then I do right, right, right and right. Whereas I can
take a path where I go right and top, right and top, top, top, right and right. Whereas I cannot do
the following. I am not allowed to do the following that I go right and right and then again come
back left and then top and then bottom.
And I cannot do all those things that is not allowed, the only movements that are allowed are one
cell at a time either to the right or to the top. So, for solving this or finding the number of valid
paths I do the following so imagine this is the set of all your valid paths and you have another set
which denote the set of all strings of length 2n. And the 2n length string has equal number of R
symbols and T symbols that is a second set, and my goal here is that I will show that there is a
bijection between these 2 sets.
And if indeed there is a bijection between these 2 sets then it shows that the number of valid
paths is nothing but the number of strings of length 2n which has equal number of R symbols and
equal number of T symbols. And we know that the cardinality of this latter set namely the
cardinality of the set of all strings of length 2n which has n number of R symbols and n number
of T symbols is C(2n, n), because it is equivalent to saying that you have 2n positions.
And out of those 2n positions you have to find out the n positions where the R symbol is going to
be there because once you identify the n symbols and positions where the R symbol is going to
be there the remaining n positions have to be occupied with the T symbols. And the bijection is
very simple here, you take any valid path it will have definitely n number of R movements and n
number of T movements because you are at position number (0, 0) and you have to go to the
position number (n, n).
The number of R movements; first of all the only movements are the right movements and the T
movements, I can imagine that each time I make a right movement I put down a character R.
And each time I make a top movement I write down the character T. So, definitely I have to
make order n number of R movements and n number of T movements. It cannot be the case that
609
the number of R movements is more than the number of T movements or vice versa because then
definitely that is an invalid path and the bijection is very straight forward here.
The third question is how many diagonals are there in a convex polygon with n sides? So, again
we can derive the formula using induction or some other mechanism but we will count it
directly. So imagine you are given a convex polygon consisting of n sides and where the vertices
are v1 to vn, now let us focus on some arbitrary vertex vi and try to count the number of diagonals
that we can have where vi is one of the end points.
Now if vi is one of the end points of the diagonal then the other end point of the diagonal cannot
be the immediate neighbors of vi, namely the vertex number vi + 1 that cannot be the end point
because vi, vi + 1 is not a diagonal, it is the edge of the convex polygon. In the same way, the
other endpoint of the diagonal cannot be vi – 1, because vi and vi - 1 constitutes an edge or a side
for the convex polygon and the side of a convex polygon cannot be treated as a diagonal.
So, that means with vi as one of the end-points of the diagonal I can have n - 3 diagonals, why n -
3? Because excluding these 3 vertices namely vi, vi - 1 and vi + 1 all the remaining n - 3 end points
can be the other end point of the diagonal with vi being one of the end points. So, now it turns
𝑛(𝑛 – 3)
out that the total number of diagonals will be .
2
610
Why over 2? Because what we did here is with vi being one of the end points I have n - 3
diagonals, so that means with v1 as one of the end points I can have n - 3 diagonals with v2 as
one of the end points I can have n - 3 diagonals and in the same way with vn as one of the end
points I can have n - 3 diagonals. So, if I sum all of them that gives me the total number of
diagonals.
But I will be counting the diagonals twice; I will be counting some of the diagonals twice namely
the diagonal with the end points vi, vj will be counted twice because with vi being one of the end
points and the other end point could be anything I would have accumulated n - 3 diagonals. One
of the diagonals there will be the diagonal where vj is one of the end points and the same
diagonal will be counted again when I will be focusing on the case where vj is one of the end
points and the other end point could be anything including vi.
So, that is why to compensate or to remove that over counting we are dividing it by 2 and this
will be the case where n is greater than equal to 4 because if the number of sides is 3 or less than
3 then we cannot have any diagonal, so for instance in a triangle you do not have any diagonal.
In question number 4 we are interested to find out the number of triangulations of a convex
polygon. So, let Tn denotes the number of triangulations of a convex polygon with n + 2 sides
and what is basically a triangulation: it is the process of dividing a convex polygon by non
611
intersecting diagonals. So, if my n is equal to 3 that means I have 5 sides, so I am taking the case
of a pentagon and there are 5 ways of triangulating it by non-intersecting diagonals.
So, now I want to find out a recurrence relation or want to find out the number of ways of
triangulating a convex polygon with n + 2 sides so I will find that by formulating a recurrence
equation and by showing that the solution for that reference equation is same as the nth Catalan
number. So, imagine you are given a convex polygon consisting of n + 2 sides which has the
vertices v1 to vn + 2.
Now to solve my problem of triangulating a convex polygon with n + 2 sides into smaller
problems I consider an arbitrary edge, so for simplicity I take that arbitrary edge to be the edge
or the side vn + 1 and vn + 2, namely the side with end points vn + 1 and vn + 2 and it turns out that this
side vn + 1, vn + 2 can be the part of only one triangle in the overall triangulation. That means if I
fix the third point vk where the vk could be any vertex in the set v1 to vn.
That will fix the triangle in the triangulation which could result in a triangle where one of the
sides of the triangle is vn + 1, vn + 2 that means I am focusing up on the case where, so for instance
if I could have a case where this vn + 2, vn + 1 being one of the sides of the triangle and the third
point would have been v2. But even in that case this edge vn + 1, vn + 2 can be part of only one
triangle; it cannot be part of multiple triangles, because of this reason that we are interested in
doing the triangulation using non-intersecting diagonals. So, I will be dividing my problem into
smaller problems depending upon the third point or the third vertex of the triangle involving the
edge vn + 1, vn + 2. So, the third vertex vk could be any vertex it could be the vertex number v1 in
which case my triangle would have been something like this or my vertex vk could be vertex
number v2 in which case my triangle would have been something like this and so.
So, now once I fix the third vertex namely the vertex vk the overall polygon with n + 2 sides will
be now divided into 2 smaller polygons. The first polygon P which has the vertices this portion
this vertex number v1, v2,v3, vk and the side vk, vn + 2 and the last side being vn + 2, v1 so it will
have k + 1 sides and how many ways I can triangulate the polygon P? Since it has k + 1 sides as
per my definition n + 2 side convex polygon is triangulated in Tn number ways.
612
So, k + 1 side polygon will be triangulated into Tk - 1 number of ways or Tk - 1 number of
triangulations are possible for the polygon P. And the other polygon being the polygon Q, which
will have n + 2 - k sides namely the starting vertex will be vk and the side vk, vk + 1 then the next
side will be vk + 1, vk + 2 then all the way to the vertex vn + 1 and the last side being the side vn + 1,
vk. And as per my definition of Tn the number of ways of triangulating Q is Tn - k.
So, these many triangulations for P and these many triangulations for Q along with the triangle
where the 3 vertices are vn + 1, vn + 2 and vk gives me all possible triangulations where one of the
triangles is vn + 1, vn + 2 and the third vertex of that triangle is vk. So, from the product rule it
comes out that the total number of triangulations with this being one of the triangles in the
triangulations is Tk - 1 * Tn - k.
But now my k can range from 1 to n, my k could be vertex number v1, my k could be vertex
number v2 and so on. So, if I take the summation over k being equal to 1 to n then I get the total
number of triangulations for n + 2 sided convex polygon and this is the same as the recurrence
relation for your nth Catalan number and your initial conditions will be this T0 is 1, T0 means the
number of triangulations for a 2-sided convex polygon.
Well a 2-sided convex polygon cannot be divided or triangulated so no way of triangulating I am

denoting as one way. T1 means the number of ways of triangulating a convex polygon of 3 sides
namely the triangle, and the triangle itself is the triangulation of itself. So, that is why there is
only one way. T2 that means the number of ways of triangulating a rectangle, so this is one
possible triangulation and another possible triangulation for the rectangle will be if you draw
this.
So, there are 2 ways of triangulating a rectangle that is why Tt will be 2 and from 3 onwards the
recurrence will trigger.
613
The last question is we want to find out a recurrence relation for the number of derangements of
n objects, so just to recap a derangement of n objects is a permutation of those n objects such that
none of the objects is at its correct position. That means the object number 1 is not at the first
position object number 2 will not be at the second position and so. So, we can divide the set of
the derangements of n objects into 2 categories.
So, and for these 2 categories we consider or focus on the element which is there at the first
position so we are considering the case where at the first position we have the element ak where
ak could be either a2, a3 or an and with ak being occurring at the first position of the derangement
we can have 2 categories of derangements. Category one, where the element a1 is occurring at
the kth position and remember element a1 is allowed to be occurring at kth position in a valid
derangement, because at the kth position we are not putting ak. ak has already occupied the first
slot and in the kth slot I can have either a2 or I can have a3 or I can have a4. So, I am considering
the case where at the kth position the element a1 is there. So, if that is the case that means I have
already deranged 2 objects the kth element is deranged and it is now shifted to the first position
and the element a1 is also deranged it is no longer occurring at the first position but rather
occurring at the kth position.
If that is the case then my problem boils down to the problem of the deranging the remaining n -
2 elements so I am still left with the problem of deranging a2 a3 ak - 1 ak + 1 and an. So, I am still
614
left with n - 2 elements and whatever way I could derange them you take any derangements of
these n - 2 elements in that the derangement you add the following positioning. You put ak at the
first position and you put a1 at the kth position that will give you a derangement for the n
elements.
And as per my definition of derangement there are Dn - 2 ways of deranging the n - 2 elements, so
this gives you the first category of derangements where ak is occurring at the first position. The
second category of derangements with ak occurring at the first position is the following. You do
not have the element a1 allowed at the kth position, that means element a1 can take any other
position of course it cannot take the position a1.
Because at the first position you have already put the element ak and you are not allowing the
element a1 to occur at the kth position as well in this second category of derangements. So, these
are the only 2 categories having fixed the first position or having reserved the first position for
element ak you have only 2 choices or 2 categories of derangements possible. One category
where at the kth position you are allowing a1 and another category of the derangements where
you are not allowing a1 at the kth position.
So, we are considering now the later category, so if a1 is not allowed to occupy the kth position
then I am still left with the problem of deranging n - 1 elements, why n - 1 elements? Because
even the element a1 is now supposed to be kind of deranged in the sense that it is not allowed to
be occupying or it is not allowed to occupy the kth position. So, that is like a restriction with
respect to element a1 itself now, so as per my definition of derangements there are Dn - 1 ways of
deranging n - 1 elements.
So, you find out those derangements namely the derangements of n - 1 elements and in that
derangement the element a1 would not be occupying the kth position. You take any such
derangement and add the element ak at the first position that will now give you a derangement of
n objects where element ak is occurring at the first position and element a1 is not occurring at the
kth position and this will be your later category of derangements.
615
And if you sum these 2 derangements the number of derangements in category 1 and the number
of derangements in category 2 that will give you all possible derangements where the element ak
is occurring at the first position. Now since there are n - 1 options for ak so my ak could be a2, my
ak could be a3, my ak could be an and for each possible ak I have Dn - 1 + Dn - 2 number of
derangements. So, that is why the overall formula for the number of derangements will be n - 1
times the summation of Dn - 1 and Dn – 2, with that I end tutorial number 7, thank you.
616
Lecture -44
Graph Theory Basics
Hello, everyone welcome to this lecture, so the plan for this lecture is as follows.
In this lecture we will introduce the basic terminologies related to graph theory. We will discuss
about different types of graphs and we will also discuss about Euler's theorem.
617
So, what is a graph? It is a collection of two sets, a set of vertices and a set of edges. So, the set of
vertices is denoted by the set 𝑉, 𝑉 ≠ 𝜙 it is a non-empty set of vertices also called as nodes and
we have another set of edges denoted by this notation 𝐸, which have edges of the form (vi, vj)
where both 𝑣𝑖 , 𝑣𝑗 ∈ 𝑉 and this edge set could be empty, 𝐸 = 𝜙. So, what it means is that in a graph
you can have a graph which has no edges but the vertex set cannot be empty. The vertices are
always there, you cannot have a graph which is where the vertex set is empty as well as the edge
set is empty.
So, we have two types of graphs directed graphs and an undirected graph. When I say directed
graphs then pictorially you can imagine that in directed graph the edges have directions associated
with it. That means we have the notion of the starting point of an edge and the ending point of an
edge. And in terms of set theoretic notations, we can say that a directed graph is a graph where the
edges are ordered pairs and as soon as I say that edges are ordered pairs, that means it matters
whether the starting endpoint is vi or the starting end point is vj, if you are talking about an edge
(vi, vj). So, for instance if I consider this directed graph, then the edges that we have here are the
ordered pairs (v1, v2) because the starting point of this edge is v1 and the end point is v2 i.e if
(𝑣1 , 𝑣2 ) ∈ 𝐸 then (𝑣2 , 𝑣1 ) ∉ 𝐸. Whereas if we consider undirected graphs then the edges have no
direction associated with them which is equivalent to saying that the edges are unordered pairs.
That means when I consider this edge from the vertex v1 to vertex v2, if (𝑣1 , 𝑣2 ) ∈ 𝐸 then
(𝑣2 , 𝑣1 ) ∈ 𝐸.
618
So, let us now define what we call as a simple graph and this definition is applicable both for the
directed graph as well as undirected graph. So, a simple graph is a graph which has no self loops
and there can be at most one edge between any two nodes. So, if I consider this undirected graph
then this is not a simple graph because between the nodes v1 and v3, you have two edges this edge
as well as this edge which is not allowed as per the definition of an undirected simple graph.
But if I consider this directed graph, then this is a simple directed graph because even though you
have two edges involving the nodes v1 and v3, this is one of the edge and this is another edge. They
are different directed edges because the ordered pair (𝑣1 , 𝑣3 ) ≠ (𝑣3 , 𝑣1 ). So, this is an example of
a directed simple graph.
So, now let us introduce some terminologies related to undirected graphs. So, if you are given an
undirected graph then a pair of vertices u, v are called adjacent or neighbors of each other. If the
edge (𝑢, 𝑣) ∈ 𝐸 b edge set, that means if u and v are the end points of an edge, then in an undirected
graph then we call the vertices u and v to be adjacent or neighbors.
Here v could be u that is allowed. That means if I have a self-loop then the vertex v will be called
adjacent to itself, right? Then if we are given an edge small e, then we will say that it is incident
with the nodes u, v if u and v are the end points of the edge. So, for instance if I consider this edge
619
𝑒, then v1 and v2 are incident with respect to this edge because they are the end points of this edge
e and so on.
Now let us define next what we call as the degree of a vertex v. So, again I am giving the definition
with respect to undirected graph, but the definition can be extended or generalized for directed
graphs as well. So, what is the degree of a vertex? The degree of a vertex is the number of edges
incident with v or in the simpler language, the number of edges which have v as one of its end
point.
And the definition has a special case if we have a self-loop incident with the vertex 𝑣. If there is a
self-loop incident with the vertex is v then that self-loop is counted as contributing 2 to the degree
of the vertex of small v. So, for instance, if I take this undirected graph then the degree of the
vertex v1, let us find out what's the degree, how many edges are there incident with the vertex v 1.
So, this is one of the edge, this is another edge, okay to the two edge and now we have this edge
again incident with v1 so, total three. Whereas if I consider the degree of the vertex v3, then we
have this edge, this edge and this edge and we have a self-loop. The self-loop will be counted
twice, while counting the degree of the vertex v3. So, total the degree of the vertex v3 will be 5 and
so on.
So, this degree of a vertex the definition is with respect to the undirected graph, the undirected
graph need not be simple. In fact in this particular example between the vertex v3 and v1, we have
two edges and both of them are counted while counting the finding the degree of the vertex v3.
620
Next, we state a very fundamental fact about an undirected graph this is also called as the
handshaking theorem. So, if you are given an undirected graph it may be simple, it need not be
simple, it is just an undirected graph. And say the graph has m number of edges. Then what the
theorem basically says is that if you sum the degrees of all the vertices in the graph then it will be
twice the number of edges always.
So, you can verify this with respect to this example graph, so you take the summation of ∑(𝑣1 ) +
deg (𝑣2 ) + deg (𝑣3 ) It will be equal to twice the number of edges in the graph, so what is on? So,
you have 1, 2, 3, 4, 5, 6, 7 and 8. So 2 ⋅ 7 = 14 and you can verify that. So, deg(𝑣1 ) = 3,
deg(𝑣2 ) = 6, deg(𝑣3 ) = 5.
So, let us prove this statement is true for any undirected graph. So, consider any arbitrary edge
𝑒 = (𝑢, 𝑣) ∈ 𝐸 in your undirected graph. It may be a self loop that means 𝑣=u or it may not be a
self-loop. Let us try to find out that what will be the overall contribution of this edge when we are
summing up the degrees of all the vertices in the graph? So, the claim is that the contribution of
this edge will be 2 to the overall summation of degrees of all the vertices.
Because this edge will be contributing 1 when we will taking the degree of the vertex u and it will
be again contributing 1 when we are taking the degree of vertex v and if u is same as v then as per
the definition of the degree the self-loop will be counted twice, because 𝑣=u. So, that is why in the
621
expression when we are summing up the degrees of all the vertices in the graph the contribution
of the edge (u, v) will be 2. And hence it is easy to see that the summation of the degrees of all
the vertices will be twice the number of edges in the graph.
So, based on this fundamental fact we can derive another conclusion about any undirected graph
and this conclusion is often called as the Euler's theorem. So, what it says is the following. So,
what this theorem basically says? It says that you take any undirected graph then the number of
vertices of odd degree will be always even. That means you will either have 0 number of vertices
of odd degrees or 2 vertices of odd degrees or 4 vertices of odd degree or so on.
That means it would not be the case that the number of vertices of odd degree is odd and for that
for deriving this conclusion we will use the previous fact namely the summation of the degrees of
all the vertices in the graph is twice the number of edges and the proof is very simple. So, let V be
the set of vertices in your undirected graph where 𝑉 = 𝑉𝑜𝑑𝑑 ∪ 𝑉𝑒𝑣𝑒𝑛 and 𝑉𝑜𝑑𝑑 ∩ 𝑉𝑒𝑣𝑒𝑛 = 𝜙.
Because you cannot have a vertex in a graph which has both odd degree as well as even degree. A
vertex will have either odd degree or even degree and hence it will belong either 𝑣 ∈ 𝑉𝑜𝑑𝑑 or 𝑣 ∈
𝑉𝑒𝑣𝑒𝑛 so these two sets constitute a partition of your set of vertices V. Now what I can say is that,
if I take the summation of the degrees of all the vertices in the graph then that is equivalent to the
summation of the following two quantities.
622
You take all the vertices in the set Vodd namely all the vertices which have odd degrees, and take
the summation of the degrees of the respective vertices in the set Vodd. And you take all the vertices
in the set Veven namely all the vertices which have even degrees and sum of their degrees and if
you sum these two quantities that will give you the summation of the degrees of all the vertices in
the graph.
Now I know that the left hand side of this equality is even, because as per the above theorem the
left hand side is two times the number of edges. The number of edges could be odd or it could be
even, but two times that number of edges will be an even quantity. So, I know that the left hand
side is an even quantity and I also know that if I sum up the degrees of all the vertices which have
even degrees, then that is also an even quantity.
The number of vertices of even degree it could be odd, it could be even it does not matter. But
since I am adding several quantities each of which is an even quantity, the overall summation will
be even. From that I can conclude the following that I can conclude that if I sum up the degrees of
all the vertices in the set Vodd that also will be even because if that is not the case then you cannot
have the difference of two even quantities.
Because that is simply coming from the fact that the summation of the degrees of all the vertices
in the set Vodd, is the difference of two even quantities and the difference of two even quantities
will be even, so what is the conclusion that we have drawn till now? We have concluded that if I
take the summation of degrees of all the vertices in the set Vodd then that is an even quantity. That
is possible only when the number of entities in the set Vodd is even, right?
So, because you are summing up several odd quantities and the summation of those odd quantities
is even. That is given to you, that is possible only when the number of quantities that you have
added is even. So, that is a fundamental fact about any undirected graph, irrespective of whether
it is a simple graph or a non-simple or whether it is a regular it is not a simple graph.
623
Now let us define some special types of undirected graphs. So, the first special type of undirected
graph is a complete graph. And the property of this graph is that you have exactly one edge between
each pair of distinct vertices. And since this is a simple graph and our property is that you cannot
have more than one edge between every pair of distinct vertices. Automatically we have here the
restriction that you cannot have a self-loop, because if you have a self-loop then that self-loop will
be violating the definition here.
So, the requirement here is that you take the vertices, all the vertices between every pair of distinct
vertices you will have exactly one edge. You do not have the option of 0 or 1, exactly 1 edge
should be there between every pair of distinct vertices and if n is the number of nodes in a complete
graph then we use this notation Kn to denote a complete graph with n nodes. So, these are the
examples some of the examples of complete graphs with various values of n.
The complete graph with 7 nodes is this, complete graph with 4 nodes is this, complete graph with
12 nodes is this.
624
Then another special type of simple and directed graph is a cycle graph denoted as C n. Here the
vertex set will be consisting of n nodes and you will have 𝐸=
{(𝑣1 , 𝑣2 ), (𝑣2 , 𝑣3 ), … , (𝑣𝑛−1 , 𝑣𝑛 ), (𝑣𝑛 , 𝑣1 )}. Now since the graph is simple, the cycle graph is
defined only when the number of vertices is 𝑛 ≥ 3. Because if I try to define a cycle graph between
involving just two nodes, then as per the definition the edge set will be the following. You have
an edge between 1 and 2 and again you have an edge between 2 to 1 that will be the definition of
the edge set as per this general definition.
But that will violate the property that it is a simple graph. So, that is why 𝑛 ≥ 3 and then only we
can define a cycle graph. Well you cannot say that, why cannot I take this to be a cycle graph
namely a graph where I have just an edge between 1 and 2? Because the interpretation of the cycle
here is the following. The interpretation of the cycle is that once you go to the nth vertex, you have
an edge from the nth vertex back to the first vertex.
So, without violating the requirement that your graph is a simple graph, this is possible only when
n is greater than equal to 3. So, if I consider a cycle graph with three nodes then that will be
something like this you, have v1, v2, v3 and then you have an edge back from 3 to 1.
625
Then there is another special simple and directed graph called as the wheel graph. It is slightly
different from the cycle graph, so what you do is you take a cycle graph involving n -1 nodes and
then you add a central vertex which is the nth vertex and the central vertex is now we add an edge
involving this central vertex and all the vertices in your cycle graph Cn-1. So, for instance if I want
to form W4 then I take the cycle graph involving three nodes. Add the fourth vertex v4 and add an
edge from this fourth vertex to every other existing vertex in the cycle graph.
We also have another special simple undirected graph called as the n cube or the hyper cube
denoted by this notation Qn. So, this graph will have 2n nodes, where each node represents a
possible n bit string. So, remember that the number of bit strings of length n is 2n, so each string is
626
represented by a node in this hypercube graph and you have an edge between the ith vertex and the
jth vertex if the bit strings represented by the ith vertex and the jth vertex differ in exactly one bit
position. Otherwise the edge is not there between the ith vertex and the jth vertex. So, for instance
if I take the graph Q1 then I can have only two strings, two bit strings of length one, the bit string
0 and the bit string 1. They differ in exactly one bit position so the edge will be there. If I want to
form the graph Q2 the way I do it is as follows.
So, there will be, this will be the graph Q2, because you will have four bit strings of length 2 and
you can see that this edge is there because the binary string is 10 differs with the string 00 in
exactly one bit position namely the first bit position. You do not have an edge between the bit
string or the vertex one representing the bit string 10 and the vertex representing the bit string 01.
Because these two bit strings differ in more than one position, so let me go back and try to
understand that what exactly is the relationship between Q2 and Q1?
How exactly I can interpret Q2 or relate Q2 with the graph Q1? So, the way I can construct the
graph Q2 is as follows. I take two copies of the graph Q1, this is your first copy and this is the
second copy. In the first copy I add the bit one at the beginning of all the nodes or all the bit strings
represented by the nodes in the copy of Q1. So, the copy of Q1 that I would have taken is the
following. I would have taken the vertex representing the bit string 0 the vertex representing the
vertex bit string 1. What I am saying is you add 1 at the beginning of these bit strings.
You take the second copy of Q1 and you add 0 at the beginning of all the strings represented by
the notes in this copy of Q1 and now you add the required edges depending upon the bit strings
differ in exactly one bit position or not. In the same way if I want to define or get the graph Q 3
what I do the following, I take two copies of Q2, one copy of Q2, another copy of Q2 and I extend
the length of the bit strings of the nodes by adding one at the beginning of all the strings in the first
copy of Q2.
And I add 0 at the beginning of all the bit strings in the second copy of the Q2 and then I add the
required edges depending upon whether the two vertices differ in exactly one bit position. So, that
is the definition of our Qn graph.
627
Now let us next define what we call as Bipartite graphs. So, if you are given a simple graph, then
the graph is called bipartite, if we can find out two vertex sets V1 and V2 such that the following
whole the vertex sets V1 and V2 should constitute a partition of your vertex set. That means 𝑉 =
𝑉1 ∪ 𝑉2 and 𝑉1 ∩ 𝑉2 = 𝜙.
And the second property that I need from this partition V1 and V2 is the following. I need the
following, you take any edge E in the graph the end points whose end points are vi and vj, then one
of the end points should be in one of the subsets and the other end point should be in the other
subset. 𝑒 = (𝑣𝑖 , 𝑣𝑗 ) then 𝑣𝑖 ∈ 𝑉1 and 𝑣𝑗 ∈ 𝑉2 . So, what it means is that you take any edge of the
graph, it cannot be the case that both its end points are in the same set. Set V1 or in the set V2, one
of the end point should be in V1 the other end point should be in V2. So, for instance if I take this
graph then, this graph is not a bipartite graph because I cannot find the required V1 set and V2 set.
This is because if I focus on the specific portion of the graph, namely this portion is the triangle
graph. So, if I call this as this edges that is e1, e2 and e3 and if I try to find out a candidate V1, V2
satisfying this condition, I cannot do that.
Say, for instance I include this red colored vertex in the set V1, so I let me call this vertices as a, b,
c. So, suppose I include a in V1 as per my requirement since the other end point of edge e1 namely
the vertex v should be the other subset or other partition. I should put the vertex b in the subset V2,
628
that is fine. Now what about the edge e2? The other endpoint of the edge e2 is c, so it cannot go in
the same set V2 because that will violate my requirement.
So, I can put c in the collection V1, but as soon as I put c in the collection V1, I see that I get a
violation. Because I now have an edge namely e3, where both the end points a and c are in the
same collection V1. So, that is why irrespective of how you try to come up with your candidate V1
and V2 you cannot find a partition V1, V2 for this graph satisfying this requirement. So, that is why
this graph is not a bipartite graph.
But if I take this graph, then this graph is a bipartite graph and what will be the partition set for the
vertex set V? What you can do is you take all the blue colored vertices and put them in the
collection V1 and you take all the red colored vertices and put them in the collection V2. And now
you can see that you take any edge e in this graph one of its end points will be in this V1 and the
other end point will be in V2. It would not be the case that both its end points are either in V1 or
V2. So, that is why this is an example of a bipartite graph.
Refer Slide Time: 29:53)
Now let us define next what we call as a complete bipartite graph? So, it is a special type of
bipartite graph. So, first of all it is a bipartite graph namely it should be possible to come up with
a partition of the vertex set, and that partition should have a special property. You take any edge
629
in the graph one of its end point should be in V1 other end point should be in V2 that comes from
the definition of the bipartite graph.
But this implication is now a bi-implication, that means it is the case that you have an edge in the
graph between each and every vertex in the set V1 and the set V2. So, if you compare this bi-
implication condition with the previous definition, in the previous definition it might be possible
that you have some vertices in V1 and some vertices in V2 such that between those vertices you do
not have an edge in the graph. So, for instance if I take this graph then this is a bipartite graph but
this is not a complete bipartite graph. Why is not a complete bipartite graph? Because if I say for
instance, you take this vertex it will be in one of the subsets given by your partition. And you take
this blue colored node this will be in the other subset in your partition, but you do not have an edge
between these two vertices in the graph.
So, that is why this implication was there only in one direction, but in a complete bipartite graph
what I am saying is the implication should be a bi-implication. So, that is why this graph is not a
bipartite graph but this graph is a bipartite graph because you can put the vertices u1, u2, u3 in one
subset and you can put the other vertices namely v1, v2, v3, v4 in the other subset namely V2.
And you can check now that you take any edge in the graph one of its end point is in V1 and the
other end point is in V2. So, that shows the implication is true in one direction and the implication
is true in the other direction as well, you take any vertex in V1 and any vertex in V2. There exists
an edge between those two vertices in your graph. So, that is why this is a complete bipartite graph.
And we use this notation Km,n to denote a complete bipartite graph, where the cardinality of the
two subsets in your partition are m and n respectively.
630
So, that brings me to the end of this lecture these are the references used for today's lecture. So,
the basic concepts related to graph theory you can find in the Rosen book, but there is also this
advanced or dedicated book for graph theory. So, this book on graph theory is very nice, it covers
both the basic concepts as well as advanced concepts. And if you are interested to explore graph
theory, I encourage you to get a copy of this book. Thank you!
631
Lecture -45
Matching
Hello everyone. Welcome to this lecture.

The plan for this lecture is as follows. So, in this lecture we will introduce the definition of
matching. We will discuss about complete matching and we will see the theorem statement for
Hall's marriage theorem.
632
So, let us begin with bipartite graphs and matching. So, just to recap in the last lecture we discussed
what are bipartite graphs? They are the graphs where the vertex set can be partitioned into two
disjoint subsets V1 and V2 where for each edge in the graph one of the end points is into the subset
V1 and the other end point is in the subset V2. And it turns out that bipartite graphs are very
important class of graphs used to model various real-world problems.
And one of the problems for which they are used extensively is that of job assignment. So, let me
demonstrate the job assignment problem with this example. Imagine you have two organizations,
organization 1 and organization 2. Both of them are trying to build a software and say the software
has four modules. It has a Requirement module, Architecture module, Implementation module and
Testing module.
And say both organizations have their respective employees. Organization 1 has employees A, B,
C and D. Each of these employees can handle one of the four modules required in the software and
the skill set of each employee is modeled by adding an edge between the node representing that
employee and the node representing the corresponding skill set which that employee can handle.
So, for instance this edge here denotes that the employee A is capable of handling Requirement
and this edge from A to Testing denotes that he is capable of handling the Testing module as well.
In the same way the employee B can handle three of the modules. It can handle Architecture, it
633
can handle Implementation and it can handle Testing and so on. Similarly, the edges are added in
this graph representing the skill set of the employees in the organization number 2.
Now the job assignment problem is the following. Since we have to build a software, we have to
ensure that each of the four modules namely Requirement, Architecture, Implementation and
Testing are taken care. So, we want to assign employees to do the jobs as per their skill set such
that each job is attended by some employee. But at the same time we do not want to be very strict
or unfair with an employee by assigning multiple jobs as per its skill set.
So, for instance I do not want to do the following. I take the employee B and assign Architecture,
Implementation and Testing all three to employee B. That I do not want to do. So, I want to assign
each employee at most one job. When I say at most one job that means it is fine if I do not assign
any job in my job assignment or when I am assigning job to the employees to a particular employee
it is fine.
I just want to ensure that each job goes unattended and at the same time I do not assign multiple
jobs to any employee when I am doing the job assignment. So, that is my job assignment problem.
So, it is easy to see that one of the possible job assignment in organization 1 is the following. I
take or I ask employee D to take care of the only job that it is capable of doing namely Requirement
and then I take the employee B.
So, employee B can handle Architecture, Implementation and Testing. So, I choose architecture
for employee B. I pick employee C and assign the Implementation module to employee C and then
I take the employee A and assign the Testing module to employee A. So, I can come up with one
job assignment in the organization 1 satisfying this criteria. But if you take the second organization
it is not at all possible to come up with the job assignment in organization 2 satisfying these
conditions. Why so?
So, let us try to come up with one possible job assignment and then we will see that we will get
stuck here. So, for instance let us start with W and W is capable of only handling Architecture. So,
I can say W is assigned Architecture. So, we have taken care of W and Architecture. Then we go
634
to X. X can handle Requirement, Implementation and Testing. So, let us assign the Requirement
module to employee X.
Now let us go to employee number Y. Employee number Y can only handle Architecture. But I
have already assigned the Architecture module. So, no need to again assign it to any other
employee. So, I can say let us not assign any job to employee Y and then if I go to employee Z he
can handle Requirement, Architecture and Testing. Requirement and Architecture are already
taken care then I can assign only Testing to employee Z.
But in this whole process in this scheduling the module Implementation is not assigned to any
employee. It goes unattended. And you can try other possible combinations as well. You will see
that irrespective of the way you try to assign the jobs you cannot come up with a job assignment
satisfying these conditions. You cannot do that in this graph.
So, whatever we have discussed in this example can be modeled by a concept called matching. So,
imagine you are given a simple undirected graph. That is important. Then a collection of edges
denoted by M is called a matching if no two edges in this collection of edges M are incident with
the same vertex. Formally what I am saying here is the following. You take any pair of edges
(𝑒𝑖 , 𝑒𝑗 ) ∈ 𝐸, if they are different then all the four end points, two end points of ei and the two end
points of ej they should be distinct, they should be different. I stress here this condition 𝑒𝑖 ≠ 𝑒𝑗 is
635
important here in this implication. Because if I take the same edge ei and the same if I take the
edge 𝑒𝑖 = 𝑒𝑗 , of course, the first one of the endpoints of ei will be the same as one of the endpoints
of ej and so on. So, this inequality would not hold if my 𝑒𝑖 = 𝑒𝑗 .
So, this condition this implication should hold for every pair of distinct edges ei and ej from my
collection of edges that I have picked. I stress here that there is no restriction on |𝑀|. I can just
pick one single edge from my edge set that will constitute a matching or I can pick two edges from
my edge set and if they can satisfy this implication that will also be considered as a matching and
so on.
So, what this implication means on a high level is the following. If I take the vertices of all distinct
edges in my collection of edges M they should be distinct. So, again if I take the job assignment
problem and organization number 1, what I actually did there was I picked the matching. My
matching was this collection of edges namely I took the edge (D, Requirement), (B, Architecture),
(C, Implementation) and (A, Testing).
And you can check here that you pick any 𝑒𝑖 ≠ 𝑒𝑗 , which are distinct in this collection of edges
M, their four end points will be distinct. So, if I have a matching M then a vertex v in my graph
will be called matched with respect to that matching if the vertex v is the end point of some edge
in my matching otherwise the vertex v is called unmatched. So, formally this means that there
should exist some edge small e and some vertex u such that the edge (𝑢, 𝑣) ∈ 𝐺 is there in my
graph and that edge is nothing but the edge e.
If that is the case then I will say that the vertex v is matched. In fact the vertex u is also matched
with respect to the matching if I have an edge of the form (u, v) if I have an edge of the form (u,
v) in my collection of edges M that I have picked.
636
So, for instance if I take this graph here and if this is my matching M then I have vertex number A
matched. Because there is an edge with one of its endpoints being A. I also have vertex B matched
with respect to M because I have an edge with B as its endpoint and so on. Now we have different
types of matching. So, we have what we call as maximum matching and it is a matching which has
the largest cardinality.
That means it is the collection of edges whose cardinality is the largest. That means you cannot
have a matching whose cardinality is bigger than the collection of edges of the matching that you
have picked. So, if I take this graph and the maximum matching here is the collection of edges (a,
b) and (c, d). Its cardinality is two, whereas if I take the matching M’ equal to (b, c) then that is
not the maximum matching. Why it is not the maximum matching?
Because the cardinality of |𝑀’| = 1. By the way the matching M’, this collection M’ is indeed a
matching because the end points are distinct here. You do not have first of all the second edge
here. So, if you just take any edge by default it is a matching. So, this is a matching, M’ is a
matching and it is not a maximum matching because you have only one edge in M’ whereas you
can have another matching M which has two edges in it.
So, M’ is not a maximum matching here. However, the matching M’ is maximal matching. So,
there is a difference between maximal matching and maximum matching. So, what is the maximal
637
matching? A maximal matching is a matching which cannot be further extended and what do I
mean by cannot be further extended? By that I mean that you cannot find another matching M’
such that 𝑀’ is a superset of your matching M.
Because if you can find another matching M’ which is a super set of M that is equivalent to saying
that you can extend, that means you can add further edges in M and still get a matching of bigger
cardinality. That is what is the interpretation of cannot be further extended in this definition. So,
that is why your matching M’ is a maximal matching because you cannot add any other edge now
in M’ and still get a new matching. Why so?
Because I cannot add the edge (a, b) in M’, because if I add the edge (a, b) to get say M’ and M’’
is not a matching. Because in M’’ you will have the edges (a, b) as well as (b, c) and you do not
have all the endpoints of any pair of distinct edges distinct. So, here you have the edges e1, e2 such
that one of the endpoints of edge e1 is same as one of the end points of the edge e2 which goes
against the definition of matching.
So, that is why M’ here in this example cannot be further extended and hence it is a maximal
matching. Whereas if I take the matching (c, d) in this graph it is not maximal. Why it is not
maximal? Because I can further extend my matching M and get a bigger matching, namely I can
add the edge (a, b) in this matching and it will still constitute a bigger matching here. So, that is
why there are two types of matching here, maximum matching and maximal matching.
Of course, every maximum matching is a maximal matching but other way around need not be
true. You may have a matching which is just maximal but it is not maximum and one example is
this. A matching (c, d) is the maximal matching. Sorry, one example here is this the matching (b,
c) it is a maximal matching but it is not maximum matching.
638
Now we have another kind of matching called as complete matching and what is a complete
matching? So, imagine you are given a bipartite graph with bipartition (V1, V2) then a matching in
this graph is a complete matching from 𝑉1 → 𝑉2. And when I define a complete matching it is very
important whether the complete matching is from 𝑉1 → 𝑉2 or from 𝑉2 → 𝑉1. They will be different
in general. So, here I am giving the definition of a complete matching from 𝑉1 → 𝑉2.
So, the matching M that I have picked is called a complete matching from 𝑉1 → 𝑉2 if every vertex
in V1 is matched with respect to my matching M or equivalently |𝑉1 | = |𝑀|. This is because if
every vertex in V1 is matched that means I have edges in my matching M where for every vertex
small v in V1 there is a corresponding edge with v as its end point in my matching M.
And as per the definition of a matching I cannot have any other edge as well present with the same
endpoint v. That is not possible if I already have an edge with v as its end point in my matching
M. I cannot include any other edge in my matching M different from this edge which has v as its
end point. That automatically means that the number of edges in my matching M that I have picked
should be exactly same as the number of vertices in my subset V1.
If that is possible if that is the property of my matching then I will say that my matching M is a
complete matching from 𝑉1 → 𝑉2. Whereas if I want to find a complete matching from 𝑉2 → 𝑉1
then the matching should be such that its cardinality should be exactly equal to the cardinality of
639
V2. Namely it should be ensured that every vertex in V2 is matched and that is possible only if the
|𝑀| = |𝑉2 |
So, for instance if I take this bipartite graph and it is easy to see that this is a bipartite graph because
I can put A, B, C, D in one subset and I can put requirement, architecture, implementation and
testing in another subset. And this is a bipartite graph but it is not a complete bipartite graph and
when I am defining complete matching I do not need a complete bipartite graph.
It is defined with just bipartite graph. Why this graph is not a complete bipartite graph? You have
many missing edges here. So, for instance I do not have the edge from D to testing and so on. This
edge is not there. So, if this is my V1 and V2 and if I pick this matching M then it constitutes a
complete matching from 𝑉1 → 𝑉2. Why it constitutes a complete matching from 𝑉1 → 𝑉2 ?
Because all the vertices in V1 they are matched, A is matched, B is matched, C is matched, and D
is matched. In fact, in this particular example this matching is also a complete matching from V2
to V1 as well. Because I can interpret this matching as if it covers each and every vertex in the
vertex set V2. So, requirement is covered or matched, architecture is matched, the node
implementation is matched and the node testing is also matched.
So, now let us talk about a necessary and sufficient condition for checking whether there exists a
complete matching in my given bipartite graph or not. So, if I am given a bipartite graph and if I
640
want to check whether there exists a complete matching from V1 to V2 or a complete matching
from V2 to V1 there should be some procedure for doing that.
Can I say that there exist some graph theoretic properties by checking which I can declare whether
the given graph has a complete matching or not? And there exists a very simple condition that
condition is also called as Hall's marriage problem. If you are wondering why the term marriage
is coming here, because the condition is given in the context of a bipartite graph where the vertex
set V1 and V2 corresponds to the set of boys and the set of girls and the edges basically in the graph
represents the preferences that the boys have for the girls.
So, if some boys prefer some girls and vice versa you will have the corresponding edges between
the vertex sets in V1 and V2. And we would like to find out a complete matching from 𝑉1 → 𝑉2 in
the sense that we would like to ensure that each boy gets a wife or a bride with respect to the
preferences that he has and we want to ensure that all the boys are covered. So, that is equivalent
to saying that I want to find out the complete matching from the set 𝑉1 → 𝑉2 .
So, that give the necessary and sufficient condition for that problem and that necessary and
sufficient condition is also called as the Hall’s marriage condition. So, I have given here two
graphs. So, this is my graph number G1 and this is my graph number G2. The interpretations of V1
and V2 are different in G1 and G2. In G1 the vertex set V1 are my employee set and the vertex at
V2 are the skill set. Where in my graph G2, I have made the skill set as the vertex set V1 and
employee set as the vertex set V2.
So, you can see that in the graph G1 I have a complete matching from V1 to V2. That means I can
ensure that all the nodes in 𝑉1 = {𝐴, 𝐵, 𝐶, 𝐷} are matched with respect to the matching that I have
formed here. Whereas if I take the graph G2 then I cannot have a complete matching from V1 to
V2. Now why it is not possible to have a complete matching from 𝑉1 → 𝑉2 ? And we already saw
when we discussed the job assignment problem that in this specific graph G2 it is not possible to
do the job assignment.
641
So, recall the job assignment problem was that we want to ensure that each of the four modules
namely requirement, architecture, implementation and testing are taken care by some employee.
But at the same time I do not want to assign more than one job to any employee. That problem can
be modeled as if you want to find out a complete matching from 𝑉1 → 𝑉2 . Because if you want to
find a complete matching from 𝑉1 → 𝑉2 that is interpreted as if you want to pick a subset of edges
in G2 such that no two distinct edges in that subset has a common endpoint.
That is equivalent to saying that there exists no employee who is assigned multiple jobs and at the
same time you want to take care of all the four modules. So, that is why complete matching from
𝑉1 → 𝑉2 . I do not want to cover all the employees, it is fine if some employee is not assigned any
job. My goal is to assign or ensure that each job is taken care. So, that is why the complete matching
is from 𝑉1 → 𝑉2 and it turns out that in graph G2 there is no complete matching from 𝑉1 → 𝑉2.
So, if you are wondering that why a complete matching is possible in graph G1 but why a complete
matching is not possible in G2, what exactly is the difference? What exactly is the source of
problem in graph G2? Well the source of the problem in the graph G2 is this specific sub portion
of the graph. If you take this specific sub portion of the graph then we have three modules namely
the requirement module, implementation module and testing module and there are only two
employees who can take care of these three modules. So, in terms of graph theoretic properties I
have a subset V1’ here and a subset V2’ here such that the number of neighbors of V1’ is less than
the number of nodes that you have in V1’. So, the number of neighbors all together is two but you
have three nodes in V1’. So, how can it be possible that you take care of requirement as well as
implementation as well as testing?
And at the same time ensure that neither the employee X nor the employee Z is assigned more than
one job. That is not possible because if at all requirement is to be taken care it can be taken care
only by either the employee X or by the employee Z. Once you assign it to employee X, say for
example, then you cannot assign employee X any other job. So, you will be now left with two
more modules to be taken care. But now you have only one employee left. How can that be
possible? You will be able to only assign testing to employee Z and now the module
implementation will go unattended. So, that is the source of problem in graph G2 which is not
642
allowing you to find out the complete matching from V1 to V2 and that is precisely the formal
statement on the necessary and sufficient condition given by the Hall’s marriage theorem.
So, to understand the necessary and sufficient condition given by the Hall's marriage theorem, let
us first introduce a few definitions here and notations. So, if I have a graph G then the neighbors
of a vertex v, small v is denoted by this notation N(v) and what exactly is the neighbor set of the
vertex v? 𝑁(𝑣) = {𝑢: (𝑢, 𝑣) ∈ 𝐸}W. So, it is basically saying that the neighbours of vertex v are
all the vertex u such that u is one of the end points of an edge incident with the vertex v. Of course,
if there is a self loop at the node v then v also will be considered as its own neighbor. And once
we have the definition of a neighbor of a single vertex then the definition of a neighbor of a vertex
at A is the following. So, if A is a set of vertices that the neighbor of this set of vertices in A is
basically the union of the neighbors of all the individual vertices in that.
So, for instance if I have this bipartite graph well by the way the neighbor and the neighbor of a
set of vertices is defined with respect to any graph but we will be using it in the context of bipartite
graph and we will be discussing the Hall’s marriage theorem. So, if I consider this bipartite graph
and then if this is my set of vertices then its neighbor set is A, C, D and B. Why? Because the
neighbor of the vertex requirement are C and D and the neighbors of testing are B and A. So, if I
643
take their union, I get the set {𝐴, 𝐶, 𝐷, 𝐵}. Whereas if I take the subset {architecture,
implementation}, so the neighbor of architecture is only C and neighbor of implementation is B.
So, what exactly is the necessary and sufficient condition given by the Hall’s marriage theorem
for complete matching? It states the following; the Hall’s marriage theorem statement is the
following. If you are given a bipartite graph where (V1, V2) is the bipartition then for complete
matching from V1 to V2 you need the following necessary and sufficient condition. You need the
following. You take any subset 𝐴 ⊆ 𝑉1.
That subset could be an empty subset,𝜙. It could be a subset with just one vertex or it could be the
whole set V1. You take any 𝐴 ⊆ 𝑉1., the number of neighbors of that subset A should be greater
than equal to the number of nodes in the set A, |𝑁(𝐴)| ≥ |𝐴|, for all 𝐴 ⊆ 𝑉1 . Of course, the
neighbors of the set A will be in the subset V2 because that is the definition of your bipartite graph.
You cannot have the neighbors of the subset A within the vertex set V1 itself.
The neighbors of A will be in the other subset in your bipartition namely V2. So, what it says is
you want to find out the complete matching from V1 to V2, this is a bipartite graph and if you have
any subset A here, it should have either the same number of neighbors in V2 as the number of
nodes in A or more neighbors. It cannot have less number of neighbors in V2 and that precisely
644
was the source of problem due to which we do not have a complete matching in the graph of
organization number 2.
So, if you take this graph this was the graph for organization number 1 and in this graph we are
able to find out a complete matching from the vertex set V1 to vertex set V2. This is because you
take any subset of V1, you take either {A, B} or you take {B, C} or you take {A, B, C}, you take
any subset. That subset has either the same number of neighbors as the number of nodes that in
the subset that you are taking or more number of neighbors in the other subset in your bipartition.
But if you take this graph, graph G2 then this condition is violated. Specifically, if you take your
subset to be {requirement, implementation, testing} namely A is equal to 3. Then the cardinality
of its neighbor set is what? So, requirements neighbor are X and Z, implementation neighbors is
only X and testing neighbor is only X. So, the number of neighbors all together for your subset A
that you have chosen here is 2. So, you have three nodes to be covered or matched namely
requirement, implementation and testing. But there are only two employees namely X and Z who
can handle those three modules.
So, the only way that you can ensure that all the three modules requirement, implementation and
testing are taken here is that you assign more than one job to employee X or Z which goes against
the definition of complete matching. So, we will prove this condition or the theorem Hall’s
marriage theorem in our next lecture.
645
I conclude today's lecture. These are the references used for today's lecture. To summarize, in this
lecture we introduced the notion of matching. We saw various types of matching like maximal
matching, maximum matching and complete matching and we saw the necessary and sufficient
condition for the existence of a complete matching in a bipartite graph namely the Hall's marriage
theorem. Thank you!
646
Lecture -46
Proof of Hall’s Marriage Theorem
Hello everyone, welcome to this lecture. The plan for this lecture is as follows.
In this lecture we will see the proof of Hall’s Marriage Theorem that we have discussed in the
last lecture.
647
So, just to recap the theorem statement of Hall’s Marriage Theorem is the following. It says
that if you have a bipartite graph with bi partition (V1, V2) and if you want to find out whether
there exists a complete matching from the subset V1 to subset V2 then it is possible if and only
if |𝑁(𝐴)| ≥ |𝐴|,for any subset 𝐴 ⊆ 𝑉1. So, this condition is both necessary as well as sufficient.
So let us first prove the necessary condition that indeed this condition is necessary for the
existence of a complete matching. So, what exactly we want to prove here? We want to prove
that complete matching from V1 to V2 is possible only if this condition is true. Of course, this
means this condition has to be true ∀𝐴 ⊆ 𝑉1. So, that is implicit here. So, recall that the way
we can interpret an only if statement is the following. Now if this is the p part and if this is the
q part, then the way to interpret this only if condition is that if the condition after only if it is
not there then whatever is there before only if that would not happen.
So, the condition that q does not happen means there exist at the least some 𝐴 ⊆ 𝑉1 such that
the number of neighbours of that subset A is less than the number of nodes. If that is the case
then we have to argue that no complete matching is possible from the vertex set V1 to the vertex
set V2. That is what we want to prove. And the contrapositive of the statement is the following:
the contrapositive says that if complete matching from the vertex set 𝑉1 → 𝑉2 is there then you
take any 𝐴 ⊆ 𝑉1, the number of neighbours of that subset A should be at least as large as the
number of nodes in the subset for any 𝐴 ⊆ 𝑉1. So, that is what we want to prove here. So, this
is the final thing we will prove by proving the necessary condition so and we will give a direct
proof. We do not need any fancy thing here.
So, imagine there is a complete matching from 𝑉1 → 𝑉2 and let that complete matching be
denoted by M. So, if that is the case, we have to show that you take any 𝐴 ⊆ 𝑉1, this condition
holds that is what we have to show. So, now let us focus on the nodes in A. So, remember we
are considering the following you have the bipartition (V1, V2) and you have a subset A and
you also have a complete matching M, match with respect to which all the vertices in V1 are
matched.
That also means that all the vertices in the subset A are also matched with respect to the same
matching M. Because 𝐴 ⊆ 𝑉1, so that means every node in A must be the end point of some
distinct edge in the complete matching that you have the found from V1 to V2 and that is
648
possible only if the number of neighbours of the subset A is as large as the number of nodes in
A.
Because if at all you are able to find out if you are able to match all the vertices in A using the
collection of edges in M and as per the definition of matching, two distinct edges have distinct
end points so that automatically means that this condition is true. So, that is the proof of the
necessary condition.
Now let us prove the sufficiency condition that means we want to prove that if this condition
is ensured that means if you have a bipartition (V1, V2) and if it is ensured that you take any
𝐴 ⊆ 𝑉1 the number of neighbours, 𝑁(𝐴) ≥ |𝐴|, that is guaranteed then we have to show that
there exists a complete matching from V1 to V2. And we will give an existential proof here.
What do we mean by existential proof? We will show that if this condition is ensured then there
exist at least one complete matching from the vertex set V1 to V2 and that existential proof will
be given by induction on the cardinality of your vertex set V1, |𝑉|. So, we will first prove the
base case. So, assume that you have a bipartite graph with bi partition (V1, V2) and where there
is only one vertex in V1 and all other vertices of your graph are in the subset V2 and this
condition is ensured for your (V1, V2). If that is the case since my vertex set V1 has only one
node call it u. The only subset A possible for V1 is the subset V1 itself. Of course, we can have
the empty subset A of V1 but that is not interesting. We take 𝐴 ⊆ 𝑉1 and 𝐴 ≠ 𝜙.
649
That is possible here is the subset V1 itself A being the V1 itself and since this condition is
guaranteed that means there is at least one node the node u has at least one neighbour in V2. It
may have more than one neighbour as well that is also possible but since 𝑁(𝐴) ≥ |𝐴| and if I
take 𝐴 = 𝑉1 , the base case ensures that the node u has at least one neighbour in the subset V2.
And if that is the case then it is very trivial to find out the complete matching from V1 to V2.
The complete matching will be, just take one of the edges with u as the one of the end points
and that will be a complete matching from V1 to V2.
So, now let us go to the inductive step and for the inductive step we first assume the inductive
hypothesis. So, my inductive hypothesis is the following. I assume here that you take any
bipartite graph with bipartition (V1, V2) such that |𝑉1 | ≤ 𝑘 and if it is ensured that for any
𝐴 ⊆ 𝑉1, the number of neighbours of A is at least as large as the number of nodes in A, then a
complete matching is there from V1 to V2. That is my inductive hypothesis. I am assuming this
to be true for all bipartite graphs where |𝑉1 | ≤ 𝑘 .
650
Now I have to go to the inductive step and I have to show that assuming the base case and
assuming the inductive hypothesis to be true, I have to prove that the statement or the
sufficiency condition is true even for a bipartite graph where |𝑉1 | = 𝑘 + 1 provided this
condition is ensured in that graph. So, I consider an arbitrary bipartite graph G with bipartition
(V1, V2).
And the cardinality of |𝑉1 | = 𝑘 + 1 such that it is ensured that for any 𝐴 ⊆ 𝑉1, the number of
neighbours of the subset A is as large as the number of nodes in A that is given to me. My goal
is to show the existence of a complete matching in my graph G from the vertex set V1 to V2
that means I have to give you a matching I have to show that there exists a matching with
respect to which all the vertices of the subset V1 will be matched.
And I have to use the inductive hypothesis because right now I am considering the case when
my cardinality of V1 is k + 1. So, as a principle of inductive proof we have to somehow reduce
a graph, a bipartite graph where V1 is of cardinality k + 1 to another bipartite graph where the
bipartition has the property that the corresponding V1 has cardinality k. And then I have to use
the inductive hypothesis on that graph and show the existence of a complete matching in that
reduced graph. And based on the complete matching that I have in the reduced graph I have to
show that I can build upon that complete matching in the reduced graph and give you a
complete matching for the bigger graph G. So, that will be the proof strategy. So, for doing that
what I am going to do is I am going to exploit this condition.
651
So, I am assuming here that my graph G is such that for any subset A of the set V1 the number
of neighbours of A is as large is at least as large as the number of nodes in A. So, now there
could be two possible cases here. Case 1 is the following, your graph G is such that for every
k-sized subset of V1 that subset has at least k + 1 neighbours in the subset V2. So, here I am
focusing on the case where A is exactly equal to k.
So, my case one is you take all your graph G is such that you take any |𝐴| = 𝑘 in your V1 that
has at least k + 1 neighbours in the subset V2. So, for instance if I take k equal to say 3. So what
I am saying here is your graph G is such that you take any subset of three nodes in your V1, it
will have 4 or more number of neighbours in V2. So, for instance if you take the first 3 nodes,
it will have 4 nodes, 4 neighbours in V2 or if you take the last 3 nodes then also it has 4 or more
neighbours in V2.
Or even if you take say for instance the first node, the second node and the fourth node that
also will have 4 or more number of neighbours in V2 and so on. So, that is case 1, that means
your graph G is such that this condition is there. And my case 2 could be the following. I have
a k-sized subset of V1 which has exactly k neighbours in V2. So, pictorially you can imagine I
am talking about the case where your graph G is such that even though this condition is true,
but as part of that condition you have a subset A of k nodes in V1 which has exactly k
neighbours in V2 that is the case that does not violate this condition. This condition is still
satisfied even for that subset A because this condition says that 𝑁(𝐴) ≥ |𝐴|. So, even if it is
equal to the number of nodes in A that means the condition is satisfied.
So, my case 2 is talking about a possibility where in my graph G, I have a subset A of k nodes
which has exactly k neighbours in the subset V2. So, again for demonstration here I am taking
the case of 𝑘 = 3. So, these are the only 2 possible cases with respect to my graph G and in
both the cases I have to show the existence of a complete matching from V1 to V2 and in both
the cases I will be using the inductive hypothesis. So, let us first consider case 1.
652
And for demonstration purpose I am taking 𝑘 = 3. So, this is the case where my graph G is a
bipartite graph with bipartition (V1, V2) and |𝑉1 | = 𝑘 + 1 and this condition is ensured in my
graph G and this condition is ensured in such a way that you take every k-sized subset of V1 in
G, it has k + 1 or more number of neighbours in V2. That is the case I am considering right
now.
And my goal is to show the existence of a complete matching from V1 to V2. So, here is how I
will find the complete matching. So, you consider any vertex u from V1 you are free to use any
vertex, just for simplicity I am taking the first vertex. And remember my goal is to reduce this
graph G where |𝑉1 | = 𝑘 + 1 to another bipartite graph where the cardinality of the
corresponding V1 is k so that I can use the inductive hypothesis.
So, for that only I am considering an arbitrary vertex u in the subset V1 and I am focusing on
one of its neighbours in V2. So, for instance let it be u and its corresponding neighbour v is
there. By the way what is the guarantee that the node u that I have picked here has at least one
neighbour v in V2, well that is coming because of the base case if I consider the case where A
is equal to one.
And the subset A being the set consisting of node u, then as per the condition the number of
neighbours of u is one or more than one. So, that means at least one neighbour of u is there in
my graph and that neighbour has to be in the subset V2 because I am considering a bipartite
graph. So, out of all the neighbours of u, I am just picking some arbitrary neighbour call it v
and then what I do is I reduce my graph to a following graph.
653
I remove the node u from my graph and I remove the node v from the graph and I remove this
edge because this edge now is part of my matching. Remember my goal is to find out the
complete matching in the overall graph, so one of the edges of that complete matching is the
edge (u, v) and that will ensure that the node u is matched. Now I have to take care of ensuring
that the remaining k nodes of V1 are also somehow matched.
So, because of this reduction now I will get a new graph and that new graph will also be a
bipartite graph because my original graph was a bipartite graph and the only thing that I have
changed is I have removed the node u, I have removed the node v and I have removed the edge
between u and v and all the edges which has u as one of its end point. And all the edges which
has v as this endpoint.
So, that will ensure that my new graph which I am calling as the reduced graph is still a bipartite
graph and the corresponding bipartition of the reduced graph will be V1’, V2’. So, 𝑉1 ’ = 𝑉1 − 𝑢
and 𝑉2 ’ = 𝑉2 − 𝑣. Now what can we say about the cardinality of |𝑉1 ’|? It will be k. And what
can I say about the cardinality or the number of neighbours of V1’ that are there in V2’?
My claim here is that, the nodes in V1’ has k or more number of neighbour in V2’ in my reduced
graph. This is because in my original graph G not the reduced graph in my original graph G, if
I take the case where 𝐴 = 𝑉1 ′ i then since I am in case 1 it would have been ensured that in my
graph G, this subset A namely the subset V1’ has k + 1 or more number of neighbours in G,
because I am in case 1. One of the neighbours of V1’ could be the node v which I have deleted
and taken as part of the edge (u, v) in my complete matching which I am trying to build. But
even if I now remove the node v from the graph G in my reduced graph it will be ensured that
the number of neighbours of V1’ will be k or more than k.
Because if 𝑁(𝑉1 ’) = 𝑘 − 1 in my reduced graph, then I get the implication that in my bigger
graph namely the original graph, the subset V1’ has exactly k neighbours. But that goes against
the assumption that I am in case 1 and in case 1, I am assuming that each k-sized subset of V1
in the graph G has k + 1 or more number of neighbours. Now if the subset V1’ has at least k
number of neighbours in V2’ then I can use my inductive hypothesis.
654
And as per my inductive hypothesis if you have a bipartite graph where the cardinality of the
first set in your bipartition is exactly k and if it is ensured that, you take any subset of V1’, it
has at least as many neighbours as the number of nodes in A. Then as per my inductive
hypothesis, I know that there exists a complete matching in my reduced graph. I say I stress
here in the reduced graph which will ensure that all the vertices in V1’ are matched.
That means it will be a complete matching from V1’ to V2’. Now take that complete matching
and to that complete matching add the edge (u, v) and that will give you now a complete
matching in the original graph G matching or ensuring that all the vertices of V1 are matched.
So, it will be a complete matching from V1 to V2. And why this is a valid matching because in
the matching M that you are finding in the reduced graph none of the edges will have the node
u or the node v as its end point.
Because the node u and node v or none of the edges incident with u or v are present in your
reduced graph. Because they were present in your original graph and you have removed the
node u, node v and all the associated edges and got the reduced graph and your matching M is
in the reduced graph and if in that matching you add this edge (u, v) that will ensure that your
original V1 which also had the node u. So, it is completely covered or it is ensured that all the
nodes in V1 are matched with respect to this bigger match. So, that is the proof for case 1.
Whereas now let us focus on case 2 and the case 2 is slightly subtle here because here we are
in the case where we are assuming that there is some k-sized subset of V1 which has exactly k
neighbours in V2 and in this case, we cannot run the argument that we used for case 1. In case
655
1, what we did is we arbitrarily picked some node in V1 and matched it by taking one of the
edges incident with that node.
And argued that even if I remove u from my graph the remaining V1 namely V1’ it will be
ensured that it has k or more number of neighbours in the reduced graph but that would not
happen because of this specific case. It might be the possible it might be possible that the node
u is part of a k size subset of V1 which has exactly k neighbours in V2. So, when you are
removing the, when you are removing the edge (u, v) from the graph and getting the reduced
graph.
Then that k-sized subset may be will be now reduced to k - 1 size subset and now in the k - 1
size subset you may not have sufficient number of neighbours in the corresponding V2’ and
you cannot run the and you cannot use the inductive hypothesis. So, you will get stuck here so
we have to handle this case in a careful fashion and still show the guarantee the existence of a
complete matching from V1 to V2.
So, what I do here is the following. Since there is at least one k-sized subset of V1 which has
exactly k neighbours in V2, I focus on that subset call it is there might be multiple such subsets
in V1. I take any one of them so take the subset |𝑆| = 𝑘 and focus on its neighbour set T, such
that |𝑆| = |𝑇| = 𝑘. So, 𝑆 ⊆ 𝑉1 and 𝑇 ⊆ 𝑉2.
So, for instance this is your set S this is your set T. Now since |𝑆| = 𝑘 , I can use my inductive
hypothesis and since the number of neighbours of S is as large as the number of nodes in S
from inductive hypothesis a complete matching is there. So, call that complete matching as M.
Now this is a complete matching from S to T not from V1 to V2. So, there is still one node left
which is not yet matched.
Because that is not part of this matching M, so now my reduced graph will be the following. I
remove the set of nodes from in S from V1 and I remove the set of nodes in T from V2 and get
the corresponding V1’ and V2’. So, V2’ may have more than one nodes as well but here for
simplicity I am left with a graph which has one node in V1’ and one node in V2’.
656
So, |𝑉1 ’| = 1, because my V1 had k + 1 nodes and I removed a subset of k node so I am left
with only 1 node and my claim is that V1’ has still at least 1 neighbour in your reduced V2
namely in V2’. If this is not the case then what it ensures the following: it ensures that in your
original graph G, the set V1 had exactly k neighbours and remember the set V1 is nothing but
this leftover node so your V1 is nothing but your 𝑉1 ’ ∪ 𝑆. So, my claim is that if in if this node
which is left in V1’ it has no neighbour in V2’ then your original graph G the subset V1 had
exactly k neighbours. And |𝑉1 | = 𝑘 + 1 remember because S is of size k and you are left with
one node in V1’ so overall V1 had k + 1 nodes.
So, I get the implication that V1 had exactly k neighbours that means there is an A where the
number of neighbours of A is less than the number of nodes in A but that is violation of this
condition it is guaranteed that you take any 𝐴 ⊆ 𝑉1 in your graph G. The number of neighbours
is as large as the number of nodes in A. So, that means this will give you a false conclusion.
If V1’ if the single node in V1’ has no neighbour left in V2’, then that gives me an implication
that in the original graph G if I take the set A to be the subset V1 itself then it has only k
neighbours namely less number of neighbours. But that goes against my assumption that in my
graph G this condition is true for every subset A.
So, from my inductive hypothesis, I know that there is now a complete matching M prime also
from V1’ to V2’ this is basically coming from the base case not from the inductive hypothesis
because I can trigger the base case as my cardinality of V1’ is 1. So, my from my base case I
know that since the number of neighbours of V1’ is as large as the number of nodes in V1’.
And V1’ is of size 1, I can use the base case and argue that there is some complete matching
M’ which ensures that all the vertices of V1’ are matched or that matching M’ is a complete
matching from V1’ to V2’. Now if I take the union of the matching M from the subset S to the
subset T and the matching M’ which is a complete matching from V1’ to V2’, that will ensure
that now I have a complete matching from V1 to V2.
So, that proves the sufficiency of the condition even for case 2. So, it does not matter whether
I am in case 1 or in case 2, in both the cases if this condition is ensured that means, you take
any 𝐴 ⊆ 𝑉1, the number of neighbours 𝑁(𝐴) is as large as the number of nodes in A then there
always exist a complete matching from the subset V1 to subset V2.
657
So, that brings me to the end of this lecture. These are the references for today's lecture. To
summarize, in this lecture we discussed the proof of Hall's Marriage Theorem. We showed the
necessary proof of this we prove the necessity condition as well as we give an existential proof
for the sufficiency condition, thank you!
658
Lecture -47
Various Operations on Graphs
Hello everyone, welcome to this lecture. The plan for this lecture is as follows. In this lecture we
will discuss.
Various operations that we can perform on graphs. We will see various mechanisms of
representing graphs. We will discuss the graph isomorphism problem and we will define the
connectivity in a graph.
659
So, it turns out that since graph is nothing but a collection of two sets we can perform various set
theoretic operations on a graph and obtain new graphs. So, let us discuss some of the important
operations which we can perform on the graphs. So, we will first define what we call as the
subgraph of a graph, if you are given a graph 𝐺 = (𝑉, 𝐸) with the vertex set being V and edge set
being E. Then, a graph 𝐻 = (𝑊, 𝐹) with vertex set W and edge set F will be called as a subgraph
of G, if the vertex set W of H is a subset of the vertex set V of G, namely all the vertices of H
should be the vertices of G and the edge set F of H should be an edge set, there should be a subset
of the edge set E of G, that means all the edges of F should be present in. So, that is a simple
straight forward definition of a subgraph. Now let us define what we call a proper subgraph of a
graph.
So, we will first give an intuitive definition what exactly we can think of a proper subgraph and
then we will see that definition is not correct. So, remember when we define the proper subset of
a set we say that 𝐴 ⊂ 𝐵, if A is a subset of B and there is something extra which is always there in
B which is not there in A. So, let us try to extend that definition in the context of a proper subgraph.
So, say my definition is that H graph H will be called as a proper subgraph of G if either the vertex
set of H is a proper subgraph,𝑊 ⊂ 𝑉 it is a proper subset of the vertex set of G and the edge set of
H, 𝐹 ⊂ 𝐸it is a proper subset of the edge set of G, suppose that is my definition. But this is my
definition, then with respect to this definition if I take my graph G and H to be this then H will not
660
be considered as a proper subset of G. This is because all the vertices of H are the vertices of G as
well, so this conditions that 𝑊 ⊂ 𝑉 is not satisfied. So, as per this definition, I will say that H is
not a proper subgraph of G but this is a proper subgraph of G. Because there is something extra in
G which is not there in H, namely the edge between the vertex b and c is there in G but that is not
there in the graph H.
So, that means the definition that I gave here is not the correct definition, so that is why the right
definition of the proper subgraph is the following. I will say that H is a proper subgraph of graph
G if it is a subgraph of G because that is definitely the requirement. And it is a subgraph which is
different from the graph G or the parent graph. If that is the case then I will say that my graph H
is a proper subgraph of the graph G. So this condition that G is not equal to H takes care of the fact
that there is something extra in the graph G which is not there in the graph H.
Now let us next define what we call as induced subgraph. So, if you are given a graph 𝐺 = (𝑉, 𝐸)
with vertex set V and edge set E and if I take a subset of vertices W, then G’ is called the induced
subgraph or the induced by the vertex set W such that the vertex set of G’ is W and the edge set
E’ of G’ consists of only those edges whose both the end points are within the subset W.
So, basically what this induced subgraph tries to do is the following. You are given a collection of
vertices W, that W could be empty or it could be the entire set of vertices V. So, you are given
661
some subset W and you are focusing only on that part of the graph G where all the edges have
endpoints within the subset W only. Even if there are all the edges where one of the end points is
outside the subset W are not focusing on those edges.
So, that is the definition of the induced subgraph, so its vertex set will be W and the edge set will
be only those edges, this should be (vi, vj), not (ei, ej), such that both the end points are members
of the subset W. So, for instance if this is my graph G and if I take my 𝑊 = {𝑏, 𝑐}, then I am
focusing only on that part of the graph G where the edges have their end points restricted within
the subset W.
So, that means I cannot take this edge, this edge is not allowed because one of the end points is a
and that a is not within my subset W. Similarly, I cannot take the edge between a and c because
one of the end points of this edge is the set a which is not there in my W. Whereas if I take my W
to be the node a only, then I get an empty graph. Empty graph in the sense which has no edges
because this edge will not be there as b is not within my W, this edge will not be there because the
node c is not within my W and this edge also will not be there.
So, now let us see some set theoretic operations that we can perform on an existing graph to get
new graphs, so imagine you are given a graph then the deletion of an edge is denoted by this
operation. So, imagine a small e is an edge, so if I delete an edge then the vertex set does not get
662
disturbed, it remains the same it is only the edge set which gets affected. That means my new edge
set will be the old edge set minus the edge e which I am excluding.
Whereas if I am removing a collection of edges, even in that case my vertex set remains intact, it
is only the edge set which gets affected, that means my new edge set will be the difference of the
old edge set and the edge set E’ which I am deleting from the graphs. So, it is like saying the
following; imagine your graph represents a computer network where the nodes are the computers
and the edge represents a cable connecting two computers. So, if you remove a cable that does not
mean that the corresponding computers also get deleted, the computers are still there, it is only the
cables which are getting removed. Similarly, if I add a new edge that is expressed by this operation
then my edge set gets affected, so I will be including a new edge, so I will be including a new edge
and the end points of the edge e will be included in my vertex set.
On the other hand, if I delete a vertex from my graph G, then definitely the vertex that gets affected
and also the edge set gets affected. So, I have to remove all the edges whose one of the endpoints
is v from my graph, it is like saying the following: again, if I take the fact that my graph represents
a computer network then deleting a vertex is equivalent to saying that I am deleting or removing a
computer itself from my network. So, if I remove a computer from the network then whichever
cable has one of its end points as that computer, those edges will not be there anymore in my
computer network, so I have to modify my edge set as well. Whereas if I remove a set of vertices
V’ then my new vertex set will be the difference of the old vertex set and the vertex of V’ and edge
set E’ will be the following.
I have to remove all the edges where one of the end points of the edge is in my subset V’. So, let
me demonstrate these operations with respect to this graph, so imagine this is my graph G, if I
remove the vertex a, then this edge e1 and edge e3 will no longer be there and I will get this reduced
graph, whereas if I remove the cable connecting the node a and node b or the computer a and
computer b then only the edge e1 vanishes, the vertex set remains the same.
663
Now let us discuss the various data structures that we can use to represent graphs and people who
have studied data structure, they must be knowing two of the common representations that we use
to represent graphs. So, again I am explaining in the context of undirected graphs, but you can
easily generalize this data structure to represent directed graphs as well, so we have what we call
as the adjacency matrix representation, so this is a boolean matrix.
So, if your graph G is consisting of vertex set V and edge set E and if the cardinality of the vertex
set is n, then this matrix is an 𝑛𝑋𝑛 boolean matrix and the (𝑖, 𝑗)𝑡ℎ 𝑒𝑛𝑡𝑟𝑦 = 1 𝑖𝑓𝑓 (𝑣𝑖 , 𝑣𝑗 ) ∈ 𝐸,
otherwise it will be 0. And this representation is preferred for dense graph. What do we mean by
dense graph? A graph which has a lot of edges, that means it is not the case that you have very few
edges, that means you have lots of edges in the graph in which case a majority of the entries in
your matrix will be 1.
Whereas a sparse graph is the graph which has lots of vertices but very few edges, for such a graph
the adjacency list is the preferred data structure for representation of the graph. So, what is this
adjacency list? So, it is a collection of linked list, each link is basically a collection of n linked
lists, so you will have the first linked list with v1 as the starting node, second linked list as with v2
as the starting node and nth linked list with vn as the starting node and in v1 you will now link all
the nodes which are incident.
664
You link all the nodes which are the endpoints of an edge with v1 as its one of the end points. That
means in your graph G, you focus on the edges which have v1 as one of the end points, so for those
edges find out the other end points and you put them in the linked list with v1 as the end point. So,
basically the linked list starting with v1 list down all the neighbors of v1, similarly the linked list
with v2 list down all the neighbors of v2 and so on.
So, now you can see why this representation is very popular for representing sparse graph, that
means if graphs which have only a very few edges, if you have very few edges then the size of
each linked list will be very small. You do not need a huge matrix with lots of entries being zeros
and only few entries being one, so these are the two popular representations for representing an
undirected graph, they can be used even for representing directed graphs as well and there is this
third data structure which is called as the incidence matrix.
So, what is this incidence matrix? It is so again I am explaining, assuming an undirected graph, so
this will be a matrix with |𝑉|𝑋 |𝐸|. And it basically represents the relationship between the edge
and its endpoint. So, what it means is the following if you have an edge 𝑒 = (𝑣𝑖 , 𝑣𝑗 ) w then the
entry number (𝑣𝑖 , 𝑒) = 1 and (𝑣𝑗 , 𝑒) = 1, otherwise and the remaining entries will be 0.
So, for instance if this is my graph G, so there are 3 vertices, so three rows and there are three
edges, so three columns. What are the endpoints of e1? The endpoints of e1 are a and b, so under
the column e1, I only mark the entry for a row and b row as 1 to show that the end points of the
edge e1 are a and b and all other column entries under e1 will be 0. Similarly, the end points of e2
are b and c, so only the entry in row number b and column number e2 will be 1. And entry in row
number c and column number e2 will be 1 and all other entries in column number e2 will be 0 and
so on, so that is the incidence matrix.
665
Now let us define a graph isomorphism. So, if you see these two graphs, pictorially they are drawn
in a different way. So, the first graph is a rectangle whereas the other graph does not look like a
rectangle graph because you might be saying that there are two edges which are crossing each
other. But if you see closely structurally, they are similar graphs. What do I mean by structurally
they are similar graphs, so both the graphs have the same number of nodes namely 4, same number
of edges namely 4 edges and even though the vertex names are different in the two graphs.
So, if I call this as graph G1 and this as graph G2, the name of the vertices of G1 are {𝑎, 𝑏, 𝑐, 𝑑}
whereas the name of the vertices of G2 are {𝑣1 , 𝑣2 , 𝑣3 , 𝑣4 }, so you will say that how can they be the
same graph because the vertex names are different. If I do not focus on the name of the vertices
and the name of the edges but mentally think in my mind whether there exists a one-to-one
correspondence between the vertices of the graph and the edges, then I find that the 2 graphs are
structurally same. What does that mean?
If I think in my mind that vertex a of G1 corresponds to vertex v1 of graph G2 and if I consider

vertex b of graph G1 corresponding to vertex v3 of G2, then you can see that there exists an edge
between a and b in graph G1 and similarly I can consider this edge as the edge e1 between the
mapping of a and the mapping of b. Similarly, this node c, I can associate with the node v2 here,
and you can see that if that is the case then the edge e2 in the graph G1 corresponds to the edge e2
of this graph G2 and so on.
666
So, in that sense they are structurally the same graphs but drawn in a different fashion. In the same
way this graph G1 and this graph G2 are structurally same, so the node 1 of graph G1 corresponds
to the node a of graph G2 and so on and you can then verify that structurally if I reinterpret graph
G2, then I can redraw it in the same way as the graph G1. So, in that sense these pairs of graphs are
isomorphic, they are structurally drawn differently, they are drawn differently but structurally they
represent the same information.
That means I can always redraw one of the graphs as another graph. So how do I formally define
whether two graphs are isomorphic or not? So, imagine you are given two graphs G1 and G2, they
are called as isomorphic and isomorphic graphs are represented by this notation. So, I will say that
the two graphs are isomorphic, if I can define a bijective mapping between the vertex set V1 and
the vertex at V2 such that the following holds.
If you have an edge between the node u and v in the first graph, then you focus on the mapped u
vertex in G2 and the mapped v vertex in G2 and there should be an edge between the mapped u
vertex and the mapped v vertex in E2 as well and this implication is bi-implication, that means
other way around should also hold. So, if you can find one such mapping, one such bijection
between the vertex sets, then we will say that two graphs are isomorphic and isomorphism is
denoted by the bijection, 𝜋: 𝑉1 → 𝑉2.
So, when we say that show me an isomorphism between graph G1 and G2 basically I am asking
you to show the one-to-one correspondence between the two vertex sets but namely the vertex set
of G1 and the vertex set of G2 such that this bi-implication is true. So, the graph isomorphism
problem is the following: you are given two graphs G1 and G2 and you have to check whether they
are isomorphic or not.
Checking whether they are isomorphic or not is equivalent to checking whether there exists a
bijection between the vertex sets of the two graphs such that bi-implication is true. So, what will
be the naive algorithm to check whether two given graphs are isomorphic or not? If |𝑉1 | = |𝑉2 | =
667
𝑛 ,the cardinality of the vertex sets of the two graphs is n. By the way a simple necessary condition
for the two graphs being isomorphic is that they should have the same number of vertices.
If one graph has more number of vertices than another how at the first place they can be
isomorphic, they can never be isomorphic. So, imagine that the vertex set of both the graphs is of
cardinality n, then a naive algorithm to check whether the two graphs are isomorphic is to try all
possible 𝑛! bijection between the vertex at V1 and the vertex at V2 and for each of those bijections
check this implication is true or not.
So, indeed if the two graphs are isomorphic one of these 𝑛! bijections will satisfy the bi-implication
and hence you can declare that the two graphs are isomorphic. But if all the 𝑛! bijections fail to
satisfy this bi-implication, we will say that the two graphs are not isomorphic. But then what is the
running time of this algorithm? You have to try 𝑛! bijections and 𝑛! is an enormously large
quantity.
So, that is why this naive algorithm will work only for the small values of n and it is still a big
open problem or to come up with efficient algorithms or feasible algorithms to check whether two
graphs are isomorphic or not.
668
So, how do we verify whether two graphs are not isomorphic or not and we can verify whether
two graphs are not isomorphic or not by checking for graph invariant properties. So, what are graph
invariant properties? They are the properties which should be preserved by isomorphic graphs, that
means these are the properties which should be there both in graph G1 as well as in graph G2 if at
all they are isomorphic, that means if any of these properties is violated then you can declare that
the two graphs are not isomorphic.
So, some of the naive graph invariant properties which should be preserved by the isomorphic
graphs are the following: they should have the same number of vertices, same number of edges
number of vertices of a particular degree should be the same in both the graphs and so on. So, for
instance if in your graph G1, there are two vertices of degree 2, then in G2 also there should be
exactly two vertices of degree 2.
Otherwise, the two graphs can never be isomorphic because you would like to associate a vertex
of degree 2 in G1 with another vertex of degree 2 in G2 and vice versa. However, it turns out that
if any of these graph invariant properties is violated and you can immediately declare that the two
graphs are not isomorphic. However, it turns out that the graph invariant properties are the only
necessary condition for the existence of isomorphic graphs.
And we do not have an exhaustive list of graph invariant properties, we do not know that all these
properties or you do not have a list of properties such that if those properties are preserved both in
graph G1 and G2, then you can declare that the two graphs are isomorphic. Unfortunately, we do
not have such graph invariant properties. So, for instance if I take these two graphs G1 and G2,
then it is slightly difficult to identify a graph invariant property which is present in G1 but not
present in G2.
In fact the graph G1 and G2 here are not isomorphic and we have to identify here a graph invariant
property which is present in one graph but not present in the other graph. So, if you see closely
here, if at all graph G1 is isomorphic to graph G2, then I need to associate the vertex a of G1 with
some vertex in G2 and that association or the vertex which could be associated with vertex a could
be either the vertex t or the vertex u or the vertex x or the vertex y.
669
Why so? Because the degree of a here is 2, so it can be associated only with a vertex of degree 2
in G2 and the only vertices of degree 2 in G2 are t, u, x and y. You cannot associate vertex a with
vertex s because a has degree 2 where s has degree 3 and so on. But it turns out that in graph G 2
the vertex t, the vertex u, the vertex x, the vertex y all of them are adjacent to another vertex of
degree 2. So, for instance; if you take t, t is adjacent to u and u has degree 2.
If you take u, u is adjacent to t and t has degree 2. If you take x, x is adjacent to y which has degree
2 and similarly y is adjacent to x which has degree 2. But if you see graph G1, all the neighbors of
the node a have degree 3, so b has degree 3 and t has degree 3. So, that means this is now a graph
invariant property which is present in G2 but not present in G1, the graph invariant property is that
all the vertices of degree 2 in G2, they have a neighbour which also have degree 2.
But that property is not satisfied in G1, namely I have a vertex of degree 2, namely the vertex a
and none of its neighbours has degree 2. So, that shows that I can never find out isomorphism or
a bijection between the vertex set of G1 and the vertex set of G2 because I cannot find a vertex
corresponding to the vertex a in the graph G2.
Now let us next define the connectivity in a graph, so for that we first recall the definition of a path
of length n between the node u and the node v in an undirected graph, where n is a non-negative
670
integer. So, a path of length n between the node u and the node v is a sequence of n edges in the
graph and say 𝑒1 = (𝑥0 , 𝑥1 ), 𝑒2 = (𝑥1 , 𝑥2 ), …., 𝑒𝑖 = (𝑥𝑖−1 , 𝑥𝑖 ), … , 𝑒𝑛 = (𝑥𝑛−1 , 𝑥𝑛 ) such that the
starting vertex or the first end point of the edge e1 is u and xn is equal to v. If that is the case then
I say that I have a path of length n between the node u and the node v. So, remember there is no
restriction on whether the edges in this sequence of n edges are distinct or they are allowed to be
repeated or not, no such restriction is there when I consider a path of length n.
A path is also called as a walk from u to v, because we are just traversing edges and going from
the node u to node v, if the starting vertex u and the end vertex v of the path are same, then the
path is also called as a circuit or a closed walk and if all the edges in my path between u to v are
distinct, (𝑒1 ≠ 𝑒2 ≠ ⋯ ≠ 𝑒𝑛 ). That means no edges are repeated then I call the path as a simple path
and if I have a circuit where the starting vertex and the end vertex are the same and if all the edges
are distinct, then the circuit is called a simple circuit.
By the way, in a simple path the vertices are allowed to be repeated, it is only the edges which are
not allowed to be repeated.
So, once we have the definition of a path, let us now define what we call connected graphs and
components, so an undirected graph is called connected if there exists a path between every pair
of distinct vertices, I stress here between distinct vertices, I am not interested to check whether
671
there exists a path from a node to itself, but between every pair of distinct vertices there should
exist at one path, there could be multiple paths as well but at least one path should be there between
every pair of distinct vertices.
Now what is the connected component of a graph? A connected component of a graph is the
maximal connected subgraph of the graph. What does the maximal connected subgraph means,
well it is a connected subgraph of the graph G and it is maximal in the sense that it cannot be
further extended. What does that mean? What does it mean that I cannot further extend that
connected subgraph, well that means that you cannot have another subgraph of the graph G which
is also connected such that the connected subgraph is a proper subgraph of that connected
subgraph.
That means, it is maximal in the sense that it is not a proper subgraph of any other connected
subgraph of the G. So, what does this mean? Imagine I take this graph, then can I call this as the
connected component and can I call this graph as the connected component. So, if I take this
triangle involving d, c and e, it is not a connected component because even though it is a connected
subgraph of your graph G, it is a proper subgraph of this connected subgraph of the G, so that is
why this is not maximally connected.
So, that is why the connected component of this graph is the original graph itself, because the
whole graph itself is connected at the first point, so if my graph G is connected at the first point
then the only connected component of the graph will be G itself. Whereas if my graph G is
disconnected, then the connected component will be the collection of maximal connected
subgraphs.
672
Now let us next define cut vertex and cut edge. So, cut vertex are also called as articulation point
or critical vertices, what does it mean when I say a vertex is critical, so if I take this graph G, the
node c is very critical here, because if I delete this node c then it will disconnect the entire graph.
So, in that sense it is a critical vertex or articulation point, so my definition here is the following:
I will say a vertex v in a graph G is a cut vertex and this cut vertex is defined with respect to a
connected graph.
So, this vertex v will be called as a cut vertex if deleting the vertex will disconnect the graph or
equivalently the number of connected components of the graph 𝐺 − 𝑣 is at least one more than
that of G, because if the number of connected components increases for the graph 𝐺 − 𝑣, then
that is possible only if my graph 𝐺 − 𝑣 becomes disconnected. Because I started with a connected
graph and even after deleting the vertex v, my new graph still has only one connected component
namely it is still connected. That means the vertex v is not a cut vertex, that means deleting the
vertex v does not disconnect the graph. Similarly, I can define a critical edge which is also called
as a bridge or cut edge. So again, if I take the same graph as above and focus on this edge c and
that connecting the node c and f, then the edge connecting the node c and f is very critical because
if I delete that edge then the whole graph gets disconnected.
So, that gives me the definition of a cut edge, I will say an edge is a cut edge for a connected graph
if deleting that edge disconnects the graph which is equivalent to saying that the number of
673
connected components of the reduced graph, namely the graph obtained after removing the edge e
is at least one more than the number of connected components of G. So, the number of connected
components of G was 1 because it was a connected graph, but now in the new graph namely in the
graph G - e, the number of connected components is 2 or more than 2. That means my graph got
disconnected because of deleting the edge e and hence I will call the edge e as a critical edge or a
bridge.
So, that brings me to the end of this lecture, these are the references. Just to summarize, in this
lecture we discussed various set theoretic operations on the graph. We discussed the various data
structures which we can use to represent graphs and we discussed the cut vertex and the cut edge.
Thank you!
674
Lecture -48
Vertex and Edge Connectivity
Hello, welcome to this lecture. In this lecture we will continue about discussion on our vertex
connectivity, cut vertices, cut edges and we will introduce vertex cut and vertex connectivity, edge
cut and edge connectivity. And we will formally prove the relationship between the vertex
connectivity and edge connectivity.
675
So, let us start with the definition of a vertex cut. It is also called as a separating set. So, imagine
you are given a graph 𝐺 = (𝑉, 𝐸). Then a proper subset 𝑉’ ⊆ 𝑉 of the set of vertices is called the
vertex cut if removing the vertices in V’ disconnect your graph. So, remember if your graph has
an articulation point then that articulation point itself can constitute a V’ of potential V’ whose
deletion will disconnect the graph.
But it might be possible that your graph may not have an articulation point in which case you may
need to delete more than one vertex in the graph to disconnect it. So, the basic idea here is we are
now try to generalize the definition of articulation point in terms of a subset of vertices V’. So, if
I take this graph and if I remove the nodes c and node f then this node g will get disconnected from
the network, because these edges will also go away.
Whereas I can remove the vertices c, b and a that will ensure that d and a separate out from current
diagram. So, here as of now the there is no criteria on the cardinality of V’, |V’|. I am just checking
with a V’ constitutes a vertex cut or not whether deleting the vertices in V’ disconnects the graph
or not. So, now the question is, can I say that every connected graph which has nodes as a cut
vertex and answer is yes, except when the graph is a complete graph.
So, if you take a complete graph of a n nodes even if you remove up to n - 1 nodes your graphs
still remains connected, the reduced graph. So, remember my 𝑉’ ⊆ 𝑉. You cannot say that you can
676
delete the entire graph itself because if you delete the entire set of vertices the entire graph
vanishes. So, in a complete graph at max, you can delete up to n - 1 nodes with the hope to get a
disconnected graph, but that is not possible.
So, now let us next define vertex connectivity of a graph. And the vertex connectivity of a graph
is also denoted by this parameter kappa, 𝜅(𝐺). So, the vertex connectivity of a graph is the size of
the smallest vertex cut. That means the size of the smallest V’ whose deletion will disconnect your
graph or equivalently it is the minimum number of vertices to be deleted to disconnect your graph.
So, consider this graph and for this graph your 𝜅(𝐺) is equal to 2. When your 𝜅(𝐺) will be 1 if
your graph has an articulation point. But if there is no articulation point and you may need to delete
more than one vertex to disconnect your graph and in this graph we do not have any articulation
point. So, we need to delete at least two nodes to disconnect the graph namely the nodes c and f.
Whereas if I take this graph G, then my 𝜅(𝐺) here will be 0, because my graph is already
disconnected, and I do not need to delete any additional node to further disconnect it. Whereas if
I take this graph, then the definition that I have given for vertex connectivity does not make sense
here. Because even if I delete up to n - 1 nodes here namely 2 nodes if I delete say a and b, I will
be left with a connected graph. So, that is why to take care of this special case of complete graph,
I slightly change my definition of vertex connectivity and my definition of vertex connectivity is
677
the following. It is now the minimum number of vertices which needs to be deleted to either
disconnect the graph or produce a graph with a single node. This later condition is to take care of
the complete graph. So, with respect to this new definition the 𝜅(𝐺) or the vertex connectivity for
this triangle graph will be 2.
So, it turns out and it is easy to verify that the vertex connectivity of a graph will always be in the
range 0 to n - 1, where n is the number of nodes in your graph. Because if your graph is already
disconnected then you do not need to delete any vertex. Your vertex connectivity will be 0 whereas
if your graph is a complete graph, then you need to delete up to n - 1 nodes to produce a graph
with a single node. Now, my graph will be called as k-connected, if the vertex connectivity of the
graph is at least k. That means the size of this smallest vertex cut is k.
Now we can keep similar theory with respect to a collection of edges whose removal will
disconnect the graph. So, we define what we call as an edge cut. So, imagine you are given a graph
and a collection of edges E’ will be called an edge cut, if deleting the edges in E’ from the graph
G disconnects your graph. So, there might be several E’s possible. Now when I give the definition
of edge cut, I am not focusing on the minimum sized E’.
I will be just given an E’ and I have to check whether deleting the edges E’ deletes or disconnects
my graph or not. So, for instance if I take this graph, if I remove the edge between c and g, c and
678
f and f and g then I get a disconnected graph because the node g now gets disconnected from the
rest of the network. Similarly, if I remove the node edges between a and d, d and e and d and c,
then I think this should not be if I remove the edges, it constitutes an edge cut. Because now the
node d gets disconnected from the entire graph. So, again similar to the question that we asked for
the vertex cut, let us answer this question whether every connected graph which has nodes as an
edge cut or not. Again, the answer is yes, except for the case when your graph is already a graph
with just a single node and no edges. Because if you have a graph with a single node and no edges
then you do not have any edge to delete at the first place.
Otherwise, you always have a set of edges in a connected graph which you can delete to disconnect
it. So, we now define what we call as the edge connectivity of a graph and this is denoted by λ.
So, what is the edge connectivity of a graph? It is the size of the smallest edge cut or equivalently
the minimum number of edges to be deleted which disconnects graph. So, if I take this graph then
the edge connectivity is 2 because I need to delete two edges to disconnect the graph.
If your graph was a bridge or cut edge then λ will be 1. But if your graph does not have a bridge
then you need to delete more than one edges to delete the graph to disconnect the graph. If you
take this disconnected graph, then edge connectivity will be 0, because it is already disconnected,
and I do not need to delete any edge to disconnect it. But now if I take this graph, which has no
edges and just a single node then this definition does not make sense here.
679
Because I cannot delete any edge in this graph to let make it disconnected graph. So, to take care
of the special case, I modify my definition and my modified definition of edge connectivity is the
following. I define edge connectivity to be the minimum number of edges which needs to be
deleted to either disconnect the graph or produce a graph with single node. This latter condition is
added for taking care of this special case.
Because if I apply this definition to the special case then I get λ equal to 0. Because my graph is
already a graph with a single node and I need to delete 0 number of edges to produce a graph with
a signal node. Again, it is easy to verify that your edge connectivity will be in the range 0 to n -
1, 0 for the case when your graph is already disconnected and or for this particular case when your
graph is just consisting of a single node. And n - 1 for the case when your graph is the complete
graph. If your graph is a complete graph, then you just delete n - 1 edges incident with any node
then that node gets disconnected from the rest of the network.
So, now we want to fix some upper bounds on the vertex connectivity and edge connectivity. So,
let us first prove an upper bound on the vertex connectivity. So, my claim here is that for any
connected, non-complete graph, I stress connected not complete graph the vertex connectivity is
always less than equal to the minimum degree that is possible in your graph. That means you take
680
the vertex v which has the least degree in the graph say the degree is d, then my claim is that 𝜅(𝐺)
is always less than equal to d.
So, a simple proof for this fact is the following. So, consider this graph or consider any arbitrary
connected non-complete graph and focus on the vertex v, which has the least degree, namely the
degree of the vertex v is d. That means the node v has d number of neighbours. Now what happens
in this graph if all the neighbors of the vertex v are deleted?
So, for instance in this graph the vertex c has the, Sorry the vertex g has the least degree. When
the vertex e also has the same degree as vertex g so what I am saying is if you remove all the
neighbors of the vertex which has the least degree then that vertex v gets disconnected from the
rest of the graph. So, that shows that you do not need to delete more than d number of nodes in the
graph to produce a disconnected graph.
The maximum number of nodes that you need to delete is d. But then now you might be wondering
that as per the argument that I have given why it is less than equal to, why not exactly equal to.
You might argue that I definitely need to delete exactly d number of nodes to disconnect the graph
or to disconnect the node v from the rest of the graph. Well that is not necessarily true if you
consider this graph then here the minimum degree is equal to 2.
Because you have the node g which has degree 2, you have a node f which has degree 2, you have
node d which has degree 2, you have node e which has degree 2 and so on. So, you might argue
that definitely I need to delete at least 2 nodes to disconnect the graph. Of course, if you can delete
all the neighbors of d or all the neighbors of e or all the neighbours of f or all the neighbours of g
you get a disconnected graph. But you do not need to do that much.
681
Because if you just delete the node c your graph gets disconnected. So, the vertex connectivity
comes by here is 1 which is strictly less than the minimum degree in the graph. Now let us put, let
us derive some upper bound on the edge connectivity and again I take the case of a connected,
non-complete graph and upper bound here remains the same. My claim is that the edge
connectivity of a graph is always upper bounded by the minimum degree that is possible in your
graph.
So, the proof is again very simple, let v be the vertex with least degree, of course, you can have
multiple vertices with the same least degree focus on one of the vertices v. And then argue that
what happens if all the edges incident with the vertex v are deleted. So, for instance in this graph,
g is the vertex v which has the least degree. So, what I am arguing here is that if you delete all the
edges which are incident with this vertex g.
The vertex g gets disconnected from the rest of the graph. So, that shows you do not need to delete
more than least degree number of edges in your graph to produce a disconnected graph. But then
again you get the same question that as per this argument one may get the feeling that λ should be
exactly equal to the minimum degree in the graph, why less than equal to? Again, consider this
graph here the minimum degree is 2.
682
So, the vertex f has degree 2 the vertex g also has degree 2. So, that means to remove these two
edges the node f gets disconnected and in the same way if you remove these two edges the node g
gets disconnected and so on. So, you might say that I definitely need to delete two edges to produce
a disconnected graph from this connected graph. The answer is no. Because if you remove this
edge which is a bridge in this overall graph your graph gets disconnected. So, the 𝜆 here is 1 not
2.
So, now we want to establish a relationship between the vertex connectivity and edge connectivity.
And there is a very nice theorem statement which says that you take any connected, non-complete
graph then the vertex connectivity is always less than equal to the edge connectivity. So, let us
prove this. The proof is very simple, so imagine you are given a connected, non-complete graph.
Why I am taking non-complete? Because for a complete graph this inequality is always true, both
𝜅(𝐺) as well as 𝜆(𝐺) are n - 1.
So, that is why I am taking the case of a non-complete graph and connected graph. Because again
if I take the disconnected graph, both 𝜅(𝐺) and 𝜆(𝐺) are zero and inequality holds. So, I take a
connected, non-complete graph G, which is an arbitrary graph and imagine that E’ constitutes an
edge cut for G, the minimum edge cut for G which has 𝜆(𝐺) number of edges. That means if I
remove the edges 𝑒1 , 𝑒2 , … , 𝑒𝜆 from the graph, my graph gets disconnected.
683
And say the end points of the ith edge ei is (ui, vi). So, these are the end points of the edge ei. So
you can imagine that if I remove the edges 𝑒1 , 𝑒2 , … , 𝑒𝜆 from my graph G, I get 2 connected
components, component C1 and component C2. So, you can imagine the structure of your graph is
something as follows. You have the vertices images structured in such a way that you can interpret
𝑒1 , 𝑒2 , … , 𝑒𝜆 as kind of bridges between component C1 and C2.
So, that is one of the endpoints e1 is in component C1 and other endpoint namely v1 is in C2 and so
on. By the way here, it is not necessary that all the nodes u1 to 𝑢𝜆 or v1 to 𝑣𝜆 are distinct. It might
be possible that you have just u1 in component C1 and all the edges 𝑒1 , 𝑒2 , … , 𝑒𝜆 has u1 as one of
its endpoints. That is also possible but similarly you might have a case where the edges are such
that you only have v1 as the only node in C2 and all this edges 𝑒1 , 𝑒2 , … , 𝑒𝜆 as v1 is one of its end
point.
But it is not necessary that we want to 𝑢𝜆 and we want to 𝑣𝜆 are all distinct. But for simplicity and
for pictorial understanding I am representing them as λ number of u vertices and λ number of v
vertices. And as per my property as per my definition E’ or this collection of edges e1 to
𝑒𝜆 constitutes an edge cut. That means even if you do not delete a single edge from this collection
of edges the graph G still remains connected.
Only when you delete all this λ edges the graph gets disconnected into components C1 and C2.
That is the property of this collection of edges E’. Now from these edges I have to show the
existence of at least, I have to show the existence of some 𝜆 number of vertices whose deletion
will definitely disconnect my graph. Because if I can show you the existence of some λ number of
vertices whose deletion will disconnect my graph then that shows that my vertex connectivity is
also upper bounded by λ, and hence it shows that the vertex connectivity is upper bounded by edge
connectivity.
So, let us do that, so first delete the 𝜆 − 1 u vertices namely to delete the end point of e1 from the
component C1, you take the edge e2 and delete the edge u2 from the component C1. And similarly,
you take the delta minus 1 edge and delete the u vertex from the component C 1. So, you have
deleted λ-1 vertices from your graph G.
684
And remember as per the definition of deletion of a vertex and I delete a vertex all the edges
incident with that vertex also gets deleted from the graph. So, my new graph I call it as the graph
H and my new graph H will look something as follows. The edge 𝑢𝜆 is still there in my graph G.
And because of that the end point 𝑢𝜆 is still there in my graph G, now what can I say about my
graph H? My graph is still connected graph.
It is not yet disconnected, because I still have one bridge or one edge going from 𝑢𝜆 to the
component C2 and that will ensure that everything in C1 is reachable to C2 via the edge 𝑒𝜆 . Now
here is the crucial claim, my claim is that the edge 𝑒𝜆 constitutes a cut edge or a bridge for the
reduced graph H. And the poof is very straight forward.
If this edge 𝑒𝜆 does not constitute a cut edge for the reduced graph H, that means that even if I
delete this edge 𝑒𝜆 , still somehow everything in C1 is reachable to C2. That is the implication but
if that is the case then I get a contradiction that the collection of edges E’ which I assumed to be
the minimum number of edges whose deletion will disconnect the graph G is not a valid
assumption. So, I remember I assume that as soon as I remove the edges e1 to 𝑒𝜆 , my graph get
splitted into 2 parts, C1 and C2 such that nothing in C1 is reachable to C2 and vice versa.
So, I have already removed e1 to 𝑒𝜆−1 . And now I mark queuing that if I remove 𝑒𝜆 my graph H
gets splitted into C1 and C2, such that C1 is completely disconnected from C2 and vice versa. If that
is not the case that means I need to further delete more edges even after removing e1 to 𝑒𝜆 to
disconnect my original graph G, which goes against the assumption that E’ was the collection of
minimum edges whose deletion will disconnect my graph.
So, I get this implication, remember I have already removed λ-1 vertices. And now I have to show
that if I add one more vertex to this collection of λ-1 vertices, which I have already deleted from
G, I get a vertex cut for my graph G. Now, you might be tempted to say that if I include 𝑢𝜆 to this
collection of vertices u1 to 𝑢𝜆−1 , it will always constitute a vertex cut for G, or you might be
tempting to say that if I include 𝑣𝜆 the collection of vertices in u1 to 𝑢𝜆−1 , that will constitute a
vertex cut for G.
685
You can not necessarily do that you have to argue here based on cases. So, since I have argued
that the edge 𝑒𝜆 constitutes a bridge or a cut edge for the graph H, I can definitely say that one of
its end point is a cut vertex for the reduced graph. Because then only the edge 𝑒𝜆 can constitute a
bridge. And this is a property which is there with respect to any bridge or any cut edge of a graph.
I can always say that if I have a cut edge in a graph, one of its end point definitely a cut vertex.
Because if none of the end points of the edge is a cut vertex, then in the first place that edge was
not a cut edge, a very simple fact. Now, I have two possible cases. If 𝑢𝜆 is a cut vertex for H, then
I can say that the collection of u1 to 𝑢𝜆−1, and the vertex 𝑢𝜆 constitutes the vertex cut for G.
Whereas case 2 is the following, if it is the if 𝑣𝜆 is the cut vertex for the graph H then I can say
that the collection of u1 to 𝑢𝜆−1 , along with 𝑣𝜆 will constitute a vertex cut for G. So, you cannot
always say that its u1 to 𝑢𝜆 which is always a cut vertex for the graph G. It depends, once you have
removed the first 𝜆 − 1, edges and one of the end points of those 𝜆 − 1 edges in one of the
components you will be left with of cut edge and your graph may be something as follows.
So, your graph may be something like this. So, you might have got 𝑢𝜆 and then you have 𝑣𝜆 and
then the rest of the graph is still, that means your reduced graph H is something like this. In that
case, you cannot say that if I just delete 𝑢𝜆 along with u1 to 𝑢𝜆−1, I get disconnected graph. No, by
removing 𝑢𝜆 you get the whole graph C2 as you reduced graph, which is connected.
So, in that case, you have not obtained a cut vertex. It is only when you remove 𝑣𝜆 that this portion
of C2 gets disconnected from 𝑢𝜆 . So, depends upon which end point of the cut edge 𝑒𝜆 is the cut
vertex for the reduced graph, and that cut vertex along with the λ-1 vertices, which you have
already removed will give you a vertex cut for the original graph G. So, that is the subtle point,
the two cases. Otherwise the rest of the proof is straight forward.
686
So, now let us unify the relationship between the vertex connectivity and edge connectivity taking
care of various cases. We just proved that for any connected, non-complete graph the vertex
connectivity is always less than equal to edge connectivity. We know that for disconnected graphs,
𝜅(𝐺) = 𝜆(𝐺) = 0. And for complete graphs, we know that 𝜅(𝐺) = 𝜆(𝐺) = 𝑛 − 1.
And you also prove that for connected non-complete graphs, individually the vertex connectivity
is upper bounded by the minimum degree, 𝜅(𝐺) ≤ min degree (𝑣) and the edge connectivity is
𝑣∈𝑉
also upper bounded by minimum, 𝜆(𝐺) ≤ min degree (𝑣). So, unifying all these things we can
𝑣∈𝑉
say the following. Irrespective of whether my graph is connected, disconnected, complete, non-
complete the vertex connectivity is always less than equal to edge connectivity and edge
connectivity is always less than equal to the minimum degree in the graph. 𝜅(𝐺) ≤ 𝜆(𝐺) ≤
min degree (𝑣).This relationship takes care of all the cases here, complete, non-complete,
𝑣∈𝑉
connected disconnected and so on.

687
So, that brings me to the end of this lecture. Just to summarize, in this lecture we discussed about
the vertex connectivity, edge connectivity and we prove the general relationship that exists
between the vertex connectivity and edge connectivity. Thank you!
688
Lecture -49
Tutorial 8
Hello everyone, welcome to tutorial number 8. So, we start with question number 1 where we
want to prove that you take any simple graph G with 6 nodes, either the complete graph 𝐾3 ⊆
𝐺 or the complete graph 𝐾3 ⊆ 𝐺̅ . So, let me first define what exactly is the complement of a
graph in general.
So, the complement of a graph G is a graph which has the same vertex set as the vertex set G.
And the edge set of 𝐺̅ is complement of the edge set of the graph G, namely if you have an
edge between the nodes vi and vj in the graph G, then the edge will not be present in the 𝐺̅ and
vice versa, where 𝑣𝑖 ≠ 𝑣𝑗 . So, that is the general definition of a complement of a graph. And
we want to prove this property in any simple graph with 6 nodes.
So, if you see closely then this question is equivalent to showing that the Ramsay number
𝑅(3, 3) or the Ramsay function R(3, 3) is 6. Why so? So, recall what exactly are Ramsay
numbers? So what we want to prove is that if you take any party where you have 6 guests and
if each pair of distinct individuals are either friends or enemies. Then, we prove that irrespective
689
of the way, the people are friends or enemies with each other, they are always exist either 3
mutual friends, namely we can find 3 friends F1, F2, F3 who are mutually friends with each
other.
That means F1 and F2 are mutual friends, F2 and F3 are mutual friends and F1 and F3 are mutual
friends or we can always find 3 mutual enemies in the party. At least we can find 3 persons,
person 1, person 2, person 3 such that p1 and p2 are not friends with each other, 2 and 3 are not
friends with each other and 1 and 3 are also not friends with each other. So, we can model the
friendship relation as an undirected graph, where I can say that there exists an edge between
the person i and person j if and only if they are mutually friends.
If we do that then the friendship status of 6 people in a party, any party can be represented by
a simple graph with 6 nodes and since we had proved that R(3, 3) = 6, that is equivalent to
showing this property that we are supposed to prove. You wanted to prove that R(3, 3) is = 6
is equivalent to showing that if you take the friendship graph, you can always find 3 nodes in
the friendship graph, such that among those 3 nodes you have edges, you can always find vi, vj
and vk such that you have the edges between vi, vj, you have the edge between vj and vk and
you have the edge between vk and vi. Or you have 3 people such that between i and j no edges
there, between j and k no edge is there and between k and i no edges there in the graph G. If
no edges are there among these three nodes in the graph G, then in the 𝐺̅ graph, you will have
an edge between the edge nodes vi and vj, the nodes vj and vk and the nodes vk and vi.
So, that is equivalent to showing that K3 is present or 𝐾3 ⊆ 𝐺̅ graph. So, this question we have
already solved in principle. It is just that we are now getting a graph theoretic interpretation of
the friendship relationship.
690
Now let us go to question number 2 here. We want to prove or disprove the following; If in a
simple graph G, (G – vi) is disconnected for every vertex vi in the graph, then it implies that
the graph G is also disconnected. Or equivalently here we want to check that if in the simple
graph G, every vertex vi is an articulation point or cut vertex, then the graph G is disconnected
because if (G – vi) is disconnected that means the vertex vi is an articulation point.
So, we want to check whether this property is true or not. To prove this, let us prove a related
statement or we prove a relative claim. So the claim is the following; If you take any connected
simple graph then they always exist at least two vertices, none of which is an articulation point,
this always holds in any connected simple graph. I can always guarantee the presence of two
vertices which are not cut vertices, of course, the number of vertices in the graph is greater than
= 2 because if the graph is just one vertex then this claim does not make any sense.
How exactly do we prove this claim? So, you focus on the nodes u, v in the graph G which are
farthest, that means the distance among the nodes u and v is the maximum in the graph. That
means you take all pair of nodes u, v find out a distance among those nodes u, v. And among
all the (u, v) pairs, you focus on the (u, v) pair such that the distance is maximum in the graph
G. My claim is that the nodes u as well as the node v are not articulation points, they are not
cut vertices and this can be proved using a proof by contradiction.
So, on contrary, assume that say the vertex v is a cut vertex. So, this is without loss of generality
the same argument can be applicable if we assume on contrary that the vertex u is a cut vertex.
So, if the vertex v is a cut vertex, that means by deleting the vertex v, my graph gets divided
691
into two connected components C1 and C2. And this means that the vertex v has degree 2 and
it will have at least one neighbor say, w in the component C2.
Because if the degree of the vertex v would have been just one that means if the only neighbor
of the vertex v would have been this vertex x in the component C1 and there is no neighbor w,
then how can it be possible that deleting v disconnects the graph was split the graph into two
component C1 and C2? So, since the deletion of v splits your graph into two connected
components, that means there is something, some node w in C2 such that v is having an edge
to that node w in the connected component C2.
But that gives you a contradiction to the fact that the nodes u and v are the farthest nodes in
your graph G, because now you can see that a distance between the node u and w is more than
the distance between the nodes u and v. That means by nodes u, v are not the farthest nodes in
the graph, but it is rather the node u and w which are the farthest nodes in the graph. So, we get
a contradiction and that proves that whatever we assume about the vertex v is not true, so we
assume that a vertex v is a cut vertex, which is not true.
So, now coming back to the question, the question is equivalent to saying that can we have a
simple connected graph where every vertex is a cut vertex? And that is not possible, because
that is precisely what we proved in this claim. We proved in this claim that if at all your graph
is a connected simple graph there are definitely and the number of nodes is greater than = 2,
then they are definitely exist two vertices vi and vj such that neither vi is an articulation point
nor vj is an articulation point, it is not possible.
692
Let us go to question number 3. So here we have to find an unknown graph G, the graph G is
not given to you but it is just given that it is a simple graph. And since the graph G is not
known, we also do not know its incidence matrix B, but we know that incidence matrix B is
such that the product of the incidence matrix and its transpose is this matrix. So, we have to
basically recover the original graph G from the product of the incidence matrix and its transpose
that is given to us.
Of course, a naive we have to intact will be you try all possible values of, for all cases of the 𝐵
matrix and the 𝐵 𝑇 matrix and multiply them and check whether your guess gives you this value
of 𝐵𝐵 𝑇 matrix or not. We will not do that. We will argue and try to get back the information
regarding the graph G. So, imagine that your number of vertices in the graph is n and the
number of edges in the graph is m.
So, the incidence matrix of the unknown graph will be an n cross m matrix, so I am denoting
the unknown incidence matrix by this notation. So, b11, b12, b1m they are the unknown Boolean
values, so remember each entry of the incidence matrix will be {0, 1}, either 0 or 1. And if the
edge e is between the vertex vi and vertex vj, then in the incidence matrix if we focus on the
eth row, then if we focus on the row number vi and if you focus on the row number vj and the
column number e.
Then under the column e in the ith row we will have the entry 1 and the jth entry we will have
entry 1 and all other entries will be 0. That is the property of your incidence matrix. Now, we
do not know as of now which entries are 0, which entries are 1. Now it is easy to see that the
693
jth row of the incidence matrix B will become the jth column in the transpose of the incidence
matrix and that comes from the property of the transpose of a matrix. And what will be the (i,
j)th entry, when we multiply the matrix B with the matrix B transpose?
So, how exactly the (i, j)th entry of the product of B and B transpose would be computed? We
would have taken the ith row and we would have multiplied the ith row with the jth column
component wise namely, bi1 would have been multiplied with bj1. And then added to the product
of bi2 and bj2. And like that you would have multiplied the entry number bim, the entry number
bjm, and if we had all these things that will give the (i, j)th entry of the matrix 𝐵𝐵 𝑇 .
And we are given the value of 𝐵𝐵 𝑇 . So now what I can say is the following; If I take any (i,
j)th entry where i is not = j, then the (i, j)th entry in the product matrix 𝐵𝐵 𝑇 will be 1 if and
only if the vertex vi and the vertex vj are incident on a common edge. That means they are the
endpoints of an edge. This is because we already argued that this is our (i, j)th entry, the (i, j)th
entry is one and only if one of the values in this sum of m values is what because of none of
the values.
So, if the first product in this sum is 0, and if the second product in the sum is also 0 and like
that if the mth product in the sum is also 0 and how come the (i, j)th entry is one? So, (i, j)th
entry is 1 only if you have bi1 = bj1 = 1, if that is the case that means the vertex i and vertex j
they are the endpoints of the edge number 1 or bi2 should be = bj2 should be = 1, which implies
that the ith vertex and jth vertex, they are the endpoints of the edge number 2, and so on.
So, if I focus on the (i, j)th entry, where i and j are distinct and checking whether they are 1 or
0, we can identify whether the vertex vi and vj are the endpoints of an edge 1. And you have
that information available in the product matrix 𝐵𝐵 𝑇 . And if I take the (i, i)th entry, that means
if I substitute j = i here and focus on the (i, i)th then the (i, i)th the product matrix will be this
expression and this is nothing but the degree of the vertex vi.
So, you have all the information available now about the graph G in the product matrix. So
your graph G is such that the degree of 1 is1, the degree of vertex 2 is 2, the degree of vertex 3
is 4, the degree of vertex number 4 is 2 and the degree of vertex number 5 is 3 and the endpoints
of each edge is also available by focusing on the (i, j)th entry in this matrix, where i and j are
694
distinct. So, this is how you can get back all the information about your unknown graph from
the product matrix 𝐵𝐵 𝑇 .
So, now let us go to question number 4. In question number 4 we have given the definition of
a tree. So, a tree is a connected acyclic graph, that means it is a graph which is connected, that
means you take every pair of distinct modes, there will be a path and it is acyclic, that means
the graph has no cycle. We have to show that if you take any tree with n nodes, then the tree
has n - 1 edges. So there are several ways to prove this theorem we will use proof by induction,
induction on the number of nodes n.
So, the statement is obviously true for the base case namely for a tree which has only 1 node.
So if you create a tree with 1 node then it will have 0 edges. Assume the inductive hypothesis
is true, that means assume the statement is true for all trees consisting of up to k nodes. And
now we are going to the inductive step, where we are going to consider an arbitrary tree
consisting of k + 1 nodes. And we focus on any arbitrary edge with endpoints u and v in the
graph G or the tree G.
My claim is that the edge (u, v) is a cut edge in the tree and this is true for any edge in the tree.
You take any edge the claim means it will be a cut edge. That means if you remove the edge
connecting the nodes u and v then your tree G gets splitted or divided into two connected
components. If that is not the case, that means even if after deleting the edge between u and v,
your tree remains connected that means there are still some way to get back to the node u from
the vertex u even if this edge is not there between u and v.
695
Then we get the conclusion that there is a cycle in the graph but that goes against the definition
of a tree. So, now if my edge (u, v) is a cut edge, I will get two components C1 and C2. I do not
know how many nodes are there in C1 and how many nodes are there in C2? So, I can assume
that they have n1 and n2 number of nodes respectively. But what I know is that if I sum up the
number of nodes in C1 and the number of nodes in C2, that will give me the total number of
nodes that I have in the tree, which is k + 1.
And I also know that both n1 as well as n2 are upper bounded by k. This is because C1 and C2
are disjoint and both of them are non empty, ≠ 𝜙. So, since n1 and n2 is less than = k and both
C1 is connected as well as C2 is connected, and I do not have any cycle in C1 and I do not have
a cycle in C2. that means both C1 as well as C2 are individually trees with n1 and n2 nodes
respectively.
So, as per the inductive hypothesis, I can apply the inductive hypothesis and claim that the
number of edges in C1 is n1 - 1, the number of edges in C2 is n2 – 1. And that gives me the total
number of edges in the original tree G is just one more than the number of edges that I have in
the tree C1 and C2. This is because the only edge which I have touched or removed is the edge
between the nodes u and v and that proves the inductive step.
So, now let us go to question number 5. And question number 5, we define what we call as a
self-complementary graph. So, a graph G is called self-complementary if it is isomorphic to its
complement. So, for instance, this is a self-complementary graph with 4 nodes. I am not
696
labeling the nodes but you can check here that the graph G and the graph 𝐺̅ , they are isomorphic
to each other and hence the graph G is self-complementary here.
So, we want to prove here a very interesting property about self-complementary graphs, we
want to prove that if your graph is a self-complementary graph, then the number of nodes in
the graph is either a multiple of 4 or it is some 4 times k + 1. That means either the number of
vertices is completely divisible by 4 or if you divide the number of vertices by 4 then you will
get the remainder 1 that is what we want to prove here.
At least you can check that this statement is true for the G and G’ that we have here; the number
of vertices in G is 4 and 4 is visible by 4. So, how do we prove this? Since the graph G is self-
complementary that means the number of edges in G and the number of edges in 𝐺̅ have to be
the same. So, the cardinality of the edge set E and edge set E complement will be the same and
I know that for any graph it may not be self-complementary.
If you take any graph then the total number of edges in E and E complement is the same as the
product of the number of vertices and the number of vertices minus 1 over 2. Because all the
edges which are in G they would not be in 𝐺̅ and vice versa and if you take the union of the
graph G and the graph 𝐺̅ , you will get a complete graph with the number of nodes same as the
number of vertices in the graph G or 𝐺̅ .
So, if you take 𝐺 ∪ 𝐺̅ then you get the complete graph with n nodes, where n is the number of
vertices in the graph G. So, remember the number of vertices in G and 𝐺̅ will remain the same,
you do not take complement with respect to the vertex set, the complement is respect to the
edge set. So, we know these two facts, one fact that is true for every self-complementary graph
and another fact which is true for every graph.
Based on these two things, if I substitute that cardinality of E is same as cardinality of E
complement then I get that two times the cardinality of E is = |𝐸 | + |𝐸̅ | = |𝑉|∙(|𝑉|−1)
2
which shows that the product of the cardinality of the vertex set and the vertex set minus 1 is
a multiple of 4 or it is divisible by 4. Now you have 2 quantities here a and b and what are the
prime factors of 4? 2 and 2, it turns out that you cannot have both a as well as b simultaneously
divisible by 2.
697
Because one of the quantities is odd, then other quantity will be even. a and b you have you
cannot have to consecutive numbers both of them simultaneously to divisible by 2 and but
since I know that the product of a into b is divisible by 4, then that is possible only if either a
is divisible by 4 or b is divisible by 4. Then both a and b would have been individually divisible
by 2, then I cannot claim this fact here.
But since I know that both a and b cannot be simultaneously divisible by 2, but the overall
product a times b is divisible by 4 then that is possible only if a times b is divisible by 4. It only
then either a is divisible by 4 or b is divisible by 4 and that shows what we wanted to prove
here. Now in the same question we want to draw a self-complementary graph with a vertex set
which has 4k number of nodes.
So, you are given a value k, k could be anything, it could be 1, 2, 3, 4 given the value of k you
have to draw a self-complementary graph, which has 4 times k number of nodes. So, if k is =
1, then this is the self-complementary graph, but you cannot draw a distinct self-complementary
graph for each and every value of k, I just want to draw a general graph which unifies all self-
complementary graphs with 4k number of nodes, so how do we do that?
So, what I do here is I take four groups of k nodes and those four groups of k nodes are disjoint.
So this is my first group, this is my second group, this is my third group and this is my fourth
group. Now this group of k nodes denotes a complete graph with k nodes, that means you have
an edge between every pair of distinct nodes in this group and similarly this copy of k nodes
denotes a complete graph with k nodes.
̅ denotes an incomplete graph of k nodes, that means it is a collection of k

And this group 𝐻
nodes with zero edges and similarly this collection of k nodes have zero edges. Now what I do
is the following my G is the following my graph G which is a self-complementary graph with
four k nodes is the following, so of course my graph G will be now having 4k nodes because
the total number of nodes are k + k + k + k, so 4k nodes.
Now what I am doing here is the following; So, this thick edge between this group of k nodes
and this group of k nodes denotes that if you have the nodes v1 to vk here and if you have the
698
nodes vk + 1 to v2k here this collection, then you have an edge between every node in this group
and every node in this group. That is what is the interpretation of this thick edge. Similarly you
have an edge between every group between every node in this group and every node in this
group.
That is interpretation of this thick edge and similarly you have an edge between every vertex
in this group and every vertex in this group, that is interpreted by this thick edge, that is
interpretation of this thick edge, so that is my graph G. Now what will be the complement of
this graph G? So, this is my graph G, so my graph 𝐺̅ will also have 4k number of nodes but
then what will happen is the following.
So, since this group was a group of k nodes with edges between every pair of nodes you will
still have those k nodes but no edges among any pair of nodes, whereas this copy of k nodes
will get converted into a complete graph of k nodes. This copy of a complete graph of k nodes
will get converted into a collection of k nodes with no edges and this copy of a graph with k
nodes and zero edges gets converted into a complete graph with k nodes. And then since these
edges are present in G, they will not be present here anymore.
So, that is why these edges have vanished and similarly you can see the edges which were not
there in G there will be now present in 𝐺̅ and vice versa. The edges which were not there in G
they will be present in 𝐺̅ , so there were no edges between this group and this group in G, but
now those edges are here and so on. So, that is how your 𝐺̅ will look like, so it is easy to see
that your graph 𝐺̅ is isomorphic to the graph G.
I can interpret or rearrange or redraw the graph 𝐺̅ in the same form as the graph G. I just have
to orient it a little bit, that is all, say if I orient this graph rotate, this graph little bit like this,
then I get the same structure as the graph G and that shows that my graph G and G is self-
complementary because it is isomorphic to its own complement.
699
In question 6, we have to either prove or disprove the following; If the graph H is a subgraph
̅ is also as the graph of 𝐺̅ . Well, the statement is not necessarily
of G then can we say that 𝐻
true. A very simple counter-example is the following: here you have a graph H and a graph G,
̅ , in 𝐻
the graph H is a subgraph of G, but if you take 𝐻 ̅ , the only the edge will be between the
nodes 1 and 3 because the edge is between 1 and 3 was not there. Where in 𝐺̅ there will not be
̅ is not a subgraph of the graph 𝐺̅ . So, this statement
any edge, so clearly H prime or the graph 𝐻
is not necessarily true.
(Refer Slide time: 31:18)
Then let us see question number 7 and 8: a simple graph is called regular if the degree of every
vertex is the same. And if the degree of every vertex in a simple graph is some value r, then we
call such a graph an r-regular graph. So, here you are given a few graphs and we have to find
out which of these graphs are regular and which are not. so Kn is a regular graph because the
700
complete graph with n nodes in such a graph the degree of every vertex is n - 1, so it is regular.
The cycle graph with n nodes is also regular because if you take the degree of every vertex, it
will be 2. But your wheel graph Wn is not a regular graph, because it is the central node which
has a huge degree compared to the other vertices of the graph. Whereas if you take the
hypercube graph, we can prove that it is a regular graph and the degree of every vertex will be
the same.
Now what we have to do in question 8 is the following; You are given a value of k. You have
to draw a simple regular graph where the degree of every vertex is 2 times k + 1 such that the
graph has a cut edge. You have to give the construction of a general graph. So, what we can do
here is the following; we take two copies of a complete bipartite graph. So we take one copy
of a complete bipartite graph where I have 2k nodes in the individual sets in the bi-partition.
So, 2k number of nodes in v1 and 2k number of nodes in v2 and I have an edge between every
vertex in v1 and every vertex and v2, that is denoted by this arrow symbol, bi-implications
symbol. Because the graph will look very ugly if I keep on adding edges between every vertex
in v1 and every vertex and v2. So, to avoid making it look ugly I just denote the existence of an
edge between every node in v1 and every node in v2 by this bi-implication.
And similarly, I take another copy of a complete bipartite graph and say I call the bipartition
of this copy of the complete bipartite graph as C1’ and the C2’ and again I have an edge, I have
an edge between every node in v1’ and every node in v2’. So, this is my v1, v2, v1’, v2’. Now I
need to ensure that my graph has a cut edge. So I try to introduce a cut edge. This will be my
overall cut edge and what I do here is, so let me call the end points of this cut edge as a and b.
I connect the node a to every vertex in the subset v2. So, that will ensure that the degree of the
vertex a is 2k + 1, why 2k + 1? Because it will be connected to all the vertices of v2, so it gets
degree 2 k through that and it is also having an edge to the node v so that ensures that a degree
of a is 2k + 1.
Using the similar argument, I can say that a degree of b is also 2k + 1 because b has a neighbor
in every vertex with v in v1’, so through that it gets 2k degree and b is also a neighbor of the
node a, so an additional degree, so the degree of b is also 2k + 1. Now all the vertices of v2 will
have degree 2k + 1. This is because you take any vertex of v2, say the first vertex, so it is a
701
neighbor of a degree 1 and it is having an edge with every node in v1.
So, through that it gets the degree 2k, so total degree 2k + 1. Due to the same argument each
node in v1’ also have degree 2k + 1 because you take any node in v1’, it is having an edge with
every node in v2’, so that it gets degree 2k and the same node is also having an edge with the
node b, so through that it gets one more degree so total degree 2k + 1. But we also need to
ensure that every vertex in v1 and every vertex in v2’ also gets degree 2k + 1.
Till now we have ensured that the degree of a is 2k + 1, we have ensured that the degree of b
is 2k + 1, we have ensured that every vertex in v2 has degree 2k + 1 and we have ensured that
every vertex in v1’ has degree 2k + 1. But right now the degree of every vertex in v1 is 2k and
similarly the degree of every vertex in v2’ is 2k. I need to increase the degree of each vertex in
v1 by 1 and each I have to increase the degree of each vertex in v2’ by 1 as well and that is
simple.
What we can do is the following; Take the vertex set v1, you pair them into k pairs, you take
the first two nodes and add an edge between them, then you take the third and the fourth note
and add an edge between them and like that you take the fifth and sixth node and add an edge
between them and so on. So, that will ensure that the degree of every vertex in v1 becomes 2k
+ 1, and if you do the similar process for v2’ as well. That will ensure that the degree of every
vertex in v2’ becomes 2k + 1. So, with that I conclude the tutorial number 8. Thank you!
702
International Institute of Information Technology - Bangalore
Lecture - 38
Euler Path and Euler Circuit
Hello everyone welcome to this lecture the plan for this lecture is as follows. In this lecture
we will discuss about Euler path and Euler circuit and we will see the characterization for the
existence of Euler path and Euler circuits in a graph.
So, let us start with the definition of Euler circuit and Euler path. So, imagine that you are
given a graph then an Euler circuit is a simple circuit which contains every edge of the graph.
So, since it is a circuit that means the starting point and the end point of the trail or the tour
703
will be the same that means you have to start at the same vertex and you have to end at the
same vertex in the tour.
And it is simple in the sense that during the tour the edges are not allowed to be repeated. So,
it is a special type of simple circuit in the sense that it contains every edge of the graph; no
edge of the graph will be absent in this simple circuit if you have the existence of such a
simple circuit and the circuit will be called as an Euler circuit. And if you have an Euler
circuit in your graph then the graph will be called as an Eulerian graph.
Whereas an Euler path is a simple path which contains every edge of the graph so the
difference between Euler path and Euler circuit is that in the case of Euler path your starting
point and end point are not same because it is just a path. However it is still simple and hence
the edges are not allowed to be repeated. Whereas in the case of Euler circuit edges are not
allowed to be repeated but you also need the fact that the starting point and end point should
be the same.
So, let us see some examples of both these concepts. So, imagine this is a graph given to you
then this graph has an Euler circuit and hence this graph will be an Eulerian graph. So, if you
follow the tour along the blue edges or the blue arrows that gives you an Euler circuit. So, for
instance suppose I start at e in fact you can start at any vertex and I first go from e to d that
takes care of this edge then I go from d to c that takes care of this edge between d and c.
Then I go from c to f that takes care of this edge then from f I go to g that takes care of the
edge between the node f and g then I go from g to c that takes care of this edge. And then
finally I stop my tour by traversing the edge between c and e. So, you can see I started at e
and ended my trip at e and in my tour all the edges of the graph are covered and no edge is
repeated. Hence this is an example of an Euler circuit.
Whereas if you see this graph then it is easy to verify here that this graph does not have any
Euler circuit you start at any vertex it is impossible to make a tour starting at the same vertex
and ending at the same vertex and traversing every edge of the graph exactly once and
without repeating any edge that is not possible. So, for instance let us try to make a tour
starting from a so if I traverse from a to c and then if I go from c to d.
704
And then if I go from d to e and then from e to b and then if I go from b to d and then if I go
from d to a. And then if I go from a to b by the time I reach b I have traversed all the edges
but now you see that my current point is b and I started my tour at a. So, the requirement of
Euler circuit is that tour should start and end at the same vertex. So, currently I am at b if at
all I want to end my tour at a I will be repeating the edge between b to a.
And hence this tour will no longer be a simple circuit. However this graph has an Euler path
because if you follow the tour along the edges highlighted in the red color then I have started
the tour from a ended my tour at b and I have traversed all the edges of the graph exactly
once. So, hence this graph is not an Eulerian graph because it does not have an Euler circuit
but it does have an Euler path.
So, Euler gave a very simple necessary and sufficient condition according to which you can
verify easily whether a given arbitrary graph as an Euler circuit or not. So, the theorem
statement is the following. Imagine you are given a connected graph that is important and
your graph need not be a simple graph it can be a multi graph by multi graph I mean that
between the same pair of vertices you might have multiple edges.
So, the graph need not be a simple graph but still I can define the notion of Euler circuit and
Euler path even for multi graph. So, imagine you are given a connected undirected graph
which is a multi graph and the graph has at least 2 nodes because if my graph is just a single
node then again the notion of Euler circuit does not make much sense there. So, imagine you
are given a multi graph which is connected and it has at least 2 nodes.
705
Then what Euler proved is that the graph will have an Euler circuit if and only if each of the
vertices in the graph has even degree. And this condition is both necessary condition as well
as a sufficient condition because this is an if and only if statement. So, we will prove both the
necessity condition as well as the sufficient condition. So, let us first prove the necessity
condition namely the only if part.
And for that we have to show the following implication we have to prove that if your graph
has an Euler circuit then it implies that each vertex of the graph has even degree you cannot
have any vertex in the graph which has an odd degree. So, and it is very simple to prove this:
so imagine, your graph. you are given an arbitrary graph which may not be a simple graph
and imagine that a graph indeed has an Euler circuit.
So, I am calling an Euler circuit which is there in your graph by T. I am denoting it by T. So

since it is an Euler circuit the tour T will start and end at the same vertex. So, I am denoting
the starting point and the ending point of the tour by the node a. So, the first thing to observe
here is that the degree of the vertex a in your graph will be even why so? Because the first
edge of the tour will be incident with a, namely it will be an edge coming out or incident with
a that means because your tour is starting from the node a.
So, the the first edge in the tour which is incident with the node a will contribute 1 to the
degree of a. And since your tour also ends with the node a that means last edge in the tour is
also incident on the incident with the node a. So, that implies that definitely the degree of a is
at least 2. And if your node a occurs as an intermediate node in the tour T then again it
contributes 2 to the degree of a.
Because each time you will be entering the node a via some edge and you will be coming out
of the node a in the tour. So, the edge through which you enter the node a in the tour that
contributes 1 to the degree of a and edge through which you are coming out of the node a in
the tour contributes again to 1 to the degree of a and it can happen multiple times. So, if your
node a is appearing p number of times as an intermediate node in that tour, then the overall
degree of a will be 2 times p and this is apart from the degree 2 which is contributed because
the starting edge of the tour was incident with a and the ending edge of the tour is also
incident with a. So, every time you enter the node a or a occurs as an intermediate node you
706
are actually counting 2 to the degree of a and since my circuit T is an Euler circuit that means
all the edges incident with a in my original graph will be covered; will be appearing
somewhere in my tour T.
And as we have argued here each time the node a occurs in the tour we are actually counting
2 to the degree of a. So that shows that the degree of the node a will be even. In the same
way I can argue that if you take any intermediate node b which is appearing in the tour it will
contribute 2 to the overall degree of b because if your tour is, if this is a part of the tour where
you enter the node b, and then by following some edge incident with b you go to some
another node and then again suppose you come back to the node b through some edge and
again you leave the node b. So, every time you enter the node b and you come out of the node
b you are counting 2 for the degree of b. So, again following the same logic as we have used
to argue that the degree of node a is even we can conclude that the degree of the node b also
will be even.
And again since all the edges incident with the node b in your original graph will be
appearing somewhere in the tour T that shows that the degree of the node b is even. So,
necessity condition is very simple.
Now we will prove the sufficiency condition if part and for that we have to show the
following: I have to show that if you are given a connected multi graph where all the vertices
have even degree then there exists at least 1 Euler circuit in your graph there might be
multiple Euler circuits also possible in your graph but at least 1 Euler circuit is definitely
707
there. And for proving the sufficiency condition I will discuss here an algorithm called as
Fleury’s algorithm.
By running this algorithm on a graph we are the degree of every vertex is even we are
guaranteed to obtain an Euler circuit and algorithm is very simple. And overall the main
principle followed in this algorithm is that when you are trying to make a tour in the graph by
using the Fleury’s algorithm try to not burn the bridges; by bridges I mean the cut edges try to
avoid traversing the cut edges until and unless it is not possible to avoid traversing the cut
edges or the bridges.
So, the algorithm is an iterative algorithm because in each iteration we will be advancing our
tour and after a certain number of iterations our tour will end and we will end up covering all
the edges of the graph. So, we can start the tour from any vertex there is no restriction that
you should start the tour only from a specific vertex you can pick any vertex to start your
tour. So, the vertex from where I am starting my tour I am denoting it as v0.
And I am defining a set W0 which is initialized to the set v0 my starting point of the tour and
in each iteration I will be picking an edge which I will be traversing next in my tour and once
I traverse that edge since I require overall; since my final output should be a simple circuit
where the edges are not allowed to be repeated once I have traversed an edge in the graph I
should not consider it in the future iteration.
So, I will keep on updating my graph once an edge is traversed I should remove it for further
consideration. So, as a result my graph also will keep on getting updated. So my initial graph
G0 will be the input graph G itself because as of now I have not traversed any edge I have just
decided the starting point of my tour. Now as I said the algorithm is iterative so imagine you
have finished the k iterations.
So, right now k = 0 but imagine that you have already obtained a partial tour and your partial
tour has already traversed k number of edges edge e1, edge e2 and edge ek where the edge e1
is incident with the node v0 and v1 the edge e2 is incident with v1 and v2 and like that the edge
ek is incident with vk - 1 and vk. So, imagine that this tour has been this is the partial tour
which you have already done through the Fleury’s algorithm.
708
Now you have to decide what should you do in the next iteration. So, as I said your graph G
also keep on getting updated because as you keep on traversing more and more edges those
edges are removed from further consideration. So, since my edges e1, e2, ek have been already
traversed and covered in my tour I will be removing those k edges from my graph and
updated graph is Gk. Remember by removing the edges we are not removing the vertices we
are just removing the edges which we have traversed vertices remain as it is.
Now for the next iteration we will do the following. So, since my current tour has stopped at
the node vk I will check whether there are more edges to be traversed incident with the node
vk in my graph Gk that means I will just check whether there are any more edges left for
traversing or not and that edge is incident with vk or not. If there are no more edges left
incident with the node vk then I stop the algorithm and the tour Wk is my output tour.
We will argue later that indeed this tour is an Euler circuit. But suppose if there are still some
more edges which are left which are not yet traversed and those edges are incident with vk
then I have to select the next edge incident with vk for traversing. Now there might be 2
possibilities here if you have only 1 edge left in the graph which is incident with vk you have
no choice you have to traverse that edge because you have to ensure that that edge is covered
as part of the tour.
So, in that case you have no other choice; no other option. But imagine you are in a scenario
where there are multiple edges which are still not traversed and incident with vk then among
all those edges which are still incident with vk you should select your next edge for
traversing. And you should give preference to a non cut edge of the graph Gk and that is what
I mean by do not burn the bridges that means say you have reached the node vk.
And you have multiple edges still left in the graph incident with the node vk. So, what the
algorithm says is you cannot arbitrarily choose any of those edges. Among all the edges
which are still incident with vk and not yet traversed, check which of the edges are non cut
edges for the graph Gk I stress for the graph Gk because now your graph is Gk not the original
graph because in the original graph you have already removed the edges e1 to ek.
So, if you have non cut edges still left in the graph Gk give preference to non cut edge but if
you have no non cut edge available incident with vk then feel free to use or traverse any of the
709
cut edge incident with the vertex vk that is what is Fleury’s algorithms. So, once you have
decided that edge ek + 1 has to be traversed next by following the preference rule dictated by
the Fleury’s algorithm you will update your tour to Wk + 1.
And that updated tour will now have this new edge ek + 1 included incident with the nodes vk
and vk + 1 . And then you will again go to the next iteration, that is the algorithm; a very
simple algorithm.
So, let me demonstrate this algorithm so imagine this is a graph given to you. And you can
easily check here that this graph indeed satisfies the sufficiency condition as dictated by the
Euler’s theorem because indeed every vertex in this graph has an even degree. So that means
if I run the Fleury’s is algorithm I should definitely get an Euler circuit. So, let us check
whether we get an Euler circuit here or not.
So, I start with the node v0 and there are 2 edges left incident with the vertex v0 my graph G0
is my original graph and both the edges are non cut edges for the graph G0 so you are free to
traverse any of them. So, suppose I traverse the edge between the node v0 and v2 and hence I
remove that edge and I updated my tour. Now this is my graph G1. . Right now I am at the
node v2 and I have 3 edges which are not yet traversed then incident with the node v2 and
none of them is a cut edge.
So, I am free to traverse any of them, so suppose I decide to traverse the edge between v2 and
v1. And hence I remove it from the graph I updated my tour and my graph gets updated to G2.
710
Now I am at the node v1 now you can see that I have 3 edges incident with v1 the edge
between v1 and v3 the edge between v1 and v4 and the edge between v1 and v0. And you can
see that the edges between v1 and v3 is a non cut edge.
And also the edge between v1 and v4 is also a non cut edge but the edge between v1 and v0 it
is a cut edge because, indeed if you remove the edge between v1 and v0 the vertex v0 gets
disconnected from the rest of the graph in G2. So that is why as per the Fleury’s algorithm
when you have the choice here between selecting the cut edges and non cut edges you should
give preference to the non cut edges that means you should either traverse edge v1, v3 or you
should traverse the edge v1, v4.
And you can check why that is the case because if you do not follow the Fleury’s algorithm
and you decide to make or include this edge namely the edge from between v1 and v0 and you
advance your tour and you reach v0 then you are stuck you still have lots of edges to cover.
And you have now reached a point where from that point if you want to come back to the
graph you have to repeat the same edge between v0 and v1.
And hence you will not obtain a simple circuit. So that is why Fleury’s algorithm says when
you have an option between cut edge and non cut edge you should give preference to non cut
edge. So, we will give preference to either the edge between v1 and v3 or the edge between v1
and v4. So, I followed the edge between v1 and v3 and now I am at v3. I have 3 edges incident
with v3 none of them is a cut edge so I can select any of them.
So, I select the edge between 3 and 5 now I am at v5. And now you can see that there is only
one edge incident with the node v5 which is not yet traversed namely the edge between v5 and
v4. And indeed that is a cut edge in the graph G4. But I do not have any choice I have to
traverse that edge because there is no other edge left incident with v5 other than the edge
between 5 and 4. So, I have to traverse that edge but that would not cause any issue.
Because if I traverse the edge between v5 and v4 that means I have now taken care of all the
edges incident with v5 in my original graph. And I do not need to come back to the node v5 in
my future iterations of the tour. So, now I am at the node v4, multiple edges are incident with
the node v4 none of them is a cut edge so we can choose any of them I choose to traverse the
edge between 4 and 2.
711
Now at vertex v2 there is only 1 edge incident namely between v2 and v3 which is indeed a
cut edge for the graph G6 but I have no other option. So, I have to traverse that edge but that
would not cause any issue. Now I am at v3 there is only 1 edge incident with v3 which is a cut
edge but again not an issue and I have to follow that edge because there is no other option.
So, I go to the node v4. Again there is only 1 edge left incident with v4 which is a cut edge.
So, I have no choice I have to traverse that edge I am now at v1 and there is only 1 edge left
in the graph, traverse that edge and now you have your tour ending. So, W10 will be the tour
T which was output tour of your Fleury’s algorithm and it is easy to see thatwe have indeed
obtained an Euler circuit. So that is a demonstration of Fleury’s algorithm.
712
So, now we want to prove that indeed the output of Fleury’s algorithm is an Euler circuit and
the proof is slightly involved. But I will try to give you a high level overview of the proof of
correctness. So, what we want to prove here is that if your graph G is a connected graph and
multi graph; remember for Euler circuit I do not need my graph necessarily to be a simple
graph. As long as all the vertices of the graph has even degree that is fine.
I will end up getting a circuit which covers each and every edge of the graph even if there are
multiple edges between the same pair of nodes. So, the theorem statement that I want to
prove here is that if your graph is connected and every vertex has even degree then by
running this Fleury’s algorithm a very simple algorithm you can see the algorithm is very
simple. So, the claim is that by running this simple algorithm the output tour that we obtain is
indeed an Euler circuit.
So, there are multiple things which we have to prove regarding the output that we obtain as
part of the Fleury’s algorithm. So, let the output be Wk that means I have run for k iterations I
have to first show that indeed the output is a simple path that means no edge is repeated.
That means all the edges e1, e2, ek which I obtain in the tour Wk are distinct edges but that is
very simple to prove because what we are doing in the Fleury’s algorithm if you check this
step.
Once I have decided the next edge to traverse I am not going to consider it in the future
iterations I am simply removing it from my original graph G and I am updating my graph. So
that ensures that in each iteration I am selecting distinct edges and hence my output will be a
simple path. The next thing I have to prove is that not only the output is a simple path it is a
closed circuit that means the starting point and the end point they are the same.
And there are multiple ways to prove this: a very simple proof will be proof by contradiction.
So, we want to prove that v0 and vk are same that is what we want to prove but on contrary
assume that vk and v0 are different. So, assuming this contrary statement I have to arrive at
some false conclusion or false statement. So let us see what is the false conclusion we can
arrive at. So, since I have terminated my tour with the node vk, that means this particular
step which determines the termination condition of your algorithm guarantees or implies that
there are no more edges incident with the vertex vk in your graph Gk . There are no more
edges left that means what I can say is the following if the node vk which is my endpoint of
713
the tour has appeared p number of times in there tour. So, remember the vertices are allowed
to be repeated in your Euler circuit.
It is the edges only which are not allowed to be repeated. So, it is not necessary that vk has
appeared exactly once it can appear multiple times in fact it can appear multiple times. So,
imagine it has appeared p number of times that means you started with your tour with v0 you
went to v1 and you continued your tour and you stuck vk somewhere and then again you came
out of vk.
And then again suppose you entered vk and then again you came out of vk and so on. So like
that assume that vk has appeared p number of times in your tour. That means the degree of the
vertex vk in your original graph is 2 times p + 1 why 2 times p + 1? Because out of those p
times where the vertex vk is appearing, the last occurrence is actually the occurrence where
you are actually terminating the tour.
So, you remember you are terminating your tour with the vertex vk. So that means out of
those p times definitely 1 time is the last occurrence. And the remaining p - 1 times you have
entered you have come out you have entered you have come out you have entered you have
come out. So, I am assuming here that the vertex vk is occurring as an intermediate node p
times apart from the final occurrence.
So, where a p number of occurrences of the vertex vk as an intermediate node and there is a
final occurrence of the vertex vk as the endpoint of your tour. So that means that the overall
degree of the vertex vk is 2 times p + 1, 1 because of the final appearance of the vertex vk in
the tour and 2 times p because it is occurring as an intermediate node and each time it is
occurring as an intermediate node we are counting 2 to the degree of the vertex vk.
So that means the overall degree of the vertex vk is 2 times p + 1 which is an odd quantity and
this is a contradiction, contradiction to the fact that in my graph it is guaranteed that all the
vertices are of even degree. So that means whatever I have assumed here is contrary; that
means my starting point and end point are the same.
714
So that means whatever output I obtain it is indeed a simple circuit now left is the tricky part
to prove. So, we have to prove that indeed the tour Wk, the simple circuit Wk, which we
obtain here it has all the edges of the graph G, no edge is missing in this tour and the proof
will be by contradiction. Again there could have been multiple ways but we will follow a
proof by contradiction approach.
So, imagine that some vertices of positive degree are still left in my graph Gk. So, remember I
have stopped my algorithm at the kth iteration because I am assuming that my tour consists of
k edges. So, the instance of the graph left at that point is the graph Gk and since I am
assuming that there are still some untraversed edges left in my graph that means there are
some vertices which do have incident edges left in my graph Gk.
That means in other words there are still some vertices of positive degrees left in my graph
Gk. So, I am denoting the set of all the vertices which still have some untraversed edges left
in the graph Gk by the set S. So, namely it is the set of all the vertices which have degree
greater than 0 in the leftover graph Gk. The first thing to observe here is that since you started
with your original graph where all the vertices have even degree, it is guaranteed that even in
the set S all the vertices still have even degree. So, if you have not at all traversed any edge
incident with the vertex in a set S that means whatever was its original degree that is still left
that means its original degree was even and that even quantity is retained as a degree in my
graph Gk as well. Whereas if you have traversed some of the partial edges incident with node
v in the set S that means if its original degree was some 2 times p, and if you have traversed
some of the edges that may say the vertex v has occurred q number of times that means you
715
have taken care of 2 q degree that means the leftover degree will be 2 p - 2 q which is still an
even quantity. So, that ensures that each vertex in my graph S still have even degree; whereas
for every vertex vk, the vertex vk with which I have stopped my tour it is not a member of S.
Because since I have ended my tour with the vertex vk and ending condition or the
terminating condition was that there are no more edges left incident with the vertex vk.
So, I can say that the vertex vk is indeed a member of the set V - S. So, now the proof by
contradiction here basically would like to derive the following : we would like to derive the
fact that if at all we have not obtained Euler circuit by running Fleury’s algorithm, that means
at some point during some iteration in the algorithm we have not followed the Fluery’s rule
namely there must have been some intermediate iteration where we would have traversed cut
edge rather than traversing a non cut edge - that is a contradiction we have to arrive at.
So, how do we arrive at that contradiction? So, let vp be the last vertex from the set S which
appears in your output tour Wk. So, pictorially imagine that this path that is indicated by this
dotted arrow that is a tour T obtained by your Fluery’s algorithm. And since we have
terminated in the kth iteration this is my graph Gk. And what I am saying here is there must
be some vertex vp which is there in your set S. That means there are still some untraversed
edges incident with the vertex vp and vertex vp is the last vertex; last vertex in the sense that
there might be multiple vertices from the set S which could have occurred along your tour
Wk. Among all those vertices from the set S which has occurred in your tour Wk I am
focusing on the last vertex which has appeared I am then calling that vertex as vp.
So, first of all you might be wondering that what is the guarantee that such a vertex vp is
there? Well if the vertex vp is not there that means the unexplored part of the graph which is
not yet covered is completely separate or not at all have any overlap with your tour Wk; that
means your original graph is a disconnected graph. But I am assuming that my original graph
is a connected graph that means there must be some overlap between the uncovered portion
and the output tour which you have obtained as part of your Fluery’s algorithm.
And I am focusing on the last overlap here; overlap in terms of the vertex. So, the last overlap
I am calling it as the vertex vp. And since there are still some edges incident with my vertex
vp in my graph Gk when I have terminated my algorithm, I denote by H the connected
716
component in my graph Gk containing the vertex vp; that means whatever is the unexplored
portion left in my graph and Gk incident with the vertex vp.
So, the vertex of vp + 1 was selected during the p + 1th iteration and definitely the vertex vp + 1
belongs to the set V - S that means in my final output when I am considering the graph Gk
there would not be any more edges left which are not yet traversed and still left and incident
with vp + 1 because vp + 1 is not a member of the set S it belongs to the set V - S because the
last appearance of a node from the set S is vp not vp + 1 that means all the edges which are
incident with vp + 1 would have been traversed as part of the tour Wk.
This implies that the edge vp and vp + 1 which you have selected during the pth iteration is a
cut edge in the graph instance Gp. Because if this edge vp and vp + 1 is not a cut edge that
means there is still a way to go to edge and then come back to the vertex vp + 1 then that
violates the assumption that vp is the last occurrence of a node from the set S which appears
along your tour Wk.
So that is why there is no way to go back to this unexplored portion H and come back which
implies that this H between vp and vp + 1 is actually a cut edge in your graph Gp. Now since
my vertex vp is a member of the set S and it has some untraversed edges left over in the graph
Gk that means it has at least one edge incident and that edge is a part of my sub graph H. So, I
am calling that vertex as vertex V so there might be still multiple edges incident with the
vertex vp left I am calling one of the edges as the edge v, vp.
And notice that H is connected because that is our definition of a connected component. And
we have already argued that whatever vertices that are there which have untraversed edges
left they still have even degree; that means every vertex in this connected component has an
even degree. So that means now you have an untraversed portion in the graph which is
connected and where every vertex has an even degree.
So, it is a very simple fact to prove which I am not proving here that if you have a connected
graph or a connected sub graph where you have every vertex of even degree. Then it would
not have any cut edge that means none of the cut edges in the graph H will be a cut edge. So
that shows that during the pth iteration you have an option of selecting a non cut edge namely
the edge between the vertex v and vertex vp this edge.
717
That edge was still there to traverse which was a non cut edge during the pth iteration but you
did not follow the Fleury’s instruction but rather followed the edge between the vertex vp and
vp + 1. And due to which you have leftover portion of the graph namely the portion H which
has not yet obtained as an output in your overall algorithm. So that means you have not
followed the Fleury’s algorithm that means you have given preference to a cut edge rather
than a non cut edge which is the contradiction. So that means if we follow the Fleury’s
algorithm systematically there is no possibility of leaving out any edge in the graph.
So that was the simple characterization of Euler circuit. Now let us quickly prove a
characterization of Euler path. So, the characterization of Euler path is that in your graph
there should be exactly 2 vertices of odd degree and remaining all vertices should have even
degree. So, the necessity condition again can be proved along similar lines as we did for the
characterization of Euler circuit. So, we want to prove that if at all you have an Euler path
then there are exactly 2 vertices of odd degree.
So, imagine your Euler path that is there in a graph is T which starts at the vertex a and ends
at the vertex b. So, it is easy to argue here that degree of and degree b will be both odd
because the first edge of the tour will be incident with the node a which will contribute 1 to
the degree of a and if a occurs as an intermediate node p number of times then that
contributes 2 times p to the overall degree of a.
718
So, hence the overall degree a will be 2 times p + 1 and same we can argue for the node b as
well. Whereas if you take any other intermediate node c different from a and b which is
occurring say k number of times in the tour then the overall degree of node c will be 2 times
k; so that proves the necessity the condition.
How do be prove the sufficiency condition? So, imagine that you have a connected multi-
graph where you have exactly 2 vertices of odd degree remaining all vertices of even degree
then I have to show that I can find out an Euler path; so imagine that 2 vertices which are
having odd degrees are a and b. So, what we do is we add dummy edge in my graph between
those special nodes a and b which have odd degrees.
And let G’ be the resultant graph. Now it is easy to see that the graph G’ all the nodes
including the node a as well as node b have even degree and that satisfies the characterization
of Euler circuit. So, I can run the Fleury’s algorithm on the modified graph G’ to obtain an
Euler circuit; I call it T. And now in that Euler circuit just remove the dummy edge imagine
or pretend thus edge is dummy edges are not there.
The resulting trail will be a simple trail which will have all edges of the original graph it
would not have the dummy edge that you have added. So, for instance suppose tour T is like
this you traversed and suppose you reached the vertex a and suppose as part of the tour that
you have obtained as an output of Fleury’s algorithm; this is a tour you started at a same
vertex you ended at same vertex and this tour has all the edges of the dummy graph.
719
So, you can imagine or you can extract out an Euler path of this term is tour as the following
you can imagine as your tour starts with b and then follow this tour and end at a and just
ignore this dummy edge that is all so that will be continue that has an Euler path.
So that brings me to the end of this lecture. These are the reference used for today’s lecture, I
also follow some of the notes from Prof. Choudum’s NPTEL lecture on graph theory
specially for the proof of correctness of Fleury’s algorithm. So, just to summarize in this
lecture we saw the definition of Euler circuit, Euler path and we proved the necessary and
sufficient condition for the existence of Euler circuit and Euler path in the graph, thank you.
720
Lecture – 51
Hamiltonian Circuit
Hello everyone welcome to this lecture and the plan for this lecture is as follows. So, in this
lecture we will discuss about Hamiltonian circuit. And we will discuss about some sufficiency
conditions for the existence of Hamiltonian circuit in a graph namely the Dirac’s theorem and
Ore’s theorem.
721
So, what is a Hamilton Circuits and Hamilton Paths? So, on a very high level it is a version of
travelling salesman problem in incomplete graphs. So, specifically a Hamiltonian circuit is a
simple circuit. So, what do we mean by a simple circuit? It is a tour which starts and ends at the
same vertex and all the edges are distinct. But it is a special type of simple circuit in the sense
that the vertices are not allowed to be repeated.
And every vertex of the graph occurs exactly once that means no vertex of your graph is missed
it will appear definitely that does not mean that all the edges of the graph are covered. So, this is
different from your Euler circuit. The Euler circuit, the requirement first at all the edges should
be covered as part of your tour. Here the requirement is that all the vertices should be covered.
And no vertex should be repeated.
Whereas Hamiltonian path is a simple path that means it may not start and end at the same
vertex. And it should cover exactly once each vertex of the graph. We call a graph as a
Hamiltonian graph if it has at least one Hamiltonian cycle. So, for instance if I consider the first
graph here this, this graph has a Hamiltonian circuit because if I make a tour like a to d, d to e, e
to c, c to b and b to a then it covers all the vertices.
So, now you can see that this edge not there as part of the tour that is fine; the requirement is that
you should start and end at the same vertex and traverse each vertex of the graph exactly once.
722
Whereas this graph, so this is the first graph it has an Hamiltonian circuit whereas the second
graph it does not have a Hamiltonian circuit, why? Suppose I start my tour at a go to b and go to
d and go to c and then go to b but I cannot repeat the edge between a to b because that is already
traversed.
So that means it will be violating my requirement of a simple circuit. So, edge will be repeated,
so I will start my tour at a and my tour at b but I have covered all the vertices but my starting
point and end points are different and hence this is an Hamiltonian path and not an Hamiltonian
circuit.
So, now the next interesting question will be is there a necessary and sufficient condition through
which we can check whether a given graph is Hamiltonian or not. Unfortunately like Eulerian
graphs where we have a single condition which was both necessary as well as sufficient. So, just
to recall we had the condition that all the vertices of your graph should have even degree and that
was both necessary as well as sufficient for the existence of an Eulerian circuits.
But that is not the case for Hamiltonian graph we do not have a single condition which is
simultaneously necessary as well as sufficient. And we do have separate conditions which are
either necessary but not sufficient or sufficient but not necessary. So, what we will discuss; we
723
will discuss in this lecture 2 important sufficient conditions for the existence of Hamiltonian
graph. I stress that those 2 conditions are not necessary conditions.
So, both those interesting sufficient conditions are based on the following intuition. The intuition
is that if your graph is such that it has sufficiently large number of edges and those edges are
uniformly distributed among the nodes, so it is a very vague term uniformly distributed. So,
imagine it is uniformly distributed among the nodes and the graph is very dense. Then we can
argue about the existence of the Hamiltonian circuit.
So, let us see what do we mean by uniformly distributed in 2 different contexts. So, the first
sufficiency condition is what we call us Dirac’s theorem. So, it says that if you have a connected
𝑛
graph where the degree of every vertex is at least 2 , I stress for every vertex then it is guaranteed
that your graph is Hamiltonian. However you can quickly verify that this condition is not
necessary that means you may have a graph which is Hamiltonian where even though the degree
𝑛
of all the vertices is not 2 .
A very simple graph could be your cycle graph which is Hamiltonian where the degree of every
𝑛
vertex is 2 which need not be . So, the Dirac’s theorem says that if your graph is sufficiently
2
𝑛
dense in the sense that degree of every vertex is then the graph is Hamiltonian. Whereas
2
another related sufficiency condition is Ore’s condition which says that if your graph is such that
for every pair of non adjacent vertices u and v the summation of the degree of u and v is at least
n then your graph is a Hamiltonian graph.
So, again what Ore’s condition says is that you take every pair of non adjacent edges and some
of their degrees it should be at least n. So, if you compare the Dirac’s condition and Ore’s
condition then it is easy to see that if Dirac’s condition holds in your graph that means if the
𝑛
degree of every vertex in the graph is at least then you take any pair of non adjacent vertices u,
2
v and some of their degrees it will be at least n.
724
So that means if Dirac’s condition is ensured in your graph then that also ensures that Ore’s
condition is also ensured in the graph but the other way around may not be true. You may have a
pair of vertices u and v, say which are non adjacent and say where the degree of u is n - 1 and say
where the degree of v is 1. So, in total the sum of their degrees is greater than equal to n. But you
will see that u is taking the bulk of the degree, whereas v is taking a very small degree namely
one but Ore’s condition says that even in this case the graph is Hamiltonian. So, in that sense
Ore’s condition is more flexible it does not put too much restriction in terms of degrees on the
vertices of the graph. It says that you take any pair of non adjacent vertices as long as you
guarantee that the summation of their degrees is n it does not matter how exactly n is distributed
as degrees across the degree of u and v.
𝑛 𝑛 2 1
It could be the case that both of them are 2 , or u is taking say 3 of n and v is taking 3 of n and
2
so on, still my graph will be Hamiltonian. Whereas Dirac’s condition is slightly stringent in the
𝑛
sense that demands that every vertex should have degree and then only I can argue that my
2
graph is Hamiltonian. I stress here that none of these 2 conditions is a necessary condition
because as I said if you take the cycle graph. Then the graph is Hamiltonian, you take cycle
graph of any number of vertices it will be Hamiltonian graph but neither the Dirac’s condition
holds not the Ore’s condition holds.
725
So, what we will prove here we will just discuss a very high level overview of the proof of Ore’s
theorem. So, what we want to argue here is the following you are given a graph which has at
least 3 vertices because then only it makes that to talk about the summation of degrees of non
adjacent pair of vertices. So, it is given that you take any pair of vertices u and v which are non
adjacent the sum of their degrees is at least n.
Then I want to argue that my graph is indeed Hamiltonian. And the proof will be by
contrapositive that means we would not give you a concrete algorithm, by running which you
can find out your Hamiltonian circuit or Hamiltonian tour but rather we will argue logically that
indeed this implication is true. That means we are trying to give a non constructive proof where
we are arguing proof by contrapositive. So, what will be the proof by contrapositive? So, we will
argue that if your graph G is not Hamiltonian then Ore’s condition is false. Now what does it
mean? When I see all Ore’s condition is false. So, Ore’s condition is a universally quantified
statement. It says that for all u, v which are non adjacent that summation of degree of u and v is
greater than or equal to n.
So, when I say that negation of this Ore’s condition because that is what will be the ‘q’ part in
the proof by contrapositive. So, the negation of the Ore’s condition is that there exists, at least
one pair of u and v for which the Ore’s condition is not satisfied. So that is what we will argue
here; if the graph is non Hamiltonian then there exists at least one pair of non adjacent vertices u
and v for which the Ore’s condition is false.
I am not going to argue that for every u, v pair in my graph the Ore’s condition is false because
that is not what we mean by the negation of the Ore’s condition. And proof is very clever here.
So, we will first transform the graph G to another graph H. And intuitively what is this graph H:
it is the maximal non Hamiltonian super graph of G. Namely this graph H will have my original
graph G and it might have some additional edges as well we will see how those additional edges
are added.
It may be the case that your graph H is the same as the graph G itself but it may not be the case.
So, if that is not the case then that means I have expanded my graph G and got a new graph H
726
and the new graph H will be still non Hamiltonian and it will be maximal we will see soon what
exactly I mean by maximal here. So, the way I construct my graph H from the graph G is the
following I keep on successively joining non adjacent pair of vertices in my graph G that means I
take my graph G.
So, imagine this is your graph G, I randomly choose a non adjacent pair of vertices in the graph
G. So, u and v and I check whether by adding the edge between the node u and v I get a
Hamiltonian cycle in the graph or not. If by adding the edge between u and v in my graph G I
still do not obtain any Hamiltonian cycle in the graph then I will add that edge, that dummy edge,
and then I keep on repeating this process.
And keep on adding more and more edges by identifying non adjacent pair of vertices and
checking whether by adding an edge between those 2 non adjacent vertices whether I get an
Hamiltonian cycle or not. If I do not get then again add that edge and keep on bombarding more
and more edges keep on adding more and more edges in the graph G till you reach a saturation
point, saturation point in the sense that you reach a point, where you identify a pair of vertices u
and v which are non adjacent such that if in the super graph H which you have obtained till now
by keep on adding more and more edges you add this new edge between the vertex u and v you
get an Hamiltonian cycle. If you reach that point that means that is the saturation state and you
should stop that means you should not now add the edge u and v.
That means you have now identified a critical pair of vertices u and v, critical in the sense that
now if you add the edge between these non adjacent vertices u and v in the graph H you will get
a Hamiltonian cycle you stop at that point. And now the proof of this proof by contrapositive will
be focusing on this u and this v that means this non adjacent u non adjacent v and we will argue
that with respect to this specific u and v the Ore’s condition is false.
That means if I sum up the degrees of u and v it would not be greater than equal to n. So that is
the proof idea. So, the proof basically tries to identify a critical pair of non adjacent vertices u
and v such that by adding the edge between u and v in the super graph you are bound to get a
Hamiltonian cycle. So, in that sense my graph H maximal that means I cannot add more edges in
727
the graph edge beyond this point because I have the guarantee of existence of at least one non
adjacent pair of critical vertices.
So, since I have now identified my node u and v which are the critical vertices what I can now
say is the following. I can say that in my super graph H, I do have a Hamiltonian path. That
Hamiltonian path will start with the node u and it will end with the vertex v and it will cover all
the vertices of the graph. Why it is a Hamiltonian Path? Because of the fact that if I would have
added the edge between the nodes u and v in my graph H then that edge if I traverse or include in
my path P that would have given me a Hamiltonian cycle.
So, because of that fact I can conclude that there exists a Hamiltonian path in my graph H
starting with u and ending with v which is simple and which covers all the vertices of the graph.
So that is why u1 to un. Now here comes a very crucial claim. We claim here that if you forget
about the starting point of the tour namely u or u1 and then focus on the remaining vertices
namely the vertex 2, vertex 3, vertex 4, vertex n which occurs in your Hamiltonian Path P then
for each such vertex uk you cannot have simultaneously the edges (u1, uk) and edge (uk – 1, un)
existing in your super graph H.
So, remember this claim is with respect to the super graph, namely the graph which you have
obtained by keep on adding or by expanding your original graph G, this claim is not about your
original graph G; this claim is about the super graph H. So, pictorially what I am arguing here is
so this is your Hamiltonian Path P. So, let me draw it in a better way. So you started with u1 then
u went to u2, u3 and like that uk -1, uk, un - 1, un where un is v and u1 is u and all the vertices are
bound to appear here.
So, my claim is if you take any k so for instance if I take k = 2 then the claim is that you cannot
have simultaneously the edges (u1, u2). So, definitely (u1, u2) is there in your super graph H
because that is why it is a part of your Hamiltonian Path P. So, the claim says that since you have
(u1, u2) in the graph you cannot have the edge (un, u1) in the graph. Because if I substitute k = 2
here it says that you cannot have simultaneously (u1, u2) as well as (u1, un).
728
So, since you have the edge between u1 and u2 you cannot have the edge between un to u1.
Similarly the claim says that either you can have the edge between u1 and u3 in the graph H or
you can have the edge between u2 and un that is the claim here. So, let us prove this claim the
proof is very simple and elegant. So, again the proof is by contradiction so imagine there is some
intermediate k such that you simultaneously have the edge between uk - 1 and un that means this
edge is there in the graph.
As well as you have the edge between u1 and uk in the graph H again I stress that the claim is
with respect to the super graph H. So, imagine both these edges are there then I can argue that
actually from this Hamiltonian Path P I can extract out a Hamiltonian circuit in my super graph
H itself which is a contradiction because as per my construction the super graph H is still non
Hamiltonian.
So, how can I extract out a Hamiltonian circuit, it will be just like doing a crossover. So that
means my extracted Hamiltonian circuit will be that you go from u1 to u2, you go from u2 to u3
you go all the way to uk - 1 as per the Hamiltonian path but in the Hamiltonian path after uk - 1 you
have traversed to uk. But what I could have done actually is the following: if the edge between uk
-1 and un is there I could have followed this edge.
And then follow the rest of the tour as per the Hamiltonian path. That means go from un to un - 1
and un - 1 to un - 2 come back all the way to uk. And since you have also the edge between uk to u1
because that is what I will obtain by assuming a contrary of this claim statement. I could follow
this edge and end my tour at u itself and I would have covered all the vertices.
So that means the proof of the claim is based on the fact that if you have both these edges then
by just crossing those 2 edges and following the rest of the things as part of your Hamiltonian
path you can extract out a Hamiltonian circuit in your super graph H which is a contradiction.
So, we have proved this claim what exactly this claim says?
729
The claim says the following if I bunch or if I iterate over k = 2 to n and then for pair of vertices
so for k = 2 the claim says that between u1 and u2 you can have the edge or between u1 and un
you can have the edge you cannot have both the edges u1, u2 as well as u1, un that is not possible.
At most one of these 2 edges is there in the graph H. The claim statement for k = 3 says that
either you can have the H between u1 and u3 or you can have the edge between u2 and un but you
cannot have both these 2 edges simultaneously.
And similarly the claim for k = n implies that you can have either the H between u1 and un or an
H between un-1 to un you cannot have both the edges simultaneously. Now that gives you the
implication that remember my u = u1 and un = v. That gives me the implication that in the super
graph H the summation of the degrees of the vertices u and v is at most n - 1 why at most n - 1?
Because I have n - 1 pairs here.
And in each pair I get only one edge guaranteed to be there in my super graph; in none of the
pairs I cannot have both the pairs of edges present in my super graph H. So, for instance I know
definitely the edge between u1 to u2 is there because that is why the Hamiltonian path P there that
means the edge between u1 and un is not there. And similarly I know that the edge between un
and un - 1 is there.
730
Because that is the part of the Hamiltonian path; that means the edge between u1 and un is not
there. So, the way I have collected up the pair of vertices here I am counting the degrees of u1.
And the degrees of un, the degree of u1, the degree of un, the degree of u1, the degree of un and
through each pair I either get one to the degree of u1 or one to the degree of un through none of
the pairs I can get simultaneously a degree for u1 as well as a degree for un.
And how many such pairs are there: I have n - 1 such pairs. And as a result if even if I take the
best case that means through each pair, remember the claim says that is it is not the case that
from each pair definitely an edge is there in the super graph H; it says at most one edge. So, even
if I take the best case for the graph H and through each pair I extract out or if my graph H is such
that through each pair I get a guarantee that exactly one edge is there in my super graph H.
I can at most get a summation of the degrees of u and v to be exactly n - 1. And this is the case in
my super graph H remember all the claims everything I made with respect to my super graph H.
Now if the claim is true for my super graph H the claim is obviously true in my sub graph G
because the nodes u and the nodes v were they are in my graph G I have not changed my vertex
set.
When I expanded my graph G I kept intact my vertex set it is only the edge set which I kept on
modifying that means I kept on adding edges between non adjacent pair of vertices till I identify
a critical pair of non adjacent vertices. And in the super graph H I have proved that the degree of
u and v if I take the summation of their degrees is at most n - 1. So, it is obviously the case that
even the summation of the degrees of u and v in my original graph will be at most n – 1, because
some of the edges between some of the edges incident on the node u and incident on the node v
in my super graph edge might be because of the expansion process those edges may not be
necessarily there in my original graph. So, since I have proved the statement for my super graph
that shows the statement is obviously true even for my base graph G. So that is the simple not
simple but that is our overall idea for the proof of Ore’s theorem.
731
So that brings me to the end of this lecture. So, these are the references used and just to
summarize: In this lecture we introduced a definition of Hamiltonian circuits and Hamiltonian
path and unlike Euler graphs, Euler circuits, Euler path where we are we have a single condition
which is both necessary and sufficient for the existence of Euler circuit and Euler path we do not
have a single condition which is both necessary as well as sufficient for the Hamiltonian circuit
and Hamiltonian path. So, we have seen 2 interesting sufficiency condition namely Dirac’s
condition and Ore’s condition. Thank you.
732
Lecture - 52
Vertex and Edge Colouring
Hello everyone, welcome to this lecture; the plan for this lecture is as following. We will
discuss about vertex colourings, vertex chromatic number and we will discuss about edge
colouring and edge chromatic number.
So, let us start with vertex colouring first and let us see some real world motivation for
studying the vertex colouring problem; the problem is that of exam time table scheduling. So,
there are n subjects in a college. Imagine there is a college with n subjects; multiple students
733
taking those subjects and we need to schedule the exams for those n subjects and we need to
schedule the exams in such a way that it should not happen that a student has 2 exams in the
same time slot appearing in the schedule.
So, for instance if I am a student and if I have taken subject number 1 as well as subject
number 2 then it should not happen that I get a schedule where subject number 1 and subject
number 2; their exams are scheduled on the same day and at the same time slot so that should
not happen. So, one obvious way of doing the scheduling is that you schedule one exam in
exactly one time slot and that will require you n time slots but we do not want to do that
because that will be an overkill and that might be a wastage of resources.
Instead we would require or we would be interested to find out the minimum number of time
slots where I can allow multiple exams to be conducted in the same time slot, but without
violating the condition that no student has two exams in the same time slot. It should not
happen that I schedule the exams in such a way that a student who has taken two subjects and
both the subjects appear in the same time slot in my schedule that should not happen. So, how
do we model this problem as a graph theoretic problem? So, the n subjects will form the n
nodes of my graph.
And what will be the edge set so I will add an edge between node number i and node number
j which you can interpret as subject number i and subject number j. So, subject number i and
subject number j will have an edge among them if there is at least one student who has
registered both for the subject i as well as the subject j. If there is no student who has taken
both subject i as well as subject j that means I can interpret or treat subject i and subject j as
kind of independent subjects.
And I can conduct the exam for both subjects number i as well as subject number j in the
same time slot. But if there is an edge between subject number i and subject number j that
means I cannot schedule the exam for subject number i as well as the subject number j in the
same time slot. So, how do I model that requirement? So, basically I will be interested to
colour the nodes of the graph by various colours.
And my colouring should satisfy the condition that no two adjacent nodes should get the
same colour. And the number of time slots or minimum number of time slots I require is
734
same as the minimum number of colours needed to colour the vertices. Of course a trivial
way to colour the vertices will you take n distinct colours and assign one distinct colour to
each of the n vertices. I do not want to do that but because that is equivalent to saying that I
conduct exams for the n subjects taking n slots.
I might like to assign the same colour to multiple vertices provided this condition is satisfied.
And the minimum number of colours that I need to colour the vertices will give me the
minimum number of time slots so for instance if this graph is given to me. So, I have subject
number s1 here, s2 to here, s3, s4, s5, s6, and s7 I have seven subjects. So, I can take 7 time slot
and in each time slot I can schedule 1 exam I do not want to do that.
It turns out that just with 4 slots I can finish off all the 7 exams namely subject number 7 and
subject number 4 can be scheduled together because there is no student who has taken
simultaneously the subject number 7 and subject number 4. In the same way, subject number
5 and subject number 3 can be scheduled together because there is no student who has taken
both the subjects and so on.
So, now coming to the vertex colouring problem what is the input? The input here will be a
simple graph it may or may not be connected and the output is basically an assignment of a
colour to each vertex such that no two adjacent vertices are assigned the same colour, What
is the vertex chromatic number of a graph? So we denote the vertex chromatic number of a
graph by this quantity χ0(G) this is a Greek character.
735
If you do Latex then this is the character \chi. It is pronounced as “khi” not as “chi”. So, it is
what is the vertex colour chromatic number of a graph. The vertex chromatic number is the
minimum number of colours needed to colour the vertices of the graph such that no two
adjacent vertices are assigned the same colour. It turns out that finding the chromatic number
of a graph is indeed a hard problem. Hard problem in the sense we do not have in general
efficient algorithms or practical algorithms for finding out the vertex chromatic number of a
graph, if the number of vertices n in the graph is arbitrarily large or very large. Of course I
can run exponential time algorithms, exponential in the number of vertices, and get vertex
chromatic number. What I mean by hard problem is basically we do not have efficient
algorithm, that is a very loose definition of a hard problem. So, what is an upper bound on the
vertex chromatic number?
It turns out that the vertex chromatic number of a graph is always upper bounded by 1 + the
maximum degree of any vertex in your graph. You do not require more than these many
number of colours in your graph to colour the vertices of the graph and to check whether
indeed this bound is true or not just take the case of a complete graph of n vertices where the
degrees of all the vertices are same.
And where Δ(G) namely the maximum degree is n - 1. So, for colouring all the n vertices in a
complete graph we will need indeed n colours because there is an edge between every pair of
nodes in the graph and that satisfies this upper bound. So, what we will do is we will give an
algorithm which will indeed need at most these many number of colours to colour all the
vertices of the graph. But that need not be the optimal colouring because it might be possible
that your graph may not need Δ(G) + 1 number of colours. So, in that sense my algorithm
need not be an optimal algorithm.
736
So, this algorithm is based on the greedy strategy which is a very popular strategy in
algorithm designs. So, what exactly is the greedy strategy here: the greedy strategy is that you
use the first available colour at every step if possible, if not possible then use a new colour.
So, more specifically the algorithm is an iterative algorithm and in each iteration we will pick
a new vertex for colouring.
And when we are picking the new vertex for colouring we have to follow a greedy strategy to
decide whether we can use a new colour for colouring the next vertex or whether you should
use an existing colour which you might have already assigned or given to some of the
existing vertices depending upon some conditions. So, to be more specific we do the
following repeat while loop so till we have uncoloured vertices left in the graph we do the
following.
We arbitrarily choose any uncoloured vertex which is not yet assigned any colour. So, there
might be several uncoloured vertices still left in your graph I arbitrarily choose one of them.
And let T be the set of colours which I have used till now for colouring the various vertices in
the graph. So, to begin with my T will be empty and all the n vertices are uncoloured. So, I
will start with any of the vertex and my T will be empty.
But in general as my algorithm proceeds my T will keep on taking new values depending
upon what colours I have chosen for various vertices. So, imagine I am at my current iteration
where I have decided to colour the vertex number vi because it has not been assigned any
colour. Now what I will do is I will check that is it possible to use one of the existing colours
737
from the set T and assign it to the vertex vi in the sense that there is no vertex incident with
the vertex vi or which is adjacent to the vertex vi to be more specific, and that has been
assigned the same colour which I am considering from the set T. That means your set T
might have already taken the colour c1 to ck. So, you have already used k number of colours
and then you will like to check whether it is possible to assign the colour number c1 to the
vertex vi or not without violating the vertex colouring requirement if not then check for c2 if
not then check for c3 and so on.
So, if at all you stuck or you find some colour which can be assigned to the vertex vi and if
there are multiple colours from the set T which can be assigned to the vertex vi without
violating the vertex colouring requirement then assign the least index colour from the set T to
the vertex vi. If not then pick a new colour which is different from all the colours c1 to ck and
assign it to the vertex vi and add that colour ck+1 to the set of colours T and repeat this
process that is the idea here.
So, now the question is will this algorithm always give the optimal colouring and what do I
mean by optimal colouring? By optimal colouring I mean the minimum number of colours
that is indeed required to colour all the vertices of my graph namely the number of colours is
exactly the vertex chromatic number. So, it turns out that this algorithm may not always give
you the optimal colouring because it depends upon the sequence in which you pick the
uncoloured vertices in every iteration.
So, remember in each iteration we are arbitrarily picking one vertex from my current set of
uncoloured vertices and then deciding to colour it. Now depending upon in what order you
pick it you may or may not get an optimal colouring.
738
So, let me demonstrate my point here so imagine this is a graph given to you and we want to
assign colours to the vertices by following this algorithm. So, if I follow the vertex ordering
{v1, v6, v3, v4, v2, v5} then I will need 4 colours why so? So suppose I start with the vertex
number 1 my set T is empty so I will assign colour number 1 then as per my sequencing
vertex v1 is done then I have 5 vertices left.
So, suppose I decide to colour vertex number 6. Vertex number 6 can be assigned one of the
colours from my set T. So, remember my set T has now taken colour number 1 so I can
assign colour number 1 to vertex number 6. So, vertex 6 is also done. Now I am left with 4
vertices my T has only 1 colour as of now. Then out of the 4 vertices which are not yet
coloured suppose I decide vertex number 3. Now vertex number 3 cannot be assigned colour
number 1 because colour number 1 has been assigned to vertex 6 which is adjacent to vertex
3.
So that is why now I have to use a new colour for vertex number 3; it is done. Now out of the
3 vertices as per my sequencing I am deciding to colour vertex 4. The vertex 4 cannot be
coloured with colour number 1 because colour 1 has been assigned to vertex 1 which is
adjacent to vertex 4 but colour 2 can be assigned to vertex 4 so I do not need a new colour,
vertex 4 done.
Now I am left with 2 vertices, vertex 2 and vertex 5. Suppose I choose vertex 2 then I cannot
use colour 1 I cannot use colour 2 so I have to add a new colour give it to vertex 2. And now
only vertex left is vertex 5 which cannot be given colour 1 which cannot be given colour 2
739
which cannot be given colour 3. So, the only option is to give a new colour namely colour
number 4. So, if this is the order in which I picked the vertices in every iteration I will need 4
colours.
On the other hand imagine that I choose the vertices in this order {v1, v2, v6, v3, v5, v4} then it
is easy to see that I will be needing 3 colours which is 1 less than the number of colours that I
used in the previous ordering, whereas, if I use this ordering {v1, v3, v5, v2, v4, v6} then I need
just 2 colours. And it is easy to see that 2 is indeed the vertex chromatic number of this graph.
Indeed you need 2 colours here because you have at least 1 edge. So, you cannot give the
same colour to the endpoints of an edge.
So, definitely 2 is the minimum number of colours needed to colour all the vertices of this
graph. And indeed I am giving you now colouring which requires 2 colours here. So, the
vertex chromatic number of this graph is 2. And I can get an optimal colouring from this
algorithm but this algorithm may also give me non optimal colouring namely it can give me a
colouring where I required 4 colours or this algorithm can also give me a colouring which
requires me to use 3 colours.
And this algorithm can also give me the optimal colouring. Now I do not know in what order
I should use or decide the next set of uncoloured vertices because if my graph has million
number of nodes to structure the pictorial representation of the graph may not be given to me.
In this example you can say it is having 6 vertices I can always follow this optimal strategy
but imagine that you are given an arbitrary graph which has a million number of nodes.
And you are not given a pictorial representation of the graph but just say that adjacency
matrix or adjacency list representation then in every iteration you have to just pick the edges
arbitrarily. And it may turn out that your ordering does not lead you to the optimal colouring.
What this algorithm guarantees is that you do not need more than maximum degree plus 1
number of edges why so? So, what is our maximum degree in this graph? The maximum
degree is 3 and this algorithm guarantees you that you do not need more than 4 colours. So,
let us argue that formally.
740
So, what this algorithm basically says is that the greedy approach is not always optimal. So
that is why a greedy approach is used very carefully in algorithm design. There are
algorithms which can be always solved and greedy approach can always give you optimal
solutions, but vertex colouring is not one such problem instance. The greedy approach may
not give you the optimal colouring here.
So, what we now want to argue here is that if we follow the greedy algorithm for vertex
colouring then even if you do not get optimal colouring at most the colouring that you obtain
will require you to use maximum degree plus 1 number of colours, You do not need more
than these many number of colours. And the proof is very simple. The proof is based on the
fact that when you are deciding a colour for a vertex vi.
741
Then there are 2 possible cases: either the vertex vi is the vertex which has the maximum
degree and say that all the neighbours of that vertex vi have been already coloured. So, in that
case it can have already Δ(G) colours used, in which case you have to use a new colour for
the vertex vi or it might be possible that you can use one of the existing colours from your
currently used set of colours T because remember that the colours of the neighbours of the
node vi can be used.
So, in any case it does not matter whether you are in this case or in this case you would need
more than Δ(G) + 1 number of colours. That is a simple proof for the fact that this greedy
algorithm will require at most these many number of colours.
So that is all about vertex colouring. Now let us see a related concept which is called as edge
colouring. And again let us first to see a motivation for studying this problem and then we
will discuss the general theory. So, the motivation here is how to schedule a round robin
tournament. So, what do we mean by that? Imagine you have n teams say n cricket teams
representing n countries and again for simplicity assume n to be even but that is not
necessary.
And in a round robin tournament each team has to play against each other. And then based on
the results we decide the semi final and then final. So, now we want to schedule the matches,
𝑛(𝑛−1)
so it is possible to schedule all the matches in 1 single day. So, you will be having 2
742
number of matches and you may schedule all the matches. In that case each team has to play
multiple matches but you do not want to put too much stress on the teams.
So, you want to schedule the matches in such a way that no team is forced to play more than
a single game on any day. And the goal is to come up with a schedule so that you finish all
the required matches satisfying this condition in minimum number of days; again you can do
𝑛(𝑛 −1)
the following: each day just schedule 1 match. And in that case you will require 2
number of days to schedule all the matches.
But that will be overkill because audience may not be so much patient, you might schedule
multiple matches in parallel and simultaneously satisfy this requirement. So, what will be the
graph theoretic formulation here so the input will be a complete graph with n nodes and
output will be an edge colouring with least number of colours and what is an edge colouring
here? We want to now colour the edges not the vertices.
And we want to colour the edges in such a way that if you have an edge ei and ej which are
adjacent, adjacent in the sense they are incident on a common node then they should get
different colours you cannot assign the same colour simultaneously to such edges ei and ej.
So, for instance if I take a complete graph of 4 nodes then I can do the colouring in the
following way. I can colour the edge between 1 and 4 and 2 and 3 with the same colour.
Because those 2 edges are not adjacent because their endpoints are completely distinct. I can
assign the same colour to the edges between 4 and 3 and 1 and 2 because the endpoints are
completely distinct. And I can assign the same colour to the edges between 1 and 3 and 4 and
2. So, I need 3 colours here and that corresponds to the fact that I can do the scheduling in 3
days: the first day we can schedule the match between team number 1 and 4 and team number
2 and 3 that would not force any of the teams to play more than 1 match.
On day number 2 you schedule the match between team 4 and 3 and team 1 and 2. Again that
would not force any team to play more than 1 match. And on the third day you schedule the
match between team 4 and 2 and team 1 and 3.
743
So, now coming to the general problem of edge colouring you are given a graph without
loops. And what we want here as an output? We want to output an assignment of colour to
the each edges of the graph so that no two adjacent edges and by adjacent edges I mean who
have common incident vertex. So, I need a colouring of the edges in such a way that no two
incident edges are assigned the same colour.
So, like vertex chromatic number we have a related concept called edge chromatic number
and this is denoted by χ1. So, χ0 was for vertex chromatic number and χ1 is for edge
chromatic number. So, what is edge chromatic number? Edge chromatic number of a graph is
the minimum number of colours needed to colour the edges of the graph satisfying this
condition that no two adjacent edges are assigned the same colour.
And again like vertex colouring, finding the edge chromatic number of an arbitrary graph
with large number of vertices is a hard problem, you do not have efficient algorithms or
practical time algorithms for finding the minimum number of colours. Of course you can do a
brute force and try to see whether 1 colour is sufficient, 2 colour is sufficiently, 3 colour is
sufficient, 4 colour is sufficient, 5 colour sufficient.
And then you will hit upon the right answer but that will require you enormous amount of
time so that is not an efficient algorithm. Now can we find a lower bound on the edge
chromatic number that means what can I say that definitively these many colours are indeed
required: it turns out that the lower bound is nothing but the maximum degree you take the
vertex which has the maximum degree say the vertex v has the maximum degree.
744
So, I have the vertex v and it has the maximum degree and how many edges are there? Δ(G)
number of edges are incident with the vertex v. So, definitely I need these many number of
colours to colour all the edges incident with the vertex v because none of these edges can be
assigned the same colours because all of them are incident with a common vertex namely v.
Definitely these many number of colours are required but I may need more than these many
colours as well.
And what can I say about upper bound so there is a very interesting theorem called as the
Gupta-Vizing theorem which says that if you have a simple graph then you do not need more
than these many number of colours: Δ(G) + 1. So, basically we get a range of values for edge
chromatic number the lower bound was the maximum degree and upper bound is 1 plus the
maximum degree.
Now finding the exact value is the hard problem. So, again if you want to verify whether this
theorem is true or not take the case of a triangle graph where you need 3 colours basically to
colour all the edges of the graph. So, due to interest of time I am not going into the exact
proof of the Gupta Vizing theorem because it is slightly advanced but if you are interested
you can refer to any standard reference.
745
So, these are the references for today’s lecture. And with that I conclude today's lecture; just
to summarize in this lecture we introduced the problems of vertex colouring and edge
colouring. We saw the notion of vertex chromatic number, edge chromatic number, we
discussed greedy algorithm for vertex colouring which may not give you the optimal
colouring always. And we discussed various bounds for vertex chromatic number and edge
chromatic number. Thank you.
746
Lecture - 53
Tutorial 9: Part I
Hello everyone, welcome to the first part of tutorial 9 so, let us start with question number 1.
So, in this question you are given 3 positive numbers l, m, n not be positive, non negative
integers. So, l, m, n such that l is less than equal to m and m is less than equal to n. And what
we want here is a simple graph where the vertex connectivity is l, edge connectivity is m and
minimum degree is n. So, remember the relationship between the vertex connectivity edge
connectivity, and the minimum degree is that: vertex connectivity is less than equal to edge
connectivity and edge connectivity is less than equal to the minimum degree in the graph.
Basically in this question we are asking you to give the construction of one simple graph
which satisfies the inequality with respect to the l, m, n values that are given to you. So, here
is how we can construct a graph. Since we need the minimum degree in the graph to be n, to
ensure that my resultant graph; my final graph has the minimum degree n, I take 2 copies of a
complete graph with n + 1 nodes. So, this is my copy number one and this is my copy number
two C1 and C2. Both of them are complete graphs with n nodes so I am not drawing the
edges within the complete graph. The whole graph I am denoting by this circle. Now I have
taken care of the minimum degree in my graph.
747
Now I have to take care of my vertex connectivity and edge connectivity. So, how do I do
that? I randomly pick l nodes and m nodes from the two copies. So, l nodes I pick from the
first copy and m nodes I pick from the second copy. Remember the values of l and m are
given to you and l and m are both less than equal to n. So, it is possible to pick l nodes from
the first copy. And it is possible to pick m nodes from the second copy.
Feel free to pick any l nodes from the first copy, m nodes from the second copy. So, I am
going to demonstrate assuming that l = 3 and m = 4. So I have picked 3 nodes arbitrarily
from the first copy. And I have picked 4 nodes arbitrarily from the second copy. Now I have
to take care; I have to ensure that my vertex connectivity should become l and edge
connectivity should become m.
So, I already have edges in these copies of complete graph which I have not highlighted here
but now I will add; I will give extra edges in my graph; those edges will be special edges and
these special edges will ensure that my vertex connectivity of the overall graph is l and the
edge connectivity of the overall graph is m. How do I do that? So, I add edges between the l
nodes which I have picked in the first copy and m nodes which I have picked in the second
copy in the way that it is ensured that: I add basically m edges between the l nodes and m
nodes that I have picked in the 2 copies respectively. And m edges are added in such a way
that those edges ensure that each of the l nodes and m nodes which I have picked in the 2
copies they occur as the end points of those edges which I am adding here. So, for
demonstration purpose m = 4. So, I am adding 4 edges apart from the edges which are
already there in my copy 1 and copy 2.
So, I am adding edge number 1, edge number, 2 edge number 3, edge number 4. And these
edges are added in such a way that if I call this vertices as v1, v2, v3 and u1, u2, u3 and u4.
Then if I take any vertex among v1, v2, v3 it is occurring as one of the endpoints out of these 4
edges. And in the same way if I take any of the vertices u1, u2, u3, u4 it is occurring as one of
the endpoints of these 4 edges. That is the way I am adding the edges.
So, if l = m or if l would have been equal to m what I would have done is I would have
picked l edges in first copy m edges in the second copy and just add distinct edges. That
means between the one node here and another node here I would have added 1 edge between
748
the second node in the first copy and the second node in the second copy I would have added
1 edge and between the third node of both the copies I would have added 1 edge.
But since my m could be more than l it might happen that some of the vertices out of the l
vertices which I have picked in the first copy are the endpoints of multiple edges or endpoints
of the multiple special edges. Now you can see that the way I have given these special edges
it is ensured that my vertex connectivity is l why vertex connectivity is l ? Because if I
delete the l vertices which I have picked or which are the endpoints of the special edges from
the first copy of the Kn + 1 graph sub graph then my entire graph get disconnected.
So, remember my entire graph is this whole graph which has 2 copies of the complete graph
of n + 1 nodes and these m special edges. So, if in this whole graph G, I removed the vertices
v1, v2, v3, v l then that will ensure that all these edges also vanishes. And that will ensure that
the first copy of Kn + 1 gets separated from the second copy of Kn + 1. So that takes care of
vertex connectivity being l.
And it is easy to see that the edge connectivity is m because the m edges which I have added
across the 2 copies of Kn + 1 that constitute the edge cut because if I remove all these m edges
again the 2 copies of Kn + 1 separates out. So that ensures that edge connectivity is m and as I
argued that since I have taken 2 copies of complete graph with n + 1 nodes which are a sub
graph of the entire graph the minimum degree in the graph is at least n so that is the
construction.
749
So, now let us go to question number 2. In question number 2 you are given an unknown
simple graph G, the graph G is not known to you known to you in the sense that you are just
given that it has 6 vertices but exact cardinality of edge set is not given. But it is given to you
that your graph G is such that if you delete the vertex v1 from the graph then you are left with
7 edges. If you delete the vertex v2 from the graph you are left with 7 edges you delete vertex
v3 from the graph you are left with 6 edges and so on.
So, if that is the case the question asks you to find out the cardinality of the edge set of the
original graph. Again do not try to do a brute force and try all possible graphs in your mind
and then hit upon the answer because that will take enormous amount of time. Instead we will
try to apply some rules of logic and properties of graph here. So, the property that we would
like to explore here is that if I take the graph G and if I have a vertex vi here.
If you have the vertex vi here and if you remove the vertex vi from the graph the cardinality
of the edge set gets decremented by the degree of vi because the deletion of the vertex vi will
delete how many edges from the graph? All the edges which are incident with the vertex vi
namely degree of vi number of edges from my edge set will be removed that means the
cardinality gets reduced by degree of vi. That is a simple fact that we are going to follow
here.
So that means what I can say is that my graph G is such that the edge set cardinality minus
the degree of v1 is 7 because it is given that after deleting vertex v1 in the graph you are left
with 7 edges. That means in the leftover graph which I obtain after deleting v1 if I would have
added the edges which were incident with vertex v1 and how many such edges would have
been there? : degree of v1 number of edges that would have given me the cardinality of the
edge set. In the same way from the 2nd fact I get this equation. And from the 3rd fact I get
this equation from the 4th fact I get this equation from the 5th fact I get this equation and the
6th fact I get this equation. Now if I sum all these 6 equations I get that 6 times the
cardinality of E minus the summation of the degree of 6 vertices in the graph is 36.
And now I can apply the handshaking theorem which says that if you take the summation of
the degrees of all the vertices in your original graph it is same as twice the number of edges.
So, now I have 1 equation just involving the unknown which is my cardinality of the edge
set. So, I get my edge sets cardinality to be 9.
750
In question 3 I want to draw a simple connected non complete graph with any number of
nodes where the vertex connectivity, edge connectivity and minimum degree are all same.
Again I think you can draw the graph from the answer from the graph which we constructed
in question number 1 but again let us do this question. So, why I am focusing on a non
complete graph here?
Because if I do not put that restriction then you can always give me the example of a
complete graph because in a complete graph with n nodes the vertex connectivity is n
because as per the definition of vertex connectivity the vertex connectivity of a complete
graph will be n - 1 because I cannot disconnect a complete graph the only thing I can do is
after deleting n - 1 nodes I am left with a graph with a single node.
And in the same way the edge connectivity is defined for a complete graph to be n - 1. And of
course the minimum degree of a complete graph is n - 1. ? So that is why complete graph is
always an example if I do not put this restriction of non complete graph. So, here is an
example of a non complete graph where vertex connectivity edge connectivity and minimum
degree are all 2 why 2?
So, you can see that my edge connectivity is 2 because if I remove this edge and this edge
then this portion of the graph gets disconnected from this portion of the graph. So, I need to
remove 2 edges if you just remove 1 edge then the graph does not get disconnected and due
to this I can say that if I delete this vertex and if I delete this vertex then again my graph gets
751
disconnected. So, my vertex connectivity is also 2 and the minimum degree is 2 because if
you take this vertex and this vertex then their degrees are 2 which is the minimum degree.
Let us go to question number 4. Here you are given 2 simple graphs G1 and G2 their vertex
sets are V1, E1, V2, E2 respectively there are n1 number of vertices in the first graph and m1
number of edges in the first graph. Whereas there are n2 and m2 number of vertices and edges
respectively in the second graph. Now I am defining a new operation on the graph which I
call as the Cartesian product of the graphs.
And it is possible to define the Cartesian product because remember the vertex set and edge
set they are sets, so I can always define Cartesian product of the sets. So, the way I define the
Cartesian product of the 2 graphs is the following the vertex set will be now ordered pairs
basically the vertex set here is the Cartesian product of the vertex set of the first graph and the
vertex set of the second graph because I am defining the Cartesian product of G1 and G2.
If I would have defined the Cartesian product of G2 and G1 then the ordered pairs would be
the first vertex from the second graph and the second vertex from the first graph that means
the vertex set would have been v2 x v1 but since I am defining the Cartesian product of graph
1 and 2 the vertex set is the Cartesian product of the first vertex set and the second vertex and
the edge set is E.
Now how the edges are defined here? So remember now my vertex set in the graph is ordered
pairs. So, I will say that 2 ordered pairs which represent 2 vertices they will be connected by
752
an edge if the following holds: either the first component of the 2 vertices should be same and
the second component of the 2 vertices should be an edge in the second graph under that
condition I could add an edge between these 2 ordered pairs or another condition in which I
can have an edge between these 2 ordered pairs is the following: the second component of the
2 vertices are same and the vertices which appear as the first component they have an edge
among them in the first graph. So, if any of these 2 conditions hold I will add an edge
between these 2 ordered pairs in my Cartesian product of the graph.
So, it might look slightly tricky so let me demonstrate with an example. So, imagine my
graph G1 is this which has 2 nodes and 1 edge so my n1 = 2 and m1 = 1 and I have a second
graph here which has 3 vertices and which has 2 edges. Now let us construct the Cartesian
product of the graph. So, the Cartesian product of the graph will be this I have not drawn it in
a very beautiful way.
But you can now see that the vertex set will be u1 paired with v1 that is one vertex, u1 paired
with v2 that is another vertex, u2 paired with v1, u2 paired with v2, u1 paired with v2, u1 paired
with v3 and so on. So, you will have 6 vertices here, so u1 paired with v1 is here, u1 paired
with v2 is here, and there is an edge between them. Why so? Because the first component u1,
u1 or same here and there is an edge between v1 and v2.
So that is why this ordered pair and this ordered pair you have an added an edge. In the same
way you can see that the second component here v1, v1 are same and there is an edge between
u1 and u2 in the first graph. So that is why this edge is added that is how we have built the
graph G1 x G2 here. So, now we want to prove that the cardinality of the edge set for the
Cartesian product of the graph is this value : namely 𝐸 = 𝑛1 · 𝑚2 + 𝑛2 · 𝑚1 .
And before going into the proof you can at least verify that this is actually the case for the
example graph that we have here. So, we have total 1, 2, 3, 4, 5, 6, 7 edges and your number
of vertices in the first graph is 2, number of vertices in the second graph is 3, the number of
edges in the first graph is 1, the number of edges in the second graph is 2 and you can check
that this is indeed the case.
But now we want to prove that cardinality of edge set is 𝐸 = 𝑛1 · 𝑚2 + 𝑛2 · 𝑚1 for a

general graph which is the Cartesian product of 2 graphs. So, for that what we are going to do
753
is the following we will first consider an arbitrary vertex in the Cartesian product of the graph
and try to argue what exactly will be the degree of that vertex. So, let us consider an arbitrary
vertex (u, v). My claim is that the degree of the vertex (u, v) in the Cartesian product of the
graph will be the summation of the degrees of the vertex u in the first graph, and the degree
of the vertex v in the second graph for that we observe here the following if I take this
ordered pair (u, v), to how many vertices it will be adjacent with? So, it will be adjacent with
2 categories of vertices: category 1 of vertices where the second component is v and the first
component is u’ such that (u, u’) constitutes an edge in the first graph that comes from the
definition of the edge set of the Cartesian product of the graph.
So, you have (u, v) here it will be adjacent to all (u1, v), (u2, v), (un, v) if u is adjacent or if it
is neighbor of u1 in the first graph, if it is a neighbor of u2 in the second graph and so on. That
is a category 1 type of neighbors for this vertex (u, v) and category 2 neighbors of this vertex
(u, v) will be all vertices of the form (u, v’) where the u component is same here. And the
second component v’ is actually neighbor of the component v in the second graph. So, these
are the 2 categories of vertices which will be adjacent to the node (u, v) in the Cartesian
product of the graph the first category will have these many number of nodes: deg 𝐺1 (𝑢),
second category will have these many number of nodes: deg 𝐺2 (𝑣) and that shows that this
will be the degree of any arbitrary vertex (u, v) in the Cartesian product.
So, I have written down this result here deg⁡

(𝑢, 𝑣) = deg 𝐺1 (𝑢) + deg 𝐺2 𝑣 which will be
useful now we will be applying the handshaking theorem on the Cartesian product and the
handshaking theorem says that if I sum over the degrees of all the vertices in the Cartesian
754
product that will give me the same value as twice the number of edges in the Cartesian
product of the graph. Now I can substitute the value of degree of (u, v) as per this formula.
And now it is easy to see that this term degree of u in the vertex G1 that will appear n2
number of times in this entire summation. Why n2 number of times? Because once it will
occur when this (u, v) would have taken the value (u, v1) again it will be encountered when
this (u, v) would have taken the value (u, v2) again this term will be encountered when I will
be considering (u,vn2) and so on.
So that is why the contribution of this term in the overall summation will be n2 number of
times. In the same way I can say that if I take this second term here in the overall summation
here this term degree of v in the second graph will be appearing n1 number of times; once it
will come when I am summing over (u1, v) again it will be coming next when I am summing
over (u2, v) and then finally it will be again coming when I will be summing over the vertex
(un1 , v).
So, based on this observation I can say the following that this overall summation can be
splitted down into these 2 individual summations. Since as I said that the contribution of this
term will be n2 times; it will be appearing n2 number of times I can take n2 outside in the
same way the contribution of the second term will be n1 number of times. So, I can take n1
outside and then individually the summations will be now over single vertices namely over
all the vertices in the first graph and all the vertices in the second graph.
755
And now I know that I can apply the handshaking lemma on the individual graphs G1 and G2
as well. So, if I take the summation of the degrees of all the vertices in the first graph I will
get 2 times m1 and in the same way if I take the summation of degrees of all the vertices in
the second graph I will get 2 times m2. And now if I cancel out 2 and 2 on both the sides I
will get the cardinality of the edge set which I claimed earlier.
The summation of the degrees of all the vertices v in the second graph is twice the number of
edges in the second graph which is 2 times m2 and 2 and 2 cancels out and hence I get my
desired result.
In question 5 we either want to prove or disprove the following. So, you are given a simple
graph G where G is the union of 2 graphs namely the graph F and graph H; your graph F and
H could be any arbitrary graph you have taken the union of those 2 arbitrary graphs and you
have obtained the graph G. Now we want to prove or disprove that vertex chromatic number
of the bigger graph G is always upper bounded by the summation of the vertex chromatic
numbers of the 2 sub graphs irrespective of what are the 2 sub graphs.
So, intuitively it might look that the theorem is true because we are taking two small sub
graphs F and H and combining them to get a bigger graph. And whatever is the number of
colours that I need for colouring the 2 small things I will not require more than the combined
number of colours to colour the bigger graph. That is an intuition you might get and you
might end up saying that this inequality is true.
756
But we will prove that this inequality need not be true by giving a counter example namely
we will give an example of a G. And an example of an F and H such that is inequality is not
true even though G is equal to the union of F and H. So, remember this is a universally
quantified statement because the claim is with respect to every simple graph. But the way I
can disprove a universally quantified statement to be true is by just giving 1 instance a
counter example for which violates that statement.
So, consider this complete graph with 6 nodes. And imagine that my G is the union of these 2
graphs. So, my F is a complete bipartite graph and H is now disconnected graph so you might
be wondering how exactly I have constructed this instance of F and H. What I have done
basically is I have taken this complete bipartite graph and I have put 3 vertices in one
collection and the remaining 3 vertices in another collection.
So, as per the property of bipartite graph I cannot have edges within the collection a, b, c.
And I cannot have the edges within the collection d, e, f. And since it is complete I need to
give the edge between every vertex in the first collection and the second collection. But if I
just take F then I am missing the edges within the first partition namely the edges involving a,
b, c.
So that is why I am taking this triangle. And similarly I am missing the edges d, e, f. That is
why I am taking the second triangle. And now I can say that if I take the union of F and H I
will get this complete graph with 6 nodes because in the complete graph of 6 nodes a will
have an edge between with b, c, d, e and f. So, a got all the edges involving d, e and f through
the bipartite graph and the missing edges it is getting to the triangle graph and so on.
Now what is the vertex chromatic number of this complete graph of 6 nodes? It is 6 I need 6
colours. But what is the vertex chromatic number of this complete bipartite graph F it is 2
because any bipartite graph can be coloured with 2 colours so do not get the impression that I
am colouring a, b, c with different colours in the graph F. So, I can colour all the vertices in
the partition v1 with colour number 1 and all the vertices in the partition number 2 with
colour 2.
So, I just need 2 colours for colouring all the vertices of the graph F. And for colouring the
graph H I need 3 colours. I can give C1, C2, C3 I can give C1, C2, C3. How many total colours
757
I need? Now for F and H 2 and 3 which is summing up to 5. So, now you can see that this
inequality is not true for this instance of G, F and H. So that means the statement is not
necessarily true.
Now based on this example I want to give a combinatorial proof for the following equality
but the proof should be based on graph theory. So, remember what is a combinatorial proof?
We give a counting argument and argue that expression in the left hand side and the
expression in the right hand side give the count for the same number of things. And we do not
do any kind of simplification or expansion and show that LHS and RHS are same.
What I am asking you here is that argument now should be based on some concepts in the
graph theory. So, this is the equality which we want to prove. So, again what I will do is the
following I can imagine that my left hand side expression is nothing but the number of edges
in a complete graph with n nodes. So, if I take a complete graph with n nodes then the
number of edges is nC2 (n choose 2). So that will be the interpretation of my left hand side.
Now I have to show that indeed the right hand side expression also counts the number of
edges in a complete graph. How do we argue that so the right hand side expression brings a
k
quantity k. So, you can imagine C2 as the number of edges in a complete graph with k
(n - k)
nodes. And the expression C2 you can interpret as the number of edges in a complete
graph with n - k nodes but I have to relate this k with it n somehow.
758
So, the way I can interpret the right hand side expression is that you have taken a complete
graph and you have divided into 2 sub graphs, sub graph 1 which has only k nodes out of
those and nodes and sub graph 2 which has the remaining n - k nodes I stress the k nodes
which you are taking in F are disjoint from the n - k nodes which you are taking in the
remaining complete graph.
Now whatever edges are there in the sub graph F they are definitely also present in your
graph G. And similarly whatever edges are present in the sub graph H they are also present in
the sub graph G. But there are still some edges which are missing; the some edges which are
still there in the graph G but they are not yet counted because you have now till now counted
only the edges which are there in F only the edges which are in H.
So, what you can do is if you imagine that there is an edge between every vertex in the sub
graph F and every vertex in the sub graph H that will take care of the missing edges. And
now if you include those edges as well in the edges which you have already counted that will
give you the total number of edges in your complete graph with n nodes. But now how many
edges I can have between every vertex in the set F and every vertex in the set H I will have k
into n - k number of nodes.
So that is why the summation of these 3 quantities can also be viewed as the total number of
nodes in a complete graph with n nodes and that shows that my RHS expression is same as
the LHS expression. So, with that I conclude the first part of tutorial 9. Thank you.
759
Lecture - 54
Tutorial 9: Part II
Hello, everyone, welcome to the second part of tutorial 9. So, let us start with question
number 7. So, here we first define what we call as the degree sequence of a graph and the
degree sequence of a graph is basically the sequence of degrees of the vertices in non
increasing order. So, you list down the highest degree vertex or the degree of the highest
vertex first followed by the next highest degree, followed by the next highest degree and so
on.
So, if you have n vertices, basically you are listing down the degrees of the n vertices in a non
increasing order. And we say a sequence of n values as a graphic sequence, if you can
construct a simple graph whose degree sequence is the given sequence, if you cannot draw
any simple graph whose degree sequence is a given sequence, then the given sequence will
not be called as a graphic sequence.
So I stress here that we need a graph only to be simple it need not be connected, it is fine if
the graph is not connected. So the first few parts of question 7 basically asks you to prove or
disprove which of the given sequences is a graphic sequence. So let us take the first sequence
5, 4, 3, 2, 1, 0. Of course, 1 obvious condition in a graphic sequence should be that values
760
should be non negative, you cannot have a vertex with a negative degree so that is a trivial
condition.
So in this case, we have to verify whether we can draw a simple graph with 6 nodes where
the highest degree is 5 and the smallest degree is 0. And it is easy to see that this sequence is
not a graphic sequence. Because you cannot have a simple graph with 6 nodes where the
maximum degree is 5 and the minimum degree is 0. Because if say v1 is the vertex which has
the maximum degree, so if its degree is 5, then it should be a neighbour of each of the
remaining 5 nodes.
That means each of the remaining 5 nodes will have a degree which is non 0, but you also
need a vertex with a degree 0 among those 6 nodes, which is not simultaneously possible. So,
now let us take the second sequence (6,5,4,3,2,1) and try to argue whether the sequence is a
graphic sequence or not. And again, this sequence is not a graphic sequence, but there are
several ways by which you can refute that this sequence is not a graphic sequence.
One simple way is that if you take the sum of the values that are given in this sequence is not
an even quantity, but we know that for any graph, it may not be a simple graph for any graph
the sum of the degrees of all the vertices is twice the number of edges which is an even
quantity. So, 1 obvious condition that should be satisfied by any graphic sequence is that if
you sum the values given in the sequence, it should be an even quantity, which is not the case
for the sequence given here. Let us consider the third sequence and the sequence is a graphic
sequence and this is a simple graph which realises or which has this degree sequence
(2,2,2,2,2,2).
761
So, now in question 8, we want to characterise that, we want to find out a characterization for
graphic sequences. So, if you are given a sequence with n values, how can you verify whether
that sequence is a graphic sequence or not we cannot keep on drawing all possible simple
graphs and then either prove or refute that a given sequence is not a graphic sequence, we
need an algorithmic characterization, a necessary and sufficient condition and that is given by
what we call as Havel-Hakimi theorem.
So here we are given the following you are given sequence S of n non negative integers in
non increasing order and you have a reduced sequence S*. It is reduced in the sense it has n -
1 values whereas the sequence S has n values. So how exactly we construct a sequence S*.
So, the way we construct S* from S is the following, we first to remove the value d1, and then
from the next d1 degrees or from the next d1 values in the sequence S we subtract 1 from each
of those d1 values in the sequence S.
So d2 gets decremented by 1, d3 gets decremented by 1, and d1+1 th term gets decremented by

1. Whereas, from d d1 + 2th term to the dn th term, the degrees remain the same as they were in
the sequences. So, that is the way we obtained the sequence S*. And what Havel-Hakimi
theorem says is the following it says that your sequence S is a graphic sequence if and only if
the reduced sequence S* when arranged in a non increasing order is also a graphic sequence.
So, for the moment imagine that this theorem is true, how exactly we can use this theorem to
verify whether a given sequence of S values is a graphic sequence or not? Well, we have to
reduce the sequence S and build a new sequence S* and then rearrange that terms in S*, so,
762
that the new degrees are in a non increasing order. And now, we have to verify whether the
reduced sequence S* is a graphic sequence or not. To do that, I can again apply the Havel-
Hakimi theorem.
Now, this reduced to sequence S* can be further reduced to n - 2 degrees, where I can remove
the first degree from S* and to compensate that I subtract 1 from the next few degrees. And
then the next reduced sequence again is arranged in a non increasing order and then we can
verify whether that sequence is a graphic sequence or not. And I can keep on repeating this
process; keep on decreasing my sequence till I obtain a very short sequence which I can very
easily verify whether it is a graphic sequence or not.
If it is a graphic sequence then I can come back all the way and declare that my big sequence
my original sequence S is a graphic sequence. Whereas if the reduced sequence or the small
sequence at which I stop and inspect and find out that it is not a graphic sequence, then I can
declare that my original sequence S also not a graphic sequence. So, that is a way I can apply
the Havel-Hakimi theorem to verify whether a given sequence is a graphic sequence or not.
So now, let us prove this theorem and this is an if and only if statement.
So, we have to prove 2 implications: let us first prove the easier one. So, we want to prove
that if S* when arranged in a non increasing order is graphic, then so is the sequence S, what
does this mean: so I will give a direct proof for this implication. And when I say I will give a
direct proof, I mean to say that I will assume that my premise is true and I will arrive that my
conclusion is also true so, assume that my premise is true.
That means, since my sequence S* as a graphic sequence, I can construct a graph, a simple
graph G* with n - 1 vertices and some edges whose degree sequence is the same as the
sequence S*, what does that mean? So, I can imagine that my vertex set V* has n - 1 nodes. I
call those nodes as v2, v3, vn. And since it realises the sequence S* that means, I have a vertex
of degree d2 – 1. Let v2 be that vertex.
I will have a vertex with the degree d3 – 1. Let v3 be that vertex and like that I will have a
vertex of degree this much. Let vd1 + 1 be the vertex with that much degree and like that I will
have a vertex of degree dn and let vn be the vertex with that degree. That is the implication of
763
assuming my premise to be true. Now, my graph G* is a simple graph remember, apart from
that, I do not know anything about G* whether it is connected or not connected and so on.
Now from G*, I have to build another graph G which has n nodes, which is simple and whose
degree sequence is the same as the sequence S, that is what is the implication. So the
construction of the graph G is very simple. I take a copy of G* as it is and since I have to give
a graph which has n nodes, but since I have taken the graph G* I have currently n - 1 nodes.
So, what I will do is I will now include a new node: call it v1 and I have to give some edges to
this vertex v1.
So, what I do is, I add the edge between the vertex v1 and the vertex v2 which has earlier
degree d2 – 1. I add an edge between the vertex v1 and the vertex v3 which had earlier the
degree d3 - 1 and similarly, I add the edge between the vertex v1 and vertex number d1 + 1
which had earlier the degree d1 + 1 and the remaining edges they remain as it is in the graph
G. Now, what can I say about the new degree for the vertex v2 it will be one more than what
it was earlier.
So, earlier the degree was d2 - 1, but now, since I have given a new edge to the node v2 its
degree will now become d2; similarly, the new degree of the vertex v3 will become one more
than it was earlier so, it will become d3 and like that degree of the d + 1th vertex will be one
more than what it was earlier. So, it will become this much and the degrees of the remaining
vertices will remain as it was earlier and what can I say about the degree of the vertex v1 : it
will be d1.
Because I have added d1 edges incident with the vertex v1 and now, you can see that this
sequence is nothing but the sequence S that means in the sequence S you need to have 1
vertex of degree d1. So, I have one such vertex namely v1. You need to have a vertex of
degree d2. I have one such vertex namely v2 you need to have a vertex of degree dn I have one
such vertex namely dn. So, I have now a simple graph whose degree sequence is same as the
sequence S. So, that shows that this implication is true.
764
Now, let us prove the implication in the reverse direction. So, I want to prove that if your
sequence S is graphic, then the reduced sequence S* when arranged in a non increasing order
is also graphic. And again I will give a direct proof; that means I will assume that my premise
is true and I will arrive at my conclusion. So, since my premise is true that means, I have a
simple graph call it G with n nodes.
And some edges whose degree sequence is same as the sequence S; that means, you have n
vertices say v1 to vn and let v1 be the vertex which has degree d1, v2 be the vertex which has
degree d2 and vn has the vertex which has the degree dn. Now, from this graph I have to arrive
at another graph, which is simple with n - 1 nodes and which realises the sequence S* : the
reduced sequence S*.
So, how do I do that, so, I will use now a proof by cases. So, once I assume my premise to be
true, I will do a proof by cases because there will be 2 cases which will be happening
depending upon what exactly is the structure of the graph G. So, your case 1 will be the
following imagine your simple graph G is such that the vertex v1 which has degree d1 is
adjacent to the vertex which has degree d2 it is adjacent to the vertex which has degree d3 it is
adjacent to the vertex v4 which has degree d4 and like that, it is adjacent to the vertex which
has degree d1 + 1. Suppose that is the case. Case 2 will be when this is not the case. So, case
1 is when v1 is adjacent to the vertex which has degree d2, it is adjacent to the vertex v3 which
has degree d3 and it is adjacent to the vertex which has degree d1 + 1. Now, let us see what
will happen if I delete this vertex v1 and the edges which are incident with the vertex v1
because if I delete the vertex v1 of course, these edges will no longer be there.
765
So, I will obtain now, a new graph G*, which will be of course, simple because my original
graph G was simple. So, I am not adding any edges I am deleting edges, so, by deleting
edges, I will still obtain a simple graph. So, my graph G* will be a simple graph and it will
have n - 1 nodes because I am reducing 1 vertex namely v1. Now, what can I say about the
new degrees of v2, v3 and vertex number d1 + 1 well the degree of v2 will be 1 less than what
it was earlier, because the edge between v2 and v1 has vanished. The degree of v3 will be 1
less than what it was earlier, because the edge between v3 and v1 has vanished and the degree
of the d + 1th vertex will be 1 less than what it was earlier, because the edge between the d +
1th vertex and vertex number v1 has vanished. The degrees of the remaining vertices will
remain as it was in the graph G.
So, now, what can you say about this sequence, I can say that this sequence is nothing but the
sequence S* in non increasing order, namely I can say that there is a graph, a simple graph
namely G*, which realises the sequence S* because in S* in order that S* is a graphic
sequence, you need a vertex of degree d2 - 1 in G* and you have one such vertex namely v2
you need 1 vertex of degree d3 - 1 in G* and you have one such vertex namely v3 and you
need 1 vertex of degree this much.
And you have a vertex in G* with that much degree you need a vertex of degree dn in G* and
you have a vertex whose degrees is dn. So, that means, now I can say that G* can realise the
sequence S* and hence my sequence S* is also graphic so, that is case 1.
766
Now, case 2 will be the following: case 2 occurs where in the graph G which realises your
sequence S the structure is as follows: there is at least 1 vertex vi in the set v2 to the d + 1th
vertex such that v1 is not adjacent to that vertex. So, what do I mean to say here is the
following in case 1 if you see the situation was that v1 was adjacent, so, v1 degree was d1 and
those d1 edges were contributed from the next d1 vertices namely the next d1 vertices which
has the degree d2, d3, d4 and d1 + 1 that was case 1.
In case 2 we are considering the case where this is not happening. That means, you have at
least 1 vertex vi outside this set v2 to vertex number d1 + 1 such that v1 is not adjacent to vi.
So, what do I mean by that for instance, imagine that your d1 is equal to say 4. In case 1, what
was happening is the following you need v1 to have 4 edges incident with v1 that means 4
edges should be incident with v1 that is why its degree was 4.
So, those 4 edges were between v1 and v2 where the degree of v2 was d2, it was between v1
and v3 where the degree of v3 was d3, it was between v1 and v4 where the degree of v4 is d4
and it was between v4 and v5 where the degree of v5 is d5 and of course the degrees are now
in non increasing order that was happening in case 1, but in case 2 what is happening is your
degree d1 is still 4.
But either the edge between v1 and v2 is missing or the edge between v1 and v3 is missing or
the edge between v1 and v4 is missing or the edge between v1 and v5 is missing where v2, v3,
v4 and v5 are the vertices with degree d2, d3, d4 and d5 in the graph G respectively. So now I
cannot run the same argument, which are used in case 1. In case 1, I simply deleted v1 due to
767
which all these edges which are there between v1 and vertex 2, vertex 3, vertex 4, vertex 5,
they vanished.
And the degrees of d2, d3, d4, d5 automatically got decremented by 1, I cannot run the same
argument here. Because, say for instance, if the edge between v1 and v2 is missing, then by
deleting v1, I cannot say that the degree of v2 gets decremented to d2 - 1, because v2 is not
adjacent to v1. Its degree will remain the same namely d2 or say for instance, the edge
between v1 and v3 is not there, then deleting v1 will not change the degree of vertex v3, it will
still remain d3 and so on.
So, I cannot run the same argument which I easily or conveniently used for case number 1, I
have to do something more to handle the case number 2, and by the way, these are the only 2
cases either case 1 could occur or case 2 could occur, there cannot be any third case possible.
So, the proof strategy here will be the following. What I will do is I will do some
transformation and we will see how exactly the transformation happens we will do some
transformation on the graph G and convert it into another simple graph H with n vertices and
with the same degree sequence S, that is important that means whatever what the
characteristics of G were they remain the same.
So, G was simple that transformed graph H also will be simple, G had n nodes, that
transformed graph H also will have n nodes, the number of edges in G will be the same as the
number of edges in H and G was simple H will be simple. G realised the degree sequences
768
and a transformed graph H also will realise the same degree sequences, but H will now have a
characteristic which was not there in the graph G.
So, in the graph G there was some node vi in this set. So, let me call this set as F so, there was
some node vi in the set F such that v1 was not adjacent to that node vi, but after
transformation what we will do is we will ensure that v1 is adjacent to each node in the set F,
that means, the degree d1 which was attributed to the vertex v1 is coming because of the edges
between the vertex v1, v2, vertex v1, v3, vertex v1, v4, vertex v1 and d + 1th vertex.
That means, what I can say now is that my transformed graph H is exactly having the same
structure as we had for the graph G in case 1 and now, I can apply the same argument that we
used for case 1. So, now, I will say that I will forget about the graph G I will say that now I
have a graph H, which is simple which has n nodes and which realises the degree sequence S
and where the vertex with the highest degree d1 is adjacent to the next immediate d1 vertices.
So, I can remove the vertex v1 and argue that because of the removal of the vertex v1 the
degree of the next to d1 vertexes will get decremented by 1 and that will be an instantiation or
realisation for the sequence S*. So, that is a proof idea. So, now, everything boils down to
how exactly we do the transformation. So, the transformation is as follows so, remember the
structure in the graph G is the following: there is at least 1 node vi in the set F such that the
edge between v1 and vi is missing.
And I also know that since the degree of the vertex v1 is d1 and edge between v1 and vi is
missing. So, to compensate this missing edge namely to ensure that the vertex v1 has the
degree d1 there must be some outside vertex and what do I mean by outside vertex namely
that vertex S not in the set F, but in the remaining n - d1 vertices. It is not among the first d1
vertices. So, this vertex vj is the outside vertex and there must be an edge between v1 and that
outside vertex vj because we have to take care of the fact that the degree of v1 is d1.
So again, for instance, what I am saying here is if d1 is 4. So in case 2, we know that either
the edge between v1 and v2 is missing, or the edge v1, v3 is missing, or the edge v1, v4 is
missing, or the edge v1, v5 is missing. But since I have to give degree 4 to the vertex v1, that
means v1 is adjacent to either vertex 6 or vertex 7 or vertex 8 and so on. So, that is the vertex
vj that is outside vertex vj in my current context.
769
And what I know is that in my graph G the vertex vi, its degree di is as large as the degree of
the vertex vj because that is the structure of my graph G. So, that means there must be some
neighbour of vi call it vk, which is not a neighbour of vj. Because if every neighbour of vi is
also a neighbour of vj and on top of that vj is a neighbour of v1. But vi is not a neighbour of
v1, we arrive at the conclusion that the degree dj is more than the degree di, which is not the
case.
So that is a very simple proof of the fact that there must be some neighbour namely vk, which
is there must be some neighbour vk of vi, which is not a neighbour of vj. So that is a structure
present in your graph. Now, what the transformation does is the following. Since the edge
between v1 and vi is missing in G, but after transformation, I want that edge to be present.
So, I add the edge but that will increment the degree vi or degree of vi, but I do not want to do
that. So to compensate this new edge, which I have given to vi, I take away the edge, which
was earlier present between vi and vk. So, that ensures that the degree of vi remains the same.
And I have to take away the edge between v1 and vj because since I am giving a new edge to
v1, the degree of v1 will get incremented, which I do not want to do.
So, to compensate that I take away the edge between v1 and vj, which was earlier there, but
that will reduce the degree of vj. again, which I do not want to do and to compensate that I
add the edge between vj and vk and this whole process, I am not disturbing the property that
my graph G or the transformed graph H is a simple graph. So, my transformed graph H still
remains a simple graph.
But by doing this transformation, what I have done is the following: earlier this vertex vi was
not immediately a neighbour of v1, but now in my transformed graph vi is a neighbour of v1.
So, I can repeatedly apply this transformation for all the outside vertices vi and after doing the
required number of transformation, I will get my graph H which will have the same structure
as in case 1 and then the proof becomes the same as it was in the case 1. So, that proves the
implication in the other direction.
770
In question number 9, we want to either prove or disprove whether the sequence is a graphic
sequence. So, there are 2 options: we can use either Havel-Hakimi theorem or we can use a
proof by induction to prove that this sequence is a graphic sequence, but we will give a
constructive proof to show that the sequence is graphic by showing a graph, a simple graph
with 2n nodes whose degree sequence is same as S.
So here are the vertices: 2n vertices and what I do is the following. I take the vertex v1 and
add the edge with all vertices with even indexes. I take the vertex v3 and I add an edge with
all even index vertices except the vertex v2 and I keep on doing this process and for the last
vertex with odd index, I will give only 1 edge namely an edge with the last vertex with even
index.
Now, what I can say about the degrees of the respective vertices here, so it is easy to see that
these 2 vertices will have degree n so indeed, I need 2 vertices of degree n. I will have this
vertex of degree n - 1 and this vertex of degree 2, so I got 1 vertex of degree n - 1 and 1
vertex of degree 2. And if I continue, I will find that I will get 2 vertices of degree 1 and then
eventually I will obtain the second vertex of degree n - 1 and so on. So the vertex here will be
of degree n - 1 and so on, that is a very simple construction to show that the sequence is a
graphic sequence.
771
Now let us come to question number 10. Here, we want to prove that if you are given a graph
with n vertices, and if you are doing edge colouring, then you cannot use a single colour to
𝑛
colour more than 2 edges. And it is a very simple fact, depending upon whether your n is odd
or even, we can prove this very easily. So let us take the case where n is even. So remember,
each edge has 2 endpoints.
𝑛
That means if I consider distinct edges of the graph, and if I focus on their endpoints, that
2
𝑛
will constitute the entire vertex set. So that means I cannot colour + 1 edges with the same
2
colour, because if I do that, then their endpoints will give me n + 2 vertices, but my graph at
𝑛
the first place has only n vertices. So at most I can colour edges, distinct edges with the
2
same colour, I cannot colour more than those many edges.
𝑛
Whereas if n is odd, then this quantity 2 is not well defined, it will not be an integer value. So
𝑛 𝑛−1
in that context of an odd value of n will be . And indeed, it is easy to see that I cannot
2 2
𝑛−1
use a single colour to colour more than number of distinct edges, because if I try to do
2
𝑛 −1
that, say for instance, I tried to colour + 1 number of distinct edges, then their endpoints
2
will give or constitute will n + 1 nodes, but my given graph has only n nodes. So that is a
maximum number of distinct edges, which can be coloured with a single colour.
772
Now, based on these information, I will try to solve question 11 where I want to find the edge
chromatic number of a complete graph with n nodes and my solution will be divided into 2
cases depending upon whether my n is odd or even. So, remember, from the previous
𝑛
question, I know that if your n is even then you can use 1 colour and colour at most edges,
2
𝑛
whether indeed you will be able to colour edges or not that depends upon the structure of
2
𝑛
your graph, but at max you can colour 2 edges using a single colour.
𝑛(𝑛−1)
Now in a complete graph, kn I have number of edges. So, with colour number 1, I can
2
𝑛 𝑛
take care of at most edges. With colour number 2, I can take care of another set of edges.
2 2
So, like that, how many colours I will require at least? So, I will require at least n - 1 number
𝑛
of colours, because to the first colour, I can take care of edges, the next colour I can take
2
𝑛 𝑛
care of another bunch of 2 edges and I have to take care of n - 1 such bunches of 2 edges.
So, that is why the minimum number of colours that will be required will be n - 1. Whereas if
I take the case when n is odd, then from my analysis of question 10, I know that through 1
𝑛 −1
colour I can take care of at most number of edges. And I have to take care of n bunches
2
𝑛 −1
of number of edges. So, that means I will require at least n colours if my n is odd.
2
Now, what I will show is I will show that these bounds on the edge chromatic numbers,
which were the lower bounds because they were the least number of colours which are
required, they are actually tight in the sense I will give you a constructive colouring, a
773
concrete colouring for colouring the edges of a complete graph with n nodes where n is even.
And where the number of colours required is exactly n - 1.
And you cannot beat this bound because the lower bound says you will need at least n - 1
colours. So that is why I am giving you an optimal colouring. So let me demonstrate the
colouring assuming the value of n = 8. So remember, edge colouring here corresponds to
scheduling of a round robin tournament. So we have 8 teams, and we have to schedule
matches among the teams.
And the requirement is that each team has to play against every other team once but at the
same time, we do not want to enforce a team to play more than a single match on any day. So
the way I do the colouring here is as follows. So on the first day, I keep v8 at centre and
engage v8 with v1 and engage v2 with 7, engage v3 with 6, engage v4 with 5. So, this is
equivalent to saying that this colour c1 is used to colour the edges between (v4, v5), (v3, v6),
(v2, v7) and (v1, v8).
That means, I have coloured the maximum number of edges using colour number 1 and now,
I have to use colour number 2 and using colour number 2, I will try to colour another set of 4
edges. So, which is equivalent to saying that I now want to find a schedule for the next day.
So, the schedule for the next day is obtained by kind of rotating this diagram by 30 degrees
and changing the assignment of or engagement of v8.
So, earlier v8 was engaged with v1, but now v8 will be engaged with the next team in the
clockwise direction which is v2. So, now, the assignment of the colours is the following. So, I
use the colour number c2 to colour these 4 edges, or equivalently I schedule these matches on
day number 2, then again I shift it by 30 degree and change the engagement of v8 . Now v8
will be engaged with v3 and so on.
So, now you can see that I have to do this rotation 7 number of times, and then I will be able
to colour all the remaining edges of my complete graph with n nodes: 8 nodes.
774
Now, let us see how exactly we can colour all the edges of a complete graph with n nodes
where n is odd. And I will be using exactly n colours, which is optimal because the lower
bound says that for n being odd, I need at least n colours. So, the idea here is I can convert the
complete graph with n nodes to a complete graph with n + 1 node by adding a new vertex and
the required dummy edges.
And since n was odd, n + 1 will be even. And I know a colouring mechanism to colour a
complete graph with n + 1 nodes where n + 1 is even using n colours, namely the colouring
that I had discussed just now. So, take that colouring and now you delete the dummy node
and the corresponding edges. That will give you the colouring for the original complete graph
with n nodes where n was odd.
So, for instance, what I am saying here is if you have only 7 teams, and you want to come up
with a schedule, you imagine that you have included a dummy team, say the 8th team and
now you want to come up with a round robin scheduled tournament for 8 teams with the
same restrictions that you had earlier. So, this will be the schedule you will require 7 days.
Now in the first day you can see that; on the first day you can see that v8 is engaged with v1.
So you can forget about that you can imagine that match is not going to be held; remaining
other matches will be held as per the colouring assignment namely v2 will play with v7, v3
will play with v6 and v4 will play with v5 . On the second day v8 is engaged with v2. So, you
can imagine that match will not be there actually and remaining 3 matches will be played and
775
so on. So, this now gives you a colouring; edge colouring for complete graph with n nodes
when n is odd.
Now, question 12 we are giving a greedy strategy for vertex colouring and we want to prove
that this strategy need not give you the optimal vertex colouring. So, the colouring strategy is
the following. We first sort the vertices according to their degrees and then use colour
number 1 to colour the vertex which has the highest degree that you have arranged as per
your degrees and then the next vertex in the list which is not adjacent to be v1, if at all it
exists and successively try to colour as many vertices as possible according to the colour
number 1, keeping in mind that the next vertex which you are selecting is selected according
to their degree. That means, you are following a greedy strategy and trying to occupy or
colour as many vertices with that colour and now do the same process with the next colour
and so on.
So now we have to give a counter example, namely a graph where Welch-Powell algorithm
will end up utilising more colour than the optimal number of colours. So consider this graph
and let us see how many colours we need. Actually we need 4 colours as per the Welsh-
Powell algorithm because this vertex has the highest degrees so I will colour it and then I can
assign the same colour to this vertex which has also the same degree.
And now I cannot use the same colour to colour any other vertex. Now, I will focus on the
next set of vertices which has the highest degree. So let us use this vertex, this vertex and
then the same colour I can assign to this vertex and this vertex. So that is the maximum
776
number of vertices which I can colour with the second colour. Now among the remaining
vertices I will pick the vertices which have according to their degrees.
So I can pick this vertex and the same colour can be assigned to this vertex. So we need total
3 colours; but optimal colouring is 2 this will require only 2 colours and 2 colours will be
sufficient to colour all the vertices in this graph so that shows this is not optimal colouring, so
with that I conclude this tutorial. Thank you.
777
Lecture - 55
Modular Arithmetic
Hello, everyone, welcome to this lecture. So, we will now shift our focus on number theory.
And number theory in itself is a very fascinating topic we can have a full course on number
theory. But we will be discussing only the relevant topics in number theory that is useful in
the context of computer science namely modular arithmetic, properties of prime numbers,
algorithm related to GCD and this will be very useful especially in cryptography.
So, the plan for this lecture is as follows. We will discuss about modular arithmetic and
various algorithms for doing modular arithmetic, especially modular exponentiation, which is
a very central or important operation in cryptographic algorithms.
778
So, let us begin with modular arithmetic. So, imagine you are given a value a, which is an
integer and you are given a modulus N. So, N will be called as the modulus where the
modulus is positive, then a modulo N is r, where r will be a value in the range 0 to N – 1,
such that a is equal to some quotient times the modulus plus your r. So basically r is the
remainder which you obtain after dividing a by N where the remainder is in the range 0 to N -
1. So, for instance, 5 modulo 4 is 1 and -11 modulo 3 is 1.
So, you might be wondering why not -11 modulo 3 is -2 because I can write -11 as q times
my N and r, why cannot I do that? Well, the requirement here is that the remainder should be
in the range 0 to N - 1. So, very easy way to visualize modular arithmetic is the following.
So, you can imagine that you have a clock with the N marks. So, basically, the possible
remainders which you can obtain after dividing any value by N.
And now, if you want to find out a modulo N. Where a is positive, then you start at 0 and you
count N times along this clock, wherever you stop, that will be the value of your a modulo N.
So, for instance, if you want to do it for 5 mod 4, and the possible remainders which you can
obtain are 0, 1, 2 and 3, you want to find at 5 mod 4. So, you start at 0 so, this is 1, 2, 3, 4, 5
you stop at 1. So, that is why 5 modulo 4 is 1 whereas, if you are a is negative, then you have
to do the same process but in an anti-clockwise direction.
Now, let us define what we call as congruence with respect to modulo. So if you have a
modulus N, then a will be; then we say that a is congruent to b modulo N if the remainder
that we obtained by dividing a by N is same as, the remainder that we obtained by dividing b
779
by N, that means if a modulo and b modulo N are same then I will say that a is congruent to b
modulo N and the notation that we use is the following : [ 𝑎 ≡ 𝑏 mod 𝑁 ].
So, in some sense, you can imagine that I am trying to say that a and b are equivalent, in the
sense that they give you the same remainder on getting divided by N. And it is very easy to
verify that if a is congruent to b modulo N, then that is possible if and only if a - b is
completely divisible by N. It is a very simple exercise to do that I am leaving that for you. So
we will use multiple definitions of a congruent b modulo and we can use this definition or we
can also use the fact that a congruent to b modulo N means a - b is completely divisible by N.
So there are some interesting arithmetic rules which your modular arithmetic follows. So
imagine that a modulo N is a’ and b modulo N is b’. Then a + b modulo N will be the same as
a’ + b’ modulo N, a - b modulo N will be same as a’ - b’ modulo N so on and similarly, a
multiplied by b modulo N is same as a’ into b’ and then you take modulo N. So, let us prove
these rules, we will just prove it for the case of addition, for the other operations you can
prove in the similar way.
So, since a modulo N is a’ that means, I can say that a is some quotient times N + a’ and
since b modulo N is b’ I can say that b is some another quotient times N + b’ and imagine
that a’ + b’ is some quotient times N + r. Let us see the left hand side of this rule; addition
rule it is a + b modulo and that means, you are adding a + b and then you are taking modulo.
So, I am just expanding a and b here and I can take out N common and then I can substitute
that a’ + b’ is some q times N + r.
780
So, this will be the overall value of a + b then what will be the remainder if I divide this value
by N. So, this term will vanish because on dividing it will give the remainder 0 and you will
get r. So, a + b modulo N will be r and same value you will obtain by dividing a’ + b’ and
then dividing a’ + b’ by N. Because if you divide a’ + b’ by N this term will cancel out
remember it will vanish and you will be left with the remainder.
So, what will be the interpretation of these arithmetic rules the interpretation here will be the
following that you can first reduce the operands namely a and b modulo N and then you can
perform the plus operation, subtraction operation, multiplication operation and if again
required and you can do a modulo instead of first adding and then taking a modulo. So, what
I am saying is that you first reduce and then do the operation and that will have the same
effect as if you are performing the operation and then you are reducing.
So, to make my point more clear, imagine that I want to ask you compute this. So, one
approach will be that you first compute the product of these 2 large numbers and then take a
modulo 100 that will give you an answer. But that will require some effort you cannot do it
easily using your paper and pen. Option 2 will be as per this multiplication rule that I reduce
the operand a modulo 100.
So, if I reduce a modulo 100 I get 28 and I reduce b modulo 100 and I will get 1 and then to
find out a b modulo N it is sufficient to find out a’ b’ modulo N so, a’ b’ are now very small
numbers which I can easily multiply and I will get 28 and I do not need to reduce it modulo
781
100 because 28 is already less than 100. So, my answer will be 28 which I will obtain very
easily.
So, these are some interesting rules which we will be encountering again and again that
means, if I want to do some modular arithmetic I can always reduce the operands first and
then apply the operation and then again if required I can reduce instead of doing the operation
and then applying the modulus.
Now, we have seen the rules for addition, subtraction, multiplication, what about division?
So, imagine a modulo N is a’ and b modulo N is b’ of course, a’ and b’ are in the range 0 to N
- 1. Now, what can I say about a over b modulo N and a’ over b’ modulo N. Can I say that
these 2 expressions will be same? Well the answer is no because at the first place the value a
over b modulo N may not be well defined.
Because a over b might be a fraction and maybe a real number for instance, if a is less than b,
or even if a is greater than b, a over b may not be an integer value. So how exactly you define
a over b modulo N. So let us see an example here to make my point more clear, imagine my a
is 3, b is 5, N is 4. Now, 3 over 1 modulo 4 is 3, because 3 over 1 will be 3 and 3 modulo 4
will be 3. But what over 3 over 5 modulo 4, 3 over 5 is 0.6 and 0.6 modulo 4 well, it is not at
all well-defined.
That means in modular arithmetic, if ac and bc are congruent to each other, that means ac
modulo N is same as bc modulo N that does not necessarily mean that a and b are also
782
congruent modulo N or equivalently I cannot say that you can cancel out c from both the
sides. No, that is not necessarily is the case, only in some cases certain conditions you can
cancel out c from both the sides and conclude that a and b are congruent modulo N but that
may not be always the case.
So, now, let us discuss some algorithms for modular arithmetic. And we will be seeing
algorithms for addition, multiplication, subtraction and modular exponentiation. So, this is
addition, this is subtraction, this is multiplication, and this is called modular exponentiation.
This is a very important operation in cryptography and our inputs a, b and modulus N are all
some n bit integers.
Now, what will be the complexity measurement? How exactly we judge whether our given
algorithm that we designed for performing this modular arithmetic operations are efficient or
not. Our complexity measurement will be how many operations are we performing as a
function of the number of bits that we need to represent as integer values a, b and N. And we
will be requiring algorithms where the number of operations that we perform is a polynomial
function in n.
Because, typically, we prefer algorithms whose running time is polynomial function of your
parameter; the parameter here is the size of your integer a, size of your integer b and the size
of your modulus N, which is the number of bits that you need to represent those values which
is n. We do not prefer any algorithm which is exponential time or sub exponential time in the
783
number of bits. So, it turns out that addition, subtraction and modular multiplication all of
them can be performed in polynomial in n number of bit operations.
So, for instance if you want to perform a + b modulo N. So, imagine that the bit
representation of a is an - 1, an - 2 and up to a0 and bit representation of b is bn - 1, bn - 2 up to b0
now to add a + b you can perform the bits of a and b bit by bit and taking care of carry and all
those things and then you will obtain a bit representation of a + b. And then, as we are
performing addition of two n bit numbers that will require polynomial in n number of
operations.
And then you have to do a + b modulo N that means, you have now got two n bit numbers.
So, this is n bit, this is n bit and you have to add one n bit number by another n bit number
which can again can be done in polynomial in n number of operations. Same you can perform
a - b and then take modulo N you can multiply a and b and then take modulo N all of them
will need polynomial in n number of operations: 𝒪(poly 𝑛 ).
Now, what about modular exponentiation? How can we compute ab modulo N? You might be
wondering that why cannot I do the following multiply a with itself and then take mod and
then again multiply with a and then take mod and so on. So, this operation DOT and then in
subscript N, means I am doing multiplication modulo N (.N). So basically I am saying that
you multiply a to itself b - 1 number of times and then keep on taking mod is that will be
equivalent to saying that I am performing b - 1 number of modular multiplications.
And one instance of modular multiplication need these many number of bit operations, since
I am doing it b number of times, this will be the overall complexity of my algorithm:
𝒪(𝑏 ∙ poly 𝑛 ) . And you might be saying this is a polynomial time algorithm. But if you see
it closely, this is not a polynomial time algorithm. Because what exactly is b, b is an n bit
number, that means magnitude wise, b could be as large as 2n.
So, for instance, if you are a n is 1024 that means if your a is some 1024 bit number, your b is
some 1024 bit number and your n is also some 1024 bit number. Then what I am basically
saying is that this naive algorithm will require me to perform b times polynomial in n number
of operations, but my b itself could be as large as 2 to the power 1024 bit number; those
many, this is an enormously large quantity, you cannot even imagine how big this number is.
784
So, even though this might look like a polynomial time algorithm that is not the case, this is
an exponential time algorithm.
So, now, we will see a very nice method, which is a polynomial time algorithm for
performing modular exponentiation and this is called as the square and multiply approach
why it is called square and multiply? It will be clear soon so let me demonstrate the
algorithm. Suppose I want to compute a53 modulo N the naive approach will be you multiply
a to itself and then take mod N.
And then again, you multiply the result with a and then again, take mod, that means you
perform 52 modular multiplications that will be the naive approach. I am saying, do not do
that, using square multiply method, we will be seeing how to compute a52 modulo N with just
9 modular multiplications. So now you can see the drop from 52 multiplications, I have
brought down it to 9. So you can imagine the level by which it drops when N is an
enormously large value.
So what I am saying is the naive algorithm will require you 21024 bit operations if n would
have been 1024, I would bring it down to only 1024 modular multiplications. So, you can see
exponential drop in the number of modular multiplications that we require, if we follow this
square and multiply approach. So the idea here will be the following, I will treat my exponent
in binary form. So I will come up with its binary representation and the binary representation
is this : 110101.
785
And now it is easy to see that a53 can be rewritten like this. So now, what it means is the
following that I have to accumulate certain powers of a. I have to accumulate the first power
of a. I have to accumulate the 4th power of a, I have to accumulate the 16th power of a, and I
have to accumulate the 32nd power of a. Which powers of a I am not accumulating: a power
2 that I am not accumulating, I am not accumulating a power 8 and so on.
So, which powers of a I have to accumulate and which powers of a I have to leave that
depends upon the binary representation of 53. So, you can see the positions at which the
binary representation of my exponent was 0 the corresponding power of a I am excluding I
am not accumulating and bit positions where it was 1, the corresponding powers of a I am
accumulating that is the idea of square and multiply. So, I am writing my exponent from LSB
to MSB. And my square and multiply approach will be an iterative algorithm.
What I will do is the following in each iteration, I will compute the next higher power of a by
squaring the current power. So, I will start with a power 1 and if I square it, I will obtain a
power 2, if I square it I will obtain a power 4, if I square it I will obtain a power 8, square it a
power 16, square it a power 32. Of course all the things are performed modulo N. So when I
am saying squaring that means I will be multiplying a power 1 with itself and then take
modulo N that will give me a2 modulo N.
I multiply a2 with a2 and do modulo N that will give me a4 modulo and so on. And now what
I will do is depending upon which bit of my exponent is 1 and which bit of my exponent is 0.
I will determine whether to accumulate the corresponding power of a or not, that is the whole
idea. So, what I will do is I will initialize an accumulator variable here which will have my
final result stored.
And as my algorithm proceeds the value of my accumulator will keep on changing depending
upon how do I accumulate the different powers of a. And in each iteration there will be a
conditional update depending upon whether I am accumulating the current power of a or not,
and which depends upon my current bit in the exponent. So, for instance, I will start with a
power 1 and I have to accumulate it because my current bit is 1.
So, I will accumulate it and my accumulator will become a power 1. Now, I will go to the
next iteration and I will do a square to get the next power of a that will need one modular
786
multiplication and I will obtain a2 and I will check whether I need to accumulate this power
or not, I do not need to accumulate. So, go to the next iteration obtain the next power of a,
that requires doing 1 modular multiplication namely I have to multiply a2 to itself and then
take mod N I will obtain the next power of a namely a power 4 and I have to decide whether I
have to accumulate this power or not, I have to so, I will do the conditional update, and my y
will become a power 5, I go to the next power by doing modular multiplication, I have to
exclude this. I go to the next power and I have to accumulate this. So, my conditional update
will be triggered I go to the next power of a and then I have to accumulate this power of a.
So, my accumulator gets modified and then I am done with all the bits of my exponent and
this will be my final answer. So, you can see that in each iteration I will be doing a
compulsory squaring that is why the name square and why I have to do a compulsory
squaring because I have to obtain the next higher power of a and there is an optional
accumulation depending upon whether my current bit of the exponent is 0 or 1.
So, in the worst case, what can happen you will be definitely doing n number of compulsory
squaring where n is the number of bits of your exponent. And it may so happen that your
exponent has a bit representation all 1s in which case you will be doing accumulation n
number of times. So in the worst case, you will be doing 2 times n number of modular
multiplications to get your final answer.
So that is the square and multiply approach and this is the pseudocode which is very simple.
Let me go through the pseudocode 1 step at a time. So, your inputs will be our base exponent
787
and the modulus and you need output ax modulo N. So I do some initializations here so this y
will be the accumulator and z will have the various powers of a as the algorithm proceeds.
Now what I have to do is, in the example I had the binary representation of my exponent
available with me, but that you need to compute in the algorithm itself. So how do you
compute the binary representation of the method that you are aware of namely, keep on
dividing x and depending upon whether you get the remainder 0 or 1. You accumulate 0 or 1
in your binary representation, the same trick we will be doing here.
So what I do is the following till my exponent is non 0. I will do the following, I will check
whether my x is odd or not. And that will determine, whether the next bit in the binary
representation of my exponent is 0 or 1, if I go from LSB to MSB. So if it is 1, that means I
have to modify my accumulators so this is the conditional update. So I update my
accumulator by multiplying whatever is the current content of the accumulator with my
current power of a.
And then I update my x to get my next bit in the binary representation of x. And I have to do
a compulsory squaring to go to the next power of a and then I have to update my x. So that I
can obtain the next bit in the binary representation of x and I have to do this process. So this
step, coupled with this step, and checking whether x modulo 2 is 1 or not, this will give you
the binary representation of x. This is the squaring step and this is the conditional update. So
that is the square and multiply trick.
788
So that brings me to the end of this lecture. So these are the references used for today's
lecture. And for number theory, there are lots of nice texts available. I find his text very
handy and very useful. It is very easy to understand and just to summarize, in this lecture we
started our discussion on modular arithmetic. We saw the rules of addition, subtraction,
multiplication we also saw that modular division is not well defined always.
We saw the square and multiply trick for doing modular exponentiation. The naive algorithm
for doing the modular exponentiation will be an exponential time algorithm. So that is why
we do not use that we instead use the square and multiply trick. Thank you.
789
Lecture – 56
Prime Numbers and GCD
Hello everyone welcome to this lecture. The plan for this lecture is as follows. So, in this lecture,
we will introduce prime numbers, we will see some properties of prime numbers, we will also
discuss about the naive algorithm for primality testing, which is a very interesting computational
problem. And we will discuss about GCD and Euclid‟s GCD algorithm.
790
So, let us start with prime numbers. So, the definition which all of you are aware of, so, we will
say an integer p which is greater than 1 is called a prime number, if the only positive factors of p
are 1 and the number p itself. Whereas if the number p is not a prime number, then we call it as
a composite number. It is easy to see that all prime numbers except 2 are odd. Because, if you
consider an even number, 2 is always a factor of that even number.
But 2 is a prime number because the only factors of 2 are 1 and the number 2 itself. So, in some
sense, except 2 all the prime numbers are odd, but we do have an even number which is also
prime namely the number 2 so you can consider to be an odd prime. It is very odd that odd in the
English sense, it is a very odd property that you have an even number, but it is still a prime
number. So, we know some nice properties of prime numbers, which we had already discussed in
this course.
So, we know the fundamental theorem of arithmetic, which says that you take any integer greater
than equal to 1 it can be written down uniquely as product of prime powers. So how do we prove
this? We proved it using strong induction, when we discussed proof by induction and we also
know that there are infinitely many primes. So, we saw there are several proofs for this, but we
discussed one of the proofs in this course earlier. So, these are 2 interesting properties of prime
numbers, which you should keep in mind.
791
Now, the next question is how exactly we check whether a given number is a prime number or
not. So, I will be discussing the naive algorithm, which many of you might be aware of. So, the
algorithm uses the fact that if at all your given number p is a composite number then it will have
a divisor which is less than equal to the square root of that number. And the proof is very simple.
So, let p be a composite number and since p is a composite number it will have a factor say a.
And the factor will be definitely greater than 1 because the definition of composite number is
that it should have at least 1 factor different from 1 and the number itself. So, the factor a will be
greater than 1 and less than p. And then since a is a factor I can say that there is a b such that a
𝑝
times b = p that means that b is basically 𝑎 . So, now, the claim of the theorem statement here is
that either 𝑎 ≤ 𝑝 or 𝑏 ≤ 𝑝 and this can be easily proved using contradiction.
So, on contrary assume that 𝑎 > 𝑝 and 𝑏 > 𝑝. Then this gives me a contradiction that
𝑎𝑏 > 𝑝, which is not the case because my premise is that p is equal to the product of a and b.
Based on this theorem, the naive algorithm is as follows. So, you are given an input number p
you want to check whether it is prime or composite, you know that if at all it is composite, then it
will have a factor within the range square root of that number.
So, that is what you try to do here you try to check whether it has at least whether you are given
input number p has at least 1 factor i within the range 2 to 𝑝. So, you range over all possible
792
values of i between 2 to square root of your number and check whether the number i divides p or
not. If you encounter any i then you can declare that p is composite, but if none of the numbers 2,
3, 4, 𝑝 divides your p then you can declare your p to be prime.
That is a naive primality testing algorithm that you are aware of. Now, what is the running time
of this algorithm? So, how many operations you perform here, so, there are 𝑝 iterations and in
each iteration, you are performing a division. So, our measurement, our complexity measurement
here will be how many divisions you are performing here to declare whether a given number p is
prime or not. So, we will be needing 𝑝 number of divisions.
So, you might be wondering, this is a polynomial time algorithm, but that is not the case. So,
𝑛
imagine p is represented by an n bit number then the magnitude of your 𝑝 will be 2 2 , because
𝑛
your p could be as large as 2𝑛 . So, 𝑝 could be as large as 22 . So, it might look like a
polynomial time algorithm, but it is not, it is an exponential time algorithm exponential in the
number of bits that you need to represent your value p.
So again, take the case when n is equal to say 1024 bit, and say your p is as large as 2 to the
power 1024 bit number, then basically your i is between 2 to something of order 512. That
means you will be basically performing these many divisions: 2512 divisions to check whether a
given number p of 1024 bit is a prime number or not. And this is an enormously large quantity; it
is not a small quantity. So, this is not a polynomial time algorithm.
Now, you might be wondering that whether we have a polynomial time algorithm or not. And
coming up with a polynomial time algorithm for checking whether a given number is prime or
not had been a long standing open problem, people thought that we do not have any polynomial
time algorithm. But in 2002 there was this algorithm proposed called as AKS primality testing,
which is a polynomial time algorithm.
Polynomial in the number of bits that you need to represent your input number p and this is
called Agarwal Kayal Saxena primality testing algorithm which in polynomial time can tell you
793
whether your given number p is a prime number or not. So, due to interest of time, I would not
be discussing the AKS primary key testing algorithm but if you are interested you can see the
original paper the paper title was “Primes is in P.”
Now let us next define the greatest common divisor or GCD. So, imagine you are given 2
numbers a and b which are nonzero integers. And the GCD of a and b is the greatest integer
which divides both a and b. So, we say integers a and b are relatively prime, we also use the term
co-prime if their greatest common divisor is 1 that means so of course, 1 is a common trivial
divisor of every a and b.
But if 1 happens to be the only common divisor, or if 1 happens to be the greatest common
divisor of a and b, then we say that a and b are co-prime or relatively prime. Whereas if we have
n values, a1 to an, and they are pairwise, we call them as pairwise relatively prime. If you take
any pair of values ai and aj in the set of n values which are given to you, they are co-prime to
each other. Now an interesting question is if we are given 2 values a and b how exactly we find
out the greatest common divisor?
One approach could be that you use the prime factorization of a and b, what do I mean by that.
So, as per the fundamental theorem of arithmetic, a will have its unique prime factorization
namely a can be expressed as product of prime powers. So, let the various powers of the primes
794
used in the representation of a are a1, a2, an and so on. And in the same way, the integer b will
have its unique prime power factorization, then it is easy to see that the GCD of a, b will be this
min 𝑎 1 ,𝑏1 min ⁡
(𝑎 2 ,𝑏2 ) min 𝑎 𝑛 ,𝑏𝑛
value : (𝑃1 𝑃2 … 𝑃𝑛 ).
But to use this algorithm at the first place, you have to come up with a prime power factorization
of a and b which in itself is a very computationally heavy task. So, we do not prefer to use this
algorithm in general if your a and b are very large quantities.
Instead what we use is Euclid‟s GCD algorithm which is probably one of the oldest algorithms
known. In fact, people believe that this is the first instance of an algorithm for any computational
task, interesting computational tasks of course addition, subtraction, they are also computational
task and you have algorithms for that. But this is probably a very interesting computation namely
the computing GCD and Euclid gave a very simple algorithm, which we will be seeing soon.
But to understand the Euclid algorithm and why it is correct, let us first try to understand some
properties of GCD. So, the first property here is that if your number a divides b, and if the same
number a divides c, then a divides b + c. And it is very simple to prove. Since a / b, then I can
write b as some a times x, where x is the quotient and the remainder is 0 and c is divisible by a I
can say c is some y times a. Now what I can say about b + c, so I can say b + c is same as x + y
times a and hence b + c is completely divisible by a.
795
Now, the crucial observation on which the Euclid‟s GCD algorithm is based upon is the
following. Our goal is to find out GCD of a, b. And for simplicity, imagine a is greater than b.
So, the idea that is used in the Euclid‟s GCD algorithm is that if a is some q times b + r, where r
may be 0, if a is divisible by b, otherwise, r will be something in the range of 0 to b - 1. So, if a is
b times q + r, then we can see that the GCD of a and b is same as the GCD of b and r.
So, you start with a and b, where a is greater than b, your goal is to find out the GCD of a and b.
What the statement says is that it is equivalent to finding the GCD of b and r. So, let us prove
this statement. So, the proof idea here will be the following. We want to prove that the greatest
common divisor of a and b and b and r are same, we will instead prove that every divisor of a
and b is a divisor of b and r; common divisor of b and r. And every common divisor of b and r is
also a common divisor of a and b.
If you prove that, then that automatically shows that the greatest common divisor of a and b is
same as the greatest common divisor of b and r. Because we are proving this, we are actually
proving a stronger statement; we are proving the statement about all common divisors of a and b,
whether it is greatest or not. So, consider an arbitrary divisor d which is a common divisor of a
and b. So, since d divides b, it will divide -b times q as well. And any how d divides a say as per
our assumption and now I can apply this Lemma.
So, you have a divisor which divides one number and another number. So, the same divisor will
divide the addition of the 2 numbers and addition of the 2 numbers here will be a - bq and a - bq
as per our definition is r. So, that means, if you have a common divisor of a and b then the same
common divisor is also a common divisor of b and r. On the other hand, assume that you have a
common divisor d of b and r we have to show that this common divisor d is also a common
divisor of a and b.
So, again we use a similar idea here since d divides b it will divide any q times b as well and
anyhow as per our assumption d divides r. So, again apply the same lemma I can say that d
divides the addition of these 2 numbers and the addition of these 2 numbers is nothing but a, so,
796
that shows that every common divisor of a, b is also a common divisor of b, r and hence, the
greatest common divisor of a, b will be the same as the greatest common divisor of b, r.
So, based on this observation, this is a very simple Euclid‟s algorithm, your input pair is a, b
where a is greater than b and idea is that in each iteration, we will use this rule: to reduce the
magnitude of our a and b till we reach a point where r becomes 0. So, what we do is we start with
x equal to a and y is being b. So, x and y will be my placeholders and my value of x and y will
keep on changing. So, the placeholder occupies the value a and b to begin with.
And i will iteratively keep on changing the value of x and y till I reach a stage where my y
becomes 0. So, as I said that I will be using this lemma again and again. So, what I will do is I
will find my r which is my current x modulo y and whatever is my current y that goes and
becomes next x and whatever the r that I obtained that will go and become the next y and I keep
on doing the process till my y becomes 0 when I reach a stage when y becomes 0, then I return
my x and that will be my GCD.
So, you can imagine that what is happening here is the following: you start and compute a
sequence of remainders and you stop when you obtain a remainder which is 0. So, I start with the
given pair of values and treat them as the 0th remainder and the first remainder respectively. So,
797
imagine that r0 modulo r1 is r2, where r2 is a value in the range 0 to r1 - 1 and then in the next
iteration, you will update your x and y.
So, this becomes x and this becomes y and you have obtained the next r and then in the next
iteration your r2 becomes your x and your r3 becomes y and then you obtain the next reminder
and you keep on doing this process till you obtain a 0 remainder, the time when you obtain the 0
remainder that means you have 0 here; you stop. And you will output rn as the overall GCD.
Now, what is the guarantee that this iteration will eventually terminate what is a guarantee that
this is not going to loop forever.
The reason that it will eventually terminate is that in each iteration, you are definitely reducing
the value of your y by at least 1 because you are now taking; you are updating the sequence of
remainders. So, you start with the remainder which r0 and the other remainder being r1. And thus
in each iteration, your remainders keep on getting decremented at least by 1. So, at most, it will
require a number of iterations to eventually obtain a remainder of 0.
So that means eventually the algorithm will terminate. And by applying this theorem, I can say
that the GCD of a and b is same as the GCD of the 0th remainder and the first remainder and that
is same as the GCD of; so you can treat this as a and b this is your b and r and then in the next
iteration your role of a and b and b and r gets changed and so on. And when you obtain GCD of
rn, 0 that means you obtain the other remainder being 0. So, the GCD of rn and 0 will be of
course, rn and that is overall GCD because you can come back all the way and say that this rn will
be the GCD of a and b as well.
798
So, the now, next question is what is the running time of the Euclid GCD algorithm: is it
polynomial in the number of bits that I need to represent my a and b or not? Because that will be
our measure get will be our measurement of time complexity. And there is a very interesting
result attributed to Lame which says the following and we will use this theorem to conclude that
Euclid‟s running time is polynomial in the number of bits that you need to represent your a and
b.
So, what Lame‟s theorem says is the following: if there are total n iterations used in your
Euclid‟s GCD algorithm then the value of b is same as the n + 1 Fibonacci number. So, very
beautiful result, which relates the number of divisions which are used in the GCD algorithm with
the Fibonacci sequence and the proof is as follows: we will give a direct proof. So, since we are
assuming that there are total n iterations or equivalently n divisions which are performed.
So, by the way if you are wondering where exactly division is involved, this is step your division
is involved. So, basically we want to find out how many times this x modulo y operation will be
performed in the Euclid‟s GCD algorithm, I do not want an algorithm where this x modulo y
operation is performed exponential number of times. So, if n divisions are performed, so, what I
have done is I have listed down the various remainders which I will keep on getting in each
updated iteration.
799
And then in the nth iteration, I obtain 0 remainder here namely rn - 1 is completely divisible by rn.
So, my claim here I will make now a series of claims and using that I will conclude Lame‟s
theorem. So, my claim is that each of these quotients q1, q2, qn - 1 is at least 1 and that is trivial
fact because they are at least 1 because at the first place we have used n divisions to get the
output of the Euclid‟s algorithm that means definitely r0 is not completely divisible by r1.
And we obtained some remainder because that only we have gone to the second iteration So, that
means q1 is at least 1, similarly r1 is not completely divisible by r2 there was some remainder that
means, q2 is at least 1 and so on. The second claim is that the last quotient here q1 is at least 2
and this is because qn is the ratio of n – 1th remainder and nth remainder. And I know that n –
1th remainder is strictly greater than the nth remainder, because in each iteration x is strictly
greater than y.
So, since now if I my quotient has to be an integer value, so I have numerator greater than
denominator. And I know that, this fraction, rn – 1 over rn has to be an integer value. So, that
means the minimum value of qn is at least 2. Now, based on these 2 facts, let us derive the proof
for Lame‟s theorem. So, I can say that the nth remainder where I stopped the algorithm is greater
than equal to 1, it is not 0, it is greater than equal to 1.
And if you see the Fibonacci sequence, the first term is 0. The next term namely f2 is 1. The next
term is the summation of the previous 2 terms, which is f3. The next term is a summation of these
2 terms which would 2 and so on. So, I can say that rn the last remainder is at least the second
Fibonacci number. Now, what about the previous remainder. So, the previous remainder rn - 1 is
qn times rn.
Now, I already know that rn is greater than equal to 1, and I also know that qn is at least 2. So,
this is an exact equality, but I know that rn is individually greater than equal to 1 (this value) and
qn is individually greater than equal to 2 and rn definitive will be now, at least 2 times rn , rn is
nothing but I have already proved that it is greater than equal to f2. So, I obtained that the n – 1th
remainder is as large as 2 times f2.
800
And if you now compare it with the Fibonacci series here, this value is nothing but greater than
equal to f3. Now, I go to the previous remainder. So, remember, my goal is to say something
about b, that is what Lame‟s theorem is. So, what I am saying is, I am going backward and trying
to find out any equality relating all the remainders and then I will finally stopped with r1 which is
the value of b.
So, in the same way, I can say that the n - 2th remainder its exact value is this, I know that qn - 1 is
at least 1 and I already now have a relation or inequality involving rn - 1 and hence, I can say that
rn – 2 is greater than equal to this. So, I can substitute qn – 1 greater than equal to 1 and then I can
substitute this inequality for rn - 1 and hence I conclude that rn - 2 is greater than equal to f4. And
now, if I keep on doing this process, I can come to the conclusion that r2 is greater than equal to
fn.
And then finally, when I come to r1 which is the same as b and r1 is this value q times r2 + r3; q2
is greater than equal to 1. So, I can say b is greater than equal to r2 + r3 but r2 in the previous step
I would have concluded is greater than equal to fn and hence, I can conclude that b is greater than
equal to fn + fn - 1 which is same as the next Fibonacci number and that precisely is the claim.
But, Lame‟s theorem directly does not help me to tell what exactly is the number of divisions,
801
Let us derive the exact number of divisions using Lame‟s theorem. So, I am stating Lame‟s
theorem here. Now, let me recall a lower bound regarding the value of the nth Fibonacci number
and relate it with the well-known Golden Ratio. So, it is a well-known lower bound that in the
Fibonacci sequence if you focus on the nth term then it is greater than the golden ratio (α) raised
1+ 5
to power n – 2 : (𝛼 𝑛−2 ) where the golden ratio, 𝛼 = and this can be proved using strong
2
induction I am not going to do that.
Now, based on this lower bound and Lame‟s theorem I can derive the number of divisions that
are required in the Euclid‟s algorithm. So, the theorem, I can conclude that the Euclid‟s GCD
algorithm will require at most these many numbers of divisions: 5*log10b. So, I am deriving the
answer in the base 10. But you can derive the answer to the base 2 as well because remember our
time complexity is in terms of n, where n is equal to the number of bits that you need to represent
your value a and b.
So, it will be basicallylog 2 𝑏 . So, whatever argument I am giving here it can be modified easily
to get the answer in terms of log 2 𝑏 as well. So, let us prove this theorem statement. So, imagine
that there are n modular divisions or n mod operations performed inside your Euclidean
algorithm then as per the Lame‟s theorem, we know that the magnitude of b or the value of b is
at least as large n + 1 Fibonacci number.
And now if I apply the lower bound on fn + 1, I can relate it with the golden ratio, so I get a
1+ 5
conclusion that b is greater than 𝛼 𝑛−1 and the value of 𝛼 = . So, if I take log on both sides
2
of these 2 equations to the base 10, you can take the log to the base 2 as well and that will give
you a different result, but more or less it will be same. So, if I take log to the base 10, I get this
equation here there this relation.
And now I can use the fact that the log of Golden Ratio log10 𝛼 is approximately 0.208, which is
greater than 1 / 5. So that is why I can write it as 1 / 5 here and that shows that n is less than
equal to 5 log10 𝑏. So, sorry for the confusion here, „n‟ I used here for the number of divisions
802
and I was continuously under the impression that n is also the number of bits used to represent
the values a and b.
So, let us use a different representation. So, let N denote the number of bits used to represent a
and b. So, what I have derived here is the following: I have derived it, the number of divisions
that you perform is upper bounded by roughly the number of bits that you need to represent your
value b. So, we are getting it in the base 10. So, log10 𝑏 is not n, and will be something else. But
whatever derivation we have done here. The step where we took the log to the base 10, if you do
it with log to the base 2, that will give us the value of n in terms of the value of n somewhere so
it would not be more than the number of bits asymptotically it would not be more than the
number of bits that you need to represent your integers a and b. So that means Euclid‟s algorithm
is actually a polynomial time algorithm polynomial in the number of bits that you need to
represent your integers a and b.
That is a very remarkable fact because at the time when Euclid invented his algorithm, there was
no notion of time complexity, there was no notion of polynomial time complexity, exponential
time complexity and so on. So, without even worrying about how efficient it will be, Euclid gave
us a very fantastic algorithm to compute GCD of 2 numbers whose running time is polynomial,
and which can be used in practice.
803
So that brings me to the end of today‟s lecture, these are the references used. Just to conclude in
this lecture, we introduced prime numbers, we discussed a naive algorithm for checking whether
a given input number is prime or not, and its running time is exponential. And we also discussed
about Euclid‟s algorithm for computing the GCD whose running time is polynomial in the
number of bits that you need to represent your input values. Thank you.
804
Lecture – 57
Properties of GCD and Bezout’s Theorem
Hello, everyone, welcome to this lecture, in this lecture we will discuss about the properties of
GCD and we will also discuss about Bezout’s theorem.
So, the plan is as follows, we will discuss Bezout’s theorem, we will discuss about extended
Euclid’s algorithm and then we will discuss about modular multiplicative inverse.
805
So, let us start with Bezout’s theorem, which is a very interesting theorem. And what it says is
the following. It says that you can always express the GCD of 2 numbers as a linear combination
of the 2 numbers itself. So, more specifically, if you are given 2 values a and b and if you have
found their GCD, then what the theorem says is that you can always find integer linear
combiners that is important, you can always find integer linear combiners s and t, such that if
you linearly combine a and b using this combiners s and t respectively, then that will give you
the GCD. And when I say integer combiners s and t, your s and t may not be positive they can be
can be negative as well, the only condition is that they should be they are integers. So, for
instance, if you take a and b to be 6 and 14, respectively then the GCD is 2.
And it is easy to see that I can write 2 as a linear combination of my a and b, namely 6 and 14,
where my linear combiners s and t are -2 and 1 respectively. So, that is the Bezout’s theorem.
And we will prove this theorem and the proof is slightly involved. So, please pay attention here
and proof will be non-constructive. Namely, I would not show you the exact linear combiners s
and t for a given a and b. But I will argue that indeed, there exist linear combiners s and t
satisfying the conditions of Bezout’s theorem.
Later, we will see a constructive proof as well when we are given a and b, I can show you how to
construct your linear combiners s and t. So, as I said the proof is non-constructive. And the goal
will be to show the existence or I will logically argue about the existence of s and t. So, to begin
806
with, let me first define a set S, which is a set of all integer linear combinations of your inputs a
and b.
Remember throughout the proof, we will be focusing only on integer linear combinations,
because the theorem says that we can find integer linear combiners s and t. So, let S be the set of
all integer linear combinations of a and b. So, S is the form x times a + y times b where my linear
combiners x and y can be arbitrary integers, 𝑆 = {𝑥𝑎 + 𝑏𝑦 ∶ 𝑥, 𝑦 ϵ ℤ} . So, it is easy to see that
S is infinite because my linear combiners x and y can be any arbitrary integers and there are
infinite number of integers.
Now, it is an exercise for you to find out whether the set S is whether it is countably infinite or
uncountable, whether its cardinality is ‫ א‬or not. Now, my goal is to show the following if I want
to prove the Bezout’s theorem, I have to show that the GCD of a, b is also an element of the set
S. So, let d denote the GCD of a and b, my goal is to show that this element d is also a member
of the set S, namely it can be expressed as some integer linear combination of a and b.
So, I will prove or I will achieve my goal using a series of claims. So, the first claim is very
simple. It says that the set definitely contains non-zero elements. Of course, it will contain 0
element as well because if I set my linear combiners x and y to be 0, then 0 times a + 0 times b
will be 0. So, 0 is of course, an element of S. But other than 0, set S also have non-zero elements.
And two trivial examples of non-zero elements which are present in the set S are a and b.
So, if I set x = 1 and y = 0, if these are my linear combiners then I obtain a belonging to S and if
I set x = 0 and y = 1 then I get b belonging to the set S. So, claim 1 is trivial to prove. Now there
are infinite number of non-zero elements in the set S. Among all those nonzero elements I denote
by smin the element which has the least absolute value, that is important I am not focusing
whether smin is positive or negative, smin could be negative as well, but it has the least absolute
value.
So, you have the element 0 present in S and you have the non-zero elements present in S the non-
zero elements are positive as well as negative among them smin denotes the element which has
807
the least absolute value. So, since smin is a member of the set S that means, there exists some
linear combiners xmin and ymin such that smin is xmin times a + ymin times b. And again this linear
combiners xmin and ymin, they can be positive negative, they are some arbitrary integer combiners.
Now, my claim 2 is the following. I claim that this element smin; it divides every element of the
set S. That is very interesting. And since this is a universally quantified statement, because I am
claiming that smin has a property with respect to all the elements of the set S, so the statement is
universally quantified. The proof strategy for proving this claim is that I take some arbitrary
element u from the set S and show that smin indeed divides that arbitrary u.
And then using universal generalization, I can conclude that indeed my claim is correct. So, let u
be some arbitrary element of the set S. So, corresponding to u let the linear combiners are xu and
yu. Now, my goal is to show that this u is completely divisible by smin. So, imagine that u is some
q times smin + r. I do not know what exactly is the remainder r my goal is to show that indeed, u
is completely divisible by smin.
So, my goal is to show that r is 0. But in general, I can write u in this form, I can say u is some
quotient times smin + r. And from this, I get that r is equal to the difference of u and q times smin.
And remember, my goal is to show r is 0 then only I can conclude that u is completely divisible
by smin. Now, what I can do is the following, I substitute the value of u in terms of linear
808
combinations of a and b and I substitute the value of smin in terms of linear combinations of a and
b.
And this overall thing I can write again as a linear combination of a and b where this will be my
linear combiner for with respect to a and this will be the linear combiner with respect to b. So
that means I get that the element r is also a member of the set S. Now, what can I say about the
range of the absolute value of r. So, since r is the remainder, obtained by dividing u by smin, the
remainder can be 0. And it can be at most smin – 1, that is a fact that I have that follows from the
rules of the division.
So, I can say that the absolute value of r could be either 0 or it is strictly less than the absolute
value of smin. But this implies that r has to be 0. And this is because of my assumption that smin is
the is least non-zero absolute valued element of S. Remember, smin is also an element of the set S
and it is a special element of the set S in the sense that among all the non- zero elements of the
set S, smin has the least absolute value.
So, how can it be possible that you have another element r which is also an element of the set S
and its absolute value is strictly less than smin. That is not possible. That is possible only if r = 0
that means r is not a non-zero element. And that is precisely what I wanted to prove I wanted to
prove that indeed u is some q times smin. So that proves my claim number 2.
809
Now, my third claim is the following. I claim that the value smin is a divisor of your GCD of a
and b. So, remember, d is the GCD of a and b. And let us prove this claim number 3. So, from
claim number 2, I know that smin divides every element in the set S and remember that the
element a belongs to the set S as well. So that means smin divides a as well. So that is a proof for
this fact. And due to the same reason, I know that b is a member of the set S and from claim 2
smin divides every element of the set S so that means smin divides b as well. That means what I
can say is the following smin is a common divisor of a and b. And if smin is a common divisor of a
and b that means smin, of course is a divisor of the common divisor of a and b which is the
greatest in the sense it is the largest common divisor of a and b.
So that proves your claim number 3, because if smin is a common divisor of a and b, but smin may
not be the greatest common divisor there might be another divisor which is bigger than smin and
which divides a and b both in that case I can say that smin divides that common divisor d as well.
So that proves my claim number 3.
810
Now, I will show that the common greatest common divisor of a and b is also a divisor of smin.
And then finally using claim 3 and 4 I will conclude that Bezout’s theorem is true. So, let us
prove claim number 4. So, since d is a common divisor of both a as well as b that means d has to
divide a and that means d divides any multiple of a as well. Similarly d is a divisor of b as well
because d is a common divisor of both a and b. So, if it is a common divisor of a and b both it
will be dividing b and hence d will divide any multiple of b as well.
Now, if d divides x times a, for every integer x, and if d divides every integer multiple of b, then
I can say that d divides x times a + y times b for every integer x and y and hence, I can conclude
that d is a divisor of smin. Because smin is also some linear combination of a and b. So, what I
have shown here is that d is a divisor of every x times a + y times b that means you take any
integer linear combination of a and b, d divides that integer linear combination.
And smin is also one of the integer linear combination of a and b. So, hence d divides smin as well.
So, these are the 4 claims that I have established. Now what I can do is the following. From
claim 3 and 4, I can conclude that the value of d is either the same as smin or it is same -smin
because claim 3 says smin is a divisor of d and claim 4 says that d is a divisor of smin that is
possible only if this condition holds, 𝑑 = ±𝑠𝑚𝑖𝑛 .
811
That means; and what is smin? smin is a linear combination of a and b; that means either d is equal
to positive xmin times a + ymin times b or d is equal to minus of (xmin times a + ymin times b). So, if
this is the case, then my linear combiners are xmin and ymin whereas if this is the case, then my
linear combiners are -xmin and -ymin. Irrespective of the case I know that d is expressible as an
integer linear combination of a and b
So, Bezout’s theorem has been proved. Specifically we have shown this. But why this is a non-
constructive proof is the following. We do not know the exact value of xmin and ymin, which will
give me smin, because the set S is an infinite set. And I cannot iterate over all possible integer
combiners, x and y and come with the minimum value xmin and ymin, because my set is infinite.
So that is why it is a non-constructive proof, but logically I have argued that the greatest
common divisor of a and b can be expressed as some linear integer combination of a and b itself.
So now, the next interesting question will be that how exactly I find those integer linear
combiners. So, if you are given a and b, by running the Euclid’s algorithm, you can find their
GCD but if I also want to find out the integer linear combiners as which are guaranteed to exist
as per the Bezout’s theorem, how exactly I can find them. And you might be wondering that why
at the first place, I will be interested to find out the Bezout’s coefficient. So, these integer linear
combiners, they are called as Bezout’s coefficient.
812
So, you might be wondering why at the first place I am interested to find them; later on when we
will discuss about modular multiplicative inverse, this Bezout’s coefficients will come very
handy, so that is why we want to find them. So, it turns out that by doing some extra book-
keeping, that means by maintaining some additional values and data structure in my Euclid
algorithm, which are used for finding out the GCD of a and b, I can find out the Bezout’s
coefficient as well.
And running time will remain more or less the same, I would not have to do significant amount
of extra work. And extra book-keeping that we have to do leads to what we call as extended
Euclid’s algorithm. So, this was not the algorithm Euclid proposed, Euclid gave only the
algorithm to compute the GCD of 2 numbers. But the reason we call it extended Euclid’s
algorithm is we do some extension. Namely, we do some extra bookkeeping. And that extra
book-keeping helps us to find out the exact values of Bezout’s coefficient.
And extra book-keeping that we have to do is that each step, we have to express the remainders
that we keep on getting in terms of our original a and b. And that is always possible to do that.
So, I would not be giving you the exact pseudocode for extended Euclid’s algorithm, but I will
demonstrate it and then I will leave it as an exercise for you to express it as an algorithm. So,
suppose my a is 252, and b is 198. And I want to find out the Bezout’s coefficient s and t for this
value of a and b.
So, now let us see how exactly the various remainders are computed during the execution of the
Euclid’s GCD algorithm. So, in my first step, this will be my x (252) and this will be my y (198).
And this will be my r (54). In the next iteration, this will become my x (198), my current r will
become next y (54) and this will be the next r (36). So, the underlined things are the remainders
that I am obtaining. And these underlined things are the quotients. So, then this becomes my x,
this becomes my y, and this will be my new r.
And then this becomes my x, this becomes my y. And finally, I obtain 0 as the remainder and I
stop. And I stop and say that 18 will be my GCD. Now, my 18 is a GCD of 252 and 198. And
now my goal is to find out the integer linear combiners s and t such that, that s times 252 + t
813
times 198 gives me the value 80. So, for that as I said that each step at each step, you express the
remainders in terms of a and b. So, let us start with the final remainder, which is 18.
And if I go back, then 18 is the difference of 54 and 1 times 36. But as I said, that everything has
to be expressed in terms of a and b, so I will go 1 step back. And then I can see that 36 is
expressible in terms of 198 and 54. So, I can substitute the value of 36 in this equation, and then
I get that 18 is represented in terms of 198 and 54. But I want to represent 54 also in terms of 198
and 252. And for that I have to go 1 step back further and 54 satisfies this equation and then I can
substitute this value of 54 in this equation.
And then I get my Bezout’s coefficients as 4 and -5. So, basically what we have to do is at each
step, we have to keep track of my quotients, and the remainder and quotients and the remainder,
that is why I have underlined them. So, in this demonstration, I have actually done a backward
pass, because we went all the way back and stopped where 54 was expressed in terms of 252 and
198. And then I substituted that value in this final equation.
But in the actual pseudo code of extended Euclidean algorithm, you do not need to make a
backward pass, everything is a forward pass. It is just a small modification. And that is all. So,
that means, anyhow, you will be performing the computation x modulo y to find out the
remainders in each iteration of the Euclid’s algorithm, what I am saying is you can also keep
track of the various quotients and that will help you to find out the exact values of Bezout’s
coefficients s and t.
814
So, now using notion of GCD and Bezout’s coefficient, we will define what we call as
multiplicative inverse modulo N. And let us first see the definition and then we will see how
exactly we can find out the multiplicative inverse using extended Euclid’s algorithm. So, I define
operation multiplication modulo N that is denoted by this notation ( . N) . So, this notation is for
multiplication modulo N, that means you multiply a and b and then take the remainder. That is
our definition.
Now, I say an integer b to be the multiplicative inverse of another integer a modulo N if a times
b modulo N gives me 1, that means, if you multiply a with b and then take the modulo N and if
the value is 1, then I will say that value b is the inverse of a (multiplicative inverse), why it is
inverse because typically in the regular arithmetic, when I say inverse of a is 1 / a, the
interpretation there is that if I multiply a with 1 / a, then I get 1.
In the same way in the modular world, I am interested to find out a number b, which when
multiplied with a and then taken modulo N and gives me 1, if that is the case, then this b can be
treated as if it is a-1. That is the interpretation here. So, it is like more or less same as your regular
inverse, but we call it modular multiplicative inverse because everything happens modulo N. So,
we use this notation b = a-1; this does not mean that b is 1/a remember very often student gets
confused.
815
This is just a notation when I say a inverse (a-1) that does not mean 1 / a, a-1 is another integer b
which when multiplied with a and then taken; and then if you do modulo N we get answer 1.
Now, it is easy to see that if b is the multiplicative inverse of a modulo N, then a is the
multiplicative inverse of b modulo N, because when you multiplied a with b modulo N and you
get 1 that means when you multiplied a with b and then take modulo N you get 1.
So, in that sense, b is the inverse of a ; multiplicative modular inverse of a. Now, another
interesting fact here is the following if b is the multiplicative inverse of a then any number of the
form b plus minus any multiple of your modulus N, 𝑏 ± 𝑘 ⋅ 𝑁 for all 𝑘 ϵ ℤ, will also be an
inverse of a and you can verify that so what will be the result of multiplying a with this number b
plus minus kN and then taking modulo N? Well this will be the same as this.
So, remember, I can take this dot inside because as per the rules of modular arithmetic, I can first
reduce my operands and then perform the operation and this . (dot) is distributive over the plus
here as well as minus. So, now, when I do akN modulo N this will give me 0, because this
number is completely divisible by N then whatever is the remainder obtained by dividing ab / N
that will be the overall answer and ab modulo N is nothing but 1 because that is the definition of
b.
This shows that if at all you have multiplicative inverse, then they are infinite in numbers that
means, once you find 1 multiplicative inverse for a you are guaranteed to have infinite number of
multiplicative inverse because your k; you extend it in the positive direction or negative direction
you substitute k = 1, 2, 3, 4 up to infinity and you substitute k = -1, -2, -3, infinity and k = 0 you
get infinite number of inverses.
816
But now, the interesting question is when can we say that the multiplicative inverse modulo N
for a number a exists - is it the case that for every a for every modulus N I can say that the
multiplicative inverse modular and exist? Unfortunately the answer is no, there is only under
certain conditions I can say that multiplicative inverse modular and exist. So, the theorem
statement is the following.
If you are given some number integer a and a modulus N then the claim is that the multiplicative
inverse of a exists if and only if a is co-prime to N namely the GCD of a and N is 1. So, this is an
if and only if statement that means, this condition is both necessary as well as sufficient. So, let
us first prove the sufficiency condition using Bezout’s theorem. That means, assume that you
have a number a and a modulus N such that they are co-prime.
If that is the case then I have to show that I can find out the multiplicative inverse of a using
Bezout’s theorem and using extended Euclid algorithm. So, using extended Euclid’s algorithm I
can find out the exact Bezout’s coefficients s and t such that the integer linear combination of a
and N as per the combiners s and t is same as the GCD of a and N and remember the GCD of a
and N is 1. Now, if I take mod N on both the sides that means, this is your LHS I take mod N and
this is your RHS.
817
I take mod N. Now 1 modulo N will give you 1 because you divide 1 by any modulus N the
remainder will be N whereas, left hand side will be as + Nt mod N So, I can take mod inside.
Now, N times t mod N will 0 because this is a multiple of N and hence it is completely divisible
by N that means, my LHS becomes as times modulo N and anyhow in my RHS, I have 1, that
means I can say that the Bezout’s coefficient s is nothing but your multiplicative inverse of a.
And as I said earlier, if you know to find 1 multiplicative inverse, you can find the others as
well, just take s plus minus equal to k times N for all k belonging to Z (𝑠 ± 𝑘𝑁 ∀ 𝑘 ∈ ℤ). That
will give you all multiplicative inverses. So, that is a sufficiency proof. If you give me a number,
which is co-prime to your modulus I know how to find out its multiplicative inverse.
Now, I want to prove the necessity condition. Namely, I want to show that if at all the
multiplicative inverse of a exist then it implies that the number a is co-prime to your modulus.
So, assume you have an a for which you can find out the multiplicative inverse how you find out
I do not care, but it exists. And suppose the multiplicative inverse of a is b, I have to show; my
goal is to show that a is co-prime to N, that is my goal. So, imagine the GCD of a and N is c.
And as I said earlier, my goal is to show that c = 1. Now, as per the definition of multiplicative
inverse, I know that ab modulo N is 1 because b is the multiplicative inverse of a. That means
when I divide ab / N, I get a remainder 1. That means I can say that a times b is some multiple of
818
N + 1 that comes from the rules of division. That means I can rewrite 1 as the difference of ab
and kN.
Now since I know that the GCD of a and N is c, that means c is the greatest, it is a common
divisor of a and N, and it is the greatest common divisor. Then I know that c divides any multiple
of a, namely, it divides b times a and c divides any multiple of N. Namely, it can it will divide k
times N, if c divides a times b, and c also divides k times N, then remember in our earlier lecture,
we showed that c divides the summation of those two numbers as well.
That summation could be positive, negative anything. So, I can say that c divides ab - kN as well,
that means I can say that c divides 1. And that is possible only when c = 1. And that is what
precisely we wanted to show.
So, we showed that indeed if multiplicative inverse for a modulo N exist, then your number a has
to be co-prime to N otherwise the multiplicative inverse does not exist. So, this is a very
powerful condition, it says that multiplicative inverse modulo N exists only if some condition is
satisfied and the condition is that your number should be co-prime to your modulus.
So, that brings me to the end of this lecture, just to summarize : in this lecture, we saw few other
nice properties of the GCD is namely, we saw that the GCD of any 2 numbers a and b can be
819
expressed as a linear combination of the numbers itself. And we know how to find those integer
linear combiners using extended Euclid’s algorithm. And we discussed the definition of
multiplicative inverse modulo N and the condition under which multiplicative inverse modulo N
exists. Thank you.
820
Lecture – 58
Linear Congruence Equations and Chinese Remainder Theorem
Hello, everyone, welcome to this lecture, the plan for this lecture is as follows: in this lecture,
we will introduce linear congruences. And we will see 2 methods for solving linear
congruences one using extended Euclid’s algorithm and another one due to the famous
Chinese Remainder Theorem or CRT.
So, let us start with linear congruences so, in regular algebra, you often come across linear
equations of the forms a times x = b, that means you are given some real numbers a and b,
and you have to find out the value of this unknown variable x such that this condition is
821
satisfied. And how do we find out the solution for the above equation; by solution of the
above equation, I mean to find out the value of this unknown x.
And if you know that the value of a is not 0 then I can say that, if you multiply both sides by
1/a, and 1/a is considered as the inverse of a in your regular algebra, then I say that x = b/a.
That is a solution for your linear equation here. Now, when I say linear congruence, we are
more or less in the same situation except that we are in the modular world, that means
everything is given some modulus.
So, we will be given some a and b and a modulus N and our goal will be to find out x such
that 𝑎𝑥 ≡ 𝑏 mod 𝑁 and that means x when divided by N and b when divided by N gives the
same remainder, you have to find out the value of x, or equivalently ax - b is completely
divisible by N. So, for instance if I say that I am given 6x congruent to 4 modulo 10 and if I
want to find out the value of x then the possible solutions are x = 4 because if x = 4 then you
get 24 congruent to 4 modulo 10 which is true.
Because 24 - 4 is completely divisible by 10, If you substitute x = 9, then you get 54

congruent to 4 modulo 10, which is again true, because 54 - 4 is 50, which is completely
divisible by 10. And it is not the case that these are the only solutions, you have infinite
number of solution. That means any number of the form 4 + 10k, where k can be either
positive or negative will also satisfy this linear congruence.
In the same way, every number of the form 9 + 10k, where k can be either positive or
negative will also be a solution of this linear congruence. So, that is an interesting thing
unlike regular algebra, where the solution was just b/a, of course you can also say 2 times b
over 2 times a is also a solution, 3 times b over 3 times a is also a solution but more or less
their primitive form is b over a. In the same way, the primitive solutions, primitive in the
sense the base solutions are 4 and 9. And now you can create infinite number of solutions out
of these 2 solutions by adding all multiples of 10.
822
So, now let us see how we can solve linear congruences using extended Euclid’s algorithm
that is our method number one. So, you are given a, b and N, goal is to find out this unknown
x. Now, as we did for our equation in the linear algebra where we said that divide both sides
by a provided a is not 0. The question is can we do something similar in the modular world as
well that is can we say divide both sides by a. And divide both sides by a by that I mean
multiplying both sides by multiplicative inverse of a.
And that is possible only if GCD(a, N) is 1. So, remember, in the earlier; in the last lecture,
we proved that the multiplicative inverse modulo N exists only if the number for which you
want to find out the inverse is co-prime to your modulus. So, if your number a is co-prime to
your modulus N, then I know that a-1 exists. And hence I can say that multiply both sides by
the multiplicative inverse. So that is the method of solving linear congruence under this
restricted condition.
So, if your GCD(a, N) is 1 then by running the extended Euclid’s algorithm, compute the
multiplicative inverse of a namely b, I stress that it is not 1/a in the modular world it is an
integer. And now I multiply both the sides of this linear congruence by this a-1. So, I will get
this linear congruence and I know that a into a-1 is 1 modulo N and 1 into x modulo N is x.
So, I get that x is congruence to b-1 modulo N that means; I can say that the value of x being
this plus any multiple of N is a solution for this linear congruence (𝑥 = 𝑏𝑎−1 mod 𝑁 +
𝑘𝑁).
823
Because all these values of x minus this value ba-1 is completely divisible by a. However, this
method will work only if your number a is co-prime to your modulus N. What if the number
a is not co-prime to your modulus N, then we have to follow a slightly different approach
which is complicated and I am not going to discuss that matter.
Instead I will discuss another way of solving linear congruences; in fact, a set of linear
congruences and this method is often called as the Chinese Remainder Theorem attributed to
the ancient Chinese but it is also believed that the ancient Indian mathematicians also used
the same technique for solving a system of linear congruences. So, what exactly we mean by
system of linear congruences.
So, very often you come across a puzzle of the following form you have an unknown number
x which is not given to you, but it is given to you that unknown number x has a property that
when it is divided by 3, it gives you the remainder 2, when divided by 5 it gives you the
remainder 3 and say when it is divided by 7 it gives you the remainder 2. Under this
condition, find out the value of x of course, again you can find out infinite number of x
satisfying this condition, but what the CRT method says is it gives you at least one x which
satisfy this condition.
And then from that you can find out the other values of x as well, so the above puzzle, above
instance of the puzzle can be formulated as a system of linear congruence namely, my goal is
to find out an unknown x satisfying the linear congruence that it is congruent to 2 modulo 3 it
is congruent to 3 modulo 5 and it is congruent to 2 modulo 7. And the special property of this
824
problem instance is that you are given the value of x modulo various modulus, those
individual modulus are pairwise co-prime.
So, let me now formally state the theorem statement of Chinese remainder theorem and then
we will prove it. So, you are given n different modulus which are pairwise relatively prime,
that means, you take any pair of modulus mi and mj they are co-prime to each other. And you
are given n number of remainders a1 to an. So, you have to find out an unknown x which is
congruent to a1 modulo the first modulus, it is congruent to a2 modulo the second modulus, it
is congruent to an modulo the last modulus.
Now, what is the Chinese Remainder Theorem : it says that this system of n linear
congruence has a unique solution modulo the bigger modulus and what is the bigger modulus
it is defined to be the product of n modulus. So, in other words, what does it mean unique
solution by unique solution I mean that there is only one value of x in the range 0 to M-1
which satisfies simultaneously all the n linear congruences but that does not mean there are
there is only one solution in this range.
But there can be other solutions as well outside this range, in fact there are other infinite
number of solutions and what you can say about other solutions: they are obtained by adding
various multiples of M namely they are congruent to modulo M to this solution x which is in
the range 0 to M-1. So, we now want to prove the Chinese Remainder theorem and there are
multiple things which we have to prove, the proof strategy is as follows, we will give the
construction of one of the solutions in the range 0 to M - 1.
825
But that does not mean that is a unique solution, remember there are 2 parts of the proof, we
have to show that there is at least one solution in the range 0 to M - 1 which we will be doing
in this lecture. And then we also need to show that, that is the only solution you cannot have
any other solution in the range 0 to M – 1. That we will do in the next lecture. By the way,
when I say unique solution again and again, I am stressing unique solution modulo M that
means unique solution in this range, Outside this range if x is a solution, any number of the
form x + l times m, where l is positive negative will also be a solution of this system of linear
congruence. But these values, these solutions will be outside the ranges of 0 to M -1. So, do
not get confused in this term unique solution.
So now, let us see how exactly we can find at least one solution that will be the goal of this
lecture. So, the construction idea of that solution will be as follows: we will define; we will
try to find out a special linear combination of the N remainders that are given to us. So
remember, we are given N remainders a1 to an, we will try to express that unknown x which
we want to find out as a special linear combination of these n remainders namely, we will try
to find out this special linear combiners c1, c2, cn.
These linear combiners will be special in the sense that if you take the ith combiner ci and
reduce ith modulo mi namely mi modulus you will get 1, but if you take the jth combiner and
try to reduce it modulo any other modulus, you will get 0. So for instance, what I am saying
is that my combiner c1 will be such that c1 modulo m1 will be 1, but the same linear combiner
c1 modulo any other modulus will be 0, namely the n - 1 other modulus, all this will be 0.
826
In the same way your c2 modulo m2 will be 1, but c2 modulo m1, c2 modulo m3, c2 modulo
m4, c2 modulo mn will be 0. So, that will be the property of the special linear combiners; how
exactly we find them that is our whole process, but imagine for the moment that it is possible
to find out this linear combiners. That means, I know how to find out this linear combiners
such that x is indeed equal to this.
Now, you can see that if I take this value, once I have found c1 c2 cn then I will have this
exact value, then if I take this RHS and compute RHS modulo m1 then that will be same as a1
modulo m1 because for all other summands I will be getting c2 modulo m1 , c3 modulo m1 , cn
modulo m1 and their effect will be 0 0 0 0 0 it will be only c1 times a1 modulo m1 and c1
modulo m1 is 1, because that will be the property for my linear combiner. And hence, this
value of x that I have obtained modulo m1 will be indeed a1.
In the same way assuming that I have found c1 c2 cn satisfying these conditions what I can say
about the x that I have obtained modulo a2, if I do x modulo a2, then it will be equivalent to
this c2 times a2 modulo m2 because the effect of this term will be 0, the effect of third term
will be 0, the effect of the nth term will be 0 and so that is the idea here. So, everything falls
down to how exactly we find this special linear combiners c1 c2 cn satisfying this properties.
So, let us see how we find; so, remember, my bigger modulus M is the product of all n
modulus. Now, I define n number of small sum modulus, so my first sum modulus M1 is the
product of all the n modulus except the first namely m1, my M2 is the product of all the n
827
modulus except m2 and so on. So, in general the sum modulus Mk is the product of all the n
modulus except the kth modulus.
Now, my claim is that if I take the modulus mk and the modulus Mk then they are co-prime to
each other and this is true for every k from 1 to n. Now, the proof is very simple assume that
the GCD of mk and Mk is not one. So, if your GCD is not one that means, there is another
common divisor and that will have some prime factor as well because every number has a
prime factorization.
So, that means, if the GCD is not one, then that means there is at least some common prime
divisor which divides both mk and Mk. Now, if this prime number p divides this modulus Mk
then since Mk is the product of n – 1 number of small modulus so, it is the product of m1, m2,
mk - 1 and mk is missing mk + 1 up to mn it is the product of n - 1 modulus. And I know and I
am assuming here that p is a divisor of Mk.
p is a divisor of this Mk it has to either divide m1 or it has to either divide m2 or it has to either
divide mk - 1 or it has to divide either mk + 1 and so on, because if p does not divide any of
these small modulus m1 m2 mk – 1 mk + 1 mn. Then how in the first place it can divide Mk
because p is a prime number. So, that means, it has to divide some small modulus call it mj
and we already know that p divides mk. Now, this mj is definitely different from mk because
mk is not present in this Mk; it is not present.
So, that means now I have obtained a pair of small modulus mk and mj which are not co-
prime because p is a common divisor of both mk and mj which is a contradiction to the fact
that the n small modulus which are given to us they are pairwise co-prime, so that is a proof
of this claim.
828
So, I am retaining this claim here, now, if this small modulus mk is co-prime to this Mk and I
can say that I can find out the multiplicative inverse of this Mk modulo mk. I am treating mk
my modulus and Mk as the number so, I am treating it as my a and this is my N and I have
shown that a is co-prime to N and hence I know that multiplicative inverse of a modulo N
exists.
So, I know that multiplicative inverse of Mk modulo mk exists and I can find it out using the
extended Euclid’s algorithm. So, let yk be the multiplicative inverse of Mk modulo mk. That
means this property holds that means you multiply yk with Mk and then you take modulo Mk
you will get the remainder 1, you will get the answer 1.
So, these are the various facts here so, this is my Mk and the corresponding multiplicative
inverse is yk. And this I can find out for every k in the range 1 to n. Now, my claim is that the
829
product of this yk and Mk modulo every other modulus except the kth modulus is 0. So, for
instance if k = 1, what I am saying is, we know that y1M1 is congruent to 1 modulo the first
modulus.
But the claim that I am now making is the following that y1 times m1 will be congruent to 0
modulo every other small modulo, that means you take the remaining n - 1 modulus says y1
times m1 will be congruent to 0 modulo those n – 1 modulus. Similarly, if you take y2 times
m2, we know that that is congruent to 1 modulo the second small modulo m2. But with
respect to the first modulo m1, the third module m3, fourth modulo m4 and so on y2 times m2
is 0 and the proof is very simple here.
So, I know that Mk is the product of n - 1 small modulus is here, that means it is a product of
all the n modulus expect the kth modulus. And hence, if I divide this Mk by m1 I will get
remainder 0 because this number is completely divisible by m1. If I divide this Mk by m2,
again it is completely divisible by m2, if I divide this Mk by mj again, it is completely
divisible by mj, if divided by mk – 1, it is completely divisible. If I divide it by the k + 1 th
modulus again it is completely that is a very simple fact.
So that is the third property that I have retained here, now remember, the proof strategy was
that I want to express my unknown x as a special linear combination of my remainders a1 to
an. And my claim is that now I have obtained those special linear combiners. So, my claim is
that if I take this linear combination of the n remainders, namely y1 times M1 times a1. So,
830
this is my first linear combiner, this is my kth linear combiner and this is my nth linear
combiner.
My claim is that this value of x (𝑥 = 𝑦1 𝑀1 𝑎1 + … + 𝑦𝑘 𝑀𝑘 𝑎𝑘 + … + 𝑦𝑛 𝑀𝑛 𝑎𝑛 ) is indeed a

solution for this system of linear congruences and you can easily verify that; what will
happen if I take the value of this x and compute modulo mk. I compute x modulo mk so, if I
compute x modulo mk then that will be same as this first summand modulo mk the second
summand modulo mk, the kth summand modulo mk and the last summand modulo mk.
Now what can I say about the first summand modulo mk, so I know that this property holds;
that means if I take the first summand here there M1 is present and M1 is congruent to 0
modulo mk that means M1 is completely divisible by my small modulus mk. So, this first
summand is completely divisible by mk, so it will give me the remainder 0. Similarly, the
second summand will have M2 which is completely divisible by Mk.
So that is why the overall second summand is completely divisible by Mk and it will give me
the remainder 0. But when I come to the kth summand here, in the kth summand, I know that
I have yk times Mk present and yk times Mk modulo mk is 1. So that is why this overall term
modulo mk will give me ak and again the remaining other terms will vanish they will turn out
to be 0 that means it tells me that if I divide x by mk, I will obtain the same remainder that ak
gives me on dividing by mk or equivalently x - ak is completely divisible by mk. So that
means, by following this process, I can find out at least one solution satisfying the whole
equation, the whole equation in the sense the whole system of linear congruences. But I want
to find out a solution in the range 0 to M – 1.
831
How do I obtain that? So, for that, again, let me reiterate what I am been saying again and
again, if you have one solution x, satisfying the n linear congruences here. Then any number
of the form x + l times your bigger modulus is also a solution for the same system of linear
congruences where l can be positive or negative. That means, let us first prove this claim and
then we will see how exactly we can find out a solution in the range 0 to M - 1.
So, since x is a solution which satisfies the system of linear congruences, that means, x has
these properties, that means x is congruent to a1 modulo m1, it is congruent to a2 modulo m2
and so on. Then what can I say about x + l M modulo m1, x + l M modulo m1 will be same as
x modulo m1 because l times M modulo m1 will give you 0 because M is completely divisible
by m1.
Because remember your M is the product of all the n modulus, that means, even though this
might look like a different number, this different number when divided by m1 will give you
the same remainder which you obtained by dividing just the value x by m1 and we know that
x on divided by m1 will give you the remainder a1. So, that means, this different number
satisfies the first linear congruence. In the same way, the same different number satisfies the
second linear congruence and so on.
So, now this claim is true, we have proved that, so assuming you have a solution x satisfying
your linear congruence. Now, if that x is not within the range 0 to M - 1, then you keep on
subtracting multiples of M from it, you make it small and small, because every time you
subtract one full multiple of M, the new number is still a solution.
832
That means, if x does not belong to the range; if this condition is not satisfied and you want to
find out an x’, which is also a solution and within the range 0 to M, then what I am saying is
that you keep on subtracting means you first compute x - M and check whether this x – M is
within the range 0 to M - 1 or not. If not, then compute x - 2M and compute x – 3M.
Because all of these new numbers also will be solution for your system of linear congruences
and eventually by appropriately choosing the value of l you will obtain an x will be in the
range 0 to M - 1 and which satisfies all the n linear congruences. So, that shows that using the
Chinese Remainder Theorem, you can obtain at least one solution modulo the bigger modulus
satisfying the system of n linear congruences, namely, you have to find out your special
linear combiners as we have seen in the last slide.
We have to find out this yk which is the multiplicative inverse of your kth sub modulus Mk
modulo mk and if you do this, then this x will be one of the solutions, if this x that you have
obtained as per the Chinese Remainder Theorem satisfies the condition that is it is in the
range 0 to M-1, then well and good else, you find out an appropriate multiple or you select an
appropriate value of l which will ensure that you obtain a solution in the range of 0 to M - 1.
So, that brings me to the end of today's lecture just to summarize, in this lecture, we
introduced linear congruences and we discussed 2 methods of solving the system of linear
congruence one using the extended Euclidean algorithm and another one due to Chinese
Remainder Theorem. Thank you.
833
Lecture – 59
Uniqueness Proof of the CRT
Hello everyone, welcome to this lecture, so, in this lecture we will continue our discussion
regarding solving linear congruences using CRT. And specifically we will focus on the
uniqueness part of the solution. So, we want to prove that there exists a unique solution in the
range 0 to M - 1 satisfying the system of linear congruence.
So, we start with some basic properties of divisibility, so the first property is the following
imagine you are given 3 positive integers a, b, c and it is given to you that a divides the
product of b and c but a is co-prime to b. Then I can conclude that a divides c that means, if a
834
divides the product of b and c but a is co prime to b then it must be the case that a divides c
and the proof is as follows.
So, we know that as per the Bezout’s theorem, we have integer linear combiners s and t, such
that I can write the GCD(a,b) which is 1 as per my premise as s times a + b times t. Now, if I
multiply both the sides of this equation by c, I get this equation: 𝑎𝑠𝑐 + 𝑏𝑡𝑐 = 𝑐, now I know
that it is given to me a divides the product of b and c and hence it divides any multiple of b
times c. So, it divides t times bc as well and any how a divides any multiple of a, so it will
divide s times c times a.
And now if I know that a divides 2 numbers, it divides the summation of those 2 numbers as
well, but the summation of the 2 numbers that I am taking here is nothing but the value c.
That is the proof of this fact, very simple fact but useful.
Now we prove another property which we often call as the Euclid’s Lemma, which is also
very useful while proving the uniqueness part of the CRT theorem. So, the Euclid’s Lemma,
is as follows, it says that if p is a prime number and if it is given that p divides the product of
n numbers. Then definitely, it has to be the case that p divides at least one of those n
numbers, it cannot be the case that p is a prime number and p does not divide a1 it does not
divide a2 it does not divide an.
But still it automatically divides the product of those n numbers; and there are again multiple
ways to prove this. I will prove it using induction because it is convenient to prove it using
835
induction, since it is a universally quantified statement for all. So, the base case will be for n
= 1 and which is trivial, because if it is given to you that p divides a1 that means p divides a1.
Now assume that the inductive hypothesis is true that means assume that if p divides the
product of k numbers where p is a prime, then there is at least one of those k numbers which
is divisible by p, I do not know which one but it is there. Assume this statement is true for
every k and every k numbers. Now let us do the inductive step and take a new number which
is the product of k + 1 number and imagine you have a prime number p which divides the
product of those k + 1 number. My goal is to show that there is at least 1 number out of this k
+ 1 number which is completely divisible by p.
So, the first thing to observe here is that since p is a prime number, what can I say about the
greatest common divisor of p and this bigger number. So let me call this bigger number as X,
so what can I say about the GCD(p, X)? The GCD will be either one or p because the only
divisor of p are 1 and the number p itself there cannot be any other third value of GCD. So,
there are 2 possible cases so let us analyse those 2 cases.
Now if the GCD of p and the product of the first k numbers is 1, then from the previous result
means in the previous slide I showed that if a divides bc and if GCD(a,b) is 1, then in the
previous slide I showed that it has to be the case that a divides c. So, my X is the product of
first k numbers and the k + 1 th number; so, I can treat my X as a1 to ak as product,. so, this is
my say A and I also have ak + 1 also in the product that is B. So, I know that p divides X that
means, p divides the product of A and B and I am in the case where p is co-prime to A
because this product of the first k terms I am calling it as A then from the previous result, I
know that p has to divide B and B is nothing but ak + 1. So, that proves my inductive step
because in the inductive step I have to show that there exists at least one number which are
involved in the product here in the number X which is completely divisible by p.
And I have shown the existence of one such number this is when the GCD(p,A) = 1. Now,
consider the case when the GCD(p, A) = p and then in that case, if the GCD(p, A) = p then p
of course divides A and if p divides A and A is a product of k numbers and since p is a prime
number, I can use my inductive hypothesis and argue that there exists at least one number
which are involved in the computation of A which is completely divisible by p that means
either p divides a1 or p divides a2 or p divides ak. So, that proves my Euclid’s Lemma,
important property which will be again useful.
836
So now, coming back to the uniqueness proof part for the Chinese remainder theorem in the
last lecture, we showed that there exists at least one solution in the range 0 to M – 1. How do
I prove that there is no other solution possible satisfying the same system of linear
congruences and which is also in the range 0 to M - 1, we have to refute the existence of
second solution.
So, again we will take the help of a helping lemma and this helping lemma will be useful
later as well. So, what this helping lemma says is the following: imagine you are given n
modulus which are pairwise relatively prime that means, you take any pair of modulus mi and
mj they are co-prime to each other. And suppose you know that you are given 2 numbers a
and b which are congruent with respect to all the n individual modulus.
837
It means they are congruent with respect to modulo m1, they are congruent with respect to
modulo m2, and they are congruent with respect to modulo m3 and so on. Then, the claim
here is that the same 2 numbers a and b are congruent with respect to the bigger modulus M,
which is the product of all the n modulus and again there are multiple ways to prove this, let
us follow the following strategy.
So, as per the fundamental theorem, I know that this bigger modulus M must be having a
unique prime factorization, that means, I can express this bigger modulus M as product of
powers of prime. So, let those powers e1, e2, eq and so on. Now, my goal is to show the
following, in my proof I will show that if I take the prime power factorization of a - b. So, a -
b also will be a number and it will have a prime power factorization.
e1’ e2’
So, I have to select the prime power factorization of a – b be p1 , p2 , and like that pq eq’
and so on. So, what I want to show is that each of the prime factors, which are involved in the
prime factorization of M, they are also involved in the prime power factorization of a - b and
with at least the same individual powers with which they were involved in the prime
factorization of M.
So, what I am trying to say here is the following: say for instance, if my M was say 23.31.56
and so on. Suppose these are the various powers of primes which are involved in the prime
power factorization of M, my goal will be to show that if I consider a - b, then the same prime
factor 2 is involved at least 3 or more than 3 times that means it will be either greater than or
equal to 3 power it is must that means 2 should have 3 or more power appearing in the prime
factorization of a - b.
Similarly, the factor 3 should appear with the power at least 1, the next prime factor 5 should
appear with at least power 6 and so on. If I show this, then that shows that a - b is completely
divisible by M; if I show this then this will imply that a - b is divisible by M. And that is what
precisely I want to show I want to show a is congruent to b modulo M. And remember, an
equivalent definition of congruence is that if a is congruent to b modulo m then that also
implies a - b is completely divisible by M.
So, if I can show that each of the prime factors, which are involved in the prime factorization
of M are also involved in the prime factorization of a - b and with at least the same powers
838
with which they were appearing in the prime factorization of M, that means, a - b is
completely divisible by M and that is what the proof strategy will be.
So, let us consider an arbitrary prime factor of the bigger modulo M and suppose it is
appearing with power e in the prime power factorization that means so, M = 2e1, 3e2 and so on
and some pe so on. So, I am taking an arbitrary prime which is appearing with some power in
the prime power factorization of M, I have to show that the same power, at least, is also
present in the prime power factorization of a - b as well.
Now, since p is occurring with power e in the prime power factorization of M, that means I
can definitely say that p divides the product of m1 to mn because M involves a prime power of
the form pe in its prime power factorization; that means, p has to divide the product of m1 to
mn which is nothing but M. That means, this condition holds. And now, I can apply the
Euclid’s Lemma which I just proved some time back that if p is a prime number and if it
divides the product of n values, then it has to divide at least one of those n values.
So that means p divides at least one of the small modulus, let us call it as mi and I know that p
does not divide any other modulus mj, I can conclude that because I know that the various
modulus m1 to mn they are pairwise prime that means, there cannot be any other modulus mj
such that p divides that other modulus mj as well because if p divides the other modulus mj as
well, then I already have the fact that p divides mi and if p divides mj as well, then I will get
the conclusion there is a common divisor other than 1 namely the prime number p which
divides both mi and mj and which goes against assumption that my modulus mi and mj are
839
pair-wise prime. And if p does not divide any other modulus mj and I know that pe is
occurring in M in the prime power factorization, then the only way the contribution pe can
come in the prime power factorization of M is because the pe was contributed in the prime
power factorization of mi itself so, remember each of the modulus m1, m2, mi, mj, mn will
individually have their own prime power factorization and I have the product of all this
modulus which is M, I know that pe is contributed in the prime power factorization of M. So,
this pe might be accumulated through several modulus m1, m2, mi, mj, mn.
But what I have shown here is that if at all p is coming from mi that means if p’s contribution
was there in the prime power factorization of mi. Because if p divides mi then p would not be
appearing in the prime power factorization of m1, p would not be appearing in the prime
power factorization of m2, p would not be appearing in the prime power factorization of mj, p
would not be appearing in the prime power factorization of mn and so on. So, that means the
only way this pe would have been accumulated in the prime power factorization of M is
because it was present in the prime power factorization of mi itself.
So that is a conclusion I have drawn that pe occurs in the prime power factorization of mi.
Now, remember, the properties of a and b is the following they are congruent with respect to
every modulus that means a - b is completely divisible by all the n individual modulus. So, a -
b is also divisible by mi as well because as per my premise a and b they are congruent modulo
mi as well, the same mi where pe occurs in the prime power factorization.
840
That means pe occurs in the prime power factorization of mi and if mi divides a - b that means
pe also occurs in the prime power factorization of a - b as well and that is what precisely I
wanted to show. So, namely if I substitute p with p1 I have shown that p1e1 will also occur in
the prime power factorization of a - b. If I substitute p with p2 in this whole proof then I
concluded that at least p2e2 also occurs in the prime power factorization of a – b; p2 can occur
with higher power as well, but at least p2e2 is definitely there that much power is always there
in the prime power factorization of a - b and so on.
So, we have proved the helping lemma now, coming back to the proof of the uniqueness of
the solution, we wanted to prove that there is a unique solution x in the range 0 to M - 1
satisfying the system of n linear congruence says, so we will prove it as follows. On contrary
imagine you have 2 solutions, 2 solutions in the range 0 to M - 1 satisfying this system of
linear congruences and those solutions be let x and y.
That means x satisfies this system of n linear congruence is that when x is congruent to a1
modulo m1, x is congruent to a2 modulo m2 and x is congruent to an modulo mn. And
similarly, since y is also a solution, for the same system of linear congruences, this set of n
linear congruences will also get satisfied. Now, from the first equation here and the first
equation here, I get that x is congruent to y modulo mk, or x is congruent to y modulo m1.
Because x - a1 is completely divisible by m1 and y - a1 is completely divisible by m1, then

what can I say about x - y, if m1 divides x - a1 completely that comes because of the first
linear congruence here, and this linear congruence tells me that m1 divides y – a1. Then I can
841
say that and m1 divides the difference of these 2 numbers as well and the difference of these 2
numbers will be x - y.
In the same way, I can say that x - y is completely divisible by m2, I can say that x - y is
completely divisible by mk and x - y is completely divisible by mn. So, I get n congruences
like, that means x and y are congruent modulo m1, m2, mn and remember that my m1, m2, mn
they are pairwise relatively prime.
So, I can take the help of helping lemma and I can conclude that both x and y are congruent
modulo M. And since both x and y were in the range 0 to M - 1, that means they were strictly
less than M, and both of them are congruent, then that is possible only when x = y that shows
that there exists a unique solution modulo M satisfying your system of linear congruence.
842
So now, let us see an example for Chinese remainder theorem. So, say we want to find out
this unknown x satisfying the system of linear congruences : 𝑥 ≡ 2 mod 3, 𝑥 ≡ 3 mod 5, 𝑥 ≡
2 mod 7. So, we will find out the bigger modulus and sum modulus so, the bigger modulus
will be the product of 3, 5, 7 and you can see your m1 is 3, m2 is 5, m3 is 7 and a1 is 2, a2 is 3
and a3 is 2. So, my bigger modulus will be 105, M1 will be the product of all the modulus
except 3, so 35. M2 will be the product of all the small modulus except 5, and M3 will be the
product of all the 3 modulus except 7 so, I found M1, M2, M3. Now, my next goal will be to
find out M1 inverse modulo m1, M2 inverse modulo m2 and M3 inverse modulo m3, which I
can do by using extended Euclid’s algorithm. So, M1 inverse modulo m1 will be 2 because,
you can see that your M1 is 35 if you multiply 35 with 2 and then you take small modulo m1
then you will get answer 1 in the same way M2 inverse modulo m2 is 1 and M3 inverse
modulo m3 is also 1. So, then as per the Chinese remainder theorem, we will compute the
value x which is the linear combination of your a1, a2 and a3 and the linear combiners are the
various m1, m2, m3 and their respective multiplicative inverse multiplied with each other, so
this will be the value of x = 233 𝑥 ≡ 2 ⋅ 35 ⋅ 2 + 3 ⋅ 21 ⋅ 1 + 2 ⋅ 15 ⋅ 1 mod 105, 𝑥 ≡
233 mod 105.
Now this x will be 233 modulo 105 so, our goal is to find out the unique solutions of course,
233 is a solution if I take x = 233 you can verify that it satisfies the system of linear equation,
but we want to find out a unique solution in the range 0 to 104 so, how I can do that I can
keep on subtracting 105 or equivalently I can directly take 233 modulo 105 because that will
tell me exact number of times 105 have to be subtracted so that I get a remainder within the
range 0 to 104, namely 23 which will be a solution for the system of given linear
congruences.
843
So now, let us see some application of Chinese Remainder Theorem, it has tremendous
applications, of course in cryptography, but in general it has other applications and our main
application is when we want to do arithmetic with large values. So, what basically CRT tells
us that if you are dealing with very big modulus and you want to do arithmetic involving
those big modulus, then instead of doing operations modulo those big modulus you can do
operations with small modulus and they will be equivalent, what do I mean by that?
So imagine, you are given n modulus m1 to mn which are relatively prime and you are given a
bigger modulo M, which is a product of these n modulus. So, consider the set ZM, the set ZM
is nothing but it has all the integers 0 to M – 1 and you have n number of small sets here, you
have Zm1 which is nothing but you have all the integers from 0 to m1 - 1, Zm2 has all the
integers from 0 to m2 - 1 and Zmn has all the integers from 0 to mn – 1, we will later encounter
these sets again.
Now what Chinese Remainder Theorem basically tells you: it establishes a bijection between
this bigger set ZM and the Cartesian product of these n sets, what exactly is the bijection? The
bijection is the following: if you are given a value a here and you want to find out the
corresponding mapping as per this bijection then the image of a is obtained by computing a
modulo m1, a modulo m2, a modulo mn that will be the representation of a.
And my claim is that this representation that we have obtained is an injective mapping
because, if you have 2 different values a and b where a is different from b then definitely
there will be at least one mi where a modulo mi and b modulo mi will be different because if a
844
modulo m1 and b modulo m1 is same, a modulo m2 and b modulo m2 is same, a modulo mi
and b modulo mi is same and a modulo mn and b modulo mn are same, then since my
modulus m1 to mn are pairwise relatively prime using the helping lemma that we have just
proved we come to the conclusion that a and b are also congruent namely, they are same
because both a and b are in the range 0 to M - 1. So, you cannot have 2 different values a and
b and at the same time their representations are also same; by the way by representation I
mean though each value in the range 0 to M - 1 or an element of ZM will be now represented
by an n tuple.
So, why n tuple because there will be n values which will be treated as the representation of
a, so, that is an injective mapping and the mapping is surjective as well, because if I give you
arbitrary values of a1, a2, an where a1 is in the range 0 to m1 - 1, a2 is in the range 0 to m2 - 1
and an is in the range 0 to mn – 1, then I can find out the corresponding a in the range 0 to M -
1 whose CRT representation will be a1, a2, an that shows that my mapping is subjective as
well.
So, basically what I can now say is the following, if you are given 2 numbers a and b and
their corresponding representations; that means, now any operation which you want to do in
the bigger set modulo the bigger modulus that can be equivalently performed in their smaller
worlds modulo small modulus, modulo m1, modulo m2, modulo mn. So, what do you have to
do is you have to focus on the first component of the representation of a and b, they will be in
this set.
845
You perform the same operation which you want to perform in the bigger set and do modulo
the small modulus, you perform the same operation in the second world modulo m2, you
perform the same operation in the nth world modulo mn and so on and same holds for product
as well. That means equivalently what it shows is that using CRT any arithmetic operations
which you want to perform over integers modulo some bigger modulus that is equivalent to
performing arithmetic over the remainders as well. And this is a very interesting fact which
we use extensively at least in cryptography. So, for instance, if your, say m1 is; each of this
modulus is m1, m2, mn are n bit prime numbers then my M is an enormously large value.
Now if I want to do a + b modulo that enormously large modulus then it will be an overkill
instead, what I am saying is that to perform a + b modulo several small modulus and that will
be the equivalent representation of whatever remainder you would have obtained by adding a
and b in the modulo the bigger modulus. So, that gives you a tremendous saving in the
computation that is involved.
So that makes CRT a very interesting theorem, it has got tremendous application especially in
cryptography. So, with that I conclude today's lecture, these are the references and just to
summarize, in this lecture, we continued our discussion on the Chinese Remainder Theorem.
And we proved that, indeed there exists a unique solution modulo the bigger modulus,
satisfying the given system of linear congruences. Thank you.
846
Lecture - 60
Fermat’s Little Theorem, Primality Carmichael Numbers
Hello everyone, welcome to this lecture, so the plan for this lecture is as follows: in this
lecture, we will discuss about Fermat's little theorem, and we will see its application to
primality testing, and we will also discuss about Carmichael numbers.
So let us begin with Fermat's little theorem so, the theorem says that, if p is a prime number
and if a is an integer such that p does not divide a. So, this notation means does not divide : ∤,
in other words, a is co-prime to p, then the theorem says that ap - 1 ≡ 1 modulo p. And this is
847
true for every integer a, which is co-prime to p. So, that is the Fermat's little theorem
attributed to Fermat.
Why it is called little theorem because we want to distinguish this theorem from another
interesting theorem attributed to Fermat's, which is also called as Fermat's last theorem. And
this theorem also forms the basis for an interesting primality testing that we will see later on.
So, this is the theorem statement but before going into the proof of the theorem statement, let
us see an interesting corollary of this theorem statement.
So assume for the moment that this theorem statement is true, let us see an interesting
corollary. So, the corollary is ap ≡ a modulo p for every integer a and prime p. And we can
divide the proof into cases so, the corollary is for every integer it is not only for those integers
a which are co-prime to p, whereas the Fermat's little theorem is strictly for those integers a
which are co-prime to p. So, let us see the proof of the corollary first, so, we have 2 cases
depending upon whether p│a or not.
So, the first case is when p│a so, if you have an integer a such that p│a, then since a is
divisible by p any multiple of a is also divisible by p. So hence, ap is completely divisible by
p. And that means I can say that ap gives you the same remainder, which a gives you on
getting divided by p namely the remainder 0 because both a as well as ap will be completely
divisible by p if a is divisible by p. So that proves that the corollary is true for case 1.
Whereas if p ∤ a, then we can apply the Fermat's little theorem so if p ∤ a then that means the
premise of the Fermat's little theorem is satisfied. That means I can say that ap - 1 ≡ 1 modulo
p. And if that is the case, then if I multiply both sides by a, I get the conclusion that ap ≡ a
modulo p so that proves that the corollary is true even for the case when a is not divisible by
p.
848
So now let us come back to Fermat's little theorem and we prove it. So, we want to prove that
you take any integer a, which is co-prime to p, then ap - 1 ≡ 1 modulo p that is what we want to
prove. So the proof is as follows: so, you consider the first p - 1 multiples of a, namely, 1
times a, 2 times a, 3 times a like that p - 1 times a, all these are different multiples of a. The
claim is that all these multiples of a namely 1 times a, 2 times a, 3 times a, p - 1 times a when
getting divided by p will give you distinct, non 0 remainders.
That means whatever is the remainder that you will obtain, by dividing 1 time a by p call that
remainder is r1 whatever remainder you obtained by dividing 2 times a by p call that
remainder as r2 and like that whatever remainder you obtain by dividing p - 1 times a by p
called that remainder as rp-1, the claim is that none of these remainders are same and of
course, all of them are non 0. So, the proof will be by contradiction, we will now prove this
claim by contradiction.
So, imagine you have 2 different multiples of a say r times and s times a where r and s
belongs to the set 1 to p - 1 such that the remainder which you obtain by dividing r times a on
dividing by p and the remainder that you obtain upon dividing s times a by p are same. The
claim says that is not the case, but assume on contrary that you have 2 such different
multiples, which gives you the same remainder.
Now, if r times a and s times a are congruent modulo p, then as per the definition of
congruence, I can say that r times a - s times a is completely divisible by p: (p│( ra – sa)) .
That means p divides r - s times a: (p│ a(r - s) ); that implies that p has to divide r – s : (p │
849
(r – s) ), because as per the premise of my theorem statement GCD(p, a) is 1 as a is co-prime
to p. So, this is the premise of my theorem statement you have a number p which is prime and
p does not divide a that means the GCD (p, a) = 1.
So now, you can recall one of the properties of divisibility that we had discussed in earlier
lecture. If p is a prime, which divides the product of 2 numbers and one of the numbers in a
product is not divisible by the prime that means the other number has to be definitely
divisible by the prime. So you have 2 numbers, A B, so you can imagine r - s as A and a as
B. So, we have p divides the product of A and B, but p does not divide B. So that is possible
only if p divides A, A is r - s.
So, we get the conclusion that p has to divide r - s or in other words, r and s are congruent
modulo p. But if r and s are congruent modulo p and since both r and s are strictly less than p,
then the only way it is possible that r is congruent to s modulo p is that r is exactly equal to s.
Of course, if r and s would have been outside the range 1 to p - 1 then it might be possible
that even though the value of r and value of s are different, but still they are congruent
modulo p.
But remember that r and s are strictly less than p and if they are congruent modulo p, then
that is possible only when r = s. So, we arrive at a contradiction because we assumed that r
and s were different. So, r times a and s times a were different multiples and that means r was
different from s that was our assumption, but we come to the conclusion that r = s. So, that
means, whatever contrary statement we assumed is incorrect that means this claim is true so,
we have proved this claim.
850
Now, my claim is that if you multiply these p - 1 multiples of a, so, you have 1 time a, 2
times a, 3 times a, p - 1 times a if you multiply them you will get one number. If you divide
that number by p you will get the same remainder which you will obtain if you multiply the
numbers 1, 2, 3 up to p - 1 and then divide the resultant value by p. That means call the value
on your left hand side as X, call the value on your right hand side is Y I am saying here that
X ≡ Y.
And this follows from your claim that we had just proved this is because as per claim the
various remainders which you obtained by dividing these p - 1 distinct multiples of a, call
them as r1, r2, rp-1, we have proved that all these remainders are non 0 and they are distinct.
That is what the claim we have just proved and all these remainders r1, r2, rp-1 they are the
remainders obtained by dividing a value by p.
So, the possible remainders that you can obtain could be 0 to p - 1. But 0 is not a possible
remainder as per the claim statement we are not going to get a 0 remainder. That means the
only remainders that I can obtain are 1 to p - 1 and at the same time, they are distinct. So that
means definitely, out of this p - 1 remainders, one of the remainders is definitely 1, out of this
p - 1 remainders that we are getting one of the remainders is definitely 2, out of this p - 1
remainders that we are getting one of the remainders is definitely p - 1.
So, that means, remember, the law of modular arithmetic says that if you want to multiply
many numbers and then want to take modulo, you want to compute the remainder, then that is
equivalent to saying that you reduce each of the numbers first modulo the same modulus and
851
then multiply them. So I can say that the value of X modulo p will be the same as multiplying
the remainders r1, r2, rp-1 and then taking modulo p.
And I know that r1, r2, rp-1 are the values 1 to p - 1 in some order; r1 may not be exactly 1, r2
may not be exactly 2, rp -1 may not be exactly p - 1. But I know that the remainders 1 to p - 1
occurs exactly once among these p - 1 remainders. So that is why I can say that the product of
these p - 1 remainders is congruent to the product of 1, 2, 3 up to p - 1 modulo p, which is Y.
So that means now I can say the following, so if you see the expression X the value a is
appearing p - 1 times, so I can take out ap - 1 outside. And then if I collect the product of 1, 2,
up to p - 1, that will be (p – 1)!. Whereas in my right hand side, namely Y, I have (p – 1)!,
because that is a product of 1, 2 up to p - 1. So that means I can say that ap - 1 times (p – 1)! ≡
(p – 1)! modulo p.
Now, what I will do is the following, let me multiply both sides of this equation by the
multiplicative inverse of (p – 1)! modulo p. Now, you might be wondering, what is the
guarantee that the multiplicative inverse of (p – 1)! modulo p exists. So recall we discussed in
one of our earlier lectures, that multiplicative inverse of a value modulo some modulus N
exists if and only if that value is co-prime to the modulus. So that means (p – 1)!
multiplicative inverse will exist only if this condition is true.
Because your modulus is p, and this is the value X, whose inverse you want to find out; let us
not call it X, because I have already used X for something else so call this value as Z. So my
claim is that this is the value Z which is (p – 1)! is indeed co-prime to n. And it is very simple
to prove that I am not going to prove that for you I leave that as exercise for you. So, since
the multiplicative inverse of (p – 1)! exist, if I multiply both sides by the multiplicative
inverse of (p – 1)!, then this (p – 1)!, when multiplied by its multiplicative inverse will give
me 1 and 1 multiplied by ap - 1 will give me ap - 1 in the left hand side whereas in the right hand
side, when I multiply (p – 1)! with its multiplicative inverse, I get 1. So I get the conclusion
that ap - 1 ≡ 1 modulo p, which proves my Fermat's little theorem.
852
So this is the Fermat's little theorem, which says that every integer a which is co-prime to p
satisfies the condition that ap - 1 ≡ 1 modulo p. And I have a corollary of this theorem that you
take any integer a which need not be co-prime to a, it satisfies the property that ap ≡ a modulo
p. So, now let us see some of the applications of this theorem it has this Fermat's little
theorem has got tremendous applications.
So let us see how exactly we can use this theorem to compute the value of some expressions
modulo some modulus which is a prime number. So, your modulo modulus p here is prime
and say I want to compute the value of 7222 modulo 11 of course, you can write down a
computer program and compute the value of 7222 modulo 11. But I want to do it very quickly
using my paper and pen and using Fermat's little theorem.
So what I can do here is the following, if I substitute a = 7 and p = 11 in Fermat's little

theorem, then I see that the condition of the Fermat's little theorem is satisfied because indeed
GCD(7, 11) is 1, namely 7 is co-prime to 11 because the GCD(7, 11) is 1 and hence, I can
say that 710 ≡ 1 modulo 11 that means, if you divide 710 by 11, you will get a remainder 1.
Now I can rewrite 7222 as follows, I can treat it as 710 modulo 11 multiplied by 710 modulo 11
multiplied by 710 modulo 11 and then finally 72 modulo 11 and then everything modulo 11.
So, basically what I am doing here is that 222 can be rewritten as 220 + 2 and now, this 220
can be written as 22 times 10 and then you have 2 anyhow, so this 7222 I have splitted it into
many blocks of 710, 710, 710 namely 22 blocks and then finally I will be left with 72.
853
And then since I have to do or I have to compute everything modulo 11 I can take modulo 11
with each block of 710. Again this comes from your rules of modular arithmetic. Now, I know
that 710 modulo 11 will give me 1 so, each block of 710 modulo 11 will give me 1, 1, 1, 1, 1
that means I will get 1 multiplied with itself 22 times which will be giving me again 1 and
then that will be multiplied with 72 modulo 11 and 72 modulo 11 is 5 and then I can say that 1
into 5 is 5, 5 modulo 11 is 5.
So, you can now see that I do not need to write any complicated program or I do not need to
do any sophisticated computation I can simply apply the Fermat's little theorem and so
conveniently I can compute the value of 7222 modulo 11.
Now, as I said at the beginning of the lecture Fermat's little theorem also forms the basis of
very interesting primality testing algorithm. We would not be seeing the full primality testing
algorithm, but we will see a part of it. So, this is the statement of the Fermat's little theorem,
which says that if you have a number p which is prime and an integer a; if you have an
integer a which is co-prime to p, then for every such integer a, the value of ap - 1 ≡ 1 modulo
p.
So now the question is can I use the theorems statement to check whether a given number n is
prime or not, of course, the number n has to be odd, because if I give you n = 2 you can easily
verify, you can easily conclude that it is a prime number because that is the only even prime
number. But other than that if at all n is a prime number it has to be odd. So now, you are
854
given an odd prime number, it might be an arbitrary large prime number and you want to
utilize Fermat's little theorem to verify whether the given number is prime or not.
So the idea here will be that I will pick some arbitrary integer b such that b is co-prime to n
and then I will check whether bn - 1 ≡ 1 modulo n or not, I do not know whether n is prime or
composite, I have to check. So what I am saying is to verify whether the given n is prime,
pick some random integer b, which is co-prime to n. And then check whether for the B that
you have chosen bn - 1 ≡ 1 modulo the given n. Now, you will get either the answer yes or no.
If you see that bn - 1 ≢ 1 modulo and then you can simply declare that the given number n is
composite, because that comes from the contrapositive of your Fermat's theorem. So the
Fermat's theorem states that if you have a number a which is co-prime to p. And if p would
have been prime, and ap - 1 will give you 1 modulo p, so the contrapositive of that will be if
you have a number a which is co-prime to p, and if ap - 1 ≢ 1 modulo p, then that implies that
p is not prime, even though a is co-prime to p, that is the contrapositive. And that is what
precisely we are using here. But what if bn - 1 ≡ 1 modulo n can I declare my n to be a prime
number, that is the problem with this primality testing. Even if bn - 1 ≡ 1 modulo the given n,
you cannot necessarily declare your number n to be a prime number.
So here is a counter example, so imagine you are given n is 341, which is an odd number, and
which is not a prime, it is a composite number because the number 341 has factors 11, 31.
Now suppose when you run this primality testing algorithm, for n = 341, you pick your b to
be 2. So indeed, GCD of 2 and 341 is 1. And it also turns out that for the given b that you
have chosen arbitrarily, bn - 1 ≡ 1 modulo the same n.
So even though this condition is true, the condition of Fermat's little theorem is true, you
cannot declare your n to be prime because the value n = 341 is indeed composite. So that is
why this primality testing algorithm is not a robust algorithm; robust in the sense, you cannot
trust the answer. If the answer is composite yes you can trust it. But you cannot trust answer
that n is prime. It may be the case that even though your value n is not a prime number, the
condition for Fermat's little theorem is satisfied.
So now, you might be wondering that why cannot I do the following? It might be possible
that I have chosen a bad b with respect to my given composite number n what if I choose a
855
good b, which is coprime to n, and for which the Fermat's little theorem condition fails. In
that case, I can declare that my n is not a prime number why cannot I do that.
However, it turns out that even if you do so, your primality testing algorithm will fail because
there are some very interesting numbers which are called as pseudo primes and Carmichael
numbers, which will cause your primality testing algorithm to fail for the case when your n is
composite, but you are not able to detect that. So let us first define pseudo primes and then
we will use it to define Carmichael numbers. So, imagine you are given positive integers b
and n and say your n is composite.
Now, if it turns out that bn - 1 ≡ 1 modulo n, then I will call my n to be a pseudo prime to the
base b. Why I am calling it pseudo prime, because it is a false prime. In the sense even
though my n is composite, it satisfies the condition of Fermat's little theorem with respect to
the integer b. That is why I am calling it base b because b is appearing in the base and n - 1 is
appearing in the exponent.
So for instance, the counter example that we just saw in the previous slide shows us that a
value n = 341 is a pseudo prime it is a false prime because it is actually a composite number,
but still it satisfies the condition of your Fermat's little theorem with respect to your base b =
2. So, now as I said earlier, you might try to run the primality testing algorithm with respect
to several bases for a given number n with a hope that indeed if your value n is composite
you hit upon some base for which the condition or the conclusion of Fermat's little theorem is
not satisfied. So, what we are doing here is we are now proposing a modified primality
856
testing algorithm where instead of picking 1 base b, which we had done earlier. We are now
randomly picking many bases say m number of bases b1 to bm each of which is co-prime to
your given number n you want to check whether the number n is prime or composite.
And now, you check whether the condition of Fermat's little theorem holds for each of the bi,
and given n namely you check whether b1n – 1 ≡ 1 or not b2n - 1 ≡ 1 or not and bmn - 1 ≡ 1 or not.
Even if for one of the basis bi, the condition of the Fermat's little theorem does not hold, you
can very confidently declare that your number n is composite.
But what it so happened that for each of the m bases which you have randomly chosen, the
condition of the Fermat's little theorem is satisfied, can you declare your given n to be a
prime? Unfortunately, we cannot do that and there are some wonderful numbers very
interesting numbers which are called as Carmichael numbers, which will actually cause your
modified primality testing algorithm to fail.
So, what exactly are Carmichael numbers, so, they are composite numbers, which are pseudo
primes with respect to every base that you can think of. So, you pick any base b or any
integer b, which is co-prime to your number n still, the condition of Fermat's little theorem
will be satisfied; that means bn - 1 ≡ 1 modulo n, it does not matter whether your base b is b1,
b2 you pick any base, that base power n - 1 will be congruent to 1 modulo n.
Namely, your n will be a pseudo prime with respect to every base that you can think of. So
that tells you that if you now input a number n, which is a Carmichael number, then definitely
this primality testing algorithm will fail because it does not matter how many bases you pick,
each of them may be co-prime to your n fine, but still, this condition will hold. And then you
will be in a dilemma whether I should safely declare n to be a prime or not.
If you declare n to be prime, then that is a false conclusion because actually your number n is
composite. So that is why primality testing algorithm based on Fermat's little theorem is not a
fool proof test. And we need to make additional tests in the modified primality testing
algorithm to get a fool proof primality testing algorithm whose details I am not going to
discuss.
857
So now, you might be wondering, do Carmichael numbers indeed exist? And if they exist, are
they finite? Or are they infinite in number? So, indeed, it turns out that we have lots of
Carmichael numbers. In fact, the study of Carmichael numbers itself is a very interesting
research topic in number theory. So, let me give you an example of a Carmichael number. So,
my claim is that 561 is a Carmichael number. So, you can see that 561 is not a prime number.
Because I have written down its prime power factorization, namely, you have 3 factors p1, p2
and p3 : (3, 11, 17) for the value n = 561. Now, I want to prove that the value 561 is indeed a
Carmichael number for that I have to prove that it is a pseudo prime with respect to every
base, you take, think of any base b, which is co-prime to your n. I will show that bn - 1 ≡ 1. So
that is this is my goal I am not focusing on the value of the base b, the only thing that I know
is that it is co-prime to your n namely 561.
So, how do I prove that b560 will be indeed 1 so, since I know that the GCD of b and 561 is 1,
I can imply that individually b is co-prime to each of the prime factors of my n. So, b will be
co-prime to 3, b will be co-prime to 11 and b will be co-prime to 17 it is very easy to prove
this implication, because, if any of these implications is false say for instance, if GCD of b
and 3 is not 1 that means, there is some common factor for b as well as 3.
So, then I can come to a contradiction that GCD of b and 561 is not 1, that is as simple as
that. Now, since b is co-prime to 3, b is co-prime to 11 and b is co-prime to 17 I can do the
following: I can rewrite b560 and my goal was to show that b560 ≡ 1, so b560 I can rewrite as b2
and then whole raise to power 280 and b560 ≡ 1 modulo 3 why so?
858
Because b560 modulo 3 is same as b2 whole raise to power 280 modulo 3, but I know that
since b is co-prime to 3, I can apply the condition of Fermat's little theorem, so, I have I can
treat 3 to be my prime p and I can treat b to be a number which is co-prime to 3. So, that
gives me that b3 - 1 namely b2 ≡ 1 modulo 3. So, now if b2 gives me the remainder 1 modulo 3,
then b560 will give me this much remainder.
So, I have basically divided b560 into several blocks of b2, b2, b2 namely 280 blocks and each
block of b2 gives me the remainder 1 modulo 3. So, basically, I get 1 multiplied to itself 280
times which will be 1 modulo 3. In the same way b560 ≡ 1 modulo 11 and this is again
because 11 is a prime number and b is co-prime to 11.
So, I can say that from Fermat's little theorem b11 - 1 which is b10 ≡ 1 modulo 11 and hence
b560 can be rewritten as 56 blocks of b10 modulo 11 each block of b10 modulo 11 will give me
1 as the remainder and 1 multiplied to itself 56 times will give me the remainder 1 and in the
same way b560 ≡ 1 modulo 17.
Because again I can apply the Fermat's little theorem here 17 is a prime number b is co-prime
to 17. So, from Fermat's little theorem b17 - 1 namely b16 will be co-prime to 1 modulo 17. So,
I can rewrite b560 as 35 blocks of b16 modulo 17 each block of b16 modulo 17 will give me
remainder 1 and hence I will get 1 multiplied to itself 35 times which will give me the
remainder 17.
So that means b560 ≡ 1 modulo 3, b560 ≡ 1 modulo 11 and b560 ≡ 1 modulo 17. So recall the
CRT helping lemma so let me rewrite the CRT helping lemma so there you had n modulus.
Where n number of modulus, which are pairwise co-prime, and you have 2 values a and b,
which are congruent to each other, with respect to all the modulus, so you have a ≡ b modulo
m1, a ≡ b modulo m2 and like that, a ≡ b modulo mn.
And it is given that all these n modulus, they are pairwise co-prime, then the conclusion is
that a ≡ b even modulo the bigger modulus, which is the product of all the individual
modulus so, that was the CRT helping lemma. So, now I can treat my A to be b560 I can treat
my B to be 1 and I have 3 modulus here, m1 being 3, m2 being 11 and m3 being 17.
859
So, A ≡ B modulo m1, A ≡ B modulo m2, A ≡ B modulo m3 and hence, I can say that A ≡ B
modulo the product of m1, m2, m3. The product of m1, m2, m3 is nothing but 561. A is anyhow
b560 and B is 1. So, I have shown that b560 ≡ 1 modulo 561 without even knowing the value of
b, so, b was an arbitrarily chosen base such that it was co-prime to your number n. So, that
shows that the value 561 is indeed a Carmichael number.
So, that brings me to the end of today's lecture and with that I also finish my discussion
regarding the number theory. As I said earlier, that number theory in itself is a very
interesting subject and we can have a full-fledged course just on number theory. But we want
to get just a flavour of number theory that is required in the context of discrete maths and
computer science. Thank you.
860
Lecture – 61
Group Theory
Hello everyone, welcome to this lecture. We will focus on abstract algebra for the next few
lectures and we will start with group theory. So, the plan for this lecture is as follows: we will
discuss the definition of groups and we will see various properties of groups. And we will also
see various examples of groups.
So, let us start with the definition of group. So what is a group? Imagine you are given a set 𝐺,
which may or may not be finite, and you are given some binary operation. By binary operation
I mean it operates on 2 operands from 𝐺. So, 𝐺 along with the operation ∘ will be called a group
861
if it satisfies certain properties, which we often call as group axioms. So, let us see what are
the group axioms.
The first axiom which we call as 𝐺! is the closure property and the closure property demands
that you take any 2 operands 𝑎, 𝑏 from your set 𝐺, if you perform the operation ∘ on 𝑎 and 𝑏
the result should be an element of the set 𝐺 itself. And hence the name closure. This is true for
every 𝑎, 𝑏 namely even when 𝑎 = 𝑏 as well.
The second property or axiom is the associativity property, denoted by 𝐺" , which demands that
your operation ∘ should be associative i.e., the order of the operands does not matter. Namely,
for every triplet of values 𝑎, 𝑏, 𝑐 from 𝐺, (𝑎 ∘ 𝑏) ∘ 𝑐 = 𝑎 ∘ (𝑏 ∘ 𝑐).
The third property or the axiom is the existence of identity denoted by 𝐺# which demands that
there should be a unique element denoted by 𝑒 present in 𝐺 called as the identity element such
that the identity element satisfies the following property for every group element. If you
perform the operation ∘ on the element 𝑎 and the identity element, you will obtain the same
element 𝑎. And this holds even if you perform the operation on 𝑎 and 𝑒 or if you perform the
operation on 𝑒 and 𝑎 i.e., 𝑎 ∘ 𝑒 = 𝑒 ∘ 𝑎 = 𝑎.
The fourth property and the last property which you require from a group is that of existence
of an inverse element, which demands that corresponding to every element from the set 𝐺 there
should exist a unique element in 𝐺, which we denote by 𝑎$! , such that the result of the group
operation on 𝑎 and 𝑎$! (or vice versa) is the identity element i.e., 𝑎 ∘ a$! = 𝑎$! ∘ a = 𝑒. I
!
stress here that 𝑎$! does not mean %. Rather it is just a notation for a special element which is
required to be present in the group for this property to hold. So, if 𝐺 along with the binary
operation ∘ satisfies all these 4 axioms, then (𝐺,∘) is a group.
Even if one of these 4 properties is violated, the set 𝐺 along with operation ∘ would not be
constitute a group. An important point to note here is that the axioms do not require the
operation ∘ to be commutative. The group axioms only demand the operation ∘ be associative.
That means, the result of performing the group operation on 𝑎 and 𝑏 need not be the same as
the result of performing the group operation on 𝑏 and 𝑎. Moreover, as discussed earlier, the
!
element 𝑎$! should not be interpreted as the numerical %.
862
So now, let us see some examples of groups. So, the set of integers ℤ which is an infinite set
along with the operation + constitutes a group. So, let us see whether all the 4 properties are
satisfied or not. So, the closure property is of course satisfied; you take any 2 integers 𝑎 and 𝑏
and add them you will again obtain an integer. The operation is associative over the integers
too since if you take any 3 integers, it does not matter in what order you add them, the result
will be the same.
The integer 0 is the identity element 𝑒 because adding 0 to any integer 𝑎 will result in the same
integer 𝑎. And the integer −𝑎 will be considered as the inverse of the integer 𝑎. So, this −𝑎 is
actually 𝑎$! as per the notation and you can see that you take any integer 𝑎, its inverse is −𝑎
because if you add −𝑎 to 𝑎 then the result will be 0, which is the identity element.
Here the set 𝐺 was the set of integers. Now, let 𝐺 be the set of non-negative integers ℤ& i.e.,
negative integers are not included. The operation is still the same, namely +. Now, it is easy to
see that this 𝐺 along with the + operation does not constitute a group. Which property is
violated? Here the closure and associative properties are still satisfied and the identity element
0 is still present in 𝐺. The issue is that the fourth group axiom is not satisfied, because the
inverse of an integer 𝑎 will be −𝑎, but −𝑎 is not an element of ℤ& because −𝑎 is a negative
integer.. Whereas the group axiom says that the inverse element also needs to be a member of
the set 𝐺 itself. So, that is why the set of non-negative integers along with the addition operation
does not constitute a group.
863
So, we have seen now a group with respect to the + operation now let us see a group with
respect to the multiplication operation. So, now my set 𝐺 is the set of all real numbers excluding
0 and my operation ∘ is the multiplication operation and now you can see that all the 4
properties of group are satisfied.
Multiplying any two real numbers will give you a real number and multiplication is associative
over the real numbers. The real number 1 is the identity element because you multiply 1 with
any non-zero real number 𝑎, you will obtain the same non-zero real number 𝑎. And you take
! !
any non-zero real number 𝑎, its multiplicative inverse will be %. And % is well defined because
!
𝑎 is non-zero. So, % indeed exists and it belongs to the set of non-zero real numbers. So all my
4 group axioms are satisfied and hence this set constitutes a group.
Whereas if I take the set of non-zero integers, then it does not constitute a group with respect
to the multiplication operation. Now, let us see which property gets violated. So the closure
property is still there, associativity property still satisfied, the identity element 1 is indeed
present in the set of non-zero integers. The problem is that the existence of inverse is not
! !
guaranteed, because the inverse of an integer 𝑎 will be %, but % may be a real number, it might
not be an integer. So that is why the fourth property is violated due to which this does not
constitute a group.
864
Now let us see some other interesting examples of groups. So let 𝑁 be a positive integer and
let ℤ' be the set of integers 0 to 𝑁 − 1. Basically, it is the set of all possible remainders which
you can obtain by dividing any integer, it could be either positive or negative, by 𝑁.
Now I define a new form of addition over this set called addition modulo 𝑁, which is denoted
by +' . So, addition modulo 𝑁 of 𝑎 and 𝑏 is defined as follows: I add a and b and then take
modulo 𝑁, the result will be called as the result of addition of 𝑎 and 𝑏 modulo 𝑁 i.e., 𝑎+' 𝑏 =
[𝑎 + 𝑏] mod 𝑁. So, now my claim is that this set ℤ' , which is a finite set because 𝑁 is a
positive integer, constitutes a group with respect to this operation of addition modulo 𝑁.
So, let us see whether the 4 properties are satisfied or not. So the closure property is indeed
satisfied. You take any integer 𝑎 and 𝑏 in the range 0 to 𝑁 − 1, you add them and then if you
take modulo 𝑁, the result will be 𝑟. And 𝑟 of course, will be in the range 0 to 𝑁 – 1, so hence
it is a member of ℤ' . So, closure property is satisfied. It is easy to see that the operation of
addition modulo 𝑁 is indeed associative because it does not matter in what order you perform
the addition modulo 𝑁 over 3 values a, b, c the result will be the same as 𝑎 + 𝑏 + 𝑐 modulo
𝑁.
The element 0 which is indeed present in ℤ' and will be the identity element because if I add
0 to any element 𝑎 from ℤ' and take modulo 𝑁 the result will be 𝑎 itself because 𝑎 is a member
of ℤ' and is in the range 0 to 𝑁 − 1. Now, if I add 0 to 𝑎, the value of 𝑎 does not get
incremented, it remains the same. And now if I take modulo 𝑁 the effect of mod will not
865
actually take place, because my value 𝑎 at the first place itself is less than 𝑁 – 1, so the result
will be 𝑎.
Now, what about the inverse? So, my claim is that the number −𝑎 which is defined to be 𝑁 −
𝑎 in the context of this operation addition modulo 𝑁, constitutes the inverse for any element 𝑎.
Recall that when the group was taken to be the set of integers and the operation was regular
addition then 𝑎 inverse was defined to be −𝑎 and −𝑎 indeed belongs to the set of integers. So,
the −𝑎 in the context of set of integers modulo 𝑁 will be defined to be 𝑁 − 𝑎 and it is easy to
see that 𝑁 − 𝑎 is again an element of ℤ' because if 𝑎 belongs to ℤ' that means 𝑎 is in the
range 0 to N-1 then that automatically implies that 𝑁 − 𝑎 is also within the range 0 to 𝑁 − 1.
Thus the inverse is a member of ℤ' and property of this −𝑎 is that if you add it with any 𝑎 and
then if you take modulo 𝑁 the result will be 𝑁 modulo 𝑁 which is 0, the identity element.
So, this is now an interesting example of a variation of addition operation with respect to a set
and together they constitute a group.
Now let us see a corresponding variation of the multiplication operation, which we call as
multiplication modulo 𝑁. So, let ℤ∗' be the set of all integers 𝑎 in the set ℤ' which are co-prime
∗
to the modulus 𝑁. So for instance, if 𝑁 = 10 then ℤ!) = {1,3,7,9} since the remaining set of
elements in ℤ!) , namely {0, 2, 4, 5, 6, 8 }, are not co-prime to 10.
And now, we define a new operation called multiplication modulo 𝑁 which is a variation of
multiplication over the elements of ℤ∗' and denoted by ⋅' . So, the result of 𝑎 multiplication
866
modulo 𝑁 with 𝑏 will be the following: I multiply 𝑎 with 𝑏 and then take modulo 𝑁 i.e.,
[𝑎 ⋅ 𝑏] mod 𝑁. And my claim is that this set ℤ∗' with respect to this operation of multiplication
modulo 𝑁 constitutes a group.
So, let us see whether the closure property is satisfied or not. So, for closure property I have to
prove that the product of any pair of integers 𝑎, 𝑏 from the set ℤ∗' modulo N is also a member
of ℤ∗' . Since 𝑎 and 𝑏 are members of ℤ∗' they are individually co-prime to 𝑁. Now let 𝑟 be the
result of 𝑎𝑏 modulo 𝑁 that means, I multiply 𝑎 with 𝑏 and then take mod 𝑁, I get the remainder
𝑟. Of course, 𝑟 will be in the range 0 to 𝑁 − 1, but that does not show the closure property
because I have to show that 𝑟 belongs to ℤ∗' . Namely, I have to show that 𝑟 is also co-prime to
my modulus 𝑁. And indeed my claim is that 𝑟 is co-prime to the modulus 𝑁 because since 𝑎
is individually co-prime to 𝑁, 𝑏 is individually co-prime to 𝑁, I get the conclusion that 𝑎𝑏 is
also co-prime to 𝑁. Because if 𝑎𝑏 is not co-prime to 𝑁 that means, if there is some common
prime factor 𝑝 which divides 𝑎𝑏 and 𝑁 then I get a contradiction that either 𝑎 is not co-prime
to 𝑁, namely the same prime is a common factor of 𝑎 and 𝑁, or 𝑏 is not co-prime to 𝑁 which
is a contradiction. So, 𝑎𝑏 is co-prime to 𝑁 and from the rules of division I know that 𝑟 is 𝑎𝑏
minus some multiple of 𝑁 because 𝑟 is the remainder which you obtain by dividing 𝑎𝑏 by 𝑁
i.e., 𝑟 = 𝑎𝑏 – 𝑘𝑁 where 𝑘 is an integer. So, now since 𝑎𝑏 is co-prime to 𝑁 and 𝑟 is 𝑎𝑏 – 𝑘𝑁,
we get the conclusion that indeed 𝑟 is co-prime to 𝑁. So that shows the closure property.
Now, let us see whether the operation multiplication modulo 𝑁 is associative or not. And it is
associative because if you take a triplet of integers 𝑎, 𝑏, 𝑐 from ℤ∗' it does not matter in what
order you perform multiplication modulo N the result will be the same that you will obtain by
multiplying 𝑎, 𝑏, 𝑐 and then taking modulo 𝑁.
The element 1 is always present in ℤ∗' and it is the identity element because you take any
element 𝑎 belonging to ℤ∗' and multiply 1 with a then if upon taking modulo 𝑁 the result will
be 𝑎 itself because 𝑎 is member of ℤ∗' and is thus strictly less than 𝑁 so the effect of mod will
not take place.
And now, I can claim that for every integer 𝑎 belonging to ℤ∗' , since the GCD of 𝑎 and 𝑁 is 1
then recall that in one of the earlier lectures we proved that if 𝑎 is co-prime to 𝑁 then
multiplicative inverse of 𝑎 modulo 𝑁 exists; that means there always exist an integer 𝑏 which
867
will be a member of ℤ∗' such that if you multiply 𝑎 with 𝑏 and then if you take mod 𝑁 the result
will be 1. And that 𝑏 you can always find out using extended Euclid’s algorithm. So that shows
that this ℤ∗' along with this variation of multiplication namely multiplication modulo N
constitutes a group.
So, now we have seen examples of several groups. Namely we have seen examples of 4 groups,
each group has a different structure; namely their elements were different and the operations
were also different. And these are not the only examples of groups, I can give you infinitely
many examples.
Now, the point is that even though they are different sets with different operations they have a
common property, and what is the common property? All of them satisfy the 4 group axioms.
So, what we can now do is instead of studying and deriving properties for each of these sets
individually and separately, we will abstract out all these sets by a common template. And all
the operations that were available with respect to the individual sets, they are also abstracted
by a single operation ∘.
And then, we will study the abstract set and along with the corresponding abstract operation
assuming that they satisfy these 4 properties and we will derive whatever interesting properties
that we can derive for the abstract set and the abstract operation. And now, I can say that
whatever properties that I have derived for the abstract 𝐺 and abstract ∘, they hold for any
instantiation of the abstract 𝐺 and abstract ∘.
868
What I mean by that is, assuming that the properties 𝐺! , 𝐺" , 𝐺# , 𝐺* are satisfied for this abstract
𝐺 and ∘ and say based on these properties, I derived several interesting properties say
𝑝! , 𝑝" , 𝑝# , 𝑝* , 𝑝+ … 𝑝, . Then I can say that all these properties 𝑝! , 𝑝" , 𝑝# , 𝑝* , … , 𝑝, holds for any
instantiation of the group.
And this is very interesting, because I am not deriving these properties separately and
individually for each set but rather deriving it once and for all for this abstract group 𝐺 with
the abstract operation ∘. So, that is why abstract algebra is a very interesting topic in computer
science because once we do this abstraction and derive algorithms or properties for this abstract
group and abstract operation, then depending upon our requirement and our application, we
can instantiate the group and operation with some concrete set and concrete operation and then
apply these properties that we have derived on those corresponding concrete instantiations.
And this is very helpful in several areas of computer science, especially in cryptography which
we will see later.
So that is why when we say abstract algebra, it might look very abstract to you, because we are
talking about an abstract set and abstract operation and keep on deriving properties, but when
we instantiate those sets and operations by a concrete group and concrete operation and then
fit it in an application, then you will see the real application of the theory that we are developing
in the abstract algebra.
So now, what we will do is from now onwards, we will not be focusing on concrete groups and
the corresponding operation but rather we will be focusing on an abstract group and
869
corresponding operation. So, I will say that, my 𝐺 is a group, I do not care what exactly are the
elements of my group 𝐺, I will just give them some names. I would not know whether they are
integers, whether they are real numbers, whether they are vectors, whether they are matrices, I
will not go into the exact instantiation.
And in the same way, I will denote my corresponding group operation by ∘, I will not go into
the details whether my operation ∘ is the numerical addition, numerical multiplication or
whether it is addition modulo 𝑁 or whether it is multiplication modulo 𝑁 or whether it is matrix
multiplication or whether it is dot product of vector or scalar product of vector and so on.
And then I will just assume that my group axioms 𝐺! , 𝐺" , 𝐺# , 𝐺* are satisfied, and derive
whatever interesting properties I can derive for the groups. So, it turns out that there are 2
popular notations which are used for the abstract group operation ∘. The first notation is the
additive notation, and I stress that this is just a notation, where instead of ∘ we will use the plus
(+) symbol and the identity element 𝑒 will be denoted by 0. So I stress here that this plus is not
a numerical plus and 0 is not the numerical 0; it is just a notation that we are following. If you
do not want to bring the plus and 0, you can just stick to your operation ∘ itself and you can
use 𝑒 as your identity element. But since we are very much habituated to plus and 0 that is why
sometimes we find it convenient to use instead the additive notation. In the additive notation
the additive inverse of any element 𝑎 in the group, namely the inverse of 𝑎 under the group
operation, is denoted by −𝑎.
So, examples of groups which come under the umbrella of additive notations are (ℤ, +) and
(ℤ' , +' ). So, you can recall that the set of integers with the operation plus was actually an
additive group, the set of integers modulo 𝑁 along with the operation addition modulo 𝑁 forms
under the umbrella of additive group and so on.
Whereas another popular notation, which is used for abstract group operation is the
multiplicative notation where the operation ∘ is instead represented by the dot (⋅). But this dot
does not mean numerical multiplication, this is just a representation for convenience. And if I
am following the multiplicative notation, then the identity element 𝑒 will be represented by 1.
!
And the multiplicative inverse of 𝑎 will be represented by 𝑎$! . Again, 𝑎$! does not mean % it
is just a representation if I am following the multiplicative notation.
870
So, now let us derive some nice properties that are true for any abstract group. So the first
property that I am going to derive are the left cancellation rule and the right cancellation rules.
So, the left cancelation rule says the following: if you take any arbitrary group elements
𝑥, 𝑦, 𝑧, 𝑎, 𝑏 from the group and if it is the case that that 𝑥 ∘ 𝑦 = 𝑥 ∘ 𝑧 then I can conclude that
𝑦 = 𝑧 and this is called as the left cancellation rule. Why cancellation? Because from the
implication I can say that this has the same effect as if I am cancelling out 𝑥.
And in the same way you have a corresponding right cancelation rule. Namely, the rule says
that if 𝑎 ∘ 𝑥 = 𝑏 ∘ 𝑥 then you can cancel out 𝑥 and conclude that 𝑎 = 𝑏. So, let us prove the
left cancellation rule, the similar proof you can give for the right cancellation rule. And I will
give the proof for any abstract group 𝐺 along with the corresponding abstract operation ∘. And
I will give a direct proof. So imagine that your premise for the left cancelation rule is true,
namely, 𝑥 ∘ 𝑦 = 𝑥 ∘ 𝑧.
Since 𝑥 is an arbitrary element of the group, the inverse of 𝑥 is also present in the group. Thus
the result of 𝑥 $! ∘ (𝑥 ∘ 𝑦) will be the same as 𝑥 $! ∘ (𝑥 ∘ 𝑧). And now, since my operation ∘ is
associative, I can rearrange the terms here on the left hand side as well as on the right hand side
but from the property of inverse we get 𝑥 $! ∘ 𝑥 = 𝑒 which is the identity element. Since the
property of the identity element is that if you perform the operation on the identity element and
any element 𝑦 you will obtain the same element 𝑦, we arrive at the conclusion that 𝑦 = 𝑧,
proving the left cancellation rule. A similar proof can be given for the right cancellation rule.
871
Now, an interesting corollary of the left cancellation rule and the right cancellation rule is the
following. Imagine you take a group which has say 𝑛 number of elements, 𝑔! , … , 𝑔, and you
take any group element 𝑔- . Now, if you perform the group operation on 𝑔- and various elements
of the group, say you perform the operation on 𝑔- and 𝑔! , 𝑔- and 𝑔" and so on till 𝑔- and 𝑔, ;
of course, from the closure property all of them will be elements of the group itself. The
question is whether the results that are obtained will be same or different. So, I will obtain 𝑛
elements, so call the first value that I obtained as 𝑎! , call the second value that I obtained as 𝑎"
and call the last result that I obtained as 𝑎, .
The claim here is that the result that I will obtain, namely 𝑎! to 𝑎, , are all distinct. This is
because, suppose the result of 𝑔- ∘ 𝑔. = 𝑔- ∘ 𝑔/ then from my left cancelation rule, I come to
the conclusion that 𝑔. = 𝑔/ . So, contra positively, if 𝑔. and 𝑔/ are different then the result of
𝑔- ∘ 𝑔. is different from 𝑔- ∘ 𝑔/ , which shows that each of this results are distinct.
A group is called as an Abelian group, if it is a group and it satisfies an extra axiom, namely a
fifth axiom, which says that operation ∘ is commutative. So, it is not the case that every group
is Abelian because the operation ∘ may or may not be commutative, but if the operation ∘ is
commutative, then my resultant group is called as an Abelian group. So, for instance, the set
of integers with respect to the plus operation is Abelian and there are other examples of Abelian
group as well.
872
The group order is basically the number of elements in the group 𝐺. Now, depending upon
whether the number of elements is finite or infinite; namely, whether the cardinality is finite or
infinite, the group order is either finite or infinite.
So with that, I conclude today's lecture. Just to summarize in this lecture, we started our
discussion on abstract algebra. We discussed about the definition of groups, abelian groups we
saw various examples and we also saw left cancellation rule and the right cancellation rule.
Thank you.
873
Lecture - 62
Cyclic Groups
Hello everyone and welcome to this lecture. So, in this lecture, we will continue our discussion
on groups and we will introduce a very special class of groups called as cyclic groups.
Let us first prove that the identity element and inverse element are unique in any group. We
first prove for the identity element. Let 𝐺 be an abstract group. 𝐺 has to have an identity element
because that is one of the group axioms. We now have to prove that it has a unique identity
element 𝑒 i.e., 𝐺 cannot have multiple identity elements. And the proof will be by contradiction.
874
So, on contrary assume that 𝐺 has 2 distinct group elements 𝑒! and 𝑒" and both of them are
identity elements. So, the proof by contradiction paradigm tries to arrive at a contradiction, so,
let us see what contradiction we can arrive at. Both 𝑒! and 𝑒" are identity elements and the
following holds from the property of identity elements: you take any element 𝑎 from the group,
the result of 𝑒! ∘ 𝑎, will be the same as the result of 𝑒" ∘ 𝑎 and both these answers will be same
as 𝑎. Since 𝑒! ∘ 𝑎 = 𝑒" ∘ 𝑎 we can apply the right cancellation rule and conclude that 𝑒! = 𝑒"
which is a contradiction since we assumed that 𝑒! and 𝑒" are distinct elements. Thus, we have
shown that every group 𝐺 has a unique identity element.
We next show that every element 𝑎 in any abstract group 𝐺 has a unique inverse element 𝐺.
You cannot have multiple inverse elements in the group. So, again the proof will be by
contradiction. So, on contrary assume that you have multiple inverse elements for this 𝑎.
Let 𝑎!#! and 𝑎"#! be two distinct inverse elements. Now, the property of the inverse element is
that, if I perform the group operation on the inverse and the element I should get the identity
element. So, the result of 𝑎!#! ∘ 𝑎 will be the identity element and the result of 𝑎"#! ∘ 𝑎 will also
be the identity element.
Thus 𝑎!#! ∘ 𝑎 = 𝑎"#! ∘ 𝑎 and from the right cancellation rule we conclude that 𝑎!#! is the same
as 𝑎"#! which goes against the assumption that 𝑎!#! inverse and 𝑎"#! are distinct. So that shows
that, every element in the group has a unique inverse.
875
Now, we want to introduce a new operation in the group which we call as, group
exponentiation. The group operation is still ∘, but we will be using that operation ∘, multiple
times on an element of the group which I can view as a some kind of group exponentiation. So,
in the regular arithmetic when I say 𝑎 $ , it is interpreted as if I want to multiply 𝑎 with itself
𝑥 − 1 times.
So, I want to abstract out that operation in the context of a group itself. So, imagine you are
given an abstract group and without loss of generality, I will now follow the multiplicative
notation. This is just for our convenience because we are accustomed to multiplicative notation
while discussing exponentiation in the regular arithmetic, so that is why I am using the
multiplicative notation.
But whatever I am discussing is true even if my group operation is additive or if it is treated as

an abstract operation ∘. So, since I am following the multiplicative notation, I will be using 1
for denoting the identity element 𝑒, 1 does not mean the numerical 1, remember. And I will use
!
𝑎#! for denoting the inverse of the element 𝑎, again 𝑎#! need not stand for %, it depends upon
my exact group.
Now, the group exponentiation for any group element is defined recursively as follows.
Because in the regular world also, 𝑎 $ can be defined recursively, so, I define 𝑎& as 1, in the
regular world, and then I define 𝑎! as a and then I can recursively define 𝑎 $ as the result of
𝑎 $–! with 𝑎. Similar definition I will now use in the context of an abstract group.
So, I will define 𝑔& to be the identity element, this is a definition. And I will define 𝑔! to be
the element 𝑔 itself. Now, I will define 𝑔( to be 𝑔 ⋅ 𝑔(–! . So, remember 𝑔(–! is also a group
element because 𝑔(–! is further recursively be defined as 𝑔" ⋅ 𝑔(#" and 𝑔(#" is again
recursively defined as 𝑔(–) ⋅ 𝑔" and so on.
So, 𝑔(#! will be a group element and 𝑔 is a group element and I am following a multiplicative
notation. So, this multiplication actually stands for the abstract group operation ∘. So, whatever
result I will obtain by performing the group operation on the element 𝑔 and the element 𝑔(–!
that will be defined as 𝑔( , for every 𝑚 ≥ 2.
876
I can define even the negative powers of my group element. So, 𝑔#! will actually stand for the
multiplicative inverse of my element 𝑔 and 𝑔#( will be recursively defined as follows. I will
take the multiplicative inverse of g and I will take 𝑔#((#!) and I will apply the group operation
and whatever is the resultant value that will be called as 𝑔#( . So, this is the way I defined a
group exponentiation assuming that I am following a multiplicative notation, corresponding
definition will be there if I am following an additive notation.
Now, it turns out that the rules of integer exponentiations that we are aware of are applicable
even for group exponentiations. Imagine I am given a group element 𝑔 and I take arbitrary
exponents 𝑚 and 𝑛 where 𝑚 and 𝑛 could be positive or negative. Now, it turns out that if I take
the group element 𝑔( and if I take the group element 𝑔, and perform the group operation then
that will give me the same group element 𝑔(-, .
And this 𝑔(-, can be obtained by recursively following this definition. So, what I am saying
is, on your left hand side you have 2 group elements and we are performing the group operation
on them. So, we will get one group element call it 𝑎 and you have 𝑔(-, which is another group
element whose value I can obtain by following this recursive definition call it 𝑏. My claim is
𝑎 = 𝑏 i.e., the group elements 𝑎 and the group elements 𝑏 are same. So, you can easily follow
that.
In the same way, let 𝑔( be 𝑎 and then if I compute 𝑎, then that will be the same as the element
𝑔(, , so 𝑔(, also will be some element, call it 𝑏. So, 𝑎 = 𝑏 and 𝑔(, will be the same as the
element 𝑐 ( where 𝑐 = 𝑔, ; so all of them will be same. It is easy to verify these rules are
applicable even in the context of group exponentiations.
877
So now, let us define order of a group element. So, imagine you are given a finite group and
for convenience, I will be using the multiplicative notation. However, whatever we define here
holds for any group. And now consider an arbitrary group element g.
We define a function from the set of natural numbers to the group and my function is the
following. The domain will be {0, 1, 2, … , ∞} and the co-domain is the group. The way I go
from the domain to co-domain is, if I want to map the element 𝑖, I go to 𝑔. . Now, it is easy to
see that since my group 𝐺 is a finite group, it will have finite number of elements whereas, my
domain is infinitely large then by pigeon hole principle, I know that there exists at least 2 non-
zero values 𝑎 and 𝑏 such that 𝑎 > 𝑏 and both 𝑎 as well as 𝑏 get mapped to the same group
element, namely 𝑔% = 𝑔/ . Now, since 𝑔#/ is also a group element, if we multiply both sides
of the equation with 𝑔#/ then we get 𝑔%#/ = 𝑔/#/ = 1. Note that 1 is the identity element in
multiplicative notation. Since 𝑎 > 𝑏, 𝑎 − 𝑏 is positive. This in turn implies that there is at
least one positive integer 𝑛, such that for the element 𝑔 which I have arbitrarily chosen here,
𝑔, is 1.
Of course, there might be multiple values of 𝑛 for which 𝑔, will be 1, it depends upon how
many (𝑎, 𝑏) pairs are there. But at least 1 positive integer 𝑛 is definitely there such that 𝑔, is
the identity element. Among all those positive integers 𝑛 such that, 𝑔, is equal to the identity
element, the smallest positive integer is called as the order of the element 𝑔.
878
So, let 𝐺 be a finite group and for convenience assume we are following the multiplicative
notation and 𝑔 is a group element then the order of the group element 𝑔 is the smallest positive
integer 𝑛 such that, 𝑔, is 1. So, the above definition or the order of our group is with respect
to a finite group because if the group is infinite and if you are now taking an arbitrary group
element then it may not be the case that you can easily find out the 𝑛, or whether, at the first
place we do not know whether such an 𝑛 exist or not if my group is infinite. So, in that case I
will say that the element 𝑔 will have an infinite order. But for the finite groups, the smallest
positive integer 𝑛 such that 𝑔, is the identity element, will be treated as the order of the group
element 𝑔.
So now, let us discuss some interesting properties of the order of a group element. So again, I
will stick to the multiplicative notation. So, imagine you are given an element 𝑔 and it is given
to you that its order is 𝑛; that means, I know that 𝑔, is 1 then my claim is the following. If you
have 𝑔 0 also giving you the identity element then that is possible if and only if, 𝑠 is a multiple
of 𝑛 that means, 𝑠 is completely divisible by 𝑛.
Of course, for 𝑠 = 𝑛 this claim is true, but my claim is that, if at all there is any other 𝑠 such
that 𝑔 0 gives you the identity element then 𝑠 has to be a multiple of 𝑛. So, I have to prove 2
implications here. So, let us first prove the implication in one direction. Assume 𝑠 is a multiple
of 𝑛. So, I want to prove that if 𝑠 is a multiple of 𝑛 then this direction implication is true.
I want to prove that 𝑔 0 will give me the identity element given that 𝑔, is the identity element
since the order of 𝑔 is 𝑛. Since 𝑠 is a multiple of 𝑛 we can write down that 𝑠 is some 𝑚𝑛 where,
879
𝑚 is some integer. Then what can I say about 𝑔 0 , as per my group exponentiation rules, it is
same as 𝑔(, .
And 𝑔(, I can break down as the group element 𝑔, being raised to the power 𝑚. But since 𝑔,
is the identity element this is the same as the identity element raised to the power 𝑚 and identity
element raised to power 𝑚 means, I am operating the identity element with itself, several types,
namely 𝑚 times which will give me again the identity element namely 1. So that proves the
implication in one direction.
Now, let us prove the implication in the other direction. I want to prove that if there is some
exponent 𝑠, such that 𝑔 0 is 1 then, definitely 𝑠 has to be a multiple of 𝑛. So, again I will give a
direct proof here. So, I can always write down my exponent 𝑠, as some quotient time 𝑛 plus a
remainder where, the remainder will be in the range 0 to 𝑛 – 1. And my goal is to show that 𝑠
is completely divisible by 𝑛, namely, I want to show that my remainder 𝑟 is 0.
Now, again I am giving a direct proof. So, I am assuming my premise to be true, since my
premise is true that means 𝑔 0 is 1 and now, I am writing down the value of 𝑠 in terms of the
quotient and the remainder to get 𝑔(,-1 . Now, I can rewrite 𝑔(,-1 and break it as per the
rules of group exponentiation like (𝑔( ), ; I can write it out as the result of group operation
being performed on 𝑔(, and element 𝑔1 and my right hand side is the identity element.
Now, I can further apply the rules of the group exponentiation and say that 𝑔(, is same as the
element 𝑔, being raised to power 𝑚 and 𝑔, is the identity element. So, this gives me 1( which
is 1. That means, this is identity element or 1 and 1 multiplied with 𝑔1 will be giving me the
element 𝑔1 itself, so, I get 𝑔1 is equal to identity element.
So, remember that my range of r is 0 to 𝑛 − 1. So, my conclusion now is the following, since
𝑔1 is equal to identity element 𝑟 has to be 0 because if 𝑟 is not 0 and if it is strictly less than 𝑛
then I get a contradiction to the fact that 𝑛 is the smallest positive integer for which 𝑔, was
identity element. So that shows the implication in the other direction as well.
880
So, now, let us define what we call as cyclic group. Let 𝐺 be a group with some abstract
operation ∘. It may or may not be a finite group. The specialty of the group is that it has an
element 𝑔 which we call as a generator. It is called a generator because when you take different
powers of this generator, again by power I mean group exponentiation, you will get all the
elements of your group. That means, this element 𝑔 has the capacity to generate all the elements
of your group by performing the group exponentiation on this generator.
A group that has a generator 𝑔 is called cyclic and is represented by the notation 𝐺 = ⟨𝑔⟩. This
notation basically says that 𝑔 can act as a seed and reproduce the entire set 𝐺 by computing
different powers of this generator. Of course, a cyclic group can have more than one generator.
However, we require a group to have only one generator for it to be cyclic.
Before proceeding further, let us see some examples of a cyclic group. So, consider the infinite
group, namely the group based on the set of integers with respect to the plus operation. My
claim is that the integer 1 constitutes your generator.
This is because if you take different powers of this element 1, it will give you all the elements
of your set of integers. So, let us see whether we can generate any arbitrary integer 𝑥 by
computing some power of this element 𝑥. And indeed, it is easy to verify that you take any
integer 𝑥, it will be some 𝑘 ⋅ 1 for some integer 𝑘. So, for instance, if you want to generate,
say, the element 0 from this element 1, then I know that 0 ⋅ 1 = 0 from the definition of group
exponentiation.
881
Since we defined 𝑔& to be the identity element when using the multiplicative notation, in the
additive notation this will mean that, if we add 𝑔, 0 number of times that will give me the
identity element for any 𝑔. So, I am now treating 𝑔 as 1, so that means, 0 ⋅ 1 will generate the
element 1 whereas, if you want to generate the integer 1 then it is same as you perform 1! .
1! in the additive notation will be treated as 1 ⋅ 1 and that will give you 1. If you want to
generate the element 2 through the element 1 then that is same as performing the operation plus
on the element 1 that will give you 2. So, this can be treated as 2 times 1 and so on. So that
means, all the values are now in the positive side can be generated by the element 1 and in the
same way you can generate the negative elements as 1. So, for instance, if you want to generate
-1 then, -1 can be interpreted as if you want to perform −1 ⋅ 1.
-2 can be interpreted as −2 ⋅ 1. So, −2 ⋅ 1 is nothing but the additive inverse of 1 namely, -1

being added to itself 2 times. In the same way you want to generate -3 that is same as −3 ⋅ 1
and −3 ⋅ 1 is nothing but -1 being added to itself 3 times. That is the definition of group
exponentiation for the additive notation. So that shows that even though this group is infinite
it is having a generator, namely the element 1.
So this is an example of an infinite cyclic group. Now, let us take an example of a finite cyclic
group. Let 𝑝 be a prime and now if I consider the set of all integers modulo 𝑝, namely the set
0 to 𝑝 − 1 and if my operation is +2 then my claim is that, this group is a cyclic group and in
fact has multiple generators. In fact, all the elements except the identity element 0 will be a
generator for this group.
Let’s verify this by taking p = 5. Then, the set here is {0, 1, 2, 3, 4} and my operation is +3 .
Now, you can check here that I can generate all the elements through 1. So, 0 can be generated
through 1 because 0 ⋅ 1 is defined to be 0, namely the identity element. 1 ⋅ 1 is also defined to
be the element itself. And now 2 ⋅ 1 is basically (1 + 1 modulo 5) and (1 + 1 modulo 50) is 2.
3 ⋅ 1 will be (1 + 1 + 1 modulo 5) which will be 3, and 4 ⋅ 1 will be 1 + 1 + 1 + 1 which is 4
modulo 5 which is 4.
882
In the same way 2 is also a generator, you can generate all the elements through 2. So, 0 ⋅ 2
will be defined as 0, 1 ⋅ 2 will be defined as 2 itself, 2 ⋅ 2 will be 2 + 2 modulo 5 which is 4,
3 ⋅ 2 will be 2 + 2 + 2, you get 6 but you have to do operations modulo 5, so, you will get 1
and then, 4 ⋅ 2 will be 2 + 2 + 2 + 2 which is 8 modulo 5 which is 3. So, 2 is also a generator
since we were able to generate all elements in the group. Similarly, you can check that 3 and 4
are also generators.
So, now, let us derive some interesting properties for cyclic groups. So, imagine 𝐺 is a cyclic
group and suppose the order of 𝐺 is 𝑛. So that means, now I am considering a finite cyclic
group since the group has a well-defined order. Let 𝑛 be the number of elements and say 𝑔 is
one of the generators. Then my claim is that the order of the generator is 𝑛. What does that
mean?
So, this means that 𝑛 is the smallest positive integer such that 𝑔, is equal to the identity
element. So, assume I follow the multiplicative notation. So, the order of 𝐺 is equal to 𝑛 means,
𝑔, is 1 and 𝑛 is the smallest such positive integer. By the way in the definition of order of an
element, why I am focusing on positive integer, because if I do not put a restriction on positive
integer then clearly 𝑔& is always defined to be the identity element. So that is why I am
interested in the smallest positive power for which 𝑔, will be 1.
So, now, let us prove this statement regarding the order of the generator. So, the proof will be
by contradiction. So, on contrary assume that the order of the 𝐺 is not 𝑛, but some positive
integer 𝑖 where 𝑖 is strictly less than 𝑛. So, now, what can I say about the elements
883
𝑔& , 𝑔! , … , 𝑔.#! ; of course, they are group elements because we have the closure property being
satisfied. But apart from being group elements, my claim is that, all these 𝑖 elements are
distinct. Again, this can be proved by contradiction. So, on contrary assume that the j-th power
of 𝑔 and k-th power of 𝑔 produce the same group element, where 𝑘 is some power higher than
𝑗 and both j-th power and k-th power are strictly less than equal to 𝑖 − 1.
Now, if that is the case then I come to the conclusion that 𝑔4–5 is the identity element. How do
I get this? By multiplying both the sides of this equation by 𝑔#5 . But then that means that the
order of G is (𝑘– 𝑗). Why 𝑘 – 𝑗? Because since, 𝑘 is strictly greater than 𝑗 then 𝑘 − 𝑗 is positive
and that means 𝑘 − 𝑗 is not 0 and 𝑘 − 𝑗 is strictly less than i.
Which is a contradiction to my assumption that, order of 𝑔 is 𝑖, so that shows that indeed, the
statement that all this 𝑖 distinct powers of 𝑔 are going to give me distinct group elements is
true. But, if these 𝑖 powers of 𝑔 are giving me the distinct elements then how come at the first
place 𝑔 is a generator. Because if 𝑔 is a generator then it has to generate all the 𝑛 elements of
the group. Right now I have generated only 𝑖 elements of the group, by raising 𝑔 or by
computing 𝑖 distinct powers of 𝑔 and 𝑖 is strictly less than 𝑛.
How do I generate the remaining elements of the group? You might be wondering, why can’t
I go for the higher powers. The problem is that if you go to the higher powers, you start getting
the elements which you have already generated through the first 𝑖 powers of your generator.
Namely 𝑔. , will give you the same element as 𝑔& . 𝑔.-! will give you the same element 𝑔! ,
𝑔.-" will give you the same element as 𝑔" and so on.
So that means, once you have computed the first 𝑖 powers of 𝑔 the next powers of 𝑔 will start
giving you the elements which you had already generated; you would not be getting any extra
or any new elements of the group. And that goes against the assumption that my 𝑔 was a
generator for the whole group. If it was a generator for the whole group then it should have the
capability to generate all the 𝑛 elements, not just 𝑖 elements. So that is why the order of my
generator 𝑔 has to be the same as the order of the finite group or the whole group which is 𝑛.
And that shows that, why we are calling this group as a cyclic group because if 𝑔 is the
generator then by raising 𝑔 to different powers from 0 to 𝑛 − 1, I will be getting the entire
884
group. Now, once I start computing the higher powers of 𝑔, namely 𝑔, , 𝑔,-! and so on, I
would not be getting anything extra. I will start getting the same elements which I have
generated by computing 𝑔& , 𝑔! … 𝑔,–! . In that sense, it is a cyclic group; cyclic in the sense,
you can arrange the elements of the group in a cycle. And that cycle can be completed by
raising 𝑔 or the generator to different powers in the range 0 to 𝑛 − 1.
So that brings me to the end of today’s lecture. Just to summarize, today we saw some more
properties of groups. We discussed about the order of the group, we discussed the properties
of the order of the group, we discussed about the order of a group element. And we also
discussed about cyclic groups and derived some properties regarding the order of the generator
of a cyclic group. Thank you.
885
Lecture - 63
Subgroups
Hello everyone, welcome to this lecture. So, the plan for this lecture is as follows. In this
lecture, we will introduce the definition of subgroups and we will see some properties of
subgroups. And then we will discuss about Lagrange’s theorem in the context of subgroups
and its applications.
So, let us start with the definition of a subgroup. So, imagine you are given an abstract group
𝐺 with operation ∘, it may or may not be finite. And imagine I take a subset 𝐻 for the set 𝐺.
886
Again it may or may not be finite. Of course, if 𝐺 is finite any subset will be finite, but if 𝐺 is
infinite then I may take a finite subset or infinite subset. Now, if the subset 𝐻 with the same
operation ∘ satisfies the group axioms namely 𝐺! , 𝐺" , 𝐺# , 𝐺$ then I will call H along with the
operation ∘ to be a subgroup of the original group.
So, an example of a subgroup will be the following. So, let group 𝐺 be the set of real numbers
with the operation integer addition then if I take the set of integers then that will be of course,
a subset of the real numbers. So, my real number was the bigger set and my integer is a subset
of the set of real numbers and I take the same operation plus here. So, it is easy to see that the
set of integers is indeed a group, it satisfies the group axioms with respect to the integer addition
and hence, I can say that this is a subgroup of the group of real numbers with integer addition.
Whereas, if I take the set of integers as my main group or the bigger group with the integer
addition operation and now, if I take the subset namely the subset of non-negative integers with
the plus operation then it does not constitute a subgroup. Because the set of non-negative
integers does not satisfy all group axioms. Namely, the additive inverse is negative and it will
not belong to the set of non-negative integers.
So now, an interesting question is, imagine you are given an abstract group and now, I give
you a subset, how do I check whether it is a subgroup or not? There are 2 options, option one
that you manually check whether all the group axioms are satisfied for the subset 𝐻 that you
are given. But that is not what we will prefer because if my subset 𝐻 is very large then it might
become very difficult to verify whether all the group axioms are satisfied or not.
Instead, what we are looking now, here, is the following. We are looking for a characterization,
some kind of condition which should be sufficient to check and declare whether the given
subset 𝐻 satisfies the group axioms or not with respect to the operation.
887
So, here is a very interesting characterization for subgroups. So, you are given a subset 𝐻, of
course, a subset 𝐻 has to be non-empty because if it is empty, it can never be a group because
you need the identity element to be present at least in your group. So, definitely 𝐻 cannot be
empty; it has to be a non-empty subset. So, imagine you are given a non-empty subset, the
characterization is the following.
You just verify whether the 2 properties 𝑆! and 𝑆" are satisfied. And if they are satisfied then
you can declare that the subset 𝐻 indeed constitutes a subgroup of the original group. Note that
I am using the multiplicative notation here. So, what are these 2 conditions? The condition 𝑆!
demands that, the closure property should be satisfied.
That means, you take any 𝑥, 𝑦 value from your subset 𝐻, the result of the group operation
should be a member of the subset 𝐻 itself. And the second property here is that every element
in the subset 𝐻 should have multiplicative inverse present in the subset 𝐻 itself. So, the claim
here is that if these 2 properties are satisfied, that automatically ensures that all the group
axioms are satisfied.
That means I do not need to check separately for the existence of the identity element. And I
do not have to check for the associative property. So, let us see the proof of this
characterization. So, I have to prove that, if indeed 𝑆! and 𝑆" are satisfied then all the group
axioms are satisfied in my subset 𝐻. We observe that the closure property follows directly from
𝑆! .
888
So, if indeed 𝑆! condition is satisfied that means, the closure property is satisfied. The operation
dot or abstract operation ∘ was indeed associative in the 𝐺 itself because 𝐺 satisfies the group
axioms. So, it will be associative in 𝐻 as well because the elements of 𝐻 are nothing but
elements of 𝐺. And closure property is anyhow satisfied guaranteed in 𝐻, so that means, the
operation ∘ or the operation dot will be associative in 𝐻 as well.
The axiom number 𝑆" guarantees you that every element in 𝐻 has its inverse in 𝐻 present. And
now, I have to show that identity element is also a part of my subset 𝐻, if the condition 𝑆! and
𝑆" are satisfied. So, for that, consider an arbitrary element 𝑥 belonging to your subset 𝐻. I can
apply the axiom number 𝑆" and claim that 𝑥 %! is also present in 𝐻.
And from the first axiom 𝑆! , I know that the result of 𝑥 operation 𝑥 %! will be an element of 𝐻
because the axiom number 𝑆! , says that the closure property is satisfied. So, 𝑥 is an element of
𝐻, 𝑥 %! is also an element of 𝐻. So, the result of group operation of 𝑥 and 𝑥 %! should be also a
member of 𝐻. But what is the result of group operation being performed on 𝑥 and 𝑥 %! ? It will
be the identity element and this shows that identity element is guaranteed to be present in my
subset 𝐻. So that shows that if 𝑆! and 𝑆" are satisfied, all the group axioms are satisfied.
So that is a very nice characterization. Now, an interesting corollary here, is the following. The
corollary says that, if your original group 𝐺 is finite then no need to check even for the second
axiom; just check whether the first axiom is satisfied or not. Namely, just check whether the
closure property is satisfied or not in your subset 𝐻. If the closure property is satisfied in the
889
subset 𝐻 that automatically guarantees you that all the remaining group axioms are also
satisfied in your subset 𝐻.
So, the proof for this corollary will be the following. We have to show that if your bigger group
𝐺 is finite and if your condition 𝑆! is satisfied in 𝐻, I have to show that, condition 𝑆" is also
satisfied in 𝐻 because we had already proved that if both 𝑆! and 𝑆" are satisfied in 𝐻 then all
the 4 properties of group hold in 𝐻. Right now, it is not given to me whether 𝑆" is satisfied or
not, it is just given to me that 𝐺 is finite and 𝑆! is satisfied in 𝐻.
I will show that if 𝐺 is finite and 𝑆! is satisfied in 𝐻, I can draw the conclusion that even 𝑆" is
satisfied for my 𝐻. And now if 𝑆! and 𝑆" are satisfied for my 𝐻, I had already proved that all
the group axioms will hold for 𝐻 as well. So, everything boils down to this proof. So, the proof
will be divided into 2 cases depending upon what is the cardinality of the subset 𝐻. If the subset
𝐻 is a singleton set then it only has the identity element.
So, consider the case when indeed the subset 𝐻 is singleton and it has the identity element then
I do not have to check 𝑆! holds, 𝑆" holds or not. Indeed, they hold because the subset H which
has only the identity element present in it along with the group operation is indeed a subgroup.
The closure property is satisfied because if you perform the group operation on the identity
element with itself, you will obtain the identity element which is again a member of 𝐻.
The operation ∘ will be associative anyhow in 𝐻, the inverse element of the identity element
will be the identity element itself and anyhow the identity element is present in 𝐻. So, it trivially
constitutes a subgroup, I do not have to check about 𝑆! , 𝑆" . On the other hand, imagine that
your subset 𝐻 is not a singleton set. So, imagine that it has some other elements and consider
one such element 𝑥 which is different from your identity element. Now, since 𝑥 is a member
of 𝐻, 𝑥 is a member of 𝐺 as well.
So, let the order of 𝑥 be 𝑚. And when I say order of 𝑥, it means, order of 𝑥 in the context of
the group 𝐺. That means, if the order of 𝑥 is 𝑚 that means, 𝑚 is the smallest positive integer
such that, 𝑥 & is the identity element. Now, if 𝑥 & is the identity element, if I multiply both sides
of this equation with 𝑥 %! and again 𝑥 %! is a group element as per my definition of the group
890
exponentiation. If I multiply both sides of this equation with 𝑥 %! then the identity element
operated with 𝑥 %! will give me 𝑥 %! only.
That means, I can say that 𝑥 %(&–!) is same as 𝑥 %! . Now, what is my goal? My goal is to show
that, if 𝑆! holds in my subset 𝐻 then, 𝑆" also holds. So that is what I am trying to do here. I
have taken an arbitrary 𝑥 here, different from the identity element whose order is 𝑚 and right
now, I have derived that 𝑥 %! is same as 𝑥 &–! . My goal is to show that 𝑥 %! indeed belongs to
the subset 𝐻.
So, to show that 𝑥 %! indeed belongs to the subset 𝐻, I have to equivalently show that 𝑥 &–!
belongs to the subset 𝐻 because, I have already proved here that 𝑥 %! is same as 𝑥 &–! . And
now, how do I prove that 𝑥 &–! is a member of this subset 𝐻? I can repeatedly apply the fact
that axiom 𝑆! holds in my subset 𝐻 on the element 𝑥.
So, remember 𝑥 is a member of the subset 𝐻 and if 𝑆! holds, it holds for 𝑥 as well. So, 𝑥 " will
be a member of 𝐻, 𝑥 # will be a member of 𝐻, 𝑥 $ will be a member of 𝐻 and hence 𝑥 &–! also
will be a member of 𝐻 and 𝑥 &–! is nothing but 𝑥 %! and that shows that 𝑥 %! is automatically
guaranteed to be present in 𝐻, if the axiom number 𝑆! is satisfied.
So now, you might be wondering where exactly the fact that 𝐺 is finite is used here. Well, the
fact that 𝐺 is finite is used here is when I use the fact that the order of 𝑥 is 𝑚. Because if 𝐺 is
infinite and I cannot say necessarily what exactly is the order of 𝑥, it may not be defined at the
first place. So that is why this proof holds only for the case when my group 𝐺 is a finite group.
So that means, if I am given a finite group 𝐺 and a subset of 𝐺 then to check whether the subset
𝐻 constitutes a subgroup or not it is just sufficient to check the closure property.
Just check whether the closure property holds in the subset or not, if it holds then you can
conclude that all the remaining group axioms will also hold, a very nice characterization.
891
So, now, based on this we will generate various cyclic subgroups of a group. So, you might be
given a group which need not be a cyclic group but by using the previous result I will try to
now derive cyclic subgroups of my original group. So, I have retained the result that I have just
proved. Namely, the characterization for the existence of a subgroup. That means, if you are
given a non-empty subset, how do you check whether that non-empty subset is a subgroup or
not.
So, now, imagine you are given a group 𝐺 and an element 𝑥, whose order is 𝑚. Its order is 𝑚
means 𝑥 & is the identity element; that means 𝑚 is the smallest positive integer such that 𝑥 &
gives you the identity element.
Now, let me define subset 𝐻 which is obtained by raising or by computing 𝑚 distinct powers
of 𝑥 namely, 𝑥 * , 𝑥 ! , … , 𝑥 &–! . So, these 𝑚 elements are distinct; we had already proved that in
one of our earlier lectures. Now, my claim is that this subset 𝐻 is a cyclic group, whose
generator is 𝑥. And it is a cyclic subgroup of your original group. So, how do we prove? First
of all 𝑥 is a generator; that is easy to see because all the elements of 𝐻 are generated by different
powers of 𝑥. So, indeed, 𝑥 is a generator. Now, I have to prove that indeed, the group axioms
are satisfied for my subset 𝐻.
And for that, I have to show that both property 𝑆! as well as property 𝑆" holds for 𝐻. If I can
prove 𝑆! and 𝑆" holds for my subset 𝐻 that I have computed like this then that shows that it is
indeed a cyclic subgroup. So, let us first prove the closure property. So, let us take 2 different
892
elements from the set 𝐻. Since they are 2 different elements of 𝐻, they are basically some
distinct powers of the generator 𝑥, say the i-th power and the j-th power.
Then I have to show that the group operation performed on 𝑥 + and 𝑥 , will also give me an
element of 𝐻. And that is very trivial to prove because the group operation performed on 𝑥 +
and 𝑥 , will give me an element 𝑥 +-, . Now, this 𝑥 +-, is same as 𝑥 (+-,)mod & . Why so? Because,
since the order of 𝑥 is 𝑚, that means 𝑥 & = 1, so I can rewrite 𝑥 +-, as several blocks of 𝑥 & ;
and the last block which may not be a full block of 𝑥 & but rather it will be 𝑥 (+-,) mod & .
Now, I know that each block of 𝑥 & will give me the identity element and the last block which
has 𝑥 (+-,) mod & that will remain. Now, the identity element being multiplied to itself several
times will give me the identity element. So, this will be same as identity element being
multiplied with 𝑥 (+-,) mod & .
And hence the result of 𝑥 +-, is same as the result of 𝑥 (+-,) mod & . But then (𝑖 + 𝑗) mod 𝑚 will
give you a remainder in the range 0 to 𝑚 − 1. So, this will be a value in the range 0 to 𝑚 − 1
because the possible remainders that you can obtain by dividing 𝑖 + 𝑗 by 𝑚 will be either 0,
1, … , 𝑚 – 1. That means this is nothing but some power of x in the range where the exponent
is in the range 0 to 𝑚 − 1 and that will be definitely an element of 𝐻 because any power of 𝑥
where in the exponent you have something in the range 0 to 𝑚 − 1 will give you an element
of 𝐻. So that shows the closure property or 𝑆! is satisfied.
Now, I have to prove the 𝑆" property. That means if I take any arbitrary element from the subset
𝐻 that I have computed, it has an inverse present in the subset 𝐻 as well. So, I take some
arbitrary element where the arbitrary element is 𝑥 + and 𝑖 > 0. Why greater than 0? Because if
𝑖 = 0 then 𝑥 + is nothing but the identity element and the inverse of the identity element will
be the identity element only. So, 𝑆" will be of course, satisfied for the identity element. I want
to prove that it is satisfied for any other non-identity element as well. So that is why I am taking
𝑥 + where 𝑖 is not zero. And I have to show that for this 𝑥 + element, the corresponding inverse
is also present in 𝐻.
So, my claim is the following. That, 𝑥 &–+ which is also an element of the subset 𝐻 constitutes
the inverse of 𝑥 + . So, it is easy to see that indeed 𝑥 &–+ belongs to 𝐻. Why? Because 𝑖 is some
893
power, of course, 𝑖 is greater than 0 but 𝑖 is also less than equal to 𝑚 − 1.That means, 𝑚 − 𝑖
will also be now a power in the range 0 to 𝑚 − 1. So that is why it is a member of 𝐻. And
what will be the result of performing the group operation on 𝑥 + and element 𝑥 &–+ ? Well, it will
be the same as identity element.
So that is why I can now conclude that you take any non-identity element in the subset 𝐻, its
corresponding inverse is also present in the subset 𝐻. And since 𝑆! and 𝑆" is satisfied for the
𝐻 that I have built that means, 𝐻 indeed constitutes a group. And it is cyclic because its
generator is 𝑥. By the way, you might be wondering that why I am verifying both 𝑆! and 𝑆"
here for the subset 𝐻. Why cannot I just verify 𝑆! ? Because I just proved some time back that
it might be sufficient to just check the condition 𝑆! .
Well that is the case when your group 𝐺 would have been a finite group but here I am proving
the property for a group 𝐺 which may or may not be finite. So, if your group 𝐺 is not finite
then I have to check for both condition 𝑆! as well as condition 𝑆" . That is why I am checking
for both 𝑆! and 𝑆" .
Now, let us next define what we call as left and right coset of our subgroup. And this notion of
cosets is very important when we perform error correction in coding theory. Of course, we
would not be discussing coding theory this course, but people who are familiar with error
correcting codes they might be knowing that when we perform error correction then we use
this concept of left and right cosets. So, let us define what exactly is coset.
894
So, imagine you are given a group and a subgroup for the group. Again, they may be finite,
infinite and so on because the definition does not put any restriction. And imagine you are
given a group element 𝑔 from the bigger group. The cosets are defined with respect to your
subgroups. But they are defined with respect to elements which are chosen from the bigger
group.
So, it might be the case that element is 𝑔 may not be a member of 𝐻; it may be present in 𝐺
but not in 𝐻. So, definition does not put any restriction that element is 𝑔 should present in the
set 𝐻; it may or may not be present. But the cosets are defined with respect to your subgroups.
So, the left coset of the subgroup 𝐻 is denoted by 𝑔𝐻. And it is basically the collection of all
group elements which I obtained by performing the group operation between the 𝑔 that I have
chosen here and all the elements of my subgroup 𝐻.
Let the elements of 𝐻 be denoted by {ℎ! , … , }. Note that it might have infinite number of
elements. So, you perform 𝑔 ⋅ ℎ! , 𝑔 ⋅ ℎ" , … , 𝑔 ⋅ ℎ+ ; you perform g operation every element of
subgroup 𝐻. Of course, from the closure property you will obtain group elements. The
collection of those group elements is your left coset.
Why left coset? Because 𝑔 is the left operand and the elements of the subgroup 𝐻 are occurring
as your right operands. Whereas the right coset is defined similarly but what you now do is the
elements of subgroup 𝐻 will occur as your left operand and each of them will be operated with
𝑔. That will be your right coset and our notation for that will be 𝐻𝑔.
Now, if you change your element 𝑔 that will give you a different left coset and a different right
coset. So it is not the case that you will obtain the same left coset and right coset every time, it
depends on 𝑔.
So, the first thing that we can prove here is the following. If 𝐻 is a finite subgroup then you
take any element from the bigger group, the corresponding left coset, right coset they have the
same cardinality as the cardinality of your finite subgroup. And the proof is very simple. So,
imagine your finite subgroup 𝐻 has 𝑛 number of elements and the left coset will have the
elements {𝑔 ⋅ ℎ! , 𝑔 ⋅ ℎ" , … , 𝑔 ⋅ ℎ2 }; where we are using the multiplicative notation for the
group.
895
From the right cancellation rule, all this elements 𝑔 ⋅ ℎ! , 𝑔 ⋅ ℎ" , … , 𝑔 ⋅ ℎ2 are distinct. Namely,
you cannot have 𝑔 ⋅ ℎ+ = 𝑔 ⋅ ℎ, where ℎ+ and ℎ, are distinct because if that is the case, you can
apply the left cancellation rule and come to the conclusion that ℎ+ = ℎ, , which is a
contradiction.
So that is a trivial proof. Now, let us see a very nice property here, regarding the coset. So, let
me first demonstrate the property and then we will prove it for any general group and general
* *
coset. So, let me take this group 𝐺 which is the set ℤ!! and remember the set ℤ!! will have all
the integers in the range 0 to 10 which are co-prime to my modulus 11. So basically, you will
have all the integers 1 to 10, except 0 because 0 is not co prime to 11.
And my operation is ⋅!! ; multiplication modulo 11. That is my concrete group operation. And
suppose I take the subgroup consisting just of elements {1, 10}. If you are wondering whether
this is a subgroup or not, well, you can apply your characterization 𝑆! on this subset 𝐻 and then
you can verify that indeed the property 𝑆! holds for this subset 𝐻. Now, let us compute the
various left cosets of this 𝐻.
By various left cosets mean, I will keep on changing my 𝑔. I will take 𝑔 to be 1 first, and then
I will take 𝑔 to be 2 next, and then finally I will take g to be 10. So, the left coset of 𝐻 with
respect to 1 will be {1, 10}. The left coset of 𝐻 with respect to the group element 2 will be
{2, 9}. Why? So, this will be because of the following reason. So, if I take g to be 2 then 𝑔𝐻
will be basically 2 ⋅ 1 mod 11 = 2 and 2 ⋅ 10 mod 11 = 9 because my operation is
multiplication modulo 11.
So that is why this left coset is consisting of the elements {2, 9}. In the same way, you can
compute the left coset with respect to the element 𝑔 = 3, 𝑔 = 4. So, you will have 10 cosets.
Because you have 10 possible values of 𝑔. But now you can see here, it is not the case that all
my 10 cosets that I have obtained here they are all distinct.
Some of them are same completely or otherwise they are completely disjoint. So, for instance,
the left coset of 𝐻 with respect to the element 1 and the left coset of the same 𝐻 with respect
to the element 10 are same. Whereas, if I consider the left coset of H with respect to element 1
896
and a left coset of H with respect to 2 they are completely disjoint, they have nothing common.
So, now, you might be tempting to prove whether this is the case with respect to any coset or
not, or is it the case that it is happening only for this 𝐺 and only this subgroup?
Well that is not the case, we will prove that this is a general result. So, what we are going to
prove here is the following. If you are given any group and its subgroup then you take any 2
elements from the parent group, call it 𝑔! , 𝑔" then the left coset of 𝐻 with respect to the
elements 𝑔! , 𝑔" will be either completely same or they will be completely disjoint. That is a
statement here.
And before going into the proof idea, let us try to recall a concept that we had earlier seen in
our course where we come across a similar situation. Where we proved something of the
following form that you have many subsets and either 2 subsets are exactly same or they are
completely disjoint. Namely, we proved that result in the context of equivalence classes. So, if
we have an equivalence relation and we formed a corresponding equivalence classes then we
know that 2 equivalence classes will be either completely same or they will be completely
disjoint.
Something similar is happening here. So that is why we are now going to prove this result by
defining an equivalence relation and proving that left cosets are nothing but equivalence classes
with respect to that equivalence relation that we will define. So, my equivalence relation that I
am defining here is the following. I say that element 𝑥 is related to the element 𝑦, if the element
𝑦 is present in the left coset with respect to the element 𝑥.
897
If that is the case, I will say 𝑥 is related to 𝑦, otherwise 𝑥 is not related to 𝑦. And I will prove
very soon that indeed this relation is an equivalence relation. Namely, it satisfies the reflexive
property, symmetric property and transitive property. Assume for the moment that it is indeed
the case, that means, this relation is an equivalence relation. Then what can I say about the
equivalence classes of this relation?
Well, I can use the property that equivalence classes constitute a partition of the original set.
So, the original set over which the relation is defined is the set 𝐺. Because 𝑥 and 𝑦 are elements
of 𝐺, I have defined a relation over the elements of the group G. I say element 𝑥 and element
𝑦 of the group 𝐺 are related if 𝑦 is present in the left coset of 𝑥. So, if at all this relation is an
equivalence relation then the equivalence classes will constitute a partition of this group 𝐺.
And it is easy to see that the equivalence classes here are nothing but the cosets. Because that
is how I have defined the relation. And that automatically proves that the theorem statement
holds; that means, either 2 cosets will be completely different or they would not have any
overlap and will be identical because they constitute your equivalence classes. So, now proof
boils down to proving that this relation is indeed an equivalence relation.
So, let us prove that this relation is an equivalence relation by proving the reflexive, symmetric,
and transitive properties. So, let us first prove that the relation 𝑅 is reflexive. That means, we
have to prove that every 𝑥 is related to itself. That means, we have to prove that every 𝑥 is
always present in its left coset where 𝑥 is a element of your parent group. So, this simply
follows from the fact that 𝑥 is always the result of group operation being performed on 𝑥 and
the identity element.
And this identity element is of course, an element of your subset 𝐻 because 𝐻 is a subgroup.
So that means when I will be forming the left coset of 𝑥, I will be encountering the element 𝑥
operated with identity element and that will give me the element 𝑥 itself. Hence, I get the
conclusion that 𝑥 is related to 𝑥 showing that my relation is reflexive.
Now, let us prove my relation is symmetric. So, imagine 𝑥 is related to 𝑦. 𝑥 is related to 𝑦

means when I operated 𝑥 with all the elements of 𝐻, I must have encountered some ℎ! such
898
that 𝑥 ⋅ ℎ! = 𝑦. Now, upon multiplying both sides by ℎ!%! we get 𝑥 = 𝑦 ⋅ ℎ!%! . Note that since
ℎ! is a member of subgroup 𝐻, ℎ!%! also will be a member of subgroup 𝐻.
That means 𝑥 is nothing but 𝑦 operated with some element of the subgroup 𝐻, say, ℎ" i.e., let
ℎ!%! = ℎ" . But 𝑦 ⋅ ℎ" is a member of the left coset of 𝑦 by definition. So, what I have shown
here is that, the element 𝑥 belongs to the left coset of 𝑦. And if element 𝑥 belongs to the left
coset of 𝑦 then that is equivalent to showing that 𝑦 is related to 𝑥, as per my definition of
relation 𝑅. So, I have proved that my relation is symmetric as well, and in the same way I can
prove it, it is transitive.
So, imagine 𝑥 is related to 𝑦 and 𝑦 is related to 𝑧. I have to show that 𝑥 is related to 𝑧. So, if 𝑥
is related to 𝑦 that means, 𝑦 is a member of left coset of 𝑥. That means, 𝑦 is 𝑥 ⋅ ℎ! where ℎ! is
a member of the subgroup 𝐻. And if 𝑦 is related to 𝑧 that means, 𝑧 is a member of left coset of
𝑦 and 𝑧 = 𝑦 ⋅ ℎ" where ℎ" is a member of my subgroup.
Substituting the value of 𝑦 in the second equation gives 𝑧 = 𝑥 ⋅ ℎ! ⋅ ℎ" . Since both ℎ! and ℎ"
are members of 𝐻, we can apply the closure property and say that ℎ! ⋅ ℎ" = ℎ is some other
element of the subgroup H. Hence, 𝑧 = 𝑥 ⋅ ℎ will be a member of my left coset of x. That
means, 𝑥 is related to 𝑧 as well. So that shows my relation 𝑅 is transitive as well.
So, now, given the definitions of cosets we will give a very nice theorem which we call
Lagrange’s theorem which will be useful later on. The Lagrange’s theorem in the context of
group is the following. If you are given a finite group whose order is 𝑛, namely there are 𝑛
899
elements in the group 𝐺. And say 𝐻 is a subgroup, of course, it has to be finite because my
parent group is finite.
And say the order of the subgroup is 𝑚. Namely, there are 𝑚 number of elements in my
subgroup 𝐻. Then the Lagrange’s theorem says that 𝑚 divides 𝑛. Why this theorem is for finite
group? Because if it is not for finite group, I cannot say anything how many elements are there
in 𝐺 and my 𝐻 also could be an infinite subgroup. So, basically Lagrange’s theorem says that,
the order of any subgroup divides the order of parent group if your parent group is finite order.
And the proof is very simple assuming that we have already proved our result regarding our
cosets. Now, since my subgroup size is 𝑚; that means, the cardinality of 𝐻 is 𝑚, the size of
each coset will be 𝑚. Because, we proved already that the size of each left coset is same as the
size of your subgroup and since the size of subgroup is 𝑚, the size of each coset will be 𝑚 and
we had already proved that the cosets constitute a partition as per the relation that I have defined
here.
So, now, if there are 𝑘 distinct cosets which you can form all together. So, your 𝐺 will have 𝑛
number of elements; so, called elements as 𝑔! , 𝑔" , … , 𝑔2 and you may form the coset
𝑔! 𝐻, 𝑔" 𝐻, … , 𝑔2 𝐻. It may not be the case that you obtain 𝑛 distinct cosets, some of them may
be repeated, but they might be same or it might be the case and or otherwise the 2 cosets will
be completely different.
So, imagine that all together they constitute 𝑘 distinct cosets. Now, in each coset you have 𝑚
number of elements and if you have all together 𝑘 distinct cosets, since the union of all the
coset, distinct cosets, will give you the parent group 𝐺, I can say that 𝑘𝑚 is nothing but the
number of elements in your parent group 𝑛. And that shows that 𝑚 divides 𝑛. A very simple
proof. There are other ways of proving the Lagrange’s theorem but they might be slightly long.
But once we have proved the general result regarding cosets, the proof is just 2 line argument
here. Now, let us see some interesting conclusions of this theorem. So, imagine if your G is a
finite group of order 𝑛 and if you take any element from the group then the order of that element
𝐺 will divide order of the group namely 𝑛. The second conclusion here is that 𝑔2 is also the
identity element.
900
And the proof again is very simple. So, imagine we construct a cyclic subgroup 𝐻 by taking
different powers of the element 𝑔. Why it will be cyclic subgroup? We already proved few
slides back that if you take any group element from the parent group and compute different
powers of that element, it will give you a cyclic group. So, imagine I constitute the subgroup
𝐻 by taking different powers of the element 𝑔, that will give me a subgroup. And say the order
of the element is 𝑔 is 𝑚.
Well, if the order of the element 𝑔 is 𝑚 then the number of elements in the subgroup 𝐻 that I
have constructed will also be 𝑚. Because I have constructed 𝐻 by computing 𝑔* up to 𝑔&–! ;
that is my cyclic subgroup that I have constructed. But what exactly the Lagrange’s theorem
says? If you have finite group and a subgroup of that then the order of the subgroup always
divides the order of the parent group.
So, what is the order of the subgroup? I have 𝑚. And what is the order of the parent group? I
have 𝑛. So, from the Lagrange’s theorem I obtain that 𝑛 is completely divisible by 𝑚. And that
shows that the order of my element 𝑔 has to divide the order of the bigger group. Now, the
order of the element 𝑔 might be 𝑚 where 𝑚 can be strictly less than 𝑛 but the second result
that we want to prove here is that, 𝑔 to the power order of the group will give you the identity
element.
And once we have derived this fact it is very easy to prove that. So, let us see what exactly we
will obtain if I compute 𝑔2 . 𝑔2 will be nothing but 𝑔4& and since the order of the element is
𝑔 is 𝑚, I note that 𝑔& will give me the identity element and identity element raised to power
𝑘 will give me the identity element. So that is an implication of the Lagrange theorem.
And if I apply this implication in the context of a prime order group, namely a group where I
have prime number of elements then I get the fact that it will be cyclic and every element except
the identity element will be a generator for that cyclic group, if my group 𝐺 is a prime order.
Why so? Because if my group 𝐺 has prime number of elements, and if I consider and arbitrary
element 𝑥 and try to generate the subgroup through that element 𝑥, the order of that subgroup
has to divide the order of the parent group.
901
But the parent group has order 𝑝 which is a prime value and the only divisors of a prime number
are 1 or 𝑝 but since the 𝑥 that I have chosen is not the identity element, the only option that is
left is the order of the 𝑥 is the prime number 𝑝 itself. And if the order of the 𝑥 is the prime
number 𝑝 itself that means the subgroup that I have generated through 𝑥 is nothing but the
whole parent group.
Because if the order of 𝑥 is prime number 𝑝 and the parent groups order is also prime 𝑝 that
means, through 𝑥 I have generated all the 𝑝 distinct elements of my group. So that is a very
powerful result that means, if you want a cyclic group where you do not want to worry about
searching for the generators then try for a group which has a prime order.
So, with that I end today’s lecture. These are the references just to summarize in today’s lecture.
We discussed about left cosets, right cosets, we derived several properties for the cosets. And
we also discussed about subgroups. We gave a characterization for subgroup. And we also
discussed properties regarding the order of the subgroup, namely, the Lagrange’s theorem.
Thank you.
902
Lecture - 64
Discrete Logarithm and Cryptographic Applications
Hello, everyone, welcome to this lecture. So, in this lecture we will continue our discussion on
cryptography. And we will see some more applications of the concept from number theory and
abstract algebra in the context of cryptography. Namely, we will see the definition of public
key cryptosystem. And we will see 2 very popular instantiations of public key cryptosystem,
namely that of ElGamal encryption scheme and RSA encryption scheme.
903
So, let us start with the definition of public key cryptography. What exactly is public key
cryptography? Why exactly we need that and so on? So, this is the Diffie Hellman key
exchange protocol which allows 2 parties, Sita and Ram to talk over the internet publicly and
agree upon a common key k. And if we perform all the operations over a sufficiently large
group where a random instance of discrete log problem is very difficult and any third party in
a reasonable amount of time will not be able to come up with the value of key that is Sita and
Ram have agreed upon.
So, even though this is a very breakthrough result because before the invention of the Diffie
Hellman key exchange protocol, people simply thought that it is not at all possible to solve the
key agreement problem. But now Diffie and Hellman showed that, indeed it is possible to agree
upon a key by talking publicly.
But the downside of the Diffie Hellman key exchange protocol is that it requires both the parties
to be online. So, imagine the 2 parties in different time zones then it hinders the spontaneity of
applications like email. So, for instance, if Sita and Rama are in 2 different time zones and Sita
wants to send an encrypted mail to Ram then Sita will initiate a Diffie Hellman key exchange
protocol instance with Ram, she will send her message namely 𝑔𝛼 to Ram.
But now it might be the case that when 𝑔𝛼 is delivered to Ram, Ram is sleeping because Ram
is in a different time zone. Until and unless Ram also gets up and communicate back, 𝑔𝛽 , Sita
cannot use the key 𝑔𝛼𝛽 beta for encrypting her email. So, in that sense, the spontaneity of the
application is lost here in. So that is why to get around this problem, Diffie and Hellman
proposed an architecture for a new type of cryptosystem which is different from symmetric key
cryptosystem.
So, remember, the symmetric key cryptosystem, the same key is used both for encryption and
decryption. So, we were looking for a mechanism where the key, 𝑔𝛼𝛽 , will be used for
encrypting the email, as well as for decrypting the email. But Diffie and Hellman were thinking
about a new form of cryptography. And today, we call such cryptosystems, such forms of
algorithms, such form of cryptographic algorithms as public key cryptosystem.
904
So, let us see the architecture of public key cryptosystem. So, in this system, the receiver will
have 2 keys, a key which we call us public key, pk available in the public domain. And there
will be another key, sk which will be secret key and available only with the receiver. Now in
this system, any person who wants to encrypt a message for this receiver will look for the copy
of the public key in some public domain, say for example, a telephone directory or the
homepage of the receiver.
Once the public key copy is available to the sender and sender has the plain text m, he will use
the encryption algorithm and produce a cipher text or the scrambled text which is
communicated to the receiver. Receiver upon receiving the scrambled text, will now use a
different key, namely the secret key which is available only with the receiver and he will
decrypt and recover back the message m.
And now the security property that we require here is that if there is a third party an attacker,
who knows the description of the public key, who knows the description of the encryption
algorithm, who knows the description of the decryption algorithm and who also knows the
description of the cipher text, should not be able to figure out what exactly in the underline
message. Because the secret key is not known to him. That is a loose Security property that we
require here.
So, the analogy that I can give here is the following, you can imagine that receiver has created
multiple copies of a padlock, all of which can be opened using a single key. And now, the
public key is nothing but copies of that padlocks, but in an open state. If I am a sender and I
905
want to communicate some message secretly to the receiver, what I will do is, I will take, I will
take one copy of that open padlock, I will take the message and keep it inside a box.
And now I will lock the box using that padlock by pressing the padlock. So that is equivalent
to saying that I have encrypted my message. Now when that locked box reaches the receiver,
receiver has to open the padlock and that, he can do by using the secret key which is available
with the receiver. So that is the analogy and this is different from your symmetric key
cryptosystem where both sender and receiver are using the same key for opening the locked
box as well as for closing the locked box.
So, for the moment, just imagine that, we have a public key cryptosystem namely, we have an
instantiation of public key cryptosystem. But even if I assume that we do not have an
instantiation of public key cryptosystem, if at all this architecture is realizable, it has got
tremendous potentials. It has got tremendous potential in the sense that, now the whole problem
of key distribution is easily solved.
If I am a receiver, and if I am a amazon, for instance, I do not have to worry, who is the potential
sender, he can be any entity from the world. Whoever wants to communicate with me, I just
have to publish my public key for him which I can do once for all. And then anyone who wants
to communicate to me has just have to use that public key, encrypt a message and communicate
to me. So, in some sense, the problem of key agreement is solved.
And I do not need to have a dedicated secret key with each and every entity in this universe, I
will just have a secret key and the corresponding public key can serve the role of the encryption
key for every potential user with whom I want to do a secure communication. So, even though
Diffie and Hellman thought about this architecture, this new system, they failed to give a
concrete instantiation.
Namely, a concrete encryption algorithm, concrete decryption algorithm, a concrete

mechanism of coming up with a public key and a concrete mechanism of coming up with a
secret key. And the race for coming up with the first instantiation of public key cryptosystem
was won by another Turing Award winner triplet namely, RSA which we will discuss very
soon. But the interesting feature here, the interesting fact here is that, even though Diffie and
Hellman failed to come up with a concrete instantiation of above architecture, it was hidden in
906
their key exchange protocol itself, it was. So, what we are going to do next is, we will again
recall their key exchange protocol, the Diffie Hellman key exchange protocol and then we will
see that how by doing a minor tweak, a minor modification to the key exchange protocol, we
can get an instantiation of public key cryptosystem. But unfortunately, Diffie and Hellman
failed to realize that.
So, this encryption scheme is called as ElGamal encryption scheme attributed to Tahir Elgamal,
who made this very crucial observation regarding the Diffie Hellman key exchange protocol
and what exact modification needs to be done. So, this was the Diffie Hellman key exchange
protocol, Sita sends her mixture namely 𝑔𝛼 , Ram sends his mixture 𝑔𝛽 , were alpha and beta
are individual components picked by Sita and Ram and the final key 𝑔𝛼𝛽 .
Now, the whole process can be visualized as an instance of public key encryption scheme as
follows. The intuition is the following. If 𝑔𝛼𝛽 is a common key which is going to be agreed
upon between Sita and Ram and we know that if the discrete log problem is difficult to solve
in my group, then any third party who has monitored the communication will be unable to
compute 𝑔𝛼𝛽 or the key k in a reasonable amount of time, then I can use the same key k for
encrypting the message.
So, for instance, if Sita is the receiver and Ram is the sender and if sender is having a plain text
m and again imagine that it is an element of the group g over which all the operations are
performed then, what sender can do is the following. It can use the key, k namely 𝑔𝛼𝛽 for
masking the message.
907
By masking the message I mean, here that perform the group operation between the plain text
which is a member of the group and the key, k as well where key k is the element of the group
as well. So that will give another group element, denote it as c. So, c will be the encryption of
the message and now, how Sita can decrypt back the plain text. So, for recovering the plain
text, Sita has to unmask the effect of key because the message is been masked with key.
So, if the key is unmasked, the effect of mask will go away and Sita will be able to recover
back the plain text. And for doing that unmasking here, is nothing but taking the element c and
performing the group operation with c and the inverse of k. Because if I perform the group
operation on c and inverse of k then the effect of k and k cancels out. And what I will be left
with is the plaintext m which sender wants to encrypt.
So, now let us see the whole thing as an instance of a public key cryptosystem. So, this was the
message which sender, so, I am treating Ram as the sender here and I am treating Sita as the
receiver. So, the crucial observation of Tahir Elgamal was the following. I can imagine Sita or
receiver sending our contribution for Diffie Hellman key exchange protocol once for all, for
every potential sender.
So, right now what is happening is, depending upon Ram, Sita was picking alpha and
contributing g power alpha. So, if there are multiple Rams, she will be picking multiple alphas,
independent alphas and will be sending 𝑔𝛼1 to first Ram, 𝑔𝛼2 to the second Ram and so on.
The key observation here is that, do not do that for every potential Ram, in fact, it is not even
908
required that who is going to be Ram, let receiver or Sita start executing her instance of Diffie
Hellman protocol once for all.
Namely, whatever is her contribution for the Diffie Hellman key and what exactly I mean by
contribution? Our contribution was 𝑔𝛼 . So, you can imagine that in the Diffie Hellman key
exchange protocol, there are two contributions contribution 𝑔𝛼 coming from Sita’s site,
contribution g power beta coming from Ram site and both these contributions are somehow
combined to get the overall key 𝑔𝛼𝛽 .
So, what Elgamal proposed is that, let receiver makes her contribution, public once for all.
Namely, she picks some random alpha as her secret key and makes 𝑔𝛼 available in the public
domain as her public key. So, it is as good as saying that, she is declaring publicly that, I would
not be again and again participating in different instances of Diffie Hellman key exchange
protocol, whoever is going to be Ram, just to think as if I am going to send 𝑔𝛼 to you, if I
would have participated in the Diffie Hellman key exchange protocol.
That is the way public key and secret key will be picked by our receiver. Now, imagine there
is a sender, Ram who has a plain text m, he wants to encrypt the plaintext m. How he can do
that? Ram will now do his part of the Diffie Hellman key exchange protocol, namely, he will
give his contribution which is g power beta.
And now Ram knows that once he sends 𝑔𝛽 to Sita, using 𝑔𝛼 which Sita anyhow has made
public and treating 𝑔𝛽 as a message coming from Ram, Sita will be able to compute the key,
𝑔𝛼𝛽 . So, what Ram can do is, once he has sent 𝑔𝛽 , he can use k namely, 𝑔𝛼𝛽 for encrypting
the message.
And the overall encryption of the plain text will be now two messages. The first message will
be Rams contribution for the Diffie Hellman key exchange protocol. And the second message
is the actual encryption of the message. How Sita will be doing the decryption? So, this is the
encryption process, sending his contribution of Diffie Hellman key exchange protocol and then
masking of the message this whole thing can be visualized as encryption of the plain text.
909
The decryption happens as follows, Sita computes the key k, assuming that Ram has
participated in an instance of Diffie Hellman key exchange protocol. So, she will be able to
compute 𝑔𝛼𝛽 . So, for that, she has to take her secret key and raised that secret key, she has to
take the first component of the cipher text, namely c1 and raise it to her secret key which will
give her the common key 𝑔𝛼𝛽 .
And now she can unmask it by taking the second component of the cipher text. And performing
the group operation with k inverse where, k inverse she has computed in the previous step. So,
that will be the decryption process for Sita. A very cool observation which unfortunately Diffie
and Hellman missed. And that is why Taher Elgamal got whole credit of inventing this
cryptosystem.
And now, why this whole process, whole mechanism will be a secure mechanism? So, imagine
there is a third party or Ravana, will he be able to find anything about the message m in the
reasonable amount of time? Well, the only way he can learn anything about the message m is
by learning the key k. But for learning the key k, he has to actually attack the Diffie Hellman
key exchange protocol or he has to solve instances of discrete log problem. So, assuming that
solving random instances of discrete log problem is difficult, this whole process is indeed an
instance of public key cryptosystem.
So, now, as I said earlier that, race for coming up with a first instantiation of public key
cryptosystem was won by another Turing Award winner triplet, namely RSA, Rivest, Shamir
and Adleman. Now, let me give you briefly a description of the RSA public key cryptosystem,
910
again which is based on several interesting results from number theory that we have discussed.
So, let me recall the group 𝑍𝑁∗ and some concepts related to the group 𝑍𝑁∗ .
So, recall the definition of the 𝑍𝑁∗ is the collection of all the values in the range 1 to N - 1
which are co prime to your modulus N. So, for instance, the set Z 10 star will have the elements
1, 3, 7, 9, it would not have the element 2 because 2 is not co prime to 10. It would not have
the element 4, it would not have the element 5, 6, 8 because all of them are not co prime to 10.
It turns out that if your modulus N is a prime then, 𝑍𝑁∗ is nothing but a set 1 to N - 1.
And we already proved in one of our earlier lectures that, the group 𝑍𝑁∗ along with operation
multiplication modulo N constitutes a group. So, the order of this group 𝑍𝑁∗ namely the
number of elements in the set 1 to N - 1 which are co prime to N which is also the order of the
group 𝑍𝑁∗ is called as the Euler totient function, denoted by this 𝜙(𝑁) function. And there are
formulas for calculating the size or the order of the group 𝑍𝑁∗ depending upon the value of N.
So, the interesting cases are the following, if N is a prime number then the order of the
corresponding group phi p star is p - 1 or equivalently they are p - 1 elements in the range 0 to
p - 1 which are co prime to p. Whereas, the case which we will be using in the context of RSA
cryptosystem is N is the product of distinct prime numbers p and q. So, if N is the product of
distinct prime numbers p and q then the size of the group 𝑍𝑁∗ is being product of p - 1 and q -
1.
And this can be proved using the principle of mutual inclusion exclusion that we had discussed
in one of our earlier modules. So, if you want to verify this, consider N = 10 which is the
product of 2 and 5. And then, we know that there are 4 elements in Z 10 star namely, the order
of Z 10 star is 4, so, I should get phi of 10 is 4 and indeed, phi of 10 is 4 because it is 2 - 1
multiplied with 5 - 1.
And we know that, if I take any element a in the group 𝑍𝑁∗ then, 𝑎𝜙(𝑁) 𝑚𝑜𝑑 𝑁 is 1. And this
can be proved in multiple ways, I can use the following result from abstract algebra. I know
that the order of the group 𝑍𝑁∗ is ϕ(N) because that is what is the definition of ϕ(N) and element
1, the numeric element, the numerical 1 is actually the identity element of this group 𝑍𝑁∗ .
911
And I know that you take any group element, irrespective of what is the order of that group
element a to the power order of the group, will always give you the identity element. So, p of
N is the order of the group, a is an element, so, it does not matter what is the order of a,
irrespective of whatever is the order of a, I know a raised to the power order of the group is 1
and a raised to the power order is nothing but 𝑎𝜙(𝑁).
Based on this theorem, I can say the following. If I want to compute 𝑎 𝑥 𝑚𝑜𝑑 𝑁 then, in the
exponent I can perform modulo ϕ(N) because I can rewrite a power x as several blocks of a
power ϕ(N) a power ϕ(N) and like that and the last block consisting of a power x modulo ϕ(N).
Each of these blocks with a power ϕ(N) will give me the identity element 1, 1, 1 and I will be
left only with the last block which has a power x modulo ϕ(N).
So, now let me introduce RSA function which forms the basis of RSA public key cryptosystem.
This is a function from the set 𝑍𝑁∗ → 𝑍𝑁∗ . Then, how exactly is this function defined? So,
imagine you have a public exponent e, this is not identity element, this is some notation, this is
an exponent e which is we are going to use in the function. And this exponent e is relatively
prime to ϕ(N), I stress it is not relatively prime to N, it is relatively prime to ϕ(N).
Now, since e is co prime to ϕ(N), it will have a multiplicative inverse. I call that multiplicative
inverse as d, so, since e and d are multiplicative inverse of each other, this relationship hold.
Now, the RSA function in the forward direction is the following. If I want to compute the
output of the RSA function for x then that is same as computing 𝑥 𝑒 𝑚𝑜𝑑 𝑁. So, remember x is
a member of 𝑍𝑁∗ and hence 𝑥 𝑒 𝑚𝑜𝑑 𝑁 will also be an element of 𝑍𝑁∗ .
912
Because my underlying operation is multiplication modulo N and x power e is like performing
the group exponentiation. Whereas, my reverse function from 𝑍𝑁∗ → 𝑍𝑁∗ to will be the
following. If I have a value y and if I want to invert it, I compute 𝑦 𝑑 𝑚𝑜𝑑 𝑁. So, I can prove
that the function 𝑓𝑑 in the reverse direction is actually the inverse of the function 𝑓𝑒 in the
forward direction.
And the way we can prove it is as follows. So, you take any arbitrary x and suppose, for that
arbitrary x, the forward direction function gives you the output y. So, y = 𝑥 𝑒 𝑚𝑜𝑑 𝑁. What I
have to show is that, now if I invert this y, namely 𝑥 𝑒 𝑚𝑜𝑑 𝑁 as per the inverse function, I
should get back my x. So, let us do that. So, let us try to invert the value of y.
So, let me write down the value of y, y is nothing but 𝑥 𝑒 𝑚𝑜𝑑 𝑁 and then whole thing raised to
the power d then, I can apply the rules of group exponentiation and say that this is nothing but
𝑥 𝑒𝑑 𝑚𝑜𝑑 𝑁. And remember that 𝑥 𝜙(𝑁) is 1 because x is an element of 𝑍𝑁∗ . That is what we
discussed in the previous slide. So, 𝑥 𝑒𝑑 𝑚𝑜𝑑 𝑁 will give the same answer as if x to the power
in the exponent, I do 𝑒𝑑 𝑚𝑜𝑑 𝜙(𝑁) that means, I can reduce the exponent itself modulo ϕ(N).
But 𝑒𝑑 𝑚𝑜𝑑 𝜙(𝑁) is 1 that means, this 𝑒𝑑 𝑚𝑜𝑑 𝜙(𝑁) is 1, so, this is nothing but 𝑥1 𝑚𝑜𝑑 𝑁
and which is strictly x because if x was a member of 𝑍𝑁∗ that means, x was strictly less than
N. So, if x is strictly less than N then, the effect of mod would not take an x modulo N will be
same as x. So that shows that these functions 𝑓𝑒 and 𝑓𝑑 they are inverse of each other. So, if
you go to the forward direction through 𝑓𝑒 , you can always come back in the reverse direction
through 𝑓𝑑 .
913
Now, based on all these things, let us introduce a computational problem which we believe is
really difficult to solve. It is like your discrete log problem. So, we know that there are certain
groups where solving discrete log instance is really difficult. In the same way RSA introduced
a computational problem which we believe to be difficult to solve. Difficult to solve in the
sense, in the reasonable amount of time, a practical amount of time, we may not be able to
solve it.
I am not saying it is impossible to solve it, you can always solve it by doing a brute force, but
the brute force algorithm will take enormously large amount of time. If we operate on very
large numbers. So, the problem instance is as follows. So, we first define what we call RSA
parameter generation algorithm. So, this is parameter generation. So, to generate the
parameters, we randomly pick some n-bit prime numbers p and q.
And then we compute the modulus which is the product of p and q, we compute a value of
ϕ(N). Since, N is the product of prime numbers p and q which are distinct, by the way, we will
ensure that p and q are distinct. So, p and q are distinct, the value of 𝜙(𝑁) = (𝑝 − 1) ⋅ (𝑞 −
1). We will pick an exponent e which is co prime to ϕ(N). And since e is co prime to ϕ(N), we
will be able to compute its multiplicative inverse modulo ϕ(N) by running extended Euclid’s
algorithm.
And finally, the output of this parameter generation algorithm is the modulus, the prime factors
of the modulus, the public exponent e and the secret exponent d. What do I mean by public and
secret it will be clear soon. So, the RSA problem is the following. If I give you the modulus,
914
but not its prime factors and if I give you the public exponent in that sense, it is public, it will
be known to you and it will be known that how exactly this parameters N and e are generated.
What would not be known to you are the prime factors of N.
And if the prime factors of N are not known, you would not be knowing the value of ϕ(N) and
the value of ϕ(N) is not known to you, you would not be knowing the value of d. So, d is not
known, ϕ(N) is not known, p and q are not known. Now, the problem instance is the following.
I will be giving you a random element from my group
𝑍𝑁∗ and your goal will be to compute the inverse function, output of the inverse function that
we had just seen, for the randomly chosen y.
Namely, your goal will be to compute 𝑦 𝑑 𝑚𝑜𝑑 𝑁 where d is not given to you and 𝑦 𝑑 𝑚𝑜𝑑 𝑁
is nothing but computing y raised to power 1 over e where 1 over e is not numeric 1 over e but
it is actually the multiplicative inverse of d, so, multiplicative inverse of d is nothing but, so, d
and e, they are multiplicative inverse of each other. So, computing d is nothing but computing
𝑦1/𝑒 .
So, basically I am asking you to compute the eth root of 𝑦 𝑚𝑜𝑑 𝑁 and I want you to solve this
problem instance in polynomial of n number of time where n is the number of bits that I used
to represent p and q that means, my p and q are n-bit numbers. So, one way of solving this
problem instance is that you are able to factorize N. So, suppose you are able to factorize N in
polynomial amount of time namely, you are able to compute p and q in polynomial amount of
time.
Once you are able to compute p and q you will be able to compute ϕ(N) easily. And since you
know e and if you know ϕ(N), you yourself can run the extended Euclid’s algorithm and
compute d in polynomial amount of time and then you yourself can compute 𝑦 𝑑 . But it turns
out that factoring very large numbers is believed to be an extremely hard problem, specifically,
with the current computing power.
If I select my p and q to be as large as say, 512 bit prime numbers then my N will be an
extremely large modulus then factorizing that extremely large modulus will be very time
915
consuming process and hence you would not be able to solve and random instance of the RSA
problem.
So, now based on whatever theory we have discussed, let us see the concrete steps of the RSA
public key cryptosystem. So, remember, we have a function in the forward direction from
𝑍𝑁∗ → 𝑍𝑁∗ namely, x power e modulo N and a reverse function is y power d modulo N. And
we also have discussed RSA problem where if I do not give you the value of the secret exponent
d then, computing 𝑦 𝑑 𝑚𝑜𝑑 𝑁 is very time consuming that is what is the general belief.
Now, the way RSA cryptosystem works is as follows. So, remember, there is a sender and a
receiver. So, what receiver will do is the following. Receiver will run the parameter generation
algorithm. Namely, it will pick a random prime number p, a random prime number q, will
compute its product, will pick an index e which is relatively prime to ϕ(N). So, he can compute
ϕ(N) because he himself has picked p and q.
And once he has picked e, he knows ϕ(N), he can compute d as well. And then he will set pk
or the encryption key to be (N, e) and he will set as the decryption key to (N, d). So, d is kept
with himself, e is made public and N is also made public. If there is a sender, who has a plain
text m and which it wants to encrypt then, encryption of m is nothing but computing the forward
direction function as per the RSA function, namely, just output 𝑚𝑒 𝑚𝑜𝑑 𝑁.
And if there is a receiver who obtains the cipher text c and who has the secret decryption key
d then to get the cipher text, tend to recover back the message encrypted in c, he has to basically
916
compute the inverse function for the c, inverse function is computable, if d is available and if
d is available with the receiver, he can easily compute 𝑦 𝑑 𝑚𝑜𝑑 𝑁 and get back the message.
Now, why this is called as a textbook cryptosystem? Because this is not precisely the way we
use RSA public key cryptosystem in practice.
There are lots of shortcomings. One very important one, a very bad feature of the way RSA
cryptosystem is proposed is given here and if I use here, the major shortcoming here is that, it
is deterministic. It is deterministic in the sense that, if there is a sender S and suppose, he wants
to send the same message m after every one hour. Then, if you encrypt the same message m
using the public key (N, e), every time he will be producing the same c. So, the first time he
wants to encrypt a message m, he will send c.
Next time he wants to send a message m using the same key, he will be again sending the c,
again next time he wants to send the same message m, he will be sending the c, this itself is a
lot of information for the third party or the attacker. He may not be able to learn the exact value
of the message m. But he will be coming to know that actually it is the same message m which
has been encrypted and communicated to the receiver.
And depending upon my underlying application, this itself can be a breach of security. Ideally,
I am looking for a process where, even if the same message m is encrypted using the same
public key multiple times, it should produce different cipher texts with high probability. But
that is not the feature available with the way RSA public key cryptosystem was invented. But
we can of course get rid of this shortcoming and actual way in which we used RSA public key
cryptosystem is different from the way it is proposed here.
But this forms the basis of the RSA public key cryptosystem, a very interesting public key
cryptosystem. So, with that, I conclude today’s lecture. Just to summarize, in this lecture, we
discussed about public key cryptosystem and we discussed two popular instantiations of public
key cryptosystem namely, we have discussed ElGamal encryption scheme and we have
discussed RSA public key cryptosystem. Thank you!
917
Lecture - 65
More Applications of Groups
Hello, everyone, welcome to this lecture. So, till now we have discussed a lot of theory regard
in the context of number theory and abstract algebra. In the next couple of lectures we will see
how to tie whatever we have learned till now in number theory and abstract algebra and see
some concrete applications in the context of cryptography.
So, the plan for this lecture is as follows. In this lecture, we will introduce this concept of
discrete logarithm and the discrete logarithm problem. And we will see some cryptographic
applications of the discrete logarithm problem namely the seminal key exchange protocol due
to Diffie and Hellman.
918
So, let us start with the discrete logarithm definition in the context of cyclic groups. So, let G
be a cyclic group and that abstract operation is o and suppose the order of the cyclic group is q
that means, you have q number of elements and for simplicity and without loss of generality, I
will follow that multiplicative interpretation, while giving the definition of discrete logarithm,
but the definition can be easily generalized even when the underlying operation is interpreted
in the additive sense.
So, since my group G is a cyclic group it must be having a generator, so, let the g be generator
and since the order of the group is q, has q number of elements that means, by raising or by
computing q different powers of the generator, I can obtain all the elements of my group. Now,
consider an arbitrary element y from the group, since the element y is a member of the group,
it can be generated by some power of your generator.
That unique power in the range 0 to q - 1 which when raised to the generator gives you the
element y, will be called as the discrete logarithm of the y to the base g. That is the definition
of my discrete logarithm. So, in some sense, it is equivalent to our definition of natural
logarithms. So, we know that if a to the power x = b then we say that log of b to the base a is
x, we are trying to come up with an equivalent definition in the context of a cyclic group.
So, g is a special element the generator because, if I keep the generator to the base and compute
different powers of the generator, I can obtain all the elements of my group that means, I can
say that you give me any element of the group there must be some power of the generator such
919
that that power x of the generator gives me the element y, that unique power x in the range 0
to q - 1 is called as the discrete logarithm.
And interestingly, like the natural logarithms, your discrete logarithm also obeys the rules that
are there in the context of natural logarithms. For instance, we know that log of 1 to the base
of any a is 0 because a to the power 0 is defined to be 1 in for natural logarithms. In the context
of discrete logarithms, we say that the discrete log of the identity element of the group to the
base of g will be 0.
Because, remember, as per the rules of group exponentiation, we have defined 𝑔0 to be the
identity element. In the same way, if I take an element h from the group and compute the
element h to the power r which will be also a group element and now if I try to find out the
discrete logarithm of the group element h to the power r to the base g then it will be same as r
multiplied with the discrete logarithm of h to the base g.
And if this value is in the range 0 to q - 1 well and good, otherwise, I take a mod and bring
down the value to the range 0 to q - 1. In the same way, if I have 2 group elements h1 and h2
then the discrete log of the product of h1 and h2 will be same as the summation of the discrete
logs of h1 and h2 individually modulo q. And the general theorem statement that we can have
is the following.
If you are given that 𝑔 𝑥 = 𝑦 where g is the generator then, either x will be the discrete log of
y if x is within the range 0 to q - 1, else if x is greater than q then, if I take x modulo q then the
resultant value will be in the range 0 to q - 1 and that will be the discrete logarithm of y to the
base g. This is because if x is indeed greater than q then I can rewrite 𝑔 𝑥 as several blocks of
𝑔𝑞 , 𝑔𝑞 , 𝑔𝑞 and the last block consisting of 𝑔 𝑥 𝑚𝑜𝑑 𝑞 .
And each of these blocks of 𝑔𝑞 will give me the identity element because g is the generator
and its order will be q, hence 𝑔𝑞 will be identity element and the last block will be 𝑔 𝑥 𝑚𝑜𝑑 𝑞 .so,
I get 𝑔 𝑥 𝑚𝑜𝑑 𝑞 . So, if 𝑔 𝑥 = 𝑦, so, is 𝑔 𝑥 𝑚𝑜𝑑 𝑞 and 𝑥 𝑚𝑜𝑑 𝑞 will be a value in the range 0 to q -
1 and hence it will be the discrete logarithm.
920
So, now an interesting problem is that how easy or how difficult it is to compute the discrete
logarithm. So, imagine we are given an abstract cyclic group of order q and this notation means
that, the number of bits that I need to represent my q is n bits. So, this notation is nothing but
the number of bits needed to represent q. That means magnitude wise q is as large as 2n. Now,
let us see how difficult or how easy it is to compute a discrete logarithm.
So, you are given the description of the cyclic group. By the description I mean, you know, the
characteristic of the elements of the group, your group might be exponentially large. It might
have exponentially large number of elements and you may not have sufficient space and
resources to store down all the elements of your group. But you may know the characteristic
or the properties of the elements of your group.
And you are given a generator. So, you know that, by computing different powers of that
generator, you can generate any element of your group. And what is given to you, a random
element y from the group. That is important, a random element. It is not a predetermined or
specific element of the group, it is randomly chosen. And a discrete log problem is to compute
the discrete log of this randomly chosen y, given that, you only have the y and the generator,
you do not have anything else.
So that means your goal is to come up with a unique power x in the range 0 to q - 1 such that,
𝑔 𝑥 would have given you y. And what we want to do is, we want to come up with an algorithm
whose running time should be polynomial in the number of bits that I need to represent my q,
921
namely, n, I do not need an exponentially large algorithm. So, here is a naive algorithm which
will always be successful to give you the discrete log of the randomly chosen y.
I call this algorithm as brute force discrete log solver because it basically does what a naive
algorithm will do, you basically try all powers of x in the range 0 to q - 1. And check whether
computing g power x gives you the element y or not, if it is then, you output that x and stop the
algorithm. And definitely you will hit upon the exact value of x which is the discrete logarithm
of y somewhere when you are iterating over all values of x.
So, you will always get the answer. But let us focus on the running time of this algorithm. In
the worst case, you may end up performing iteration over all candidate values of x. So, I can
say in the worst case, the running time is order of q. But q is not a polynomial quantity in n, it
is actually an exponentially large value in the number of bits that I need to represent my value
q. So, this algorithm is not a polynomial time algorithm but rather it is an exponential time
algorithm.
So now, the next question is does there exist a better algorithm? And the answer is both yes, as
well as no. Yes because as we will see soon, there are indeed certain cyclic groups where I can
efficiently find out the discrete logarithm of any randomly chosen y, without doing the brute
force. But at the same time, we do have some candidate cyclic groups where it is for which it
is conjectured that we do not have any better algorithm other than brute forcing over all
candidate values of x. So, this is a instance of one such cyclic group.
922
So now, let us see the case where the discrete log can be easily computed. So, I consider the
cyclic group ℤ𝑝 where ℤ𝑝 is the set of all integers modulo p, namely, it has integers 0 to p - 1.
And my operation here is addition modulo p. You are given a generator, by the way, since the
order of this cyclic group is a prime quantity because it has prime number of elements then,
from the results that we know till now, every element of this set ℤ𝑝 , except the identity element
0, will be a generator.
So, 1 is a generator, 2 is a generator, 3 is a generator, p - 1 is also a generator. So, now let us

see an instance of discrete log problem and how efficiently we can solve it. So, you are given
the generator, you are given the random y and your goal is to come up with a unique exponent
x, such that, 𝑔 𝑥 would have given you y, namely, you want to compute a discrete log of y.
By the way, the interpretation of 𝑔 𝑥 here, will be in the additive sense, we are not going to
multiply g because our underlying operation here is addition. So, 𝑔 𝑥 should be interpreted as
𝑥 × 𝑔. So, imagine x is the discrete logarithm of y and that is the case then, y is nothing, but g
added to itself modulo p, x number of times and our goal is to find out what exactly is x.
So, I know that y is nothing but x times g modulo p because g added to itself modulo p is
equivalent to saying that, I multiply x with g and then take mod p. And my goal is to find out
this unknown x. Now, it is easy to see that this unknown x is nothing but the product of y with
the multiplicative inverse of g modulo p. So, this g inverse is now not the additive inverse. This
is now the multiplicative inverse of g modulo p.
And now you might be wondering whether the multiplicative inverse of g modulo p exists or
not. Indeed, it exists because the generator g is an element in the range 0 to p - 1 and it is co-
prime to p. In fact, all the elements 0, 1, 2 up to p - 1 are co-prime to p because your p is a
prime. So, the generator g is also co-prime to p. And then, since the generator is co-prime to p,
we know that its multiplicative inverse modulo p exist which we can easily find out using
Euclids algorithm.
So, now you can see that here, I do not need to do a brute force, I do not need to check whether
x = 0 satisfies relation y = x times d modulo p or not, I do not need to check for x = 1, x = 2
and all the way x = p – 1. I do not need to do that because I know that, just by multiplying y
923
with the multiplicative inverse of g, I can hit upon that right x. So, this is an instance of a cyclic
group where D log is easy, very easy to solve.
Now, let us see another cyclic group where it is conjectured that discrete log problem is really
difficult to solve for a random instance. So, consider the cyclic group ℤ𝑝∗ where ℤ𝑝∗ , will have
all the integers which are relatively prime to p and p itself is a prime number. So, if that is the
case then ℤ𝑝∗ will have all the numbers except 0 from ℤ𝑝 . And my operation is now
multiplication modulo p in that group.
So, we know that this is a cyclic group. And, in fact, later we will prove it concretely that this
group is indeed a cyclic group, but for the moment, you have to believe me that this group is
indeed a cyclic group. So, now let us see a random instance of the D log problem in this group.
So, you are given a generator, you are given a random value y and your goal is to compute a
discrete log of y.
So, this should be ℤ𝑝∗ because y is an element of your group and the group that we are right
now considering is ℤ𝑝∗ . So, imagine 𝑦 = 𝑔 𝑥 𝑚𝑜𝑑 𝑝. And it turns out that we do not have any
better algorithm to compute x, other than naive brute force algorithm that we have discussed
in the last slide. And this is because there is no pattern available because of the mod operation
which I am performing. What do I mean by no pattern available?
So, by that, I mean that, if I keep on increasing the value of x and check the value of y where
𝑦 = 𝑔 𝑥 to the power x modulo p then it is not necessary, it is not the case that as your value
of x increases, the value of y also increases. You compare this with the corresponding function
where I do not do a mod p operation. Suppose my function 𝑦 = 𝑔 𝑥 then for such a function,
I can confidently say that as the value of x increases, the value of y also increases.
But as soon as I redefine my y to be 𝑔 𝑥 𝑚𝑜𝑑 𝑝 then because of the mod p operation, it is no

longer the case that, as the value of x increases, the value of y also increases, it will increase,
decrease, increase, decrease, increase, decrease and there will be absolutely no pattern in which
the values of y increases and then suddenly dips and then suddenly increases and so on.
924
So, there will be a complete chaos if you plot a chart or a graph between x and ys with respect
to the fixed g. And it turns out that we cannot find out any pattern and hence, if my x is not
known to you, if I do not give you the value of x and just give you the value of y and it will be
very difficult in general to compute the value of x, if my group is a sufficiently large group.
So, now, let us see some applications of the discrete log problem in the context of cryptography.
So, let me tell you something about cryptography. So, it is a mathematical science, and the
main goal of the cryptography is to establish a secure communication channel between 2
entities say, Sita and Ram, who do not know anything about each other, they are meeting for
the first time over the internet, and they want to talk over the internet by exchanging public
messages.
And at the same time, they would like to ensure that, no one else should be able to find out
what exactly they are communicating. So, main application of cryptography is that we would
like to run some algorithm. And using those algorithms we would like Sita and Ram to
exchange messages, so that, it should give them the effect of a secure channel, secure channel
in the sense, it would look like as if Sita and Ram are doing conversation over very secure
channel which provides 3 properties.
It should provide the privacy of the communication, namely it means that, even if a third party
observes whatever communication is happening between Sita and Ram and even if the third
party knows the protocol description, according to which Sita and Ram are doing the
925
conversation, still that third party should not be able to figure out what exactly Sita and Ram
actually are talking about that means, the actual contents of their messages.
So that is a rough definition of privacy, we also need the authenticity property namely any
message or any packet which is coming to Sita, it should have a proof that indeed it came from
the person called Ram and in the same way any packet which goes to Ram, there should be a
proof that or there should be a mechanism to verify that indeed that packet came from the
person called Sita. So that is a rough definition of authenticity property.
And the third requirement of this secure channel is that of integrity. That means, if there is a
third party which messes some of the bits or contents which are exchanged over the secure
channel then it should be detected at the receiving end. So, through cryptography, we achieve
all these 3 properties. So, basically cryptography gives you a set of algorithms, a set of
protocols, according to which Sita can convert her messages in some format and communicate
to Ram and vice versa.
So, there are lots of applications of cryptography. So, for example, if you are a user and if you
are doing a net banking transaction then you are supposed to give your net banking password,
at that time, you do not want your net banking password to be revealed to a third party, it should
be securely communicated to the bank. So that is an application of cryptography.
In the same way, whenever you are buying something on the internet on Amazon, you are
asked to enter your credit card details; again, you would like to exchange or send your credit
card information in a secure way without any third party knowing about the exact details of
your credit card information. So, again, cryptography is coming into picture. So, it turns out
that cryptography is now used left and right in each and every application because slowly and
slowly each and everything is becoming digital. So now, you might be wondering what sort of
algorithms we use in cryptography?
926
So, now I will give you very simple algorithms based on number theory and abstract group
algebra, abstract algebra that we have seen till now. So, the 2 core problems that are addressed
by cryptography are the following. The first problem is that of key agreement. So, what exactly
is the requirement in the key agreement problem? So, the setting is the following.
We have 2 entities Sita and Ram, who do not have any pre shared information that means, no
secret question, secret date of birth, nothing. They are meeting for the first time and they are
going to talk publicly over the internet. So, we need a protocol here according to which Sita
and Ram should talk to each other, and the protocol description also will be publicly on that is
also important.
It is not the case that, process by which Sita is going to decide her message is known to Ram
beforehand and vice versa. Because I am assuming that they do not know anything beforehand.
So that means, if at all they are going to use a protocol that will be publicly available. So, we
need a publicly available protocol according to which Sita and Ram should talk to each other.
And at the end of the protocol, magically, both Sita and Ram should arrive at a common key k
which is a binary string of some length. And the interesting property, the security property that
I need from this key agreement protocol is that, if there is any third party who has monitored
the communication between Sita and Ram and who knows the protocol description should not
be able to figure out what exactly is the key k which Sita and Ram has output. It might look
like an impossible task, but we will see soon, how exactly key agreement can be achieved?
927
And assuming that the key agreement has been achieved, the second problem that is addressed
by the cryptography, the second core problem, I should stress here, it is not the case that secure
communication is the only problem, the second core problem addressed by cryptography
startup secure communication. So, the setting here is the following, we will assume that Sita
and Ram has already executed the key agreement protocol over the internet, and they have
agreed upon a common key.
And now using this common key, we would require Sita and Ram to come, we would require
some algorithms which are publicly known, according to which Sita can convert or encrypt her
message into some garbled text into some garbage and communicate to Ram and Ram should
be able to convert back those garbage or scrambled text back to the original contents using the
same key, k which Sita has. So, namely we want to come up with algorithms which should help
me to do secure communication.
And by secure communication here I mean that, if there is a third party or Ravana, who knows
the public description of your algorithm but does not know the value of key then even after
observing the communication happening between Sita and Ram and even after knowing the
full protocol description according to which these messages have been computed, the Ravana
should not be able to come up with the values of m1, m2, m3 and so on. So that is the second
problem addressed by cryptography.
928
So, it turns out that there are two kinds of, two classes of cryptographic algorithms which we
use. The first category is that of private key or symmetric key encryption. In the symmetric key
encryption, the setting is the following. It will be ensured that a common key is already shared
between Sita and Ram by some mechanism, say, by running a key agreement protocol and no
one else apart from Sita and Ram knows the value of that key.
Now, if that is the case assuming this setup has been done, the way symmetric encryption works
is as follows. So, imagine Sita has some message, it could be an email, it could be just a hi
message, it could be anything, it could be her net banking password. So, she has some message
which is abstracted as a binary string, we call her message as plain text. We want to design an
algorithm which we call as an encryption algorithm which takes a message m and the key k
both of which are binary strings.
And it should produce another binary string which we call a ciphertext. And this ciphertext will
be the scrambled message because it will have absolutely no meaning, in the lose sense and
Sita will compute this ciphertext and communicate it over the internet and send it to Ram. Now,
once Ram obtains this scrambled message, he will have a decryption algorithm, he will have
in a sense, he will know that Sita has used an encryption algorithm whose details are publicly
known and the corresponding matching decryption algorithm also will be publicly known.
So, Ram will use the corresponding decryption algorithm. And the inputs for the decryption
algorithm will be the ciphertext that he has received and the same key which has been used by
929
Sita to produce the scrambled text. And this decryption algorithm will magically produce back
the same message m which Sita has used or wanted to communicate.
So, the reason it is called symmetric key encryption is because of the symmetry, namely, the
same key is used both for encrypting the message as well as for decrypting the message. Now,
the system might look very neat, very clean, just you encrypt your message and send a message,
encrypted message, Ram receives encrypted message and decrypt and recover back the
message.
So, the analogy could be that, assume Sita and Ram have already exchanged a key for a physical
lock. If Sita has a message, what she can do is, she can take a box, keep her message written in
a paper inside the box and close the box with a lock and using the key that she has. And now
she can send this lock box by a courier or anything. So, if there is a third person who does not
have the key for opening the lock of the box, he would not be able to do that.
Now, once the courier is delivered to Ram, since Ram also have the same key, he can use it,
unlock the lock and see what exactly is the content kept inside that box. So, the same message
which Sita wanted to send will be delivered to Ram. So that is the analogy. But the system will
work if both Sita and Ram have already agreed upon this common key for the lock. How at the
first place they can do that?
Because everything will be now happening over a public channel because it is not the case that
Sita and Ram knew beforehand in advance. It is like saying the following, if I want to do a
transaction over the internet; Amazon may not be knowing well in advance that a person called
Ashish Chowdhury, would like to do a transaction with Amazon. So, I will be doing my
transaction at a run time, how at the first place I establish a secure key with Amazon? And that
too, by communicating over the internet, so that is a big question. How at the first-place key
agreement has taken place?
930
So, it was a folklore belief that it is not possible to agree upon a common key by interacting
over a public channel. But the Turing Award winner, Diffie and Hellman, proved this belief to
be incorrect, by coming up with their seminal key exchange protocol. So, I would not be going
into the full details of security proof and other details of the key exchange protocol, I will just
try to give you the underlying idea.
So, the main idea used in their key exchange protocol is the following. They observed that there
are plenty of tasks in this universe which are asymmetric, they are asymmetric in the sense,
they are very easy to compute in one direction. That means, it is very easy to go from one state
to another state but extremely difficult to reverse back the effect of that action. So, for instance,
if I take a padlock in an open state then it is very easy to lock the padlock, I just have to press
it I do not need any key.
But now, once I go to closed state of this padlock and if I asked you that, can you open it, until
and unless you do not have the key it will be extremely difficult for you. So, I am saying it is
extremely difficult to open it without a key, there might be some other mechanisms to open it
as well. You might have a Jugaad method but that will be extremely difficult, very time
consuming. I am not saying it is impossible. So, there is a difference between extremely
difficult and impossible.
In the same way, consider this task; I take a publicly known color. And then I prepare a secret
mixture. In the sense I do the public color, I add a secret color. And then once the mixture is
prepared, I give it to you. So, the mixture preparation is very easy. And now if I ask you that,
931
okay, I give you this mixture, I also tell you the public color with which I started with, can you
tell me what exactly was the secret color that I added?
Again, it is not an impossible task, you yourself can take a tumbler with a publicly known color
and keep on adding various colors which you can add and see whether that gives you the same
secret mixture that I have. But that might be a very time-consuming affair. And most
importantly, it is very easy to break someone’s heart by saying very bad words, but it is very
extremely difficult to win the love and confidence of that person back.
So, based on this idea that asymmetry is there in lots of tasks. This is the underlying idea of
Diffie Hellman key exchange protocol. So, I will be first explaining the protocol assuming that
Sita and Ram want to agree upon a common secret mixture which should be random at least
should be decided and no one else should be able to learn what exactly is the secret mixture.
So, to begin with, both Sita and Ram will be starting with some common publicly known color.
And now, what they will be doing is the following. They will prepare independently some
secret mixtures. So, Sita will prepare her secret mixture independently and Ram will be
preparing his secret mixture independently, by adding a secret color, individually and then they
will publicly exchange their mixtures. So, Sita will send her a copy of the mixture to Ram, Ram
will send his copy of the mixture to Sita.
And here I am assuming that mixture separation is an extremely difficult task that means, if
there is a third party who is observing the communication here, who knows the entire process
932
according to which Sita and Ram are acting, so, he knows that both Sita and Ram started with
a secret colour. He also knows that, Sita has added a secret component but the exact value of
that secret component is not known to this third party.
In the same way, he knows that Ram has added a secret component but he does not know what
exactly is that secret component? And now of course, he is seeing the public mixtures being
exchanged. Now, what is the goal? The goal for Sita and Ram is to come up or agree upon a
common mixture which should be known only to them. So, what they can do is, they can
individually add the secret component that they have added, to the copy of the mixture that
they are receiving from the other party.
So, whatever Ram’s mixture that Sita has received, she takes that and to that she adds whatever
components she has added to prepare her secret mixture. And same task is done by Ram. He
takes Sita’s mixture and to that he had the secret color that he added to prepare his copy of the
secret mixture. And what this will give? This will give both of them a common mixture because
it does not matter in what order you add the 3 colors finally, it will give you the same mixture.
And it will be random, it will be random in the sense, next time Sita and Ram again runs the
same protocol, they will start with the same copy of the public mixture. But now the secret
components with Sita and Ram are going to add might be different because every time every
execution of the protocol they will be preparing independent mixture. So that is why, the output
at the end of each instance of this protocol will be a random mixture. And why it will be secret?
It will be secret because any third party who is monitoring the communication, he would not
be able to separate out the secret contribution of Sita and Ram. So that is why he would not be
knowing what exactly is the final mixture that Sita and Ram have obtained. Now, we have to
convert this whole process, this whole color exchanging idea into a concrete algorithm,
mathematical algorithm and protocol.
933
So, on your left hand side, I have returned the blueprint of the color based key exchange
protocol. And now, I will instantiate each and every step by concrete mathematical step. So,
both Sita and Ram started with some public information. That public information is the
description of a cyclic group, its order and the description of the generator. So that is a public
information.
So, Sita and Ram both knows that, okay, they are going to use this cyclic group and every third
party who wants to derive the key that Sita and Ram have agreed, are going to agree upon will
also know the description of the cyclic group. So, it is like saying the following, if Sita is
considered as a user and Ram is considered as a Amazon and if Sita wants to do a transaction
with Ram and want to agree upon a common key then, a third party who wants to break the
communication knows what exactly are the description of the cyclic group that Sita and Ram
are going to use.
So now, the first step of the protocol was that, Sita and Ram prepare some random secret
mixtures independently. The corresponding instantiation of that step is that Sita randomly picks
a group element and Ram randomly picks a group element, how they can do that?
So, Sita picks a random 𝛼 ∈𝑟 ℤ𝑞 in the range 0 to q - 1. So, this notation means that 𝛼 is
randomly chosen, this r denotes that it is randomly chosen. So, she picks the α randomly from
the set 0 to q - 1 and computes 𝑔𝛼 . And independently Ram picks a random 𝛽 in the range 0 to
q - 1 and computes 𝑔𝛽 . So that is their independent secret mixtures which they now
communicate.
934
And now, what was the final step? So, Sita upon receiving Ram’s mixture, she adds her own
secret component, her secret component was α. So, adding here will mean that, she will take
Ram’s mixture namely 𝑔𝛽 and raise the whole thing to α which will give her 𝑔𝛼⋅𝛽 . And what
Ram is going to do? He will take Sita’s mixture, namely 𝑔𝛼 and to that he will add his own
contribution.
Again adding in this context means, raise it to the power 𝛽 which will result in 𝑔𝛼⋅𝛽 . But what
about a third person, an attacker, an eavesdropper, who has monitored the communication, will
he be able to compute 𝑔𝛼⋅𝛽 because that is a common key which Sita and Ram are going to
agree upon. Well, for the adversary or for the third person to compute 𝑔𝛼⋅𝛽 , he should know α
or 𝛽.
Because if any of these 2 values is learned by the attacker, he can easily compute 𝑔𝛼⋅𝛽 . But for
learning α or 𝛽 he has to basically solve, either this instance of discrete log or this instance of
discrete log, namely, upon saying 𝑔𝛼 , he should be able to compute α in the reasonable amount
of time or given 𝑔𝛽 he should be able to compute 𝛽 in a reasonable amount of time.
That means, if I ensure that computing discrete log is extremely time consuming for this
attacker and by time consuming means, at least it takes say 10 years or 15 years then, I can say
that this protocol is safe because I do not care after 15 years if attacker comes to know, what
exactly I communicated 15 years back. Because I will not be interested to keep the privacy of
my communication for so long, so, as long I ensure that it is extremely difficult.
How extremely difficult it is? So, in loose tense, it is a order of several years, if it is extremely
difficult for an attacker to come, solve an instance of discrete log, then this protocol gives me
a mechanism according to by which Sita and Ram can agree upon a common key. So, now,
you might be wondering that what should be the choice of the group, how big it should be and
so on.
So, it turns out that, if we instantiate this protocol with my group being ℤ𝑝∗ that means, if I
ensure that my group G is the set ℤ𝑝∗ where, p is some 2048 bit prime number then, using
current best computing speed machines for solving a random instance of discrete log, it will
935
take order of several years and hence, an adversary who tries to attack the scheme will fail to
do that.
And that ensures that Sita and Ram now, can safely use the key 𝑔𝛼⋅𝛽 as the common key and
run an instance of symmetric key encryption scheme to do the secure communication of their
messages. So, now you can see that how exactly the concept that we have seen in the context
of cyclic groups are useful to come up with a very important practical solution for a practical
problem namely that of key agreement.
So, with that, I conclude today’s lecture. Just to summarize, in this lecture, we introduced the
problem of discrete log and we saw that in some groups, solving discrete log might be very
easy, in some groups, it is conjecture that solving a random instance of discrete log is extremely
difficult. And if we work with those groups then, we can design practical algorithms for real
world problems like the key exchange problem. Thank you!
936
Lecture - 66
Rings, Fields and Polynomials
Welcome to this lecture. So, the plan for this lecture is as follows. In this lecture we will
discuss about rings, fields and polynomials over rings.
So, let us begin with the definition of a ring. So, we denote our ring by this notation ℝ and it
is an algebraic structure. So, namely a set, set of values, it could be a finite set, it could be
infinite set, so, it is a set and there are 2 operators plus (+) and dot (∙) which are defined over
the elements of this set. I stress that these are not integer plus and integer dot they are just
some abstract operations.
937
But for the sake of notation we are using this plus notation and dot notation. So, we will say
that this set ℝ along with these 2 operations plus and dot will be called a ring if all the
following ring axioms are satisfied. Axiom number 1: we need the set ℝ to satisfy the
properties of an abelian group with respect to your + operation and what are the properties
that we require from an abelian group.
We need closure property, we need plus operation to be associative, we require the presence
of some special identity element which we denote by this ‘0’ such that if you perform the
plus operation with element a you should get back the element a for every element a from the
set ℝ. We need the presence of additive inverse and we need the operation plus to be
commutative. So, this is the first requirement from the set ℝ and operation + namely it has to
satisfy the properties of an abelian group.
The second axiom that set ℝ should satisfy is the following: we require that the dot operation
should satisfy the closure property namely, you take any pair of elements (a, b) from your set
ℝ and you perform the dot operation you should get back again an element from the same set
ℝ. We require the dot operation to be associative. That means, it does not matter in what
order you perform the dot operation on 3 elements, you should get back the same answer.
And we demand the presence of an identity element with respect to the dot operation. So, I
denote the identity element if at all it exists by this element 1. I again stress that in abstract
algebra, this is just a notation this does not mean numeric 1 or integer 1. So, this element 1
should satisfy the property that if you perform the dot operation with 1 and any group
element; any element from the set ℝ you should get back the same element a. So, this is the
second ring axiom and the third property or the so called third ring axiom that needs to be
satisfied is that your dot operation should be distributive over the plus. Namely, if you take
any triplet of elements from the set ℝ called them as a b c, then it does not matter whether
you first perform the plus operation on b and c. So, you will get one element from the set ℝ
because the set operation plus will satisfy the closure property and if you now perform the dot
operation on the result and a then the result should be the same as if, if you perform the dot
operation involving a and b, you perform the dot operation involving a and c and then if you
perform the final plus operation and we need the distributive property to hold both in the left
sense as well as right sense.
938
And namely, we need to satisfy both the left distributive property as well as the right
distributive property. So, the distributive property that I had just discussed is called as the left
distributive property. We need the dot to be distributed over plus even if dot is after the plus.
So, if all these 3 properties R1 R2 R3 are satisfied, then we will say that the set ℝ along with
the abstract operations plus and dot constitutes a Ring (ℝ, +, ∙).
So, let us see some examples for ring. So, recall our set ℤN is the set of integers 0 to N - 1 and
suppose I take 2 operations here: plus operation is the addition modulo N (+N) and my
multiplication operation is multiplication modulo N (∙N) and my claim is that with respect to
these 2 operations, my set ℤN satisfies all the ring axioms. So, it is easy to verify that indeed
the collection 0 to N - 1 along with the operation addition modulo N constitutes an abelian
group we had already proved in our earlier discussion.
And it is also easy to verify that if we consider the multiplication modulo N operation then it
satisfies the closure property, the operation is associative and identity element is actually the
numeric 1, integer 1 which is actually present in your set ℤN. If you multiply 1 with any
element from the set ℤN and then take modulo N you will get back the same element and it is
easy to that the distributive property is indeed satisfied.
That means you can distribute this multiplication module N over plus modulo N both in the
left sense as well as in the right sense. So, that is why all my ring axioms are satisfied and this
ring ℤN with respect to the operation plus modulo N and multiplication modulo N is a very
939
special ring. Because typically in our computers in our programming in our computers we
have registers we have either 32 bit registers, 64 bit registers or even if you have powerful
processor then you have 128 bit registers, 256 bit registers, where you can save values using
either 32 bit or 64 bits or 128 bits or 256 bits and so on and if you add any two 32 bit number,
then again you get back a 32 bit answer and so on. So, if you consider say for instance your C
programming language; in older version the integers used to have 32 bit representation and
you add any 2 integer values; again you used to get back an integer value which can be
represented by 32 bits.
So, implicitly there the operation that we are performing are addition module N and
multiplication module N; namely if I say int a, b, c and if I perform c = a + b then internally I
am actually performing a + b modulo N where N is 232 if the integers a b c are represented by
32 bits whereas my N will be 264 if my a b c are represented by 64 bits and so on. So, this is a
very useful ring used because our internal processors in computers perform operation with
respect to addition modulo N and multiplication modulo N.
So, the next thing that we want to discuss is the invertible elements of a ring. So, imagine you
are given a ring and if you see closely the ring axioms, it turns out that it is not the case that
every element should have a multiplicative inverse. Again, without loss of generality, I am
considering this dot operation in the multiplicative sense. But again and again I stress that this
is not the usual integer multiplication.
940
So, if you see the ring axioms it is not necessary that every element have a multiplicative
inverse, you need the presence of identity element 1, but it is not necessary that every element
has a multiplicative inverse. So, for instance, if I take this ring, ℤ4, where my ℤ4 is the
collection 0, 1, 2 and 3, then there are several elements which do not have any inverse with
respect to the multiplication modulo 4 operation.
So, for instance the element 2 does not have any inverse. You multiply 2 with 0, you get 0.
You multiply 2 with 1 and then take mod 4 you get 2, you multiply 2 with 2 and take mod 4
you get 0 and so on, you never get a result which is 1. So, the element 2 does not have
multiplicative inverse here. So, now what we are going to do is we are going to define a
special set which I call as U(ℝ) and U(ℝ) is basically the collection of all invertible elements
with respect to the multiplication operation.
So, it is the collection of all elements x from your set ℝ for which you have the guarantee of
presence of some element u such that if you perform the dot operation with x involving x and
u, you get back the multiplicative identity element and if you have an element x present in the
set U(ℝ) then the corresponding multiplicative inverse we denote by this notation (x-1) again I
stress that this does not mean, that I am talking about 1/x this is just a representation;
representation of the unique value u which when multiplied with x gives you the identity
element 1. So, let us see some examples of the set U(ℝ); if I again consider the same ring ℤN
then we know that the invertible elements of the set ℤN are those elements x in the range 0 to
N - 1 which are co-prime to N.
Because we have proved that multiplicative inverse with respect to multiplication modulo N
of a number x exists if and only if x is co-prime to N, that means, if I consider my N to be a
prime number p then this set you U(ℤp) will be nothing but all the elements except 0 in the
range 0 to p - 1 because 0 would not be co-prime to p.
941
So, now we can prove some interesting properties for this set U(ℝ). We can prove that if you
are given a parent ring and parent ring in the sense that my set U is defined with respect to
this ring ℝ then if I consider the set of all invertible elements from this ring with respect to
the multiplication operation then that collection constitutes a group and we are going to prove
this.
So it constitutes a group in the sense that it will be actually a subgroup of your set ℝ. Because
your set U(ℝ) will be a subset of your bigger set ℝ. So, basically what we want to prove here
is that the collection of invertible elements with respect to the dot operation constitutes a
subgroup of your original ring ℝ and for this what we have to do is we have to recap the
characterization for a subgroup which we had seen in our earlier discussion.
So, recall we proved that if you are considering a group, parent group and dot operation and
if you take any subset H of that parent group then it will constitute a subgroup with respect to
the dot operation if the closure property is satisfied, if you just prove that the closure property
is satisfied that automatically ensures that all other properties are also satisfied. So, what we
have to prove is the following. To prove this theorem that the collection of all invertible
elements of an abstract ring constitutes a subgroup with respect to the dot operation, we just
have to prove that your closure property is satisfied in this collection U(ℝ). So, for proving
the closure property we have to show the following : you take any 2 elements x and y from
this set U(ℝ), the element x ∙ y is also a member of U(ℝ) that is what we have to prove this is
our goal.
942
And recall our definition of U(ℝ) is that an element is considered to be present in U(ℝ) if it is
invertible. So, basically we have to show that x ∙y is also invertible. So, the first thing to
observe is that since I am considering that element x as well as element y are invertible let me
denote the multiplicative inverse by x inverse and y inverse : x-1 and y-1 respectively.
Now, my claim here is that the elements x-1 and y-1 they also individually belong to this set
U(ℝ) that means, I can say that element x-1 itself is invertible and I can say that element y-1
itself is invertible this is because, the definition of inverse says that if x multiplied with x -1
gives you 1 I can interpret it as if that x inverse. So, here this x -1 is considered as the inverse
of x.
I can also consider that x is the inverse of x-1. This is because if b is the inverse of a then I can
consider a as the inverse of b and vice versa. So, that is why I can say that since x is
invertible and I have the guarantee of presence of x-1; similarly if y is invertible I have the
guarantee that y-1 is present, I can say that both these elements are themselves individually
invertible.
And hence they also belong to the set U(ℝ). Now, what we have to show remember our goal
is to show that x ∙y is also invertible because we have to show that it belongs to the set U(ℝ)
that means, I have to show that there exists some element x ∙y-1 which when multiplied with x
∙y will give you the identity element and my claim is that the inverse of x ∙y is nothing but y-1
∙x-1.
This is because if you multiply y-1 ∙x-1 with x ∙y and then if you use the associative law and
then rearrange the terms you get the identity element. So, that shows that this element is the
inverse of this element and the second thing that we have to prove is that this element y-1 ∙x-1
is also an element of U(ℝ) and that simply comes from the fact that we have proved that it is
the inverse of x ∙y.
So, since we have so what we are basically saying here is that since I have shown that; call
this element (x ∙y) as A and call this element (y-1 ∙x-1) as B. So, what we have shown here is
that A is the inverse of B and B is the inverse of A that means, I can say that both A is an
element of U(ℝ) because it is invertible and I can say that B is also an element of U(ℝ)
because it is invertible and that is what we wanted to show here.
943
So now what we are going to do is we are going to extend our definition of ring to another
interesting algebraic structure which we called as a field. So a field is an algebraic structure it
is a set of values and there are 2 operations; 2 abstract operations plus and dot, which are
defined over the elements of this set F and we will say that all together this collection F
along with the operation + and ∙ is a field if the field axioms are satisfied.
So, what are the field axioms: the axiom number 1 is that the set F along with the plus
operation should constitute an abelian group. The second property that we demand here is the
following if I exclude the additive inverse and see I have written down here 0 in quote and
unquote - it is not integer 0 it is just a representation for denoting the identity element with
respect to the plus operation.
So, what I am saying here is that if I exclude the additive identity element, then all the
remaining elements of the set F, together with the dot operation satisfies the properties of an
abelian group and the third property the third field axiom that needs to be satisfied is that
your dot should be distributive over plus. So if you see; closely these axioms F1 F2 F3 and
compare it with your ring axioms R1 R2 R3.
You can easily identify that your field is a special type of a ring, where every non-zero
element is invertible with respect to the multiplication operation. So remember recall in
axiom number R2 is when we considered a ring axioms, we never demanded that with respect
to dot operation, all the elements should be invertible and so on, no such demand was there,
944
but now, I am putting a demand that, I am giving you the liberty, I am giving you the freedom
to exclude the 0 element 0 element in the sense the additive identity element; all other
remaining elements should have multiplicative inverse and if that is the case then I can say
that my ring is a field. So in other words, with respect to the set U that we have defined just
now, so remember U is a collection of all elements which are invertible with respect to your
dot operation. So I will say that my set F is a field if all the elements except the 0 elements
are invertible, namely, the set U(F) is entire set F excluding the elements 0. So now it is easy
to verify that if I consider a prime modulus then the set of integers, 0 to p - 1, which is
nothing but the set ℤp along with the operation addition modulo p and multiplication modulo
p satisfies the field axioms.
Because the set of all invertible elements in this collection is the entire collection excluding
the numerical 0 because here the numerical 0 is actually your additive identity. Now an
interesting property in the field is the following. If I am given a field with an abstract plus
operation and an abstract dot operation and if it is given that the result of x ∙ y is 0 again I
stress this is the abstract dot and this is the abstract 0.
Do not consider it as to be the usual multiplication and numerical 0. So, if x ∙ y is 0, then we

can safely conclude that it is either the case that your element x is 0 or the element y is 0 that
means it will never happen that you take 2 non-zero elements and if you perform the dot
operation you get a 0 element that would not happen in a field I stress this is true for a field
you can verify this may not be the case for a ring.
So, how do we prove this. So this is an if-then statement, your if condition is this and this is
if-then part. So, I will give up proof by contrapositive. So contrapositively; what will be the
contrapositive here since an OR is involved here and you have x = 0 OR y = 0, if I put
negation in the in front of this x = 0 OR y = 0, I get x ≠ 0 and OR gets converted into AND
because remember logically if you take negation of a OR b, then that is logically equivalent
to negation of a AND negation of b. So contrapositively I want to show here that if x ≠ 0 and
if y ≠ 0, then x ∙ y ≠ 0. That is what I want to show here. So, let us prove that so since x ≠ 0
and as per the field axioms, every non-zero element has a multiplicative inverse. So let me
denote that multiplicative inverse by x-1. Similarly, y is non-zero. So, it will have a
multiplicative inverse let me denote it by y-1. Now, as we had proved earlier, that, if you
945
multiply x ∙ y with y-1 ∙ x-1 you will get back the identity element. That means, I can consider
the product of y-1 and x-1 to be the inverse of the element x ∙ y.
That means, I can say that the element x ∙ y is invertible and if the element x ∙ y is invertible,
then from the field axiom F2 we can conclude that element x ∙ y was not a 0 element and that
shows that whatever we claimed here is correct.
Now, the next thing that we want to discuss here is the polynomials over rings, which is a
very important concept used in computer science. So, imagine you are given a ring with some
abstract plus an abstract dot operation and imagine that your dot operation is commutative.
Now, if I want to define a polynomial of degree n over this ring ℝ it will be of this form
𝑎 𝑥 = 𝑎𝑛 ∙ 𝑥 𝑛 + 𝑎𝑛−1 ∙ 𝑥 𝑛 −1 + ⋯ + 𝑎0 and this is very much similar to the notion of
polynomials that we are familiar with.
We are very much familiar with polynomial over integers or real numbers and so on; if I
consider a polynomial of the form 2x2 + 3x + 1 then I say that this is a polynomial of degree 2
because the highest power is x2 with coefficient 2 we are more or less trying to define the
same concept where in the case of usual polynomials the plus operation is the integer plus
operation and 2 times x2 should be treated as 2 multiplied with x2.
So, we are just extending those plus and dot operation with respect to an abstract ring, that is
what is the generalisation here. So, we are basically generalising the existing definition of
polynomials that we are aware of with over integers to any abstract ring. So, here we have
946
saying that the polynomial will have the form an ∙ xn, an - 1 ∙ xn-1 and so on and everything
added together.
Since my degree is n my restriction will be that the coefficient an will be non-zero all other
coefficients are allowed to be 0 and each of these coefficients are from your set ℝ and your
plus operation and your dot operation are the ring operations; they are not integer plus or
integer dot operations. So, now, if I consider the set of all polynomials of various degrees that
are possible over this ring ℝ, I denote that infinite set by this notation ℝ [x]. So, now, my x is
no longer in regular parenthesis they are now within square parenthesis. So, this is just a
notation that you are taking polynomials of degree n = 0 followed by all polynomials of
degree n = 1 followed by all polynomials of degree n = 2 and so on. So, it is easy to see that
this is an infinite set because I am taking all possible polynomials of various degrees which is
a never ending process.
Now, let us see how we define the operations of polynomials over ring. So, we know how to
add 2 integer polynomials. So, if I am given 2 polynomials, say a polynomial 2x2 + 3x + 1
and say another polynomial 5x + 2 say this is my a(x) polynomial this is my b(x) polynomial
the usual way to add these 2 polynomials is the following since there is no term is involving
x2 in b(x), I can imagine that it has some 0 times x2 present implicitly, so, that I can now
safely assume that both a(x) and b(x) are of the same degrees and then I can say that you
position wise add the various coefficients, so, 2 + 0 will give you 2, 2 x 2 then the coefficient
of x will be 5 + 3 and the constant coefficient will be 2 + 1 = 3, that is the usual way of
performing addition of integer polynomials, we are now extending the same definition to
polynomials over abstract rings.
So, imagine you are given 2 abstract polynomials arbitrary polynomials over a ring and for
simplicity and without loss of generality assume that the degree n is greater than equal to m.
Now, if n is not equal to m, then as I have done for the case of integer polynomials, I can
substitute higher order coefficients in the b polynomial with 0 and then I can safely assume
that n = m then the way I perform the addition of these two ring polynomials is that, I
component wise take the coefficients of various powers of x from the a polynomial b
polynomial and I add them where my addition will be now, the addition operation over the
ring and of course, these dot operations are the dot operation over the ring in the same way, I
extend my notion of multiplication of 2 integer polynomials to multiplication of 2 ring
947
polynomials. So, again what I do here is if I take the product of 2 polynomials of degree n
and m respectively, the polynomial will be now of degree n + m and these will be the various
coefficients and again and again I stress that all this plus and dot operations are the dot and
plus operations over my ring R, they are not the usual or traditional plus and dot operations.
So, if I denote s(x) to be some polynomial then let us denote the coefficients of this sum
polynomial to be sn, sn - 1 up to s0 and it turns out that the coefficient of xi which is denoted by
si will be nothing but summation of the coefficient of xi in the a polynomial and coefficient of
xi in the b polynomial where the plus operation is over the ring.
And in the same way, if I denote p(x) as the product polynomial and if I denote the
coefficients of the product polynomial as pn + m, pn + m - 1 up to p0 and it turns out that the
coefficient of xi will be this expression (𝑎0 ∙ 𝑏𝑖 + 𝑎1 ∙ 𝑏𝑖−1 + ⋯ + 𝑎𝑖 ∙ 𝑏0 ) where all the dot
and plus operations are over the ring. So, let me demonstrate for you, whatever we have
discussed with some example here.
So, imagine I consider my ring to be ℤ3 and my plus operation is addition modulo 3 and my
multiplication operation is multiplication modulo 3 and ℤ3 means the elements 0, 1 and 2.
Now, I am taking 2 arbitrary polynomials. One polynomial is of degree 2 (2x2 + 2x + 1) and
another polynomial of degree 1 (x + 2) belonging to this set ℤ3[x]. Remember ℤ3[x] denotes
the set of all possible polynomials of various degrees, where the coefficients are from the set
0, 1, 2 and where my plus and dot operations are addition modulo 3 and multiplication
modulo 3. So, now let us see the result ofsummation of these 2 polynomials. So if I sum these
948
2 polynomials so you can imagine that in b(x) there is no term with x2, so, you can implicitly
assume that you have 0 times x2 present. So, the coefficient of x2 and the sum polynomial
will be 2 + 0 modulo 3.
Because the; plus operation is plus modulo 3. So, 2 + 0 modulo 3 will give you 2. Now, the
coefficient of x will be 2 + 1 but now this is not the usual plus this is now the plus modulo 3.
So, that is why the coefficient of x will now vanish and in the same way, the constant
coefficient will be 2 + 1 but now this is 2 + 1 modulo 3 which will become 0. So, you can
now see that the summation of these 2 polynomials will be 2x2.
Whereas if I multiply these 2 polynomials then the coefficient of x3 will be this 2, why x3?
So, this polynomial is a(x) is of degree 2 and b(x) is a polynomial of degree 1, so, the
resultant product polynomial will have degree 3 at most. So, the coefficients of various
powers will be this; now 2 times 1 and multiplication here is multiplication modulo 3, so 2
times 1 modulo 3 will be 2.
But now if you see this term, this is 2 into 2 = 4 and 4 modulo 3 will be 1 plus 2 into 1 will be
2 and 2 + 1 modulo 3 will become 0 so that is why the coefficient of x2 will become 0. In the
same way this 2 into 2 will be treated as 1, 1 into 1 will be treated as 1 and 1 + 1 will be 2 and
so on. So now we can prove here that if my dot operation over the ring is commutative, then
if I consider the set of all polynomials of various degrees where the coefficients of the
polynomial are from this ring ℝ then even with the dot operation, namely, the product of the
polynomials, if I consider this operation, then it will satisfy the; and of course, the summation
of 2 polynomials the way I have defined here. So, basically what I am saying here is that if I
consider the set of all possible polynomials over the ring. Then along with the addition of the
polynomials and multiplication of the polynomials the overall algebraic structure will satisfy
your ring axioms provided the dot operation in your ring ℝ is commutative.
And this can be proved very easily, you can at least see that we have already seen an example
here, but it is not very difficult to prove it. We can of course prove the closure property,
associative property with respect to the polynomial addition. we can prove the closure
property, associative property with respect to the multiplication of the polynomials the
constant polynomial 0, which belongs to ℝ[x] will be treated as an additive identity because
you add the 0 polynomial to any polynomial you again get back the same polynomial;
949
whereas the polynomial 1 the constant polynomial 1; will be treated as the multiplicative
identity. And if you have a polynomial a(x) say with coefficients an, an - 1 up to a0 then it is
easy to see that, I can find out a corresponding –a(x) polynomial and –a(x) polynomial will
be nothing but will have coefficients –an and –an - 1 and up to –a0 where the minus elements
are the additive inverse elements with respect to my ring then if I perform the addition of
these 2 polynomials, I will get a constant polynomial namely the 0 polynomial. Similarly, I
can show that my plus operation is distributive over the dot operation.
Namely the addition of the polynomial is distributive over the multiplication of the
polynomials and so on, I am leaving the proof of this theorem as an exercise for you, but it is
very easy to prove.
So, that brings me to the end of today’s lecture these are the references used. Just to conclude,
in today’s lecture we saw the definition of fields. rings and we also discussed about
polynomials over rings. Thank you.
950
Lecture - 67
Polynomials Over Fields and Properties
Hello everyone, welcome to this lecture so, in this lecture we will continue our discussion on
polynomials over rings. And we will see in this lecture, polynomials over fields and we will
also discuss about how to divide polynomials over fields. And we will also discuss about
factorization of polynomials using which we will define the notion of reducible and
irreducible polynomials.
So, let us start with the usual division of polynomials that we are familiar with. So, if I
consider 2 arbitrary polynomials where the coefficients are integers and we are my plus and
951
dot operations are the usual integer addition and integer multiplication then this is the way we
perform the division. So, you will be given a divisor and you will be given a number which
you want to divide.
And then what we do is basically say in this example my power is currently x4 my divisor has
x2. So that is why I try to multiply my divisor with something so that I can get something of
the form x4 so that x4 and x4 cancels out and whatever is left that will be treated as my next
value or the next thing which I want to divide. So, in each step basically we are slowly and
slowly and slowly trying to reduce the power of the number which I want to divide.
And I keep on doing this till I cannot do anything further in the sense that the remainder that I
obtain is a polynomial whose degree is less than the degree of my divisor. If I reach that stage
then I cannot proceed further. And then I can say that safely that the number which I wanted
to divide; that my original number is basically the product of divisor and quotient plus some
remainder. That is the usual way of performing division of polynomials over the integers.
Now the interesting question here is the following in the last lecture we have defined or we
have extended the notion of polynomials to rings and we have seen how to do addition of
polynomials over rings which are more or less similar to the way we perform addition and
multiplication of polynomials over the integers. What about the division operation? Can we
do something similar for polynomials which are defined over rings or fields.
952
So, for that again let us recall the definition of polynomials over rings and further explore
some more properties regarding the polynomials over rings. So, imagine you are given 2
arbitrary polynomials over an abstract ring ℝ. That means all your coefficients an, an - 1, a0,
bm, bm - 1, b0 are elements of this set ℝ. And all this plus and dot operations are your ring plus
and dot operations.
Now as we have demonstrated in our last lecture itself it might be possible that the
summation of a(x) and b(x) has a degree which is less than n even if your n >= m it might be
possible that when you take the summation of these 2 polynomials the resultant degree is less
than n, that is quite possible. So, again to demonstrate my point let us take these 2
polynomials a(x) and b(x) over, where the coefficients are elements of the set 0, 1, 2.
And this plus operation is plus modulo 3 and this dot operation is multiplication modulo 3.
And if these are my a(x) (2x2 + 2x + 1) and b(x) (x2 + x + 2) polynomial then the coefficient
of x2 will be 2 + 1 and 2 + 1 will be 3 and 3 modulo 3 will be 0; in the same way the
coefficient of x will be 0 the constant coefficient will be 0. So, even though a(x) and b(x)
none of them is numerical is 0, the 0 polynomial, in this case the summation of these 2
polynomials actually turns out to be a 0 polynomial. In the same way if I multiply 2
polynomials over rings then again it is not necessary that its degree will be exactly n + m
which is the case if I multiply 2 integer polynomials but when I multiply 2 ring polynomials
that may not be the case because the coefficient of xn + m when I do multiplication of ring
polynomials will be this value anbm, and now since my a(n) is an element of the ring and my
b(m) is also an element of the ring it might be possible that neither a(n) nor b(m) are 0
elements but still their product is a 0 element, this is quite possible if you are taking the
coefficients over the ring remember this is not possible over a field. In a field if the product
of 2 elements is a 0 element and definitely 1 of them has to be 0 but that may not be the case
in a ring again let me demonstrate this.
So, my ring ℝ here is set ℤ6 namely the elements 0 to 5 and of course my plus operation is
plus modulo 6 and my dot operation is multiplication modulo 6 then what can I say about the
product of these 2 polynomials (2x2 + x + 4) (3x + 1). So, if I consider the coefficient of x3
the coefficient of x3 will be 6x3 but what will be 6? So the coefficient of x3 will be basically 2
into 3 where this into is multiplication modulo 6 and 2 multiplied with 3 modular 6 is
basically 0.
953
So, the coefficient of x3 vanishes and now you can see that the degree of the product
polynomial here is less than the summation of the degrees of a(x) and b(x) polynomial.
Now, what we can say about the addition of polynomials and multiplication of polynomials
over fields? So, we can extend the definition of polynomial addition and multiplication that
we have given for rings to fields as well because remember fields after all is a special type of
ring. Now in a field we can definitely say that the degree of the product polynomial will be
exactly equal to the sum of degree of the individual polynomials.
Which was not the case if I perform the multiplication of polynomials over rings we have
already demonstrated that; this is because now the coefficient of xn + m
in the product
polynomial will be the product of an and bm. And remember in the last lecture we proved that
the product of an and bm can be 0 only if an was 0 or bm was 0 that means you are at the first
place your polynomial a(x) was not of degree n or your polynomial b(x) was not of degree m.
It can never happen that even though your an and bm are both non 0 but still when you
multiply an and bm over a field you get a 0 element that is not possible that is why the
coefficient an times bm will survive. And that is why the x term with xn + m will be present in
your product polynomial. Now based on this observation we can give a theorem regarding the
division of polynomials over a field.
954
And this is kind of generalization of your usual division property. What do we mean by the
usual division property? We know that in the integer world if you are given 2 arbitrary
integers a and b then I know; that of course where b is not 0. And I know that I can always
express a in the form of some quotient times b plus some remainder r where the remainder r
will be in the range 0 to b - 1. That is the usual divisibility theorem with respect to the
integers.
We are now trying to extend that property that theorem in the context of polynomials over
fields so you can interpret that now a is replaced by a polynomial. It is no longer just a single
value but rather it is a polynomial where the quotients of the polynomial are from some field.
In the same way your element b; the number b; is now generalized to a polynomial of some
degree. And similar to the case of divisibility property that we have in the integer world
where b is not allowed to be 0 because division by 0 is not well defined.
So that is why we are not allowing b(x) to be 0 here then this theorem basically says that you
can express your a(x) as some quotient times divisor plus some remainder namely some
quotient polynomial times your divisor polynomial plus some remainder polynomial where
the degree of your remainder polynomial will be strictly less than the degree of your divisor
polynomial.
So, this is your divisor polynomial, q(x) is the quotient polynomial and r(x) is your remainder
polynomial. Again we can prove this very easily but due to interest of time I am not going to
prove that and the interesting part of the theorem here is that the quotient and the remainders
will be unique here similar to the case of integer division. If you divide a by b you get a
unique quotient and a unique remainder where the remainder is in the range 0 to b - 1.
Now if my remainder turns out to be 0 then I will say that a(x) is completely divisible by b(x)
or in some sense b(x) is a divisor or factor of your a(x) so in other words consider a
polynomial g(x) I will say that g(x) is a divisor of f(x) if f(x) is completely divisible by g(x)
that means you get no remainder that means if you get a remainder but that remainder is
actually a 0 polynomial.
In other words there exists some polynomial h(x) over the field which when multiplied with
your g polynomial will give you the polynomial f(x) if that is the case then I will say g(x) is a
955
factor of your f(x) polynomial, of course if g(x) is a factor of f(x) polynomial then so is h(x)
polynomial.
So, once we have given the definition of division of polynomials over field the next thing that
we want to define is the GCD of polynomials over field. So, again this will be a
generalization of the GCD of 2 numbers that we had discussed earlier in our module on
number theory. So, d is said to be the GCD of a and b it if it is the greatest common divisor of
both a and b; in the same way imagine you are given 2 arbitrary polynomials a(x) and b(x)
over the field then another polynomial d(x) over the field will be considered as the GCD of
these 2 polynomials and the following 2 properties are satisfied. Of course the d(x)
polynomial has to divide both the a(x) polynomial as well as the b(x) polynomial because
after all it is a common divisor and it is greatest in the sense that you take any divisor of a(x)
and b(x) it is also a divisor of d(x) in that sense you can imagine that d(x) is actually kind of a
maximal possible common divisor of both a(x) and b(x).
So, pictorially you can imagine that you are given a(x) polynomial b(x) polynomial and there
can be multiple common divisors of both these 2 polynomials call them d1(x), d2(x) and so
on. Among all those common divisors you can interpret that there is another divisor d(x)
which is kind of sitting on top of the hierarchy in the sense that all these divisors d1(x), d 2(x),
d3(x), dn(x) they also divide d(x) in that sense d(x) is sitting at the top of the hierarchy among
all the common divisors of a(x) and b(x).
956
And that sense it is the maximal possible common divisor of a(x) and b(x). The reason we are
defining GCD in this sense is because we cannot define what we call as maximum when we
are considering polynomials over the fields. So, again in the case of integer GCDs where we
are given 2 integer values a and b, d was the GCD in the sense it has the maximum possible
common divisor there is no other common divisor whose value is more than d.
There the notion of more is very well defined but when it comes to polynomials over field I
cannot define that d(x) is the maximum possible common divisor of a(x) and b(x). I cannot
define a notion like maximum common divisor polynomial; this is because it turns out that if
I consider polynomials over fields then the GCD of 2 polynomials need not be unique at the
first place. And that is why you can have multiple possible GCDs of 2 arbitrary polynomials.
This is because if I consider polynomials over fields and if you are having a situation where
you have 2 divisor polynomial say d1(x) is a common divisor of both a(x), b(x) and so is the
d2(x) polynomial that is also a common divisor of both a(x) and b(x) and say the divisor
polynomial d1 divides d2 and say d2 is a divisor of d1 then I cannot conclude that the
polynomials d1(x) and d2(x) are same.
And this is not the case in the usual integer world. In the usual integer world I know that if
there is a number d1 which divides d2 and if it is the case that d2 divides d1 as well then I can
conclude that d1 and d2 are the same numbers; but when I consider polynomials over fields
and if I have 2 polynomials where the first polynomial divides the second polynomial and the
second polynomial divides the first polynomial I cannot necessarily conclude that both
polynomials are identical.
So, for instance if I consider the field ℤ7 namely my elements are 0 to 6 and my operations
are addition modulo 7 and multiplication modulo 7 and suppose I take these 2 polynomials
(3x + 5) and (6x + 3). It turns out that these 2 polynomials divide each other because if you
divide 6x + 3 by 3x + 5 you will get the quotient polynomial 2 which is the constant
polynomial and 0 remainder whereas if you divide 3x + 5 by 6x + 3 you will get the quotient
polynomial 4 and 0 remainder, this is your d1(x) and this is your d2(x).
But you can see here that even though d1 divides d2, d2 divides d1 they are different
polynomials and that means if say for instance both 3x + 5 as well as 6x + 3 are common
957
divisors of a(x) and b(x) then it can be very much possible that 3x + 5 as well as 6x + 3 are
both GCD of a(x) and b(x), that is possible. That means I cannot say that when I consider
polynomials over fields that I will have unique GCDs you can have multiple possible GCDs.
And this is unlike GCDs over integers so that is why I cannot define what I call as a
maximum possible polynomial which is a common divisor polynomial of both a(x) and b(x).
That is why I define what I call as maximal. You take any divisor of a(x) and b(x) it will be a
divisor of your d(x). Now once we have defined GCD of polynomials of over field and next
question is how do we find out?
And it turns out that the beautiful Euclid’s GCD algorithm that we had discussed in our
module on number theory it can be extended to even find GCD of 2 polynomials namely the
GCD algorithm based on repeated division. The same algorithm if I extend to the case of
polynomial, if I just extend it will work and it will give you GCD of 2 polynomials defined
over the fields.
In fact we can get what we call as the extended Euclid GCD algorithm for polynomials over
fields and find out what we call as Bezout coefficients which we had seen in our module on
number theory. So, remember if d is the GCD of 2 integer values a and b then the Bezout’s
theorem says that I can always find out linear combiners s and t such that integer linear
combiners such that s times a + t times b is actually your GCD d.
958
That means GCD is always; I can always express GCD as the linear combination of my 2
numbers a and b itself and we know how to find out this Bezout’s coefficients using the
extended Euclid algorithm. The same theorem holds even for the case of polynomials over
fields. So, basically what the theorem says here the following: if you are taking 2
polynomials over the field and say d(x) is one of the GCDs, again remember there can be
multiple GCDs possible.
So, if d(x) is one of the GCDs then I can always find out the corresponding Bezout’s
polynomials. So, you can find out this combiners λ(x) and μ(x) which will be now
polynomials because now everything is extended to polynomials such that your GCD can be
expressed as a combination of your original polynomial say a(x) and b(x) where the
combiners will be your λ(x) polynomial and μ(x) polynomials.
So, I would not be going into the exact details of how to find out this Bezout’s coefficients
and how to find out the GCD’s but I will work out an example here. So, say for instance I
want to find out the GCD of this a(x) (x3 + 2x2 + x + 1) and b(x) (x2 + 5) polynomials where
all the operations are performed over ℤ7. So, ℤ7 has all the elements from 0 to 6 and my plus
operation will be addition modulo 7, my multiplication operation will be multiplication
modulo 7.
So, what we do in the GCD algorithm: we take the higher number and divide it by the smaller
number but the numbers now here are the polynomials. So, we will take the polynomial with
the higher degree and divide it by the polynomial with the lower degree. So, this is my a(x)
this is my current a and this is my current b. And now you can see the way I am performing
the division here, it is something similar to the way we perform division over the integers.
But instead of all the operations being performed over integers, the operations will be
performed over ℤ7. So, for instance my current power here is x3 and my divisor has the
highest power x2 so that is why I am putting an x here if I multiply x with x2 + 5, I will get x3.
The coefficient here will be 1, 1 modulo 7 will be 1 and 5 into x. So, coefficient here will be
5, 5 modulo 7 will be 5.
Now I have to take the difference here. I have to subtract so x3 and x3 cancels out 2x2 is
taken as it is here. And now what will happen if I subtract 5x from x so x - 5x will be -4x.
959
But there is no -4. -4 in this field ℤ7 will be treated as +3 because by -4 I mean 7 - 4.
Basically I am talking about because remember all the operations are plus mod 7, and all the
multiplication operation are multiplication modulo 7. So that is why -4x will be treated as
+3x so that is why will have +3x and this +1 will be retained as it is. Now I have to take care
of this 2x2 so that is why I take my next term in the quotient as 2 so if I now multiply 2 with
x2 + 5, 2 times x2 will be retained as it is whereas 2 into 5 will be 10 but 10 modulo 7 will be
3.
So that is why this is 3 this is not 10 and now I will be taking the difference here. So, 3 of x
retains goes as it is now 1 - 3 will be -2 and -2 will be treated as 7 - 2 namely 5. If you are
wondering how exactly we are getting this so -3 is basically +4. And 1 + 4 is 5, -3 in this field
ℤ7 is +4 and +4 and +1 modulo 7 will be 5 so that is why I get 3x + 5.
And now you can see that the degree of 3x + 5 is less than the degree of my divisors. So, this
will be treated as my remainder and I cannot proceed further. So, this will complete the first
iteration of your Euclid GCD algorithm.
So, this was my b(x) this was my a(x) and this is my r(x). In the next iteration my b(x)
becomes a(x). And my remainder becomes the next b(x) and now you can see that if I
perform the division. Here my highest term is x2 here it is 3x. So, if I multiply 3x with 5x I
will get 15x2 but this coefficient 15 has to be reduced modulo 7, 15 modulo 7 will be 1 so 1x2
which is what I want to cancel out x2.
960
So that is why I multiply with 5x and 5x with 5 will give you 25x. Now this 25 has to be
reduced to modulo 7 so it becomes 4 so 4x so that is why I get 4x. Now if I subtract 5 goes as
it is and I will get -4x. Now this coefficient -4 modulo 7 will be +3 so that is why -4x goes
and becomes +3x. And now 3x + 5 is completely divisible by your divisor. If I put 1 as the
next term in my quotient so, I get my remainder polynomial r(x) which is now a 0
polynomial. And that is why now I can say that my GCD is 3x + 5.
Now if I want to find out the corresponding Bezout polynomials for this GCD I can do the
following. So, I can say that if I take my original a(x) in terms of divisors, quotients and
remainder; I can write my a(x) in this form. And my goal is to express my GCD in terms of
a(x) and b(x) polynomial. So, I can write down my GCD here and the remaining things I take
to the left hand side I get this expression.
And now what I can do here is the following: I can say that this is my a(x) (x3 + 2x2 + x + 1)
and this is my b(x) (x2 + 5). So, I can treat my λ(x) to be the constant 1. And I can treat my
μ(x) to this value, so I can say that my λ(x) is the polynomial 1 and my μ(x) is the
polynomial –(x + 2) but wait –(x + 2) is -x -2 but -x means the coefficient is -1 times x and -
2. So -1 is not there in ℤ7 I have to reduce it modulo 7.
So, -1 becomes 6 so it becomes 6x and in the same way -2 goes and becomes +5. So that is
why my final Bezout polynomials will be this. That is the way I can find GCD and the
corresponding Bezout combiners.
961
So, now the next thing that once we have seen polynomial division over the fields and GCD
of polynomials we can define what we call as factorization. So, a trivial factorization that is
possible for any polynomial is the following form. So, you are given the polynomial f(x), a
trivial factorization will be the following: you take any constant from the field and you take
that constant α multiplied with the multiplicative inverse of α of course α is not 0 here.
Otherwise the inverse is not well defined it does not exist. So, if I take any non 0 α from the
field and multiply α and α-1 with that polynomial f(x) I will get back the original polynomial
f(x) itself in that sense I can always say that there is a trivial factorization of f(x) namely α
and α-1 are trivial factors for any f(x). So, now what we want to define is what we call as
irreducible polynomial.
Namely polynomials which cannot be factored into products of lower degree polynomials.
That is a rough idea of what we call as irreducible polynomial. So, let us now formally
define, so intuitively it is a non constant polynomial which cannot be factored into product of
two non constant polynomials. And why we are taking the case that it cannot be factored into
product of two non constant polynomials.
Because of this trivial factorization, because if you give me any polynomial f(x) I can always
factorize it, I can always say that α and α-1 are trivial factors of f(x) but when I am defining
irreducible polynomials I am not interested in the trivial factorization so that is why to
exclude the trivial factorization I am explicitly putting a condition that the polynomial f(x)
should not be factored into the product of non constant polynomials.
962
So, more formally I take a polynomial over a field I will call it irreducible if the following
holds. First that polynomial has to be a non constant polynomial that means it should have
some term of the form xi. It is not a constant polynomial that means it is not of the form say
some γ where γ my f(x) is not of the form f(x) equal to some γ where γ is an element of the
field that is not the form of f(x).
I am not interested in such polynomials when I am defining irreducible polynomials. And the
second condition is that I can always decompose f(x) as per trivial factorization. But when I
am defining irreducible polynomials my requirement is that I should not be able to write f(x)
into the product of 2 other polynomials g(x) and h(x) where either g(x) or h(x) is a constant
polynomial because I can always write f(x) in the form of f(x) α into α-1.
And α into α-1 can be treated as an element 1. So, I can always say that f(x) = f(x) into 1. So,
namely 1 is always a trivial factor of f(x) that is not considered as a violation of this
definition. So, my irreducibility property demand here that my f(x) should not be factorized;
it should not be possible to factorize f(x) into non trivial factors. If at all it can be factorized
only when one of the factors is a constant polynomial that is allowed that would not be
considered as a violation of the property of an irreducible polynomial.
So, to give you some examples here if I consider my field to be ℤ3 namely the set 0, 1, 2
where all the operations are addition modulo 3 and multiplication modulo 3 and if I consider
polynomials over this field then this polynomial (x2 + x + 2) is irreducible, we can prove that.
We will see later how do we show whether a polynomial is irreducible or not.
But you can see, you have to believe me that this is not reducible in the sense I cannot
factorize it out into other than the trivial factorization; but this polynomial x4 + 1 is reducible
because I can write it in the form of this product of 2 polynomials (x2 + x + 2) (x2 + 2x + 2)
where none of these 2 polynomials is a constant polynomial. So that is why it is reducible,
reducible in the sense it is non trivially factorisable.
963
So, the next thing that we want to define is the factor theorem for polynomials over fields. So,
the factor theorem states the following. If you take any polynomial over the field then the
polynomial (x – α) will be considered as a factor of your polynomial f(x) if and only if the
polynomial f(x) when evaluated at x = α gives you the element 0, where 0 is the additive
identity.
And since this is an if and only if statement we have to give 2 proofs here we have to give
proof both in the forward direction as well as in the reverse direction. So, let us see the proof
in the direction where we assume that (x – α) is a factor of f(x), assuming that I have to show
that the polynomial f evaluated at x = α will give you 0. So, we will give a direct proof a very
simple proof here.
So, since (x – α) is a factor of f(x), that means I can express my f(x) as the product of two
polynomials with one of them being (x – α) the remaining thing I can write it as some g(x)
polynomial; that means what can I say about the value of the polynomial f evaluated at α that
will be same as the product of α - α with the polynomial g(α) but α - α will give you the
additive identity 0.
Because -α is the additive inverse of α and 0 multiplied with g(α) will give you 0 element,
that is a simple proof. Let us see the proof in the reverse direction. So imagine your
polynomial f is such that when evaluated at α gives you 0. If that is the case then I have to
show that (x – α) is a factor of f(x). So, I utilize my division theorem and as per the division
964
theorem I can say that my f(x) when divided by (x – α) will give me some quotient and some
remainder. This comes from your division theorem.
And my goal is to show that r(x) is 0 that is my goal. So, how do I show that r(x) is the 0
polynomial well I know that the degree of r(x) polynomial is strictly less than the degree of (x
– α) because (x – α) is treated as a divisor here or I know that r(x) = 0. If r(x) is definitely 0
my proof is done because then I show that f(x) is equal to the product of q(x) and (x – α) and
(x – α) is a factor of f(x).
But I cannot definitely claim that r(x) is always 0 I have to logically conclude that. So, again
if this is the form of f(x) I can say that f(α) will be this. And now I use my premise my
premise says that f(α) is 0. So, if f(α) is 0 I substitute this value and this term is anyhow 0. So,
I get that r polynomial evaluated at α is 0. And now I can logically argue that indeed if r
polynomial evaluated at α is 0 then I get a contradiction to this case.
So, remember my 2 possible cases for r(x) is that either its degree is less than the degree of (x
– α) or r(x) is 0. Now if r(x) polynomial has a degree less than the degree of (x – α)
polynomial then what can I say about r(x) polynomial. I can say that r(x) polynomial is a
constant polynomial because the degree of (x – α) is 1 and if degree of r(x) is less than 1 the
only possible degree which is less than 1 is 0.
That means I am actually considering the case when r(x) is a constant polynomial but if r(x)
is a constant polynomial then it does not matter whether I evaluate it at α, β I should get the
same value. But here I am getting that r polynomial evaluated at α is a 0 value. So these 2
things these 2 conditions goes against each other that means this case is not at all possible
that means you cannot have degree of r(x) strictly less than the degree of (x – α).
Because if that is the case then that goes against this conclusion that r polynomial evaluated
at α is 0. That means the only case that is left is that your r(x) polynomial is 0 polynomial
which shows that this is 0 polynomial and hence f(x) is completely divisible by (x – α)
showing that (x - α) is a factor of f(x).
965
Now based on this factor theorem what we define is the root of a polynomial. So, a value α
from the field will be considered as the root of this equation f(x) = 0 where f(x) is a
polynomial over the field provided the polynomial evaluated at α gives you the value 0 over
the field. So, again this is kind of a generalization of the notion of roots that we are familiar
with.
Now the next question is that how many roots I can have if my polynomial has degree n, so I
can prove a familiar result. So, we know that for the regular polynomials if we have degree n
then it can have at most n roots in the same way I can show that if my polynomial is over a
field then this equation f(x) = 0 can have at most n roots how do we prove this? So, imagine
that α1 to αm are the roots of your equation f(x) = 0.
And we want to show that m is less than equal to n upper bounded by n that is my goal. Now
as per the definition of a root since α1 α2 each of them is a root I know that the f polynomial
evaluated at α1, f polynomial evaluated at α2, f polynomial evaluated at αm all of them will
give you the value 0 over the field. And if they give the value 0 over the field that means I
can say that each of these polynomials as per the factor theorem are divisors of my
polynomial f(x).
That means I can say that I can write down my f(x) polynomial as the product of these m
individual degree 1 polynomials followed by some leftover polynomial g(x) where g(x) could
be some polynomial over the field. And an important thing to notice here is that each of these
966
polynomials (x - α1), (x – α2), (x – αm) they are irreducible polynomials because they are
already any how polynomials of degree 1.
And they are non constant polynomials and I also know that over a field if I multiply several
polynomials then the resultant polynomial will have a degree which is actually the sum of the
degrees of the individual factors. This is possible over a field but this is not true over a ring
we already proved that. So, now if I apply this fact over this statement.
I know that the degree of f(x) is n. And I know that this f(x) is definitely at least a product of
m factors plus there is something else as well g(x) is another additional factor. So, the degree
of f(x) is n that is given to me and since it is the product of at least m factors, I can say that
definitely n is greater than equal to m which is what I wanted to show because each of these
factors contribute 1 to the overall degree of f(x) which is n.
So, now the next thing is; next question is how exactly we find irreducible factors of a
polynomial how exactly we do the factorization. So, it turns out that we do have some simple
methods for finding irreducible factors for the special case where my polynomials are over
ℤp[x] and where my polynomials are of small degrees. And if you are wondering what
exactly is the whole purpose of finding out the irreducible factors basically the problem is
something similar to finding the prime factorization of an integer.
So, you are given an integer you want to factorize it in the same way you are given
polynomial and say if it reducible I want to find out its irreducible factors. So, there are some
967
well known methods for that we will discuss one of the methods for a special case when the
polynomial that we want to factor is a monic polynomial as well as the factors which we want
to find out they are also monic.
By the way what is a monic polynomial? A polynomial is called as a monic polynomial and
degree d if the coefficient of xd is 1. So, now let us see this method, by the way this method is
mechanical and it will work only when your polynomial is of small degrees. So, suppose this
polynomial (x4+1) is given to me and I already showed you the factors of this polynomial but
now let us try to find it out.
So, if at all this polynomial has a factor it will have either linear factors; it will have several
linear factors possible or it can have 2 possible quadratic factors or it can have 1 linear factor
and 1 cubic factor and so on these are the various possibilities because the degree of f(x) is 4.
So, now let us check for linear factors that means can I write f(x) as product of some (x – α)
and another polynomial where α is some element from ℤ3.
So, remember that if at all (x – α) is a factor of this f(x) then this f(x) polynomial evaluated at
α should give me the value 0. So, let us check whether α can be 0 or not or α can be 1 or not
or α can be 2 or not why only these 3 values? Because everything is allowed to be from ℤ3.
Now it turns out that neither f(0) nor f(1), nor f(2), is 0 by the way all the operations are
performed over ℤ3 namely all the addition and multiplications are modulo 3.
That means I can rule out the possibility of linear factors. Now since linear factors are not
possible the next thing that we I want to check is whether it can have 2 quadratic factors
possible. Now if at all it has quadratic factors it can have 2 quadratic factors. So, let me write
down it in the form of 2 quadratic factors and remember I am interested to find out the monic
factors so that is why one of the monic quadratic factors I am writing like this (x2 + Ax + B).
And another unknown monic quadratic factor I am writing like this (x2 + Cx + D); my goal is
to find out whether I can find out A, B, C, and D or not. Where A, B, C, D are allowed to be
from ℤ3 not ℤ3[x]. So, now if at all this f(x) polynomial can be expressed as the product of 2
quadratic factors these 4 conditions should be satisfied : (1) A + C = 0 (2) AD + BC = 0 (3) B
968
+ D + AC = 0 (4) BD = 1. Why so? Because you see what will be the coefficient of x3 as per
my right hand side? It will be (A + C) x3.
Because Ax into x2 will give me A times x3 and Cx multiplied with x2 will give me Cx3. So,
the coefficient of x3 will be A + C and I want that A + C should be 0 because in my left hand
side there is no coefficient of x3 in the same way I am basically just comparing the various
powers of x in the LHS and RHS and arriving at these 4 equations. Now I have to check
whether I can satisfy all these 4 equations simultaneously given that my A, B, C, D can take
values from the set ℤ3.
Now from the first equation I can conclude that A should be minus C. And from my fourth
equation I can conclude that B should be equal to D why B should be equal to D? Because
my B and D are allowed to be only from the set 0, 1, 2. So, B and D are both allowed to be
from the set 0, 1, 2. So, I cannot have different values of B and D from this set 0, 1, 2. But
when multiplied together and then taken modulo 3 gives me the value 1. So, the only
possibility that B, D gives me 1 is when both B and D are 1 or both B and D are 2. Now I can
simply rule down the possibility of B = D = 0 that is not allowed because if that is the case B
into D cannot be 1. But I cannot have B and D both equal to 1 because a both B and D are
equal to 1 then my third equation gives me that product of A and C should be 1.
And then as per the same logic A should be equal to C but that goes against the conclusion
that I get from the first equation A = - C. So, I can rule down B and D equal to 1 as well. So,
the only option left is check whether B and D can be 2 and if B and D and are 2 then as per
the equation 3 I get AC should be equal to 2. And if AC = 2, I can satisfy it by saying that A
= 1 and C = 2 and that does not violate this condition that A = - C.
969
That means I can now safely say that A = 1, B = 2, C = 2, D = 2 satisfies these condition and
hence I can factorize my x4 + 1 as product of 2 quadratic monic factors with that I conclude
today's lecture. Thank you.
970
Lecture - 68
Finite Fields and Properties I
Hello everyone, welcome to this lecture. The plan for this lecture is as follows. In this
lecture, we will discuss finite fields and their properties specifically we will discuss the
characteristic of a field.
So, let us do some warmup and see how exactly we construct finite fields. So we will see a
construction of a finite field with 9 elements. I denote set F9 which is a collection of these
polynomials. So, these are basically polynomials of degree 0 and degree 1 where the
971
coefficients are from ℤ3 and remember ℤ3 is the set { 0, 1, 2 }. So, you can see that, if I
consider the operation of polynomial addition over the set ℤ3[x], then it satisfies the closure
property namely, you take any 2 polynomials from this collection and add them you will get
again a polynomial in the same set F9 but it turns out that with respect to the operation of
polynomial multiplication, the closure properties not satisfied namely, suppose I take these 2
polynomials (x + 2) (2x + 1) and if I multiply them, then remember that when I multiply
polynomials over fields where my field is ℤ3, then the degree of the product polynomial will
be the sum of the degrees of the individual polynomials. So, I have degree 1 polynomial here
degree 1 polynomial here, so that is why the sum of the product polynomial will be 2 and this
polynomial is not a member of the set F9. So, that is why now, what I am going to do is I am
going to define a modified addition and multiplication operation, where I will be doing all the
addition and multiplication of the polynomial as I was doing earlier, but my resultant answer
will be computed modulo this polynomial (x2 + 1) and if you see closely here this is an
irreducible polynomial; irreducible monic polynomial actually. So, the modified operation
namely addition and multiplication is the following: I first do the usual addition and
multiplication over ℤ3[x].
And then I do a modulo x2 + 1 and that will be my resultant answer. So, for instance, if I
again perform the multiplication of these 2 polynomials over ℤ3[x] as I said, I will obtain this
polynomial (2x2 + 1), but now what I am going to do is I am going to divide this polynomial
by my x2 + 1 and focus on the remainder. So, if you see 2x2 + 1, I can express as 2 times x2 +
1 + 2.
So, 2 will be the remainder polynomial, namely the constant polynomial and hence as per the
modified multiplication operation the product of these 2 polynomials will be 2 which is now
a member of the set F9. So, it turns out that with respect to the modified addition and
multiplication operation namely addition and multiplication modulo this irreducible
polynomial the collection F9 satisfies my ring axioms.
972
Now, I would be interested to check whether this collection F9 indeed satisfies the axioms of
field as well, with respect to the addition and multiplication operation modulo this irreducible
polynomial. And for field axioms we need that each non-zero element should have a
multiplicative inverse. So I have to check whether every element of this set F9 except the
element 0 whether it has a multiplicative inverse.
And it turns out that indeed each non-zero element of this set F9 has a multiplicative inverse.
So, for instance the multiplicative inverse of 1 is 1 because if you multiply 1 with 1, you get 1
and then if you do a modulo x2 + 1 you will get 1 in the same way, inverse of 2 is 2 because 2
into 2 is 4 and 4 you will first reduce over ℤ3 you will get 1 and then if you reduce 1 modulo
x2 + 1 you will get 1.
If you do the product of x and 2x you will get 2x2. Now, the coefficient 2 when reduced
within ℤ3 will give you coefficient 2 itself and now if you reduce 2x2 modulo x2 + 1 so, you
will first multiply with 2; you will get this (2x2 + 2) and now, if you do a subtraction you will
get -2 but -2 over ℤ3 is actually plus 1 which is the identity element; multiplicative identity.
So, like that you can verify easily that under each element I have written down its
corresponding inverse element and each of the inverse element is actually a member of the
collection F9. So, that means each of the non-zero element here indeed has a multiplicative
inverse. So, now I want to show you some another interesting property of this field F9.
So, we have already proved now that this collection F9 satisfies the field axioms with respect
to the addition and multiplication operation modulo this irreducible polynomial, I have not
973
shown you the distributive property namely, addition distributes over multiplication. But it is
easy to verify that, but I would like to show you another interesting property of this field. So,
here I am going to focus on all the non-zero elements of this set F9.
And now, what I have computed here is the following I have computed various powers of this
element 2x + 1, of course, modulo the irreducible polynomial. So, the polynomial 2x + 1
power 1 will give you the same element (2x + 1)2 modulo x2 + 1 will give you what x so, if
you want to verify that let us do that. So, you have (2x + 1)2 so, you will first expand it so,
you will get 4x2 + 4x + 1.
But each of the coefficients has to be first reduced over ℤ3 so 4 becomes 1. So, you get x2, 4
becomes 1 again. So, you get x and then you get +1 and now, you have to reduce x2 + x + 1
modulo x2 + 1 that is the way we have defined our modified multiplication operation. So, you
will get x2 + 1 and now, if you subtract you get x; 1 and 1 cancels out.
Now, you cannot further divide x by x2 + 1 because the degree is less. So, x will be the
remainder. In the same way (2x + 1)3 will give you x + 1 and so, on. So, what I have shown
here is if you take the various powers of 2x + 1 and compute the powers as per the modified
multiplication operation, then you get basically all the non-zero elements of this collection
F9 .
That means, I can treat this element 2x + 1 as a generator which can generate all the non-zero
elements of this collection F9 and as per our notation of generators and cyclic group I can
basically say here that if I consider the field F9 and focus on the multiplication operation
modulo x2 + 1 then it is actually a cyclic group where 2x + 1 is a generator. I will touch upon
this fact later.
974
But this was just for your demonstration. So, now next we want to define what we call as
characteristic of a field. So, imagine you are given an abstract field. So, this is your abstract
plus operation and abstract dot operation; need not be your integer plus and integer dot
operation and my elements 0 and 1 are the additive and multiplicative identity respectively.
Again they are they need not be the numeric 0 and 1.
They are the representation of your additive and multiplicative identity element. Now, what I
am going to focus on is the following. I will see what are the various elements I can generate
as per the dot operation from this multiplicative identity element 1. I will be focusing on the
cyclic subgroup as per the addition operation. So basically, I am going to add 1, 0 times,
which will give me the element 0, so 0 times 1 will give me 0, 1 times 1 will give me 1 and 1
+ 1 which is same as 2 times 1 will give me 2, 1 + 1 + 1 3 times will give me 3 times 1 which
is same as 3 and so on. Again I am using; I am denoting 2, 3 as results but this may not be the
element numeric 2, numeric 3 they are basically representation of the result of adding the
multiplicative identity 1 to itself.
So, if I focus on the cyclic subgroup namely the various elements which I can generate by
adding the element 1 to itself several times then that will be a subgroup of my original group.
So remember, this collection F with respect to the plus operation constitutes a group, because
that is one of the axioms of the field and since, I am taking an element 1 belonging to the set
F and computing the various powers.
Then as per the rules of group theory, this will be considered as a subgroup and it will be a
cyclic group where 1 is the generator. So, the characteristic of the field is the smallest
975
positive integer m such that 1 the multiplicative identity, 1 is added m times I get the element
0. So, why I am focusing on the positive integer and why not 0 is allowed here? because as
per the definition here, 0 times 1 will of course, give you the element 0.
So, that is why I am interested in the smallest positive integer. So, it turns out that as per the
definition of our characteristic, if your field F is a finite field then of course the subgroup; the
cyclic subgroup generated by the element 1 also will be finite and in that case, what I can say
is that the characteristic of the field is nothing but the order of the cyclic group generated by
the element 1.
Because, whatever is the number of elements generated by this element 1 say if there are m
number of elements that means, starting from the 0th power to the (m – 1)th power, I can
generate all the elements and then as soon as I take the mth power, I will get back the identity
element namely 0. Whereas, if the field F itself is infinite and the characteristic of F may not
be well defined. So, typically we will be interested in the characteristic of a field when our
field is a finite field.
So, let us see some examples of characteristic of a field. So, let us first take this example,
namely the field consisting of the elements 0 to p - 1 that is my set ℤp and my plus operation
is addition modulo p and my multiplication operation is multiplication modulo p and here the
identity elements are indeed the numeric 0 and 1 respectively, the additive and multiplicative
identity elements.
976
So, now let us try to find out the characteristic of a field. So, for that we have to focus on the
size or the order of this subgroup namely the subgroup generated by the element 1 and if I
consider the subgroup generated by the element 1 it will be the entire ℤp because 0 times 1
will give you 0, 1 added to itself only once we are going to give you the element 1. 1 added to
itself again we will give you 2 and so on. So basically the characteristic here will be p
because if I add 1 to itself p times and remember by add I mean addition modulo p. So, if I
add 1 to itself p times the result will be p and p modulo p as per the plus modulo p operation
will give me the element 0. So that is why the characteristic of this field will be p. Now let us
consider the field that we had constructed at the beginning of this presentation, this lecture,
this was the field consisting of 9 polynomials of degree 0 and 1 over ℤ3 and all my operations
are modulo x2 + 1 then here the additive identity is the numeric 0 or the constant polynomial
0 and the multiplicative identity is the constant polynomial 1. Now if I want to find out
characteristic of the field F9 basically I have to find out the size of the cyclic subgroup
generated by the element 1.
And it is easy to see that subgroup generated by the element 1 will be the constant
polynomial 0, polynomial 1, and polynomial 2. So there are 3 elements that is why the
characteristic will be 3 indeed if you add 1 to itself thrice you will get 3. Now if you reduce 3
modulo 3 you will get 0 and now if you 0 modulo x2 + 1 you will get the element 0. That is
why the characteristic of this field is 3.
Let us consider an abstract field F4 where my elements are letters here w, y, z and t. And
now I define an addition and multiplication operation as per this table. So this table basically
tells you the result of performing the plus operation and multiplication operation. So for
instance if I consider this entry. This entry basically means that if I add y and z then my
result is t. In the same way as per the multiplication table, the interpretation here is that if I
multiply ℤ with the w my result is w and so on. That is the definition. That is my definition of
the plus operation and the dot operation here and it is easy to verify that all the field axioms
are satisfied: the closure property with respect to the plus is satisfied because you take any 2
elements of this collection F4 and add them you will get again an element of the collection
F4 .
977
Similarly dot or multiplication is closed. And you have the identity elements here. Let us
identify the additive identity element 0, what exactly is the element 0 here the element 0 here
is actually the element w, because you add w to any element you get the same element back
so you add w with w you get w you add w to y you get y and so on. So even though w is
additive identity whenever I want to refer to additive identity instead of saying w I will use
the notation 0.
In the same way the multiplicative identity here is y because if you see the column under y in
the multiplication table each element when multiplied with y gives you back the same
element. So now what will be the characteristic of this field so finding the characteristic of
this field; since the field is finite I will basically focus on the cyclic subgroup generated by
this multiplicative identity element namely the elements generated by various powers of y.
So 0th power of y will give me the element y itself and y added to itself as per the additive
table gives me the element w and that is all after that I cannot generate any new element. So
that is why since there are 2 elements here the characteristic of this field is 2 and indeed you
can check here that y additive to itself will give you the element w which is my 0 element.
So that is the definition of characteristic of a field. So we have seen examples of 3 fields in

this lecture, the field ℤp, the field F9 and the field F4 and the characteristic of each of these
fields is a prime number. Now, you might be wondering, is it accidental, or is it in general
always a case. So it turns out that this is not accidental and indeed, this is the case for every
978
finite field. So we can prove the following. That if you take any finite field F with an abstract
plus and dot operation.
Then it’s characteristic is always a prime number. It cannot be a composite number and at
least the theorem is true with respect to the examples that we had seen already in this lecture.
So now let us try to prove this theorem. So the proof will be by contradiction. So the theorem
says that the characteristic should prime number but as per the proof by contradiction
strategy, I will assume the contrary and I will assume that the characteristic is not a prime
number. If it is not a prime number; so it is not a prime value then it will be composite. So let
characteristic be m and suppose it is a composite value. Since it is a composite value it will
have prime factors or some factors. So let m1 and m2 be the factors here and none of them is
actually m because that is the definition of a composite number.
Now since the characteristic is m where m is the product of m1 and m2 that means in the field
F, the element 1 when added m1 m2 times will give you the additive identity is 0. That is that
comes from the definition. Now if this is the case then I am going to prove that either the
element 1 added m1 times will give you the element 0 or the element 1 added to itself m2
times will give you the element 0.
That is what I am going to show next. Assuming that this claim is true then this goes against
the assumption that the characteristic of the field was m1 times m2 because if 1 added to itself
m1 times gives you the element 0 then it implies that the characteristic is m1 or if 1 added to
itself m2 times gives you the element 0 then that means the characteristic is m2 and both m1 as
well as m2 are individually less than m.
So that goes against the assumption that the characteristic of the field was m1 times m2 at the
first place. So everything now boils down to proving this claim. That means assuming that 1
added to itself m1 m2 times gives you 0, I have to show that either this statement is true or this
statement is true. And again I will use a proof by contradiction to prove this claim. So my
goal is to show that 1 added to itself m1 times or 1 added to itself m2 times gives you 0 but on
contrary assume that 1 added to itself m1 times gives you a non-zero element a.
979
And 1 added to itself m2 times gives you another non-zero element say b. Now if that is the
case I have to arrive at a contradiction somehow and how do I arrive at a contradiction, so I
utilize the fact that 1 added to itself m1 m2 times can be splitted as follows: I can say that let
me add 1 to itself and m1 times and then again let me add 1 to itself and m1 times and then
again let me add 1 to itself m1 times.
And like that if I do this operation of adding 1 to itself m1 times total m2 times that will give
me the effect of as if I have added element 1 to itself m1 m2 times. Now as per my assumption
1 added to itself m1 times will give me a non-zero value a. In the same way the next operation
of element 1 added to itself m1 times will give me again element a and in the same way the
last operation of performing the addition operation over the element 1 m1 times will also give
me the element a.
So what I can say is that the result of adding 1 to itself m1 m2 times is equivalent to adding
this non-zero element a to itself m2 times. Now what I can say is the following since the
element a is a non-zero element I can say that as per the definition of multiplicative identity if
I multiply the element a with the multiplicative identity namely the element 1 I will get the
element a itself. So I can write a as dot of 1 and a and like that each of the a I can replace by
1 dot a and how many times I can do that: m2 times. Now remember I am considering right
now a field and over a field the plus and the dot operation satisfies the distributive property.
So what I can say is the following I can take out this dot outside and distribute; inside I can
collect all the plus.
And how many plus I have inside? m2 because this whole operation of 1 dot a was performed
m2 times. Now I utilize the fact that 1 added to itself m2 times will give me the non-zero
element b that means this value is nothing but b and overall I get the conclusion that 1 added
to itself m1 m2 times gives me the element b dot a and b dot a will not be 0 because as per my
assumption a is not equal to 0, b is not equal to 0 and recall in a field if you have 2 nonzero
elements then their dot is also not 0. So since b dot a is not 0, I get a contradiction that the
characteristic of the field is m1 m2 because if the characteristic of the field was m1 m2 then the
result of 1 added to itself m1 m2 times should give me the element 0. But what I have shown
here is that 1 added to itself m1 m2 times is not 0. So I get a contradiction and that shows that
my claim is correct and since my claim is correct, then that contradicts the assumption that I
made here namely the characteristic is a composite number. That is an incorrect statement
980
that means the characteristic m was actually a prime number. So with that I conclude today’s
lecture: just to summarize in today’s lecture we discussed about the characteristic of a field
and we proved that if your field is a finite field then its characteristic is always a prime
number.
981
Lecture - 69
Finite Fields and Properties II
Hello everyone, welcome to this lecture. The plan for this lecture is as follows. In this
lecture, we will continue our discussion on finite fields and we will discuss what we call as
order of a finite field and what are the properties of an order of a finite field.
So, basically order of a finite field is the number of elements in your set F if F is the field
and we can prove a very strong property; a very strong statement regarding the order of a
finite field. So, the statement here is the following: imagine your field F is a finite field and
982
suppose its characteristic is p. Now, as per the discussion that we had in the last lecture, we
know already that this number p is a prime number.
What we can prove actually is that the number of elements in this field is of the form pr
where r greater than and equal to 1. That means you take any finite field, the number of
elements in the field will be of the form pr where p is a prime number, the characteristic of
your field. Before going into the proof of this theorem, you can recall easily that the fields
that we had discussed in the last lecture.
We saw there a field consisting of 9 polynomials, its cardinality is 3 2, because its

characteristic was 3. In the same way, we saw the abstract field consisting of 4 symbols its
cardinality is 22 because the characteristic of that field was 2, if you take the field ℤp , its
cardinality is p1 because its characteristic is p and so on. So, what we are actually going to
prove is that it is not the case that it is happening accidentally only for F9 F4 ℤp.
But you take any prime any finite field with characteristic p, the number of elements in the
field will be of the form pr. So, before going into the proof of this theorem, we are going to
introduce some notations which we will be using in the proof. So, remember the additive
identity of the group is 0 and the multiplicative identity of the group is 1. I will use the
notation n with different font here (n) to denote the element, which I will obtain by adding
the multiplicative identity n number of times. So, typically in regular integer arithmetic where
this element 1 is the numeric 1 if I add 1 to itself n times we get n but, this element 1 need not
be the numeric 1 it is the multiplicative identity. If you add this element 1 to itself n number
of times and as for the closure property of the field you will obtain an element from the field,
that can be any abstract element. But just for the sake of simplicity, I will use this notation n
with a different font to represent a result of 1 added to itself and n number of times. In the
same way. 1 added to itself p number of times will be represented by this special font p and
since the characteristic of the field is p itself this element p with a special font is nothing but
the element 0.
That comes from the definition of the characteristic of a field. In the same way if I take any
abstract element, an arbitrary element f from the field and add f to itself n number of times, I
can obtain the same result by saying that each of these f s can be replaced by the product of 1
983
and f because 1 is the multiplicative identity and this I am doing n number of times and then I
can distribute the + over dot.
And 1 added to itself n number of times as per my notation is n, which is an element from
the field and that element multiplied with f will be the overall result. So, now let us go into
the proof of this theorem; the proof is very interesting here. So, you consider any element of
the field and any multiple n from the set of natural numbers, my claim is that the operation or
the result of adding f to itself n number of times, which will give me this element (n ∙f) is
also an element of F and that comes from your closure property. So, as per our notation; as
per our definition the element n added it to itself f will give me this element (n∙f) and as per
the closure property this element will be an element of the field itself. Now, what are the
relevant values of n; relevant in the sense which will give me nonzero elements.
So, the relevant values are 0 to p - 1, by relevant I mean only those multiples which will give
me distinct elements. So, 0 times f will of course, give me the element 0 as per the definition.
Now f added to itself, or 1 times f actually, to be more precise is just element f. Now, f + f
will give me some element from the field. So, I can call it as two times f; this is not the
numeric 2. But some 1 added to itself two number of times that is a representation here and
like that if I continue, then I can say that f added to itself (p – 1) number of times will give
me (p – 1) ∙f and after that if I add f to itself once more I will get the element 0 because p is
the characteristic of the field. In that sense the only relevant multiples of f are 0 to p - 1
because after that you take the higher order multiples of the element f you will start getting
the same elements which you could have generated by taking the multiples of f in the range 0
to p - 1.
Next let me define what I call as the span of the field. So, a collection of k elements, so, here
is your field F which is finite and which has some number of elements. So, if I focus on a
collection of values which are called as f1 f2 fi fk I will call the collection of these elements as
the span of the field if the following hold. You take any element x from the field that can be
expressed as a linear combination of the elements from your collection f1 to fk, where the
linear combiners are from set 0 to p – 1. Why I am focusing on the linear combiners which
are in the range 0 to p - 1 because as I said here, the relevant multiples of any element from
the field are where when you take the multiples to be in the range 0 to p - 1. So, basically
984
span means that it is actually the subset of those elements from the field in terms of which
you can express any element of the field, by taking various linear combinations. And when
you say linear combination by that I mean that I am doing the plus operation and the dot
operation as per the field. So it is easy to see that a trivial span of the field is the entire field
itself. You take any element x from the field that can be always represented as 1 times x + all
other elements from the field being multiplied with 0.
So 0 times the first element and so on. That is why the entire field is of course a span of
itself. Now let me next define what we call as the minimal spanning set of the field. So the
minimal spanning set of the field is the collection of elements from the field which is minimal
in the sense that you cannot remove any element from this collection.
If you remove any element from this collection then it is no longer the case that reduced
collection still spans the entire field. That means no proper subset of this collection spans the
entire field F in that sense it is minimal it is essential collection. And there could be multiple
minimal sets spanning your field, it is not the case that it is always unique. It may be possible
that a collection of first 3 elements from the field constitutes a minimal set spanning the field
or say the last 2 elements from the field they are the essential elements and so on.
Now why I am focusing on value r here because remember our goal is to show that the order
of the field is of the form pr that is why I am taking r here. So I am basically saying that a
collection of r elements from your field f will be considered as a minimal set spanning the
field if it is the bare minimal collection elements whose presence is required to express every
element from your field as a linear combination.
Now, what I am going to define is the following: I am going to now define a mapping g from
the ℤpr to the field F. Now, what is the ℤpr ? so as per the definition of Cartesian product, ℤpr
is nothing but the Cartesian product of ℤp which itself r times. That means if I consider an r
tuple present in ℤpr then by that I mean that I am talking about r elements where each of the
elements are from set ℤp.
Now how exactly this mapping g is defined? So if you want to map an r tuple as per the
mapping g then what basically you have to do is the following, you have to take a linear
985
combination of the elements in your minimal spanning set as per the linear combiners in your
r tuple. That is the way I have defined my mapping g. Right now I am not making any claim
about this mapping g whether it is injective, bijective, surjective.
It is just a function right now, I am just giving you the definition of the function that
definition is you give me any r tuple then I will match that r tuple to a finite field where the
mapping is obtained or where the image is obtained by taking a linear combination of the
elements in the minimal spanning set as per the linear combiners in my r tuple. Now I am
going to make certain claims about this function g.
I am going to prove that this function g is a bijection and if it is a bijection then as per the
rules of cardinality it shows that the cardinality of F is same as the cardinality of ℤpr and what
is the cardinality of ℤpr ? The cardinality of ℤpr is nothing but pr because as I said the
definition of ℤpr is you take the Cartesian product of ℤp r times. So there are pr possible
elements or pr number of r tuples present in the Cartesian product of ℤp r times.
And assuming that g is a bijection which I am going to show assuming that this statement is
true, it shows that the cardinality of F is same as the cardinality of ℤpr and hence it shows that
the number of elements in my field F is some pr so that is the proof strategy here. Now
everything boils down to proving that my mapping g is indeed a bijection and as per the
definition of a bijection I have to prove that the mapping g is a surjection and it is an
injection.
Well, proving that g is a surjection is trivial. That comes from the definition of your spanning
set. Since as per my definition, the collection of f1 to fr is a spanning set. That means you give
me any element x it will have a pre-image. Why? Because as per the definition of a spanning
set this element x can be expressed in terms of these r elements as per a linear combination.
Where the linear combiners will be from ℤp and how many such linear combiners I will need?
I will need r such linear combiners and if each of them is an element of ℤp basically the
collection of the corresponding linear combiners is going to be an r tuple from this ℤpr. So,
that trivially proved that is function g is a surjective function. Now, I want to prove that this
function g is also an injective function and that I will prove by contradiction.
986
So as per the contradiction assume that the mapping g is not injective. That means imagine
you have 2 different r tuples so you have an r tuple say n1, n2, nr. So, let me write down this
different r tuple here itself because I will need the space. So, imagine you have 2 different r
tuple n1 up to nr and another r tuple m1 up to mr and say both of them gets mapped to the
same element x as per the mapping g.
What does that mean? It means that you take the linear combination of the elements of your
spanning set as per the combiners n1 to nr and if you take the linear combination of the
elements of your spanning set as per the linear combiners m1 to mr you get the same element
same field element that is what it means when I say the mapping g is not injective.
If this is the case, I have to arrive at a contradiction. Basically, I will try to arrive at a
contradiction that the collection of r elements, which you assumed to be the minimal
spanning set is actually not a minimal spanning set that means there are some unnecessary
redundant elements which have been added unnecessarily in this collection which can be
simply removed.
I will arrive at that contradiction. How do I arrive at that contradiction? Well what I can; what
I know about this r tuples is that they are different. That does not mean that the entire set of
the r values in the first r tuple and all the r values in the second r tuple they are different;
there might be some of them which are same. So it might be the case that, say, the first r
elements in both the r tuples are same.
So you have say n1 n1 and n2 n2 occuring. But suppose i is the first index where in the first r
tuple you have the value ni and in the second r tuple you have the value mi where ni and mi
are different but the first i - 1 components in both r tuples suppose they are the same. So, I am
focusing on the first index i where the r tuple n and r tuple m they are different; that index i
could be any index in the range 1 to r.
And there definitely is one such index i because as per my assumption the entire n tuple and
the entire m tuple they are different. So, if they are completely different definitely there must
be some component, some index i where the component in the n tuple component and the
component in the m tuple, they are different. I do not know what exactly is that index, but
987
that index i definitely exists. So, I am focusing on that index i and assuming that the first i - 1
components they are same in both the n tuple and m tuple.
Then I can cancel them out both from the LHS and RHS because if I have n1 times f1
occurring in the LHS and imagine m1 is same as n1 then I can cancel out n1 f1 from both
sides. In the same way if n2 is same as m2 I can cancel out into n2 times f2 both from LHS
and RHS and so on. But then when I come to the ith term what I have done here is I take the
term mi times fi here to the LHS here.
And whatever is the remaining part of the expression in the LHS part I took it and bring it
into RHS. I have simply arranged the terms here. Now, if this is the case, if I get this
equation what I can say is the following: if I multiply both sides of the equation by the
multiplicative inverse of this element and the multiplicative inverse of the element ni – mi
exist because as per the definition and ni is not equal mi, that means ni – mi is not 0.
And if it is not 0 then as per the definition of a field, I do have a multiplicative inverse of this
element that means, I do have an element which I can denote by this notation (ni – mi)-1,
which when multiplied with the difference of ni and mi will give me the multiplicative
identity namely 1. So, if I multiply with the multiplicative inverse on both the sides, I
basically get the fact that fi can be expressed in terms of fi + 1, fi + 2 ... f r.
That means, it shows that I can remove fi safely from my supposedly minimal spanning set of
f, it is not necessary to keep fi in this collection, because I can express fi in terms of the
remaining elements in this spanning set which spans the finite field F and that goes against
my assumption that this collection of r elements f1 to fr was the minimal spanning set of the
finite field and why I came to this contradiction. Because I assumed that my function g is not
an injective mapping. So, that means, whatever I assumed about g is incorrect and that shows
that indeed my mapping g is an injective mapping and that shows that my function g is a
bijection that means the cardinality of ℤpr and the finite field are same and that proves this
theorem.
988
So, we have proved that you give me any finite field with characteristic p it will have pr
number of elements. Now, let us see how exactly we can construct finite fields for any given
pr where p is a prime number and this is very interesting because it says the following you
give me any prime number p, I will show the existence of a finite field whose characteristic
will be that prime number p.
And the number of elements in the field will be pr and how exactly we construct such a field.
So, for constructing such a field we will take the help of some irreducible monic polynomial
where the coefficients are over ℤp and the degree of the polynomial will be r. Why r? Because
r is also given as part of your input. So you are given a prime number p and value r, my goal
is to show the existence of a finite field with characteristic p and with pr number of elements.
So, to do that I am basically taking a monic irreducible polynomial with coefficients over ℤp
whose degree is r, if you are wondering whether indeed such polynomials always exist for
any given r and p, the answer is yes. Such polynomial always exists for every r and p and
there are some standard methods for doing that; getting such polynomials but for some well
known values of p and r such polynomials are publicly available.
Now, my goal is to construct a field F, so, my set F will be the set of all polynomials with
coefficients over ℤp modulo k(x). In other words, basically the set F is the collection of all
polynomials of degree 0, degree 1, degree 2, degree 3 and up to degree r - 1 where the
coefficients of the polynomial are from ℤp. So in general, I can say that F is the collection of
all polynomials of degree at most r - 1.
989
So this means degree is at most r - 1 where the coefficients are allowed from the set ℤp, why I
am saying it is at most r - 1, because since each of the coefficients are from the set ℤp and my
ℤp have the elements from 0 to p – 1 that means I can have a polynomial where all the
coefficients are 0 that means I can also have a polynomial which is the 0 polynomial. So it is
not necessarily the case that ar - 1, namely the coefficient of the r - 1 th power of x is always
supposed to be there, it can also be 0,
So, it turns out that how many elements I can have; how many such polynomials I can have
in my collection F. Since I can have each of the coefficients taken from the set ℤp. Namely
each of the coefficients can take p possible values and each of them are picked independently
that means it is not the case that the coefficient a1 depends on the coefficient is a0, it is not the
case that the coefficient a2 depends on the coefficient is a0 and a1 they are picked
independently.
So, I can say that from the product rule of counting there are pr number of possible
polynomials in my collection F. So I have defined my collection F. Now I have to give the
definition of the abstract plus operation and abstract dot operation. So, my plus operation here
is defined to be the addition of polynomials where the coefficients are added as per ℤp namely
addition modulo p and then I take the resultant polynomial modulo the irreducible
polynomial.
So that will ensure that my resultant polynomial will have coefficients over ℤp and its degree
will be at most r - 1, because the degree of k(x) is r. To begin with my a(x) and b(x)
polynomials both those polynomials will have degree r - 1 and if I add any 2 polynomials of
degree r - 1, at most I will still obtain a polynomial of degree at most r - 1.
So in fact, I do not need to take a modulo k(x), because in the sense, the effect of modulo k(x)
would not take place. And my multiplication operation is defined to be the product of 2
polynomials, the corresponding 2 polynomials, where the coefficients are multiplied with
respect to ℤp and if the degree becomes more than r, I take modulo k(x). That is my definition
of the abstract plus operation and abstract dot operation.
990
And my claim is that the way I have constructed my F and the way I have defined my plus
operation and dot operation they satisfy the properties or they satisfy the field axioms, it can
be verified easily. To check, specifically I want to show you that any non-zero polynomial
here will have a corresponding multiplicative inverse. Otherwise, remaining properties are
easy to verify: the closure, associative, distributive law, existence of identity elements and so
on.
The additive identity element will be the 0 polynomial, the multiplicative identity element
will be the constant polynomial 1 and so on. Let us see the existence of multiplicative
inverse. So, imagine you are given a non empty, non empty means non-zero, a non-zero
polynomial. I want to show it has a multiplicative inverse and the multiplicative inverse is
guaranteed because of the following: since I am taking k(x) to be an irreducible polynomial,
so till now you must have been wondering that why I am taking k(x) to be irreducible why
cannot I take k(x) to be any polynomial of degree r, there is a reason. If I take k(x) to be
irreducible then I know that the only monic GCD of a(x) and k(x) will be the constant
polynomial 1. Why so because since k(x) is irreducible, I cannot factorise out k(x). That
means I do not have non constant factors of k(x) and hence the only possible monic GCD, the
common divisor of a(x) and b(x) could be a constant polynomial 1. That means I can say that
I can now apply the Euclidean GCD theorem and as per the Euclidean GCD theorem, the
GCD can be expressed in terms of the individual polynomials itself.
So my individual polynomials are a(x) and k(x), then as per the Euclidean theorem I can find
out “linear combiners”, they are actually not linear combiners they are some polynomials
when multiplied with a(x) and k(x) respectively and added will give me the GCD where the
GCD in this case is 1 and what can I say about this multiplier polynomials f(x) and g(x)?
Each of them are actually polynomials over the fields ℤp of some degree, need not be of
degree at most r - 1. Now, if this is the case, if this equation holds then if I take modulo k(x)
on both LHS and RHS, then in my RHS 1 modulo k(x) will give me the polynomial 1 itself.
Whereas in my LHS if I divide the LHS by k(x) the effect of k(x) and k(x) cancels out k(x) is
completely divisible by k(x). So, overall I get that the product of a(x) f(x) modulo k(x) will
give me the constant polynomial 1 and in other words, I have found here 2 polynomials
which when multiplied modulo k(x) will give me 1.
991
So, I can say that f(x) can be treated as the multiplicative inverse of a(x). Now, if f(x) has
degree up to r – 1, at most r – 1, well and good. But if that is not the case I can reduce f(x)
modulo k(x) and that will give me a polynomial of degree at most r - 1 which when
multiplied with a(x) will give me the identity element 1. So, if you see here closely the way I
have argued about the existence of multiplicative inverse for a(x) polynomial is precisely the
same in which where actually when I showed that if GCD of 2 numbers a and n is 1, then we
can find out multiplicative inverse of a and there we argued that as per the Bezout’s theorem I
can express the GCD of a and n in terms of linear their combination. So, say s and t are
Bezout coefficients and then I do modulo n on both sides and I get s times a modulo n is 1
and then I say that a inverse is actually s modulo n. That is what we did in our number theory.
The same thing we have generalised in the context of polynomials.
So, that is a general template for constructing a finite field for any given p and r where the
order is pr. Now let us see how exactly this framework can be applied to construct fields of
various fields of order pr for some given values of p and r. So, imagine I take p to be 3 and r
to be 2. So, I need a irreducible polynomial of degree 2.
So, this is an irreducible polynomial (x2 + 1) and my collection F9 will have all the
polynomials over ℤ3 namely the coefficients are from ℤ3 and the degree of the polynomials
can be 0 or 1. So, I get total 9 such polynomials: {0, 1, 2, x, x + 1, x + 2, 2x, 2x + 1, 2x + 2}
and my plus operation and multiplication operation will be defined modulo x 2 + 1. If I want
to construct a finite field of order 4 so characteristic should be 2.
992
So I will take a irreducible polynomial of degree 2, so if you take this irreducible polynomial
(x2 + x + 1) and this to be your F4 set {0, 1, x, x+1} and my plus and dot operation will be
defined modulo this irreducible polynomial and so. So with that I conclude today’s lecture.
Just to summarise in this lecture we continued our discussion regarding the properties of
finite fields and we proved a very nice property about order of a finite field.
We showed that the order of a finite field is always of the form some prime number raised to
the power r where the prime number is actually the characteristic of that field. Thank you.
993
Lecture – 70
Primitive Element of a Finite Field
Hello everyone, welcome to this lecture, in this lecture, we will continue our discussion over
finite fields and we will focus in this lecture on the multiplicative group of a finite field and
we will prove some nice properties regarding the multiplicative group of a finite field
specifically we show that it is always a cyclic group, it will have some generators and those
generators are called as the primitive element.
So, let F be a finite field with an abstract plus and dot operation and I denote by F* the set
consisting of all elements of the field except the 0 element where 0 is the additive identity
994
element. And my claim here is that if I focus on the nonzero elements of the field and dot
operation, then that constitutes a cyclic group. And of course, it constitutes a group that
comes from the properties of your field axioms.
But what I am claiming here is that it is actually a cyclic group and by cyclic group, I mean
that has at least 1 generator. That means, there exists at least 1 special element g or say f here
such that all the different powers of f will give you the elements of this set F*. So, the proof
strategy here will be the following, so basically I want to show the following: imagine that
the nonzero elements are f1 to fn, there are n such elements.
My goal is to show the following, I am claiming here that there exists at least 1 element f in
this collection F* whose order is n. By that I mean that fn is your identity element and n is the
smallest such positive integer. If I can show that this claim is true, then that shows that indeed
my set F* constitutes a cyclic group. It is already a group as I said, but if this claim is true, I
end up showing that it is actually a cyclic group.
And the proof is slightly involved we will be taking help of several lemmas which are
actually independent properties of groups and so on. And then we will finally arrive at the
proof of this claim, so the claim is actually proved by contradiction. So, we want to show that
there exists at least 1 element in this collection F* whose order is n, the contradiction will be
that there exists no element in the set F* whose order is n, that is a contradiction.
If that is the case, then I have to arrive at some contradiction or some false statement. So, the
proof strategy will be the following. I will be taking help of a simple fact regarding
polynomials over the field. My fact here is the following, my claim is that if I take this degree
n polynomial which is a monic polynomial over the field, then it has exactly n roots in the
field F. That is my fact 1, I will very I can prove this very easily, but this is my fact.
Actually if you see; if you recall the properties of the roots of polynomials, we know that
since the polynomial here has degree n, the number of roots can be at most n, but my fact
here is that it has exactly n roots from the field F and the proof of this fact is the following.
My claim is that you take any element from the set F*, that means you take any nonzero
element from the field F it will satisfy this equation (xn – 1 = 0).
995
If that is the case, then basically I am showing you that all elements of set F* are actually the
roots of this equation or roots of this polynomial. So, let us prove this fact that each element
of the set F* is a root here. So imagine that the order of the element fi is k. By that I mean
that the number of distinct field elements which I can generate by computing different powers
of fi is k; that is another equivalent definition of an order. That means fik is your identity
element.
Now I can invoke here the Lagrange’s theorem of groups and subgroups. Since this cyclic
subgroup <fi> is a subgroup of your parent group F*, of course, with the dot operation that I
am not writing down separately, then as per the Lagrange’s theorem, the Order(<fi>) or the
order of cyclic subgroup <fi> generated by fi should divide the order of your parent group and
the order of the parent group is n because as per my definition F* cardinality is n.
So, n is divisible by k that means, n can be written down as some c times k. That means, I can
say that, since fik is 1, then fin will give me the same element which I obtain by raising (fi)ck.
Then as per the rules of group exponentiation I can take k inside and keep c outside and fik as
per the definition of order of fi will give me the identity element 1 and identity element 1
raised to power c will give me the identity element itself. That means, I have shown that f1 is
a root of xn - 1.
I have shown that f2 is also a root of the polynomial xn - 1

and fn is also a root of the
polynomial xn - 1. That means, I have shown you n roots so, that shows that this fact is true.
Now, coming back to this claim, I want to show that among the elements f1 to fn there is at
least 1 element whose order is n. I will show that if the order of none of the elements f1 to fn
is n, then I will show that this polynomial xn - 1 do not have n number of roots. But that will
contradict fact number 1 because fact number 1 has been proved, I have established fact
number 1. To prove this claim, my strategy will be to show that if there exists no element in
the collection F* whose order is n that means the order of f1 is strictly less than n order of f2
is strictly less than n and like that order of fn is also strictly less than n.
Then I will show that this polynomial does not have n number of roots, which will contradict
my fact 1 and that will show that indeed this claim is correct that is the proof strategy. But as
996
I said to prove this implication, so, now, the proof boils down to proving this implication
assuming that the statement in the claim is incorrect. To prove this implication, I will take
help of several lemmas several related properties.
So, let us prove those independent related properties. So, this is helping lemma number 1.
So, here I want to prove some property regarding the Euler totient function denoted by φ. So,
remember, recall that φ(n) is basically the cardinality of the subset {1, ... , n}, where the
elements are co-prime to n. Basically you want to focus on the number of elements in the
range 1 to n which are co-prime to n, the number of such elements is denoted by φ(n).
Now we can prove a very nice property in regard for this Euler totient function. The property
here is that if you take various divisors here, so, this notation d | n that means d divides n. So,
the property here is that if you take various divisors of n, call them as d1, d2, dk and so on and
then take the summation of φ of those divisors that will give you the number n. Let us prove
this. So, let S be my collection 1 to n and imagine that d1, d2, dk they are the distinct divisors
of n.
Now, I am defining a collection Cdi is basically all those elements from the set S whose GCD
with n is di. So, what basically I am trying to do here is the following: if I take any number x
from the set S and try to find the GCD of that number x along with n, then the GCD has to be
one of these divisors of n, because the GCD has to be first of all a divisor of n and the only
divisors of n are d1 or d2 or dk. So, that means if I take the GCD of any x here from the set S
and number n, it has to be either d1 or d2 or dk.
997
So, I am basically trying to bucket or put all various elements of the set S according to the
GCDs that they have with the element n. And the various buckets are Cd1, Cd2 and Cdk. Now,
as per the definition of this set Cdi, it is easy to see that this collection is actually a partition of
S. It is easy to see that the intersection of these sets is actually empty, because you cannot
have a number x whose GCD with n is both di as well as dj. So that trivially shows that the
intersection of these collections Cd1, Cd2 and Cdk is empty.
And it is also easy to see that if I take the union of various collections here that will give me
the entire set S, because you take any element x, either it will go to the bucket Cd1 or it will
go to the bucket Cd2 or it will go to the bucket Cdk, because if you take the GCD of x with n,
it has to be either d1 or d2 or dk a very simple fact here. That means I can say that the
summation of the cardinality of these individual buckets is nothing but the cardinality of your
set S and the cardinality of your set S is n.
998
Now, comes a very crucial claim, my claim here is that the cardinality of the bucket Cdi is
𝑛 𝑛
same as the value of the Euler totient function for input . And remember is an integer
𝑑𝑖 𝑑𝑖
value because n is divisible by di and di is a distinct divisor of n. Now, assuming for the
moment this claim is true, then, if I apply this claim on this equation n = |Cd1| + ... + |Cdk|, so,
call this equation as equation number 1 if I apply this claim on equation number 1, I basically
𝑛
get that n is same as the summation of the Euler totient function for , the Euler totient
𝑑1
𝑛 𝑛
function for 𝑑 and like that the Euler totient function for . That is what I have written here
2 𝑑 𝑘
your n is summation of various divisors of n and then you sum over the Euler totient function
𝑛
for various . But now, if you see closely here, if you divide n by a divisor of n, you will
𝑑𝑖
obtain a divisor of n itself because d1, d2, dk are the different divisors of n. So, if you divide n
by one of the divisors you will get another divisor.
999
So, what I can say is I can rewrite this equation and apply the logic that since d here runs
𝑛
through the various divisors of n, this will run through those divisors itself. That means
𝑑
whatever effect I can obtain here the same effect I will obtain if I run through the divisors of
𝑛
n in this summation and instead of taking the summation over φ(𝑑 ), I simply take the
𝑖
summation over φ of the various divisors itself.
So, it is a very simple fact I am not going to demonstrate; you can easily verify that. The
proof for this is that since, di is one of the divisors of n and if I divide n by one of those
divisors, I will again obtain a divisor in the list d1 to dk itself. And that shows the proof of my
lemma, but now I have not yet proved this claim. So I have used this claim and then proved
my lemma now, the question boils down to how exactly we prove this claim.
So, my goal is now, to prove that there are indeed these many number of elements in the ith
bucket, so for that let us try to analyse the property of each of the elements in the ith bucket.
So, an element x will be present in the bucket Cdi, if and only if the GCD(x, n) = di that is the
definition of the ith bucket. But, if the GCD(x, n) = di, then that is possible if and only if the
𝑥 𝑛
GCD(𝑑 , 𝑑 ) = 1, very simple.
𝑖 𝑖
𝑥 𝑛
Because if the GCD(𝑑 , ) ≠ 1 and now at the first place the GCD(x, n) was di, that means, I
𝑖 𝑑𝑖
can say that only those elements x will be present in the bucket Cdi such that for those x the
𝑥 𝑛
GCD(𝑑 , ) = 1, that means, I can say that ith bucket consists of all the elements in my
𝑖 𝑑𝑖
1000
𝑛 𝑛 𝑛
collection 1 to 𝑑 which are co-prime to 𝑑 because any number in the collection 1 to 𝑑 which
𝑖 𝑖 𝑖
𝑛 𝑥
is co-prime to 𝑑 , say, call that number as 𝑑 . You multiply that number with di that will give
𝑖 𝑖
you actually a number x which is having a GCD di with the element n. And how many
𝑛 𝑛
elements I have I can have in the collection 1 to which can be co-prime to ? As per my
𝑑𝑖 𝑑𝑖
𝑛
definition of the φ function it will be φ (𝑑 ) and that shows my claim is correct. So, I have
𝑖
proved my helping lemma number 1.

Helping lemma 2 is the following: imagine I take a multiplicative group and imagine there is
an element from the group G whose order is d, so, my element is x whose order is d. Then
my claim is that for the same element x, if you consider the element x k and remember xk as
per the rules of group exponentiation is obtained by multiplying the x to itself k number of
times, which will be an element of the group itself because my group; since G is a group it
satisfies the closure property with respect to the dot operation.
So, the element xk is actually an element from the group itself. Now, my claim here is that
𝑑
since xk is an element of the group and it will have some order. Its order will be 𝐺𝐶𝐷(𝑑,𝑘). So,
to prove this statement, I will take the help of some property from the abstract group theory
which we had discussed earlier. The property that I am going to use here is that if the element
x has order d, and then if you find that xy is giving you the identity element then that is
possible if and only if the exponent y is a multiple of d. So, you can recall the proof of this
fact from one of our earlier lecture. Now, my goal is to show that the order of xk is this value
and for that we have to prove two things. The definition of order is, you have to prove that if
1001
you indeed compute this power of the element xk, you will get the identity element and that is
trivial to prove.
Because since the order of x is d, that means xd is the identity element, then I can say that the
element xk raised to this power will give you the identity element because, if I take x k raised
𝑑 𝑘
to power 𝐺𝐶𝐷(𝑑,𝑘), this is same as xd whole raised to the power 𝐺𝐶𝐷(𝑑,𝑘) and xd is the identity
element. Identity element raised to power anything will give me the identity element, this is
trivial.
The second thing that we have to prove to show that indeed the order of element xk is this is
the following: I have to show that among all possible different positive powers of x k such
that the sth power or the corresponding power gives you the identity element. The power
𝑑
where s is actually is the minimum, what basically I am saying is that it is not the
𝐺𝐶𝐷(𝑑,𝑘)
case that xk raised to the power just single s gives you the identity element, there can be
multiple exponents s. You can have an exponent s1 which gives you the identity element, you
can have another exponent s2 which also gives you the identity element and like that, you can
have another exponent sn which also gives you the identity element. So, what I am basically
trying to argue here is: in order to show that the order of xk is this value, you have to show
𝑑
that among the various powers s1 to sn the power where the value of the power is is
𝐺𝐶𝐷(𝑑,𝑘)
the minimum one, so, the proof here is as follows.
1002
Since the order of xk is s, assuming that s indeed is the order of xk, I know that xks is 1. And
if xks is 1, I can trigger this result regarding the order of x and I can argue that k times s is a
multiple of d. In the same way, k times s1 is also a multiple of d, k times s2 is also a multiple
of d, k times sn is also a multiple of d. So, what basically I am arguing here is that if xk and
whole raised to power s1 is giving you 1, that means k times s1 is a multiple of d, k times s2 is
a multiple of d and like that k times sn is also a multiple of d.
Now, I have to focus on the smallest si such that this smallest k times si which is a multiple of
d satisfies the condition that xksi is giving you the identity element 1.
So, what I can say here is the following: if s is the smallest index or the smallest power
among these various powers s1 to sn satisfying the condition that xks is 1, then the property of
s is that this is the least positive integer of the form k times s which is a multiple of d. That
means, I can say that another property of the order s is that: it is such that k times s is the least
common multiple of both d and k, of course, k times s is a multiple of k. And k times s will
be also a multiple of d, but since s is the order of element xk that means it is the smallest
positive integer such that k times s constitutes LCM(d, k). Now, I can trigger or use the
following relationship regarding the least common multiple and the GCD. If I take the
LCM(d, k) that will be same as the product of the two numbers divided by their GCD.
And then I can rearrange the terms. Since the LCM(d, k) is k times s. I can substitute LHS
by k times s and then I get the conclusion that the smallest positive integer s such that xks is
1003
𝑑
the identity element is actually this index (𝐺𝐶𝐷(𝑑,𝑘)) and that shows the helping Lemma
number 2 is also correct.

And my third helping lemma is the following: the lemma says that if you have a
multiplicative group and if you focus on e(d) : all the elements of the group whose order is d;
for a given d; then the cardinality of e(d) will be φ(d), if the set e(d) is non empty. Of course,
your set e(d) could be empty itself that means there might be no element in the group whose
order is d. The lemma says that if your set e(d) is non empty, that means, if there exists at
least 1 element in the group whose order is d then actually there are φ(d) of such number of
elements.
So, the proof will be as follows: we will take the help of helping lemma number 2 which we
have just proved. So, imagine g is an element of the group whose order is indeed d. That
means, your set e(d) is not empty and my goal is to show this property regarding the
cardinality of the set e(d). Since the order of g is d that means, I can say that element gd will
give you the identity element.
And now, if you see closely here, each of these powers of the element g also will give you the
identity element. Say for instance, g2d. g2d can be rewritten as gd raised to power 2, gd is 1.
So, it will give 12 and 12 is 1. That means, if I take this polynomial xd - 1 over the group G, I
have shown here that the elements g0, g1, gd - 1 are the distinct d roots of this polynomial xd - 1.
1004
And that is the maximum number of roots that I can have for this polynomial xd - 1 because
this polynomial xd - 1 is of degree d. So, it can have at most d roots, but I have shown you
actually d distinct elements from the group which constitutes the roots of this polynomial.
That means, I can say that any root, you take any root of this polynomial, I can relate that root
to the element g. What I am saying is that if h is any root of xd – 1, then I can say that h is
either g0 or h is g1 or like that h is gd – 1, because I have shown that only roots which are
possible for this polynomial are g0, g1, gd - 1. That means, one of these powers of g will give
– 1
you the element h where h is some root of the polynomial x d . That is a relationship
between any root of this polynomial and element g that I have established. And what I also
know is that you take any element whose order is d apart from g. So, you take any element
say r such that order of r is also d. Then whatever argument I have used here I end up
showing that element r also will be the root of this polynomial. Because if g has order d then
g constitutes a root of the polynomial xd - 1. In the same way if r is an element different from
g and its order is d as well, then r is also going to satisfy the polynomial xd - 1 and so on.
But, I already argued here that you take any root of the polynomial x d - 1, it is related to the
element g namely, it has to be of the form either g0 or g1 or g2 or some gk. So, tying these 2
properties together, this property and this property, I can come to the following conclusion, if
your goal is to find out various elements whose order is d, then it is equivalent to finding
various elements of the form gk whose order is d.
Because any element whose order is d will be a root of this polynomial and if it is a root of
this polynomial xd - 1 it will be of the form gk. So, my goal was to find out the number of
elements whose order is d. I have reduced that problem to another problem namely finding
the number of elements of the form gk whose order is d, but my helping lemma2 says is that
𝑑
the order of the element gk will be 𝐺𝐶𝐷(𝑑,𝑘).
So, when can it be possible that the order of gk is precisely d? If your denominator becomes
1 namely the GCD(k, d) becomes 1. Because if the GCD(k, d) becomes 1 then I get the order
of gk is d divided by 1 which will be 1.
1005
That means, I can say that the number of elements of the form gk whose order is d is equal to
the number of elements of the form gk such that the GCD(k,d) = 1, and how many such k can
be there whose GCD with d will be 1. There will be precisely φ(d) number of such k values
and that shows that the number of elements in my collection e(d) will be φ(d).
So, coming back now to the proof of the main theorem, which I wanted to prove. So just to
recall I wanted to prove that if I focus on the nonzero elements of the field it constitutes a
cyclic group. Basically, I have to show, I have to argue about the existence of a generator. I
had already proved this fact and these are my 2 helping lemmas which we had proved. The
goal was to show that among the n elements in your collection F* at least 1 element has order
n. The proof will be by contradiction.
1006
Namely, we will show that if there exists no element in F* whose order is n then this
polynomial xn - 1 has less than n roots and that will go against this fact number 1. So, let us
prove this claim now. Assume that none of the elements from F* has order n. So, let the
various orders which are possible, namely, I have listed down the orders of various elements
from your set F* and let those orders be d1 to dk.
So, you have n elements, it is not the case that all of them have distinct orders. It might be
possible that order of f1 is same as order of f2, order of f3 and so on. So, it is not necessary
that since you have n elements, you have n distinct orders and few of the orders may be
repeated. So that is why let k be the possible orders for various elements in F*. And since I
am assuming that there is no element in F* whose order is n, that means none of these orders
d1 to dk is n.
Now I also know that each of these possible orders d1 to dk is a distinct divisor of n. They are
distinct because they are the various possible distinct orders and why it is a divisor of n
because I know that order of any element from F* which actually is a group divides the order
of F*, the order of F* is n. So, that is why order of f1 will be a divisor of n, order of f2 will
be a divisor of n, order of fn will be a divisor of n.
Now, when I proved the fact number 1, I also argued, I also showed there that you take any
element from F* f1, f2, fn each of them is a root of this polynomial. And as per our
assumption, that order of F will be either d1 or d2 or dk. So, by tying these two facts together,
what I can say about the number of possible roots for this polynomial? The number of
possible roots will be, namely, the number of elements with order d1, the number of elements
with order d2 and the number of elements with order dk. If I sum the number of elements
with these orders that will basically give me the number of roots for this polynomial xn – 1.
Because among the elements from F*, the orders that are possible are either d1, d2, or dk.
And each element from F* is actually a root of xn – 1. So, that is why I get this equation (# of
roots of xn – 1 = e(d1) + ... + e(dk)). Now, I will use this helping lemma here and I can say
that the number of elements in F* whose order is d1 is nothing but φ(d1).
In the same way the number of elements from F* with order dk is nothing but φ(dk) and so
on. And what can I say about the summation in my RHS? The summation in the RHS is
1007
strictly less than n: why it is strictly less than n? Because even though d1, d2, dk they are
distinct divisors of n, as per my assumption, none of them is actually n.
That means, neither d1 is n, nor d2 is n and so on. And as per the helping lemma 1, only when
I sum over φ of various distinct divisors of n, I will get the value n. But since none of these
divisors d1 to dk is n that means I am missing at least one distinct divisor of n. And that is
why I can say that if I take the summation of these quantities φ(d1), φ(d2), φ(dk), I would not
be getting the full n. If there would have been a divisor, if that means if I would have
included φ(n) here as well, then I can say that the summation of all these things is n, but since
φ(n) is missing here because as per my assumption there is no number, no element, from F*
with order n, I can say that my RHS is actually strictly less than n. My RHS is actually the
number of roots of this polynomial. So, this goes against the fact 1 because I have separately
shown already that indeed there are n number of elements from F which constitutes the root
of this polynomial. In fact, all the elements of F* satisfy this polynomial and that is possible
only if at least 1 of the elements from F* has order n. So that proves the theorem.
So now, let us apply this theorem here. So we have proved that you take any finite field and
if you focus on the nonzero elements, we have shown it constitutes a cyclic group. That
means, the collection F* along with the dot operation will have a generator and the generator
is also called as the primitive element of your entire field. And how many such primitive
elements will be there? You will have φ(|F*|) number of such primitive elements.
1008
Because as per our helping lemma there are φ(d) number of elements with order d. So, we
basically want to find out how many elements are there with order same as the order of your
F*. So, it will be same as φ(|F*|). And now if I apply this theorem for the special case of the
field, ℤp. So, your ℤp will have all the elements from 0 to p - 1 and if I say ℤp*, then it will
have p - 1 elements. All the elements except 0 are present here.
Since ℤp constitutes a field, if I focus on the nonzero elements, I get ℤp* and if I apply this
theorem, I get the conclusion that your ℤp* is a cyclic group and it will have these many
number of generators (φ(p – 1)). And this is a very crucial property which if you recall we
utilized to during our discussion on Diffie-Hellman key exchange protocol and Elgamal
encryption scheme; there we performed operations over ℤp* and there I assumed that it is a
cyclic group with some generator.
There you might be wondering what is a guarantee that indeed ℤp* is a cyclic group. Now,
we have proved that indeed ℤp* is a cyclic group and it will have many generators it will have
φ(p – 1) number of generators.

So, with that I conclude today's lecture. Just to summarize today, we discussed about the
multiplicative group of a finite field and we proved that it is a cyclic group. The generators
of that cyclic group are also called as the primitive elements of your finite group.
1009
Lecture - 71
Foundations of Cryptography
Hello everyone, welcome to this lecture. The plan for this lecture is as follows. So, in this
lecture we will see a very nice cryptographic application based on the concepts related to
finite fields, namely secret sharing. So, earlier my plan was to cover both additive secret-
sharing, as well as Shamir’s secret-sharing. We will see depending upon the availability of
time whether we are going to cover both of these topics or not.
So, let us start with the problem of secret sharing what exactly it is motivation real world
application. So, for that imagine a banking application and the way a locker is operated. So, I
1010
do not know whether you have a locker account in a bank or not, but I do have and the way
locker account is maintained or operated in the bank is as follows. Whenever you want to
open or get access to your locker, you have to go along with your key. And apart from your
key there is another key which is held by the manager.
And only when both I enter my key as well as the manager enters the key the locker can be
opened, that means in my absence, the manager cannot open the locker and the same way I
alone cannot go and open the locker myself just using my copy of the key. So now, in the
same way, here in this particular example, I consider a scenario where you can imagine that
the locker can be opened by pressing the key word or the key phrase, whatever you can call
and the key phrase is not available with a single person.
So I imagine here that we have 3 managers: manager 1, manager 2 and manager 3. And the
way this system works here is the following only when at least 2 of the managers come
together and enter their respective passwords, the locker can be opened. But if only a single
manager goes and tries to open the locker by entering his or her password, the locker should
not be opened. So for instance, if m1 just simply goes and try to enter, the locker should not
open.
Similarly, if m2 goes and try to open the locker alone, he should fail. If the manager m3 alone
tries to open the locker, she should fail and so on. But say if the first and second manager
goes together and enter their respective passwords, then the locker should be opened. In the
same way the first and third manager goes together and enter their respective passwords the
locker should be opened.
1011
Another interesting motivation for secret sharing problem is the following. So it is believed
that in the 1990’s, the access to Russia's nuclear weapon was done in the following fashion.
So the password or the credential for launching the nuclear weapon was shared among 3 top
entities of the country, namely the President, Prime Minister and the Defence minister. And it
was shared in such a way that nuclear weapon could be launched or could be accessed only if
at least 2 of the above 3 entities come together and enter their respective credentials or
passwords. But if only 1 entity comes and try to launch the nuclear weapon, then that entity
should fail.
So, in some sense, you can imagine that this is the kind of system gives you more security
more robustness in the sense that if one of the 3 entities say either the President or the Prime
minister or the Defence minister gets compromised and leaks the password, then an enemy
country can launch the nuclear weapon. But if we operate our nuclear weapon in this kind of
system, then in order that the system gets compromised, the enemy country has to
compromise at least 2 entities. That means, if it just compromises one of the entities say
either the President, or the Prime minister, or the Defence minister, then still no harm will be
caused only when 2 of the entities are compromised the harm can be caused. And the system
in that sense, the system is more secure.
1012
So, now, let us abstract both the examples that we had seen by this general problem of what
we call as (n, t) secret sharing and this problem or this primitive was independently
introduced by Turing award winner, Adi Shamir in 1979 and by Blakley in the same year. So,
I am taking this slide from Juan Garay. So, the problem definition is the following you have a
set of parties P1 to Pn. So, n is a given parameter everyone knows the identity of the parties,
we call this parties as shareholders.
And among these n parties, there is a designated entity or a special entity whom we call as a
dealer denoted by D. And dealer has some private input some secret input, let us denote it by
s which is a bit string or it could be any abstract value, but you can always assume that it is a
bit string and this value s belongs to a bigger set namely this set S which I call as the secret
space.
So, the knowledge of the secret space is publicly known, namely, it denotes the set of all
possible secrets which dealer can have. What exactly is the value of the secret from the
secret space which dealer has no one will be knowing, depending upon what kind of prior
information we may have about the secret s. But it will be a public knowledge that whatever
is the input of the dealer, it is from a bigger space namely this set fancy S which we call as a
secret space.
So, for instance, a simple example of the secret space could be the set of all possible bit
strings of length say l bits. That is the case then everyone will be knowing that dealer is going
to invoke or use this primitive with some value s belonging to the set of all possible l bit
1013
strings. And everyone will know the identity of the dealer. Now, the goal is the following: we
want a mechanism according to which dealer should share the secret s among the
shareholders. By that I mean we need a mechanism according to which dealer should
compute n pieces of information, denote them as s1 to sn and ith piece of information namely
si will be given to the ith party. So, we say that si denotes a share of the secret s for the party
pi . That is a requirement, but then if that is just my requirement it is very easy to solve this
problem. Dealer can do some kind of distribution, give some piece of information regarding
s to the first shareholder some piece of information regarding the secret to the second
shareholder and so on.
So, that is why to make the problem interesting and to model what exactly we had seen in the
previous 2 examples, we need the following requirements from this sharing mechanism. We
require that it should be impossible for any set of t or less number of shareholders to pool
their shares and reconstruct back the secret s. So again, like n, t is also some given parameter.
So, I need my sharing mechanism to be such that the vector of n shares which I denote as s1
to sn, they should be such that if any t shares from this vector are taken, then it should be
impossible to reconstruct back the secret s, irrespective of how much time I give to you how
much resource I give it to you. And this should hold even if the description of the sharing
algorithm, the description of the secret space, everything is publicly known, that is important.
I am not assuming here that the sharing mechanism, the algorithm by which the shares are
computed they are hidden.
It is known only to dealer no, that is not the case, because dealer could be any party, the end
shareholders could be any party I cannot afford to design an algorithm which is known or
which is secret and available only with the dealer so, that is my first requirement. The second
requirement is that if t + 1 or more shares are available from this vector of n shares, then it
should be possible to efficiently and uniquely reconstruct back the secret s.
So, you can imagine here that t is kind of acting as a threshold here. Any number of shares
up to t or less will fail to give you back the secret. Any number of shares which are t+1 or
more in numbers should give you back the secret s unambiguously. So, for instance, the
previous 2 examples that we had seen, there my threshold t was 1. And my n was 3. So, if
you take the banking example, there were 3 managers.
1014
So, there are we want a mechanism where the master password should be shared among 3
managers, namely 3 shareholders. So, that if any one of the 3 managers try to access the
secret, the manager should fail. In the same way in the nuclear weapon example, we had 3
entities namely the President, the Vice President and the Prime Minister. So, they are my
shareholders and my t was 1.
I do not want any one of those 3 entities to be able to access the nuclear weapon, but only
when 2 or more numbers of entities come together they should be able to access the nuclear
weapons. That was my requirement. So, this is your problem of (n, t) secret sharing. Now, the
question is how exactly we can solve this?
So, let me discuss this (n, t) secret sharing scheme due to Shamir. So, this is also called as
Shamir’s (n, t) secret sharing scheme. He gave a very nice and very elegant solution for
solving the (n, t) secret sharing problem. Independently Blakley gave another solution for
solving the (n, t) secret sharing problem. But, since we want to see the solution based on
applications of or concepts learned for finite field, I am going to demonstrate Shamir’s secret
sharing scheme for you.
So, this algorithm was published way back in ’79, in this very short paper. But this is one of
the highly cited research papers in cryptography and this is a very simple and elegant
construction. In fact, this is my personal favourite, when I will explain you the solution you
cannot even imagine that how can the solution be so simple and at the same time elegant. So,
1015
the idea behind Shamir’s secret sharing scheme is the following so, imagine dealer as this
secret s.
So, to share the secret s what dealer can do is the following: it can pick a polynomial of
degree t say in variable x. And it will be a random polynomial, which will be chosen by the
dealer. When I say a random polynomial, by that I mean only dealer will be knowing the
coefficients of the polynomial f(X). So, say for instance, if f(X) is of the form say a0 + a1
times X, and like that, the t + 1 th coefficient is at∙Xt.
So, when I say randomly choosing the polynomial by that I mean that the coefficients a0 to at
are known only to the dealer and it is not known to any shareholder. This does not violate the
assumption that the algorithm description is publicly known. The algorithm is publicly
known. What is the algorithm? The algorithm here is choosing a polynomial randomly that is
the process, that is the step, that step is publicly known. That means, the shareholders will be
knowing the actions of the dealer.
Actions of the dealer, by that I mean they will be knowing that dealer is going to pick a
polynomial f(X). But what exactly is the polynomial, the value of the polynomial that will not
be known to the shareholders. That will be random, that will be known only to the dealer.
Because if everything is known in the public domain regarding what are the values which are
picked by the dealer and so on, then how at the first place you can hope to solve this problem.
There has to be some component of randomness in this whole solution and that is
incorporated by saying that dealer picks the polynomial f(X) randomly. Now, it is a random
polynomial except that the constant term of this polynomial is the secret which dealer wants
to share, that means, the coefficients a1, a2 and at they are randomly chosen but a0 is not
randomly chosen, but rather a0 is actually the secret s which dealer wants to share.
In some sense, you can also imagine this as if the polynomial f(X) is a random polynomial,
except that the polynomial when evaluated at x = 0 will give you the value s because if I pick
a polynomial of the form f(X) where the constant term or the coefficient is the secret s. And
then you have the remaining coefficients, then that polynomial when I evaluate at x = 0 will
give me the value s.
1016
And then, once dealer has chosen this polynomial f(X) randomly, the shares are nothing but
distinct points on the polynomial chosen by the dealer. So, let me demonstrate what I am
trying to say here. And for demonstration, I assume that I need a secret sharing mechanism
where t = 1. And imagine dealer's secret is s. So, a polynomial of degree t = 1 is nothing but a
straight line. So, what I am saying here is that dealer in his mind is picking a random straight
line.
And that straight line has the property that, when evaluated at x = 0 will give you the value s,
namely his secret. And now the shares for the respective shareholders are the following: we
imagine here that α1, α2, α3. So by the way, I am assuming here n = 3. So, we need to
compute 3 shares and for computing 3 shares for the 3 shareholders, we assume here that α1,
α2 and α3 they are some publicly known distinct non 0 values.
And the share for the first shareholder is nothing but the value of the straight line at x = α1.
Namely, we compute the point (α1, s1), we compute a point (α2, s2). And we compute a point
(α3, s3), which is equivalent to saying that I am evaluating the straight line at x = α1, x = α2
and x = α3 and getting the values s1, s2 and s3. And s1 will be the share given to the first
shareholder. s2 will be the share given to the second shareholder. And s3 will be the share
given to the third shareholder. Again here, everyone will know the value of α1, α2 and α3 that
is not hidden. And everyone will know that the first shareholder is getting the value of
dealer’s straight line at x = α1. But what exactly was the straight line that is not known, that is
important. Similarly, everyone will know that second shareholder is getting the value of
dealer’s straight line at x = α2.
And everyone will know that the third shareholder is getting the value of dealer's straight line
at x = α3, because the steps of the algorithm are publicly known. Now, let us see here why
this constitutes a valid (n, t) secret sharing scheme. So, remember I am considering the case
where n = 3 and t = 1. The reason it constitutes (n, t) secret sharing scheme is because of the
following 2 facts.
Since the dealer's polynomial f(X) which was known only to the dealer is unknown and its
degree is t, it follows from fundamental properties of polynomials that if I give you t + 1 or
more number of distinct values of the polynomial, then you will be able to uniquely
1017
reconstruct back your polynomial. So, for instance since t = 1 here and the straight line which
was chosen by the dealer is not known to you.
What I am saying is the following: if I give you 2 distinct points on the straight line which
dealer has chosen, then you will be able to uniquely get back the straight line which was
chosen by the dealer. And if you can get back the straight line which was chosen by the
dealer, well, you will be knowing the secret which dealer has shared because the constant
term of that straight line is nothing but the dealer’s secret. So, for instance, if I give you the
first 2 shares and the first 2 shares are nothing but (α1, s1) and (α2, s2).
Now, using these 2 points, you will be able to get back this straight line uniquely. And once
you get back this straight line, you can get back the secret s. In the same way if I give you
say the second share and the third share. You will be able to get back the dealer's straight line
uniquely and hence dealer's secret. So, that is the first observation. So, that shows that t + 1 or
more number of shares will indeed give you back dealer’s secret uniquely that satisfies one of
the properties of (n, t) secret sharing.
Now, the second observation here is that if instead of t + 1 shares, I give you only t shares
and t shares here are nothing but in this context, they constitute t distinct values of an
unknown polynomial whose degree was t. So, the second observation here is that if I give you
t distinct values on an unknown t degree polynomial, then you cannot uniquely recover back
the polynomial f(X).
So, what does that mean here in the context of this example, where t = 1, it means the
following. Suppose, the first shareholder who has the share s1 and of course, it knows α1. The
question here is it possible for the first shareholder to get back the dealer’s secret? Well, no,
because it is possible that through the point (α1, s1), this blue straight line is actually the line
which dealer was which dealer has selected that means, it could be possible that dealer has
actually selected this blue straight line for sharing.
And this blue straight line when evaluated at α1 gives you the value s1 for namely, the point
α1, s1 lies on this straight line that could be the case. If that could be the case then the secret
would have been s’’ where, s’’ would be the constant term of this polynomial. Or it could be
equally likely the case that dealer has actually used this straight line which also pass through
1018
(α1, s1) or it could be the case that dealer has used say another straight line which also passes
through (α1 , s1) and so on.
So, that means, just using the first share s1, it is simply not possible to exactly identify the
straight line which dealer has used and hence it could be any straight line and hence it could
be any secret which dealer would have shared. So, that means you do not have sufficient
information if you are just given t shares to uniquely reconstruct back the dealer’s unknown
polynomial.
And hence you cannot uniquely get back dealer's secret. That is the intuitive idea here, a very
basic fundamental fact of polynomials of t degree namely, t + 1 or more points are sufficient
to get back the polynomial; t or less number of points are not sufficient to uniquely get back
the polynomial. Now, in the Shamir secret sharing scheme we perform or we use the above
idea where all the computations are performed over a finite field, namely my secret space will
be a finite field, my polynomials will be selected over a finite field and my shares also will be
elements of a finite field and this is done because of the following 2 reasons.
We need to maintain security, intuitively what do I mean here is that if instead of picking the
polynomial over a field, we pick polynomial over integers namely if my coefficients of the
polynomial are integers, then based on the magnitude of the shares I may end up revealing
some information about the possible range of the secret. I may not be able to leak the exact
value of the secret, but I may be leaking some information regarding the possible range of the
secret and also if I pick my coefficients of the polynomial and the shares from the integers
then actually I will be working over an infinite domain, which I would not like to do. So, that
is why everything is embedded to a finite field.
1019
So, to understand the actual Shamir’s secret sharing protocol, let us again recap the concepts
regarding polynomials over a finite field. So, if you are given an abstract field which is finite
with an abstract plus and dot operation, then a polynomial over the field is exactly a
polynomial over the integers where the difference is that now, all the coefficients are from the
field and all the plus and dot operations are your field operations.
A value x from the field will be called a root of this polynomial f(X), if the polynomial f
when evaluated at x gives you the additive identity 0 or the 0 element of the field, then we
have actually shown in our discussion on abstract algebra that you take any polynomial of
degree t then it can have at most t roots. And using this theorem, we can actually show that
you take 2 different polynomials f(X) and g(X) over the field then they can agree on at most t
points that means, you can have at most t common points lying on both f(X) as well as g(X).
It is like saying the following you take 2 straight lines, there can be at most 1 point which is
lying both on the first straight line as well as on the second straight line, you cannot have 2
points which are common and lying on both the first straight line as well as on the second
straight line because if that is the case, that means the 2 straight lines are the same straight
line at the first place.
So, that is why we can extend that idea in the context of polynomials over field as well and
then conclude that if you take 2 distinct t degree polynomials, they can agree on at most t
points. Now, another interesting result from the abstract algebra is the following: if I give you
t + 1 number of (x, y) values where the x components are distinct, then you can always find a
1020
unique t degree polynomial over the field such that these (x, y) values constitute distinct
points on that f polynomial.
So, this is often called as Lagrange’s interpolation theorem. So, basically the process by
which we can compute this f polynomial is called as the Lagrange’s interpolation. And here is
how we can get back this unique polynomial f(X). I define several t degree polynomials. So,
the ith t degree polynomial in X is called as the δi(X) polynomial and it will be of this form.
So, why it will be of degree t because in the numerator you have t factors of the form X
minus some value and in the denominator you have the product of several differences.
So, since this whole polynomial is over a field do not interpret this division as your numerical
division or integer division, the interpretation of this division is the following. In the
denominator, I have the terms (xi - x1) (xi - x2) (xi - xi - 1) and so on all these are elements
from the field and if I take the differences here they will be elements of the field. So, in the
denominator I have the product of several field elements.
So, let the final result of the product of the elements in the denominator is A then the
interpretation of this δi(X) polynomial is that instead of saying I divide the numerator by A I
should interpret it as if the numerator is multiplied with A-1 where A-1 is the multiplicative
inverse of the element A because I am performing all the operations over the field. So, in
field I have just a plus operation and a dot operation. So, division should be interpreted as if I
am multiplying with the multiplicative inverse.
So, the property of this δi(X) polynomial is the following. If I substitute X = xi, then I get the
value 1 because if I substitute X = xi, then both the numerator and the denominator becomes
same. Whereas, if I substitute X to be any other value of x say x1, x2 and so on, then I get a
value 0 because one of the factors in the numerator will become 0 and 0 multiplied with any
element will give me 0.
So, now, the unknown f(X) polynomial which I can recover from this t + 1 distinct (x, y)
pairs is the following: it is the product of these t + 1 δ(X) polynomials multiplied with the
corresponding y values. So, the first δ polynomial multiplied with y1, second δ polynomial
multiplied with y2 and the t + 1 th δ polynomial multiplied with the yt + 1 value.
1021
And you can check easily here that you take this polynomial f(X) and evaluate it at xi. You
will get the value yi because when you evaluate this f(X) polynomial at xi, then only the δi(X)
polynomial will survive and give 1 and that will be multiplied with yi. So, you will get yi and
all other remaining δi(X) polynomials will vanish. That is the idea of Lagrange’s
interpolation.
So, now, based on this concept from the finite field, here is the actual Shamir’s secret sharing
scheme. So, the setup here will be the description of a finite field. We require the size of the
field to be at least n namely the number of shareholders and there will be n distinct x values
from the field which are non 0 and which will be publicly available. To share a secret s which
is an element from the field which ensures that my secret space S here is actually the finite
field.
So, if there are multiple elements from the field which dealer wants to share, he has to invoke
this protocol multiple times. But imagine he has only 1 element from the field which it wants
to share, and to do that dealer is going to do the following. It is going to pick t coefficients for
the sharing polynomial randomly from the fields. So, this notation (ϵr) belongs to subscript r
denotes that the polynomial is randomly chosen. So, the polynomial f(X) is randomly chosen
except at its constant term is the secret which dealer wants to share.
And now, the share for the ith party is the evaluation of this polynomial at x = xi, of course,
all operations done over a finite field. Now, let us see whether this scheme satisfies the 2
requirements of secret sharing. The first requirement is that, if you take any t + 1 shares, you
1022
should be able to get back the original secret and this comes from our Lagrange’s
interpolation because t + 1 shares are nothing but t + 1 distinct evaluations of the unknown t
degree polynomial.
Why they are distinct evaluations because we have chosen our x values namely the values at
which the polynomial is evaluated to be distinct. Whereas for privacy, namely, we have to
show that you take any t shares, it leaks no information about the underlying secret s, even if
the t shareholders who combine their shares, they are computationally unbounded; they have
infinite resources and time. And intuitively, this follows from the fact that, if I take the vector
of n shares, which are computed by the dealer.
The property of these n shares is that you take any subset of t shares from this vector, its
probability distribution is independent of the actual secret s. That means, say for instance, if I
take the first t shares here s1 to st, then what I am saying is, it does not matter whether your
secret that was shared was s or the secret that was shared was s’; with equal probability the
first t shares could be s1 to st even for the secret s and with equal probability the same set of
shares s1 to st could be the shares for the secret s’ as well.
So, let us prove this formally. So, for that, let me define this set F s,t, which denotes a set of
all possible t degree polynomials over the field, whose constant term is the secret s. And how
many such polynomials I can have? Well, I can have order of the field raised to power t (|F|t)
such polynomials. This is because any polynomial in this set will be of this form where the
constant term is fixed, but you have the flexibility to choose the remaining t coefficients.
1023
And for each of the remaining t coefficients of any polynomial in this set, you have order of
field number of options because they could be any element from the field. So that is why you
can have these many number of possible polynomials over the field whose constant term is
the secret s. Now, my goal is to show that the probability distribution of any t shares
computed in an instance of Shamir’s secret sharing is independent of the actual secret.
So, for proving that I imagine here that let the first t shareholders are the corrupt
shareholders. That means, I want to prove that the probability distribution of the first t shares
are independent of the actual secret which is shared. You can prove that the probability
distribution on any subset of t shares is independent of the actual secret, but for simplicity, I
am proving with respect to the first t shares here.
So, my claim here is the following: if I give you arbitrary value of the first t shares, there
exists a unique polynomial of degree t with constant term being the secret s such that this s1
to st could occur as the first t shares. And this is because in order that s1 to st constitutes the
shares, or the first t shares for the secret s, it should be the case that there should be a t degree
polynomial whose value at 0 should be s, whose value at α1 should be s1 whose value at α2
should be s2, and like that, whose value at αt could be st. And there could be only one such
polynomial, you cannot have 2 different polynomials f1(X) as well as f2(X) simultaneously
passing through all these points, because you have t + 1 of these points in number. And we
have proved that 2 different t degree polynomials can have common values at at most t
points. That means at most t points could be common both to f1(X) as well as f2(X), you
cannot have t + 1 points common to both f1(X) as well as f2(X). That is the simple fact here.
Now to prove that the probability distribution of the first t shares is independent of the actual
secret, let us take 2 possible candidate secret which could be shared in an instance of
Shamir’s secret sharing scheme call them as s and s’. My claim is that it does not matter
whether the secret is s or whether the secret is s’ with equal probability you could get s1 to st
as the first t shares in an instance of Shamir’s secret sharing scheme.
Basically, I want to calculate the following 2 conditional probabilities. The first conditional
probability here is that given that the secret shared is s in an instance of Shamir’s secret
sharing scheme, what is the probability that the first t shares are s1 to st? And the second
1024
conditional probability here is that given that the secret shared is s’, where of course, s’ is
different from s, what is the probability that the same values s1 to st occurs as the first t shares
in that instance of Shamir’s secret sharing scheme.
And it is easy to see that both these conditional probabilities are exactly the same namely,
1
(|𝐅|𝑡 ), because in order that the shares s1 to st occurs in an instance of Shamir’s secret
sharing, where the shared secret is s, it should be the case that dealer should have chosen the
unique t degree polynomial f(X) during the sharing phase for sharing s, which passes through
the points (0, s), (α1, s1) and like that (αt, st).
But what is the probability that among all possible polynomials from this set, F s,t
,. dealer
actually chose this polynomial f(X)? Well, it is 1 (|𝐅|𝑡 ). In the same way, what is the
probability that if dealer wanted to share the secret s’, it would result the first t shares to be s1
to st the probability of that is exactly the same as the probability that dealer chooses that unique
polynomial of degree t say g(X) from this collection F s’,t
, passing through the points (0, s’), (α1,
s1) and like that (αt, st).
But there are field size raised to power t number of polynomials which dealer could have
used for secret sharing s’. Among those field size raise to power t polynomials, the
probability that dealer has actually chosen the polynomial g(X) is again 1 (|𝐅|𝑡 ). So, that
shows it does not matter whether the secret shared is s or s’ with equal probability the shares
s1 to st could occur as the first t shares. That means, the probability distribution of those t
shares is independent of the actual secret and which formally proves that you are Shamir’s
secret sharing scheme does not reveal any information if t shares are compromised.
1025
So, with that I conclude today's lecture, I have not given full formal proofs. I have given you
intuitive arguments for the privacy proof here. If you want to see the references for today's
lecture, then you can refer to my NPTEL course titled Foundations of Cryptography and this
is the link for this course. Thank you.
1026
Lecture – 72
Goodbye and Farewell
Hello everyone, so this is the farewell lecture with I have already concluded the course,
whatever I have promised at the beginning of the course, I have covered hopefully. So, let me
again quickly go through what we have learnt in this course. The main objective of the course
was to think logically and mathematically and we have covered various topics in this course,
starting with mathematical reasoning, where we have seen how to write various types of
proofs, understand the proof and so on.
We have done lots of combinatorial analysis, we have seen various advanced counting
mechanisms like counting by formulating recurrence equations and solving them. We have
seen various discrete structures like sets, relations and we have also touched upon basic
concepts from graph theory. We have also discussed about abstract algebra, number theory.
And as I said at the beginning of this course that the concepts that we learned in this course,
they are very useful in any area of computer science like algorithms, machine learning,
artificial intelligence, cryptography etc.
So this is to conclude I hope you have learnt a lot in this course, I would like to apologize for
any grammatical errors or mistakes which I might have done or made during the recording, it
1027
becomes very difficult to correct each and every mistake. So, you often might find
grammatical errors. So please pardon me for the same.
And I would like to dedicate this course to by beloved gurus of IIT Madras who built my
foundations in theoretical computer science, namely Professor Kamala Krithivasan, Professor
C. Pandu Rangan and Professor S.A. Choudum.
And some advertisements from my side, so apart from the course on discrete maths, I also
offer a course on foundations of cryptography. So, you can find the details here and it covers
in detail all the foundations for modern cryptography. And as we have seen briefly in this
course, cryptography is nothing but a mathematical science to keep your data secure and we
had seen some cryptographic applications like key exchange, public key cryptography and so
on.
1028
And so in this course, we actually cover in detail, not only encryption, key exchange and so
on, we cover the foundations and fundamentals of modern cryptography namely we deal with
formal definitions, constructions and detailed mathematical proofs for various cryptographic
primitives. And there you will find that the concepts of discrete mathematics that we have
learnt in this course are very much useful.
Second advertisement that I am always looking for motivated full time MS and PhD research
scholars who want to work in cryptography. If you are interested to work with me, you can
apply in response to the advertisements, which come out twice a year. Advertisements are
published at this website and I am not interested in research assistant or internship or offering
research assistant, internship and project positions. So do not write to me for the same and
with that I conclude this course. Thank you.
1029
THIS BOOK
IS NOT FOR
SALE
NOR COMMERCIAL USE
(044) 2257 5905/08

nptel.ac.in
swayam.gov.in

Discrete Maths by Ashish C

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Discrete Maths by Ashish C

Uploaded by

Copyright:

Available Formats

DISCRETE MATHEMATICS - IIITB

Prof. Ashish Choudhury

S.NO TOPICS PAGE.NO

1 Introduction to Mathematical Logic 5

9 Rules of Inferences in Predicate Logic 112

10 Proof Strategies I 133

11 Proof Strategies II 144

13 Tutorial 2: Part I 173

14 Tutorial 2: Part II 186

18 Transitive Closure of Relations 247

19 Warshall’s Algorithm for Computing Transitive Closure 263

21 Equivalence Relation 289

22 Equivalence Relations and Partitions 299

23 Partial Ordering 308

25 Tutorial 4: Part I 339

26 Tutorial 4: Part II 352

27 Countable and Uncountable Sets 363

28 Examples of Countably Infinite Sets 376

29 Cantor’s Diagonalization Argument 395

30 Uncomputable Functions 413

32 Basic Rules of Counting 435

33 Permutation and Combination 452

34 Counting Using Recurrence Equations 472

35 Solving Linear Homogeneous Recurrence Equations – Part I 489

36 Solving Linear Homogeneous Recurrence Equations – Part II 503

38 Tutorial 6: Part II 529

39 Solving Linear Non-Homogeneous Recurrence Equations 547

40 Catalan Numbers 562

41 Catalan Numbers – Derivation of Closed Form Formula 575

42 Counting Using Principle of Inclusion-Exclusion 590

44 Graph Theory Basics 617

46 Proof of Hall’s Marriage Theorem 647

47 Various Operations on Graphs 659

48 Vertex and Edge Connectivity 675

50 Euler Path and Euler Circuit 703

51 Hamiltonian Circuit 721

52 Vertex and Edge Coloring 733

53 Tutorial 9: Part I 747

54 Tutorial 9: Part II 760

55 Modular Arithmetic 778

57 Properties of GCD and Bézout’s Theorem 805

58 Linear Congruence Equations and Chinese Remainder Theorem 821

59 Uniqueness Proof of the CRT 834

Fermat’s Little Theorem, Primality Testing and Carmichael 847

61 Group Theory 861

62 Cyclic Groups 874

64 More Applications of Groups 903

65 Discrete Logarithm and Cryptographic Applications 918

66 Rings, Fields and Polynomials 937

67 Polynomials Over Fields and Properties 951

68 Finite Fields and Properties I 971

69 Finite Fields and Properties II 982

70 Primitive Element of a Finite Field 994

71 Applications of Finite Fields 1010

72 Goodbye and Farewell 1027

Hello everyone, welcome to this lecture on introduction to mathematical logic.

Now let us next define compound propositions. So a compound proposition is a larger

And a truth value of p conjunction q is defined as follows; p conjunction q is defined to be true

So whenever we write a conditional statement of form, if then, p → q, then the p statement or

So in English Language, this is an absurd statement. This is a completely incorrect statement.

More formally X is logically equivalent to Y provided the X bi-implication Y is a tautology,

Hello everyone. Welcome to this lecture on SAT Problem.

Just to recall in the last lecture. We discussed tautology, contingency, contradiction, we

So here is an example of an expression which is in conjunction normal form. You have