TOC - Unit-1

Department of Computer and Science Engineering (CSE)
UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE
ENGINEERING
Bachelor of Engineering
Theory of Computation
(CST-390)
Topic: Introduction
DISCOVER . LEARN .
Course Co-ordinator: Ms. Tanvi
APEX Institute of Technology (AIT) EMPOWER
Learning Objectives & Outcomes

Objective:
• To understand the basics to automata.
Outcome:
• Student will understand the
 Basics of automata

APEX Institute of Technology (AIT)
What is Automata Theory?

• Study of abstract computing devices, or
“machines”
• Automaton = an abstract computing device
• A fundamental question in computer science:
– Find out what different models of machines can do and
cannot do
– The theory of computation
• Computability vs. Complexity

3
(A pioneer of automata theory)

Alan Turing (1912-1954)
• Father of Modern Computer

Science
• English mathematician
• Studied abstract machines called

Turing machines even before
computers existed
• Heard of the Turing test?

4
Theory of Computation: A Historical Perspective
1930s • Alan Turing studies Turing machines

• Decidability
• Halting problem
1940-1950s • “Finite automata” machines studied

• Noam Chomsky proposes the
“Chomsky Hierarchy” for formal
languages
1969 Cook introduces “intractable” problems
or “NP-Hard” problems
Modern computer science: compilers,
1970-
computational & complexity theory evolve
5
Languages & Grammars
• Languages: “A language is a
Or “words” collection of sentences of finite
length all constructed from a
finite alphabet of symbols”
• Grammars: “A grammar can
be regarded as a device that
enumerates the sentences of a
language” - nothing more,
nothing less
• N. Chomsky, Information and

Control, Vol 2, 1959

Image source: Nowak et al. Nature, vol 417, 2002
APEX Institute of Technology (AIT) 6
The Chomsky Hierachy
• A containment hierarchy of classes of formal languages
Regular Context- Recursively-

(DFA) Context-
free enumerable
sensitive
(LB (TM
(PDA) )
A)

Finite Automata
1. Finite automata are used to recognize patterns.
2.It takes the string of symbol as input and changes its

state accordingly. When the desired symbol is found,
then the transition occurs.
3.At the time of transition, the automata can either move

to the next state or stay in the same state.
4. Finite automata have two states, Accept

state or Reject state. When the input string is
processed successfully, and the automata reached
its final state, then it will accept.
Finite Automata : Examples

• On/Off switch action
state
• Modeling recognition of the word “then”
Start state Transition Intermediate Final state

state
Formal Definition of FA
A finite automaton is a collection of 5-tuple (Q, ∑, δ, q0, F),
where:
Q: finite set of states
∑: finite set of the input symbol
q0: initial state
F: final state
δ: Transition function

Finite Automata Model:
Finite automata can be represented by input tape and finite

control.
Input tape: It is a linear tape having some number of cells.

Each input symbol is placed in each cell.
Finite control: The finite control decides the next state on

receiving particular input from input tape. The tape reader
reads the cells one by one from left to right, and at a time only
one input symbol is read.

Types of Automata:
There are two types of finite automata:
DFA(deterministic finite automata)
NFA(non-deterministic finite automata)

Some important points about DFA and NFA:
Every DFA is NFA, but NFA is not DFA.
There can be multiple final states in both NFA and DFA.
DFA is used in Lexical Analysis in Compiler.
NFA is more of a theoretical concept.

Alphabet
An alphabet is a finite, non-empty set of symbols

• We use the symbol ∑ (sigma) to denote an
alphabet
• Examples:
– Binary: ∑ = {0,1}
– All lower case letters: ∑ = {a,b,c,..z}
– Alphanumeric: ∑ = {a-z, A-Z, 0-9}
– DNA molecule letters: ∑ = {a,c,g,t}
– …

Powers of an alphabet
Let ∑ be an alphabet.
– ∑k = the set of all strings of length k
– ∑* = ∑0 U ∑1 U ∑2 U …
– ∑+ = ∑1 U ∑2 U ∑3 U …

Strings
A string or word is a finite sequence of symbols
chosen from ∑
• Empty string is  (or “epsilon”)
• Length of a string w, denoted by “|w|”, is equal to

the number of (non- ) characters in the string
– E.g., x = 010100 |x| = 6
– x = 01  0  1  00  |x| = ?
– xy = concatentation of two strings x and y

FAQ
:
1. What are types of finite automata.
2. Explain model of finite automata.
3. Explain tuples of finite automata.

Languages
L is a said to be a language over alphabet ∑, only if L  ∑*
 this is because ∑* is the set of all strings (of all possible length
including 0) over the given alphabet ∑
Examples:
1. Let L be the language of all strings consisting of n 0’s followed by n
1’s:
L = {,01,0011,000111,…}
2. Let L be the language of all strings of with equal number of 0’s and
1’s:
L = {,01,10,0011,1100,0101,1010,1001,…}
Definition: Ø denotes the Empty language
• Let L = {}; Is L=Ø? NO

3
Language:
A language is a collection of appropriate string. A language
which is formed over Σ can be Finite or Infinite.
Example: 1
• L1 = {Set of string of length 2}
= {aa, bb, ba, bb} Finite Language
Example: 2
• L2 = {Set of all strings starts with 'a'}
= {a, aa, aaa, abb, abbb, ababb} Infinite
Language
Finite Automata
• Some Applications
– Software for designing and checking the behavior of
digital circuits
– Lexical analyzer of a typical compiler
– Software for scanning large bodies of text (e.g., web
pages) for pattern finding
– Software for verifying systems of all types that have a
finite number of states (e.g., stock market transaction,
communication/network protocol)

5
Computer and Science
Engineering
Set (CSE)
operations on languages :

Engineering (CSE)

Example of Formal Language:
• A language is a collection of appropriate string. A language

which is formed over Σ can be Finite or Infinite.
Example: 1
• L1 = {Set of string of length 2}
= {aa, bb, ba, bb} Finite Language
Example 2 :
• Example of Finite Language: L1 = { set of string of 2 } L1 = { xy,
yx, xx, yy }

Example: 3
• L2 = {Set of all strings starts with 'a'}
= {a, aa, aaa, abb, abbb, ababb} Infinite

Language
Example 4 :
• Example of Infinite Language: L1 = { set of all strings
starts with 'b' } L1 = { babb, baa, ba, bbb, baab, ....... }

FAQ
:
1. Construct the powerset for the following sets.
• {a, b}
• {0, 1}𝖴{1, 2}
• {z}
• {0, 1, 2, 3, 4}∩{1, 3, 5, a}
• {0, 1, 2, 3}−{1, 3, 5, a}
• ∅ (the empty set)
2. Prove that 12 +22+32+……+ n2= ∑n2 =
(n(n+1)(2n+1))/6.

Equivalence of DFA & NFA

• Theorem:
– A language L is accepted by a DFA if and only if it is
Should be accepted by an NFA.
true for
any L • Proof:
1. If part:
• Prove by showing every NFA can be converted to an
equivalent DFA (in the next few slides…)
2. Only-if part is trivial:

• Every DFA is a special case of an NFA where each state has
exactly one transition for every input symbol. Therefore, if L is
accepted by a DFA, it is accepted by a corresponding NFA.

Proof for the if-part

• If-part: A language L is accepted by a DFA if it is
accepted by an NFA
• rephrasing…
• Given any NFA N, we can construct a DFA D such
that L(N)=L(D)
• How to convert an NFA into a DFA?
– Observation: In an NFA, each transition maps to a
subset
of states
– Idea: Represent:
each “subset of NFA_states”  a single “DFA_state”
NFA to DFA by subset construction

• Let N = {QN,∑,δN,q0,FN}
• Goal: Build D={QD,∑,δD,{q0},FD} s.t.
L(D)=L(N)
• Construction:
1. QD= all subsets of QN (i.e., power set)
2. FD=set of subsets S of QN s.t. S∩FN≠Φ
3. δD: for each subset S of QN and for
each input
symbol a in ∑:
• δD(S,a) = U δN(p,a)
Course Co-ordinator: Ms.
APEX Institutep inTanvi
of
s Technology (AIT)
NFA to DFA construction: Example

Idea: To avoid
• L = {w | w ends in 01} enumerating all of
1 0 power set, do “lazy
creation of states”
NFA DFA 0 1
: : {q0} {q0,q1}
0
{q0,q2}
0,1 1
0
q0 q1 q2 1
δD 0 1 δD 0 1
δN 0 1
Ø Ø Ø {q0} {q0,q1} {q0}
q0 {q0,q1} {q0} {q0} {q0,q1} {q0} {q0,q1} {q0,q1} {q0,q2}
q1 Ø {q2} {q1} Ø {q2} *{q0,q2} {q0,q1} {q0}
*q2 Ø Ø *{q2} Ø Ø
{q0,q1} {q0,q1} {q0,q2} 1. Enumerate all possible subsets
*{q0,q2} {q0,q1} {q0} 2. Determine transitions
*{q1,q2} Ø {q2} 2. Retain only those states
reachable from {q0}
*{q0,q1,q2} {q0,q1} {q0,q2}

NFA to DFA: Repeating the example

using LAZY CREATION
• L = {w | w ends in 01}
1 0
NFA DFA 0 1
: : {q0} {q0,q1}
0
{q0,q2}
0,1 1
0
q0 q1 q2 1
δN 0 1
δD 0 1
q0 {q0,q1} {q0} {q0} {q0,q1} {q0}
q1 Ø {q2} {q0,q1} {q0,q1} {q0,q2}
*q2 Ø Ø *{q0,q2} {q0,q1} {q0}
Main Idea: Introduce

states as you go(on a
need basis)

Correctness of subset construction

Theorem: If D is the DFA constructed from NFA N by subset
construction, then L(D)=L(N)
• Proof:
– Show that δD({q0},w) ≡ δN(q0,w} , for all w
– Using induction on w’s length:
• Let w = xa
• δD({q0},xa) ≡ δD( δN(q0,x}, a ) ≡ δN(q0,w}

A bad case where #states(DFA)>>#states(NFA)

• L = {w | w is a binary string s.t., the kth symbol
from its end is a 1}
– NFA has k+1 states
– But an equivalent DFA needs to have at least 2k

states
(Pigeon hole principle)

– m holes and >m pigeons
• => at least one hole has to contain two or more
pigeons
Applications
• Text indexing
– inverted indexing
– For each unique word in the database, store all locations
that contain it using an NFA or a DFA
• Find pattern P in text T
– Example: Google querying
• Extensions of this idea:
– PATRICIA tree, suffix tree

A few subtle properties of DFAs and NFAs

• The machine never really terminates.
– It is always waiting for the next input symbol or making transitions.
• The machine decides when to consume the next symbol from
the input and when to ignore it.
– (but the machine can never skip a symbol)
• => A transition can happen even without really consuming an input
symbol (think of consuming  as a free token) – if this happens,
then it becomes an -NFA (see next few slides).
• A single transition cannot consume more than one (non-) symbol.

FA with -Transitions
• We can allow explicit -transitions in finite
automata
– i.e., a transition from one state to another state without
consuming any additional input symbol
– Makes it easier sometimes to construct NFAs
Definition:  -NFAs are those NFAs with at

least one explicit -transition defined.
•  -NFAs have one more column in their transition
table

Example of an -NFA
L = {w | w is empty, or if non-empty will end in 01}

0,1
• -closure of a state q,
q0 0 q1 1 q2 ECLOSE(q), is the set of

all states (including
start q’0 itself) that can be
δE 0 1
 reached from q by
*q’0 Ø Ø {q’0,q 0}
repeatedly making an
q0 {q0,q1} {q0} {q0} arbitrary number of -
q1 Ø {q2} {q1} transitions.
*q2 Ø Ø {q2}

Example of an -NFA
To simulate any transition:

Step 1) Go to all immediate destination states.
Step 2) From there go to all their -closure states as well.

0,1
Simulate for w=101:
q0 0 q1 1 q2
 q0’
 
start q’0
q0’ q0
1

δE 0 1
1q
Ø 0
*q’0 Ø Ø {q’0,q0} x 0
ECLOSE(q’0)
q0 {q0,q1} {q0} {q0} q1
q1 Ø {q2} {q1} ECLOSE(q0) 1
*q2 Ø Ø {q2} q2

Example of another -NFA
To simulate any transition:
Step 1) Go to all immediate destination states.
Step 2) From there go to all their -closure states as well.
0,1 Simulate for w=101: ?

0 1
q0 q1 q2
  1
start q’0 q3

δE 0 1
*q’0 Ø Ø {q’0,q0,q3}
q0 {q0,q1} {q0} {q0,q3}
q1 Ø {q2} {q1}
*q2 Ø Ø {q2}
q3 Ø {q2} {q3}
15
Equivalency of DFA, NFA, -NFA

• Theorem: A language L is accepted by some -NFA if
and only if L is accepted by some DFA
• Implication:
– DFA ≡ NFA ≡ -NFA
– (all accept Regular Languages)

Eliminating -transitions
Let E = {QE,∑,δE,q0,FE} be an -NFA
Goal: To build DFA D={QD,∑,δD,{qD},FD} s.t. L(D)=L(E)
Construction:
1. QD= all reachable subsets of QE factoring in -closures
2. qD = ECLOSE(q0)
3. FD=subsets S in QD s.t. S∩FE≠Φ
4. δ : for each subset S of QE and for each input symbol a∑:
• D Let R= U δE(p,a) // go to destination states
• δD(S,a) = U ECLOSE(r) // from there, take a union

p in s of all their -closures
r in R

Example: -NFA  DFA

0,1
q0 0 q1 1 q2

start q’0

δE 0 1
δD 0 1
*q’0 Ø Ø {q’0,q0}
q0 {q0,q1} {q0} {q0} *{q’0,q0}
q1 Ø {q2} {q1} …
*q2 Ø Ø {q2}

Arden’s Theorem
In order to find out a regular expression of a Finite
Automaton, we use Arden’s Theorem along with the
properties of regular expressions.
Statement −
• Let P and Q be two regular expressions.
• If P does not contain null string, then R = Q + RP has a
unique solution that is R = QP*

PROOF
R = Q + (Q + RP)P [After putting the value R = Q + RP]

= Q + QP + RPP
When we put the value of R recursively again and again, we get the
following equation −
R = Q + QP + QP2 + QP3…..
R = Q (ε + P + P2 + P3 + …. )
R = QP* [As P* represents
(ε + P + P2 + P3 + ….) ]
Hence, proved.

Engineering (CSE)
Construct a regular expression corresponding to the
automata given below −

• Solution −
• Here the initial state is q1 and the final state is q2
• Now we write down the equations −
• q1 = q10 + ε
• q2 = q11 + q20
• q3 = q21 + q30 + q31
• Now, we will solve these three equations −
• q1 = ε0* [As, εR = R]
• So, q1 = 0*
• q2 = 0*1 + q20
• So, q2 = 0*1(0)* [By Arden’s theorem]
• Hence, the regular expression is 0*10*.
Construct aEngineering (CSE) to the
regular expression corresponding
automata given below −

• Solution −
• Here the initial state and final state is q1.
• The equations for the three states q1, q2, and q3 are as follows
−
• q1 = q1a + q3a + ε (ε move is because q1 is the initial state0
• q2 = q1b + q2b + q3b
• q3 = q2a
• Now, we will solve these three equations −
• q2 = q1b + q2b + q3b
• = q1b + q2b + (q2a)b (Substituting value of q3)
• = q1b + q2(b + ab)
• = q1b (b + ab)* (Applying Arden’s Theorem)
• q1 = q1a + q3a + ε
• q1a + q2aa + ε (Substituting value of q3)

• = q1a + q1b(b + ab*)aa + ε (Substituting value of q2)
• = q1(a + b(b + ab)*aa) + ε
• = ε (a+ b(b + ab)*aa)*
• = (a + b(b + ab)*aa)*
• Hence, the regular expression is (a + b(b + ab)*aa)*.

Engineering
Find regular (CSE)DFA using
expression for the following
Arden’s Theorem-

•
•
Solution-
Engineering (CSE)
• Step-01:
•
• Form a equation for each state-
• A = ∈ + B.1 ……(1)
• B = A.0 ……(2)
•
• Step-02:
•
• Bring final state in the form R = Q + RP.
•
• Using (1) in (2), we get-
• B = (∈ + B.1).0
• B = ∈.0 + B.1.0
• B = 0 + B.(1.0) ……(3)
•
• Using Arden’s Theorem in (3), we get-
• B = 0.(1.0)*
•
• Thus, Regular Expression for the given DFA = 0(10)*

Engineering
Find regular (CSE)DFA using
expression for the following
Arden’s Theorem-

• Solution-
•
•
Step-01: Engineering (CSE)
Form a equation for each state-
• q1 = ∈ ……(1)
• q2 = q1.a ……(2)
• q3 = q1.b + q2.a + q3.a …….(3)
• Step-02:
• Bring final state in the form R = Q + RP.
• Using (1) in (2), we get-
• q2 = ∈.a
• q2 = a …….(4)
• Using (1) and (4) in (3), we get-
• q3 = q1.b + q2.a + q3.a
• q3 = ∈.b + a.a + q3.a
• q3 = (b + a.a) + q3.a …….(5)
• Using Arden’s Theorem in (5), we get-
• q3 = (b + a.a)a*
• Thus, Regular Expression for the given DFA = (b + aa)a*

Regular Languages
Definition
• We define regular languages as the smallest class of
languages which contains all finite languages and closed
with respect to union, concatenation and Kleene closure.
• Every regular expression denotes a regular language.
Example
• (a*b) is a regular expression that denotes the set of
strings formed by a sequence, also empty, of a's followed
by a b.

Regular Expressions vs. Finite

•
Automata
Offers a declarative way to express the pattern of any
string we want to accept
– E.g., 01*+ 10*
• Automata => more machine-like

< input: string , output: [accept/reject] >
• Regular expressions => more program syntax-like
• Unix environments heavily use regular expressions

– E.g., bash shell, grep, vi & other editors, sed
• Perl scripting – good for string processing
• Lexical analyzers such as Lex or Flex

Regular Expressions
Regular = Finite Automata

expressions (DFA, NFA, -NFA)
Syntactic
al Automata/machines
expressions
Regular
Languages
Formal language
classes

Language Operators
• Union of two languages:

– L U M = all strings that are either in L or M
– Note: A union of two languages produces a third
language
• Concatenation of two languages:

– L . M = all strings that are of the form xy
s.t., x  L and y  M
– The dot operator is usually omitted
• i.e., LM is same as L.M

6
Kleene Closure (the * operator)

• Kleene Closure of a given language L:
– L0= {}
– L1= {w | for some w  L}
– L2= { w1w2 | w1  L, w2  L (duplicates allowed)}
– Li= { w1w2…wi | all w’s chosen are  L (duplicates allowed)}
– (Note: the choice of each wi is independent)
– L* = Ui≥0 Li (arbitrary number of concatenations)
Example:
• Let L = { 1, 00}
– L0= {}
– L1= {1,00}
– L2= {11,100,001,0000}
– L3= {111,1100,1001,10000,000000,00001,00100,0011}
– L* = L0 U L1 U L2 U …

7
Kleene Closure (special notes)
• L* is an infinite set iff |L|≥1 and L≠{}

• If L={}, then L* = {}
• If L = Φ, then L* = {}
Σ* denotes the set of all words over an

alphabet Σ
– Therefore, an abbreviated way of saying there is
an arbitrary language L over an alphabet Σ is:
• L  Σ*

Building Regular Expressions

• Let E be a regular expression and the language
represented by E is L(E)
• Then:
– (E) = E
– L(E + F) = L(E) U L(F)
– L(E F) = L(E) L(F)
– L(E*) = (L(E))*

Example: how to use these regular

expression properties and language
operators?
• L = { w | w is a binary string which does not contain two consecutive 0s or two
consecutive 1s anywhere)
– E.g., w = 01010101 is in L, while w = 10010 is not in L
• Goal: Build a regular expression for L
• Four cases for w:
– Case A: w starts with 0 and |w| is even
– Case B: w starts with 1 and |w| is even
– Case C: w starts with 0 and |w| is odd
– Case D: w starts with 1 and |w| is odd
• Regular expression for the four cases:
– Case A: (01)*
– Case B: (10)*
– Case C: 0(10)*
– Case D: 1(01)*
• Since L is the union of all 4 cases:
– Reg Exp for L = (01)* + (10)* + 0(10)* + 1(01)*
• If we introduce  then the regular expression can be simplified to:
– Reg Exp for L = ( +1)(01)*( +0)

Precedence of Operators
• Highest to lowest
– * operator (star)
– . (concatenation)
– + operator
• Example:
– 01* + 1 =( 0 . ((1)*) ) + 1

Finite Automata (FA) & Regular

Expressions (Reg Ex)
• To show that they are interchangeable, consider
the following theorems:
– Theorem 1: For every DFA A there exists a regular
Proofs expression R such that L(R)=L(A)
in the book – Theorem 2: For every regular expression R there exists
an  -NFA E such that L(E)=L(R)
 -NFA NF
A Kleene Theorem
Theorem 2
Reg Ex DFA

DFA to RE construction
DF Reg Ex
Theorem 1
A
Informally, trace all distinct paths (traversing cycles only once)
from the start state to each of the final states
and enumerate all the expressions along the way
Example: 1 0 0,1
q0 0 q1 1 q2
(1*) 0 (0*) 1 (0 + 1)*
1* 00* 1 (0+1)*
1*00*1(0+1)*
RE to -NFA construction
Reg Ex  -NFA
Theorem 2
Example: (0+1)*01(0+1)*
(0+1)* 01 (0+1)*
 

0
 
0

 0 1
 1
  1

 

Algebraic Laws of Regular Expressions

• Commutative:
– E+F = F+E
• Associative:
– (E+F)+G = E+(F+G)
– (EF)G = E(FG)
• Identity:
– E+Φ = E
– E=E=E
• Annihilator:
– ΦE = EΦ = Φ

Algebraic Laws…
• Distributive:
– E(F+G) = EF + EG
– (F+G)E = FE+GE
• Idempotent: E + E = E
• Involving Kleene closures:
– (E*)* = E*
– Φ* =
– * =
– E+ =EE*
– E? =  +E

True or False?
Let R and S be two regular expressions. Then:
1. ((R*)*)* = R* ?
2. (R+S)* = R* + S*?
3. (RS + R)* RS = (RR*S)*?

Applications of interest
• Comparing two DFAs:
– L(DFA1) == L(DFA2)?
• How to minimize a DFA?

1. Remove unreachable states
2. Identify & condense equivalent states into one
Course Co-ordinator: Ms. Tanvi 3

When to call two states in a DFA
“equivalent”?
Two states p and q are said to be equivalent iff:
i) Any string w accepted by starting at p is also accepted by starting
at q;
p
w
AND
q
i) Any string w rejected by starting at p is also rejected by starting at
q.
p
w
q
 p≡q
Computing equivalent states in a DFA
(Myhill- Nerode Theorem )
A =
0 1
B = =
0 1 0
A C E G C x x =
1 0 1
0 1 D x x x =
B 1
D 1
F 0
H E x x x x =
Pass #0 1 0 F x x x x x =
1. Mark accepting0states ≠ non-accepting states G x x x = x x =
Pass #1
1. Compare every pair of states H x x = x x x x =
2. Distinguish by one symbol transition
3. Mark = or ≠ or blank(tbd)
A B C D E F G H
Pass #2
4. Compare every pair of states
5. Distinguish by up to two symbol transitions
(untiluntil
(keep repeating different
tableor same or tbd)
complete)
….
Table Filling Algorithm - step by step
A =
0 1
B =
0 1 0
A C E G C =
1 0 1
0 1 D =
B 1
D 1
F 0
H E =
1 0 F =
0 G =
H =
A B C D E F G H

A =
0 1
B =
0 1 0
A C E G C =
1 0 1
0 1 D =
B 1
D 1
F 0
H E X X X X =
1 0 F X =
0 G X =
1. Mark X between accepting vs. non-accepting state

H X =
A B C D E F G H

A =
0 1
B =
0 1 0
A C E G C X =
1 0 1
0 1 D X =
B 1
D 1
F 0
H E X X X X =
1 0 F X =
0 G X X =

H X X =
2. Look 1- hop away for distinguishing states or A B C D E F G H
strings

A =
0 1
B =
0 1 0
A C E G C X X =
1 0 1
0 1 D X X =
B 1
D 1
F 0
H E X X X X =
1 0 F X =
0 G X X X =

H X X X =
strings

A =
0 1
B =
0 1 0
A C E G C X X =
1 0 1
0 1 D X X X =
B 1
D 1
F 0
H E X X X X =
1 0 F X X =
0 G X X X X =

H X X = X =
strings

A =
0 1
B =
0 1 0
A C E G C X X =
1 0 1
0 1 D X X X =
B 1
D 1
F 0
H E X X X X =
1 0 F X X X =
0 G X X X = X =

H X X = X X =
strings

A =
0 1
B =
0 1 0
A C E G C X X =
1 0 1
0 1 D X X X =
B 1
D 1
F 0
H E X X X X =
1 0 F X X X =
0 G X X X = X X =

H X X = X X X =
strings

A =
0 1
B =
0 1 0
A C E G C X X =
1 0 1
0 1 D X X X =
B 1
D 1
F 0
H E X X X X =
1 0 F X X X =
0 G X X X = X X =

H X X = X X X X =
strings

A =
0 1
B = =
0 1 0
A C E G C X X =
1 0 1
0 1 D X X X =
B 1
D 1
F 0
H E X X X X =
1 0 F X X X X X =
0 G X X X = X X =

H X X = X X X X =
2. Pass 1:
ng A B C D E F G H
Look 1- hop away for distinguishing states or s
stri
3. Pass 2:
Look 1-hop away again for distinguishing
states or strings
continue….
A =
0 1
B = =
0 1 0
A C E G C X X =
1 0 1
0 1 D X X X
1 1 0 =
E X X X X
B D F H
1 =
F X X X X X =
0
0
G X X X = X X =
1. Mark X between accepting vs. non-accepting state H X X = X X X X =
2. Pass 1: A B C D E F G H
Look 1- hop away for distinguishing states or strings
3. Pass 2:
Look 1-hop away again for distinguishing states or strings Equivalences:
continue…. • A=B
• C=H
• D=G
0 1 0 1
0 1 0 0 1
A C E G A C E
1 0 1 1 0
0 1 0
1 1 0 1
B D F H D F
1 0 1 0
0
Retrain only one copy for

each equivalence set of states
Equivalences:
• A=B
• C=H
• D=G
Table Filling Algorithm – special case
A =
0 1
B =
0 1 0
A C E G C =
1 0 1
0 1 D =
B 1
D 1
F 0
H E ? =
1 0 F =
0 G =
H =
A B C D E F G H

How to minimize a DFA?
• Goal: Minimize the number of states in a DFA
• Algorithm:
1. Eliminate states unreachable from the start state
2. Identify and remove equivalent states
3. Output the resultant DFA

DFA Minimization using Equivalence Theorem
• Example
• Let us consider the following DFA −

q Engineering
δ(q,0) (CSE)
δ(q,1)
a B C
b A D
c E F
d E F
e E f
f F f

• Let us apply the above algorithm to the above DFA −

• P0 = {(c,d,e), (a,b,f)}
• P1 = {(c,d,e), (a,b),(f)}
• P2 = {(c,d,e), (a,b),(f)}
• Hence, P1 = P2.
• There are three states in the reduced DFA. The reduced
DFA is as follows −

Engineering (CSE)
Q δ(q,0) δ(q,1)
(a, b) (a, b) (c,d,e)
(c,d,e) (c,d,e) (f)
(f) (f) (f)

The Pumping Lemma for Regular
Sets
Proved that a given language

is not regular
3
Pumping Lemma for Regular Sets
Let L be a regular language

Then there exists some constant N such that for
every string w  L s.t. |w|≥N, there exists a way
to break w into three parts, w=xyz, such that:
1. y≠ 
2. |xy|≤N
3. For all k≥0, all strings of the form xykz  L
This clause should hold for all regular languages.
Definition: N is called the “Pumping Lemma Constant”

Pumping Lemma: Proof
• L is regular => it should have a DFA.
– Set N := number of states in the DFA
• Any string wL, s.t. |w|≥N, should have the
form: w=a1a2…am, where m≥N
• Let the states traversed after reading the first N
symbols be: {p0,p1,… pN}
 ==> There are N+1 p-states, while there are only N
DFA states
 ==> at least one state has to
repeat i.e, pi= pJwhere 0≤i<j≤N (by
PHP)

Pumping Lemma: Proof…
 => We should be able to break w=xyz as follows:

 x=a1a2..ai; y=ai+1ai+2..aJ; z=aJ+1aJ+2..am
 x’s path will be p0..pi
 y’s path will be pi pi+1..pJ (but pi=pJ implying a loop)
 z’s path will be pJpJ+1..pm
yk (for k loops)
 Now consider another
string wk=xykz , where k≥0 x z
p0 pi pm
 Case k=0
=pj
 DFA will reach the accept state p m
 Case k>0
 DFA will loop for yk, and finally reach the accept state pm for z
 In either case, wk L

Pumping Lemma: Proof…
• For part (1): yk (for k loops)
– Since i<j, z
x
y≠ p0 pi pm
=pj
• For part (2):
– By PHP, the repetition of states has to occur within the first N
symbols in w
– ==> |xy|≤N

The Purpose of the Pumping Lemma for RL
• To prove that some languages cannot be regular.

How to use the pumping lemma?
Think of playing a 2 person game
– Role 1: You claim that the language
cannot be regular
– Role 2: An adversary who claims the

language is
regular
– You show that the adversary’s statement will lead to a

contradiction that implyies pumping lemma cannot
hold for the language.
– You win!!
Note: This N can be anything (need not necessarily be the #states in the DFA.
It’s the adversary’s choice.)
Example of using the Pumping Lemma to

prove that a language is not regular
Let Leq = {w | w is a binary string with equal number of 1s
and 0s}
• Your Claim: Leq is not regular
• Proof:
 By contradiction, let Leq be regular
 P/L constant should exist
 Let N = that P/L constant
 Consider input w = 0N1N
(your choice for the template string)
 By pumping lemma, we should be able to break w=xyz, such
that:
1) y≠ 
2) |xy|≤N
3) For all k≥0, the string xykz is also in L

Template string w = 0N1N = 00 … . 011 … 1
N N
Proof…
 Because |xy|≤N, xy should contain only
 you
0s (This and because y≠ , implies y=0+)
 Therefore x can contain at most N-1 0s
 Also, all the N 1s must be inside z
 By (3), any string of the form xykz  Leq for all k≥0
 Case k=0: xz has at most N-1 0s but has N 1s
Setting k=0 is
referred to as  Therefore, xy0z  L eq
“pumping down”
 This violates the P/L (a contradiction)
Another way of proving this will be to show that if

Setting k>1 is the #0s is arbitrarily pumped up (e.g., k=2),
referred to as
“pumping up” then the #0s will become exceed the #1s
Exercise 2
Prove L = {0n10n | n≥ 1} is not regular
Note: This n is not to be confused with the pumping

lemma constant N. That can be different.
In other words, the above question is same as proving:

– L = {0m10m | m≥ 1} is not regular

Closure properties of Regular
Sets
13
Closure properties for R egular Sets ( RL)
This is different
from Kleene
• Closure property: closure
– If a set of regular languages are combined using an
operator, then the resulting language is also regular
• Regular languages are closed under:
– Union, intersection, complement, difference
– Reversal
– Kleene closure
– Concatenation
– Homomorphism Now, lets prove all of this!
– Inverse homomorphism

RLs are closed under union
• IF L and M are two RLs THEN:
 they both have two corresponding regular expressions, R

and S respectively
 (L U M) can be represented using the regular expression

R+S
 Therefore, (L U M) is also regular
How can this be proved using FAs?

RLs are closed under complementation
• If L is an RL over ∑, then L=∑*-L
 To show L is also regular, make the following
construction
Convert every final state into non-final, and
every non-final state into a final state
DFA for DFA for

L qF1 L qF1
q0 qi q0 qi
qF2 qF2
…
qFk qFk
Assumes q0 isCo-ordinator:
Course a non-final state.
Ms. IfTanvi
not, do the opposite. 16
RLs are closed under intersection
• A quick, indirect way to prove:

– By DeMorgan’s law:
– L ∩ M = (L U M)
– Since we know RLs are closed under union and complementation,
they are also closed under intersection
• A more direct way would be construct a finite
automaton for L ∩ M

DFA construction for L ∩ M
• AL = DFA for L = {QL, ∑ , qL,FL, δL }

• AM = DFA for M = {QM, ∑ , qM,FM, δM }
• Build AL ∩ M = {QLx QM,∑, (qL,qM), FLx FM,δ} such
that:
– δ((p,q),a) = (δL(p,a), δM(q,a)), where p in QL, and q
in
QM
• This construction ensures that a string w will be
accepted if and only if w reaches an accepting
state in both input DFAs.

DFA construction for L ∩ M
DFA for DFA for

L qF1
M pF1
a a
q0 qi qj p0 pi pj
qF2 pF2
…
DFA for
LM (qF1 ,pF1)
a
(qi ,pi) (qj ,pj)
…
(q0 ,p0)

RLs are closed under set difference
• We observe: Closed under intersection
– L-M=L
∩M Closed under
complementation
• Therefore, L - M is also regular

RLs are closed under reversal
Reversal of a string w is denoted by wR
– E.g., w=00111, wR=11100
Reversal of a language:
• LR = The language generated by reversing all strings in L
Theorem: If L is regular then LR is also regular

 -NFA Construction for L R
New -NFA for LR
DFA for
L qF1

q0 qi
a
qj qF2  q’0 New start
state
Make the 
…
old start state
as the only new qFk
final state
What to do if q0 was Reverse all transitions

one of the final states Convert the old set of final states
in the inputCourse
DFA?Co-ordinator: Ms. Tanvi into non-final states
22
If L is regular, LR is regular (proof using
regular expressions)
• Let E be a regular expression for L
• Given E, how to build ER?
• Basis: If E= , Ø, or a, then ER=E
• Induction: Every part of E (refer to the part as “F”) can
be in only one of the three following forms:
1. F = F1+F2
• FR = F1R+F2R
2. F = F1F2
•FR = F2RF1R 3.
F = (F1)*
• (FR)* =
(F1R)*
Finite State Machines with Output (Mealy
and Moore Machines)
• Finite automata Machine receive input and process the
input by changing states.
• The only output that we have seen finite automata
produce so far is a yes/no at the end of processing.
• We will now look at two models of finite automata that
produce more output than a yes/no.

Moore machine
• Basically a Moore machine is just
a FA with two extras.
1. It has TWO alphabets, an input and
output alphabet.
2. It has an output letter associated with
each state. The machine writes
the appropriate output letter as it
enters each state.
This machine might be

considered as a
"counting" machine.
The output produced by the machine contains a 1 for each
occurrence of the substring aab found in the input string.
Engineering
Moore Machine: (CSE)
State Transition diagram & Transition table
Transition Diagram Transition

Table

Mealy machine
• Mealy Machines are exactly as powerful as Moore machines

– (we can implement any Mealy machine using a Moore machine, and
vice versa).
• However, Mealy machines move the output function from the
state to the transition.
• This turns out to be easier to deal with in practice, making
Mealy machines more practical.

Engineering
Mealy Machine: (CSE)
State Transition diagram & Transition table
Transition Diagram Transition

Table

A Mealy machine produces output on a
transition instead of on entry into a state
• Transitions are labelled i/o where
– i is a character in the input alphabet and
– o is a character in the output alphabet.
The following Mealy machine takes the one's

complement of its binary input. In other words,
it flips each digit from a 0 to a 1 or from a 1 to a
0.
• Mealy machines are complete in the sense that there is a

transition for each character in the input alphabet leaving
every state.
• There are no accept states in a Mealy machine

Automata networks
• (Parts of the) output of a Mealy/Moore automaton can be
input to another automaton
– Pipeline
– Dataflow architecture
– Feedback network

Automata as transition systems
• Output string can be seen as a

characterization of what holds
in a particular state Example transition system
• Output language e.g.
propositional logic
• Given a set of atomic p,q 1
propositions, states are labelled
with propositions that “hold”
in a state
(Moore style) q 2 3 p

Example: Parity checker
• Serial input string
– OUT=1 if odd # of 1s in input
– OUT=0 if even # of 1s in input
• Let’s do this for Moore and Mealy
11
Equivalence of Moore and Mealy Machines

Conversion from Mealy machine to Moore Machine
• In Moore machine, the output is associated with every state, and in
Mealy machine, the output is given along the edge with input symbol.
To convert Moore machine to Mealy machine, state output symbols
are distributed to input symbol paths. But while converting the
Mealy machine to Moore machine, we will create a separate state for
every new output symbol and according to incoming and outgoing
edges are distributed.
The following steps are used for converting Mealy machine to the Moore
machine:
 Step 1: For each state(Qi), calculate the number of different outputs that
are
available in the transition table of the Mealy machine.
 Step 2: Copy state Qi, if all the outputs of Qi are the same. Break qi into n states
as Qin, if it has n distinct outputs where n = 0, 1, 2....
 Step 3: If the output of initial state is 0, insert a new initial state at the starting
which gives 1 output.

Conversion from Mealy machine to Moore Machine
• Example 1:
• Convert the following Mealy machine into equivalent
Moore machine.

• Solution: Engineering (CSE)
• Transition table for above Mealy machine is as follows:

• For state q1, there is only one incident edge with output
1. So, we don't need to split this state in Moore machine.
• For state q2, there is 2 incident edge with output 0 and 1.
So, we will split this state into two states q20( state with
output 0) and q21(with output 1).
• For state q3, there is 2 incident edge with output 0 and 1.
So, we will split this state into two states q30( state with
output 0) and q31( state with output 1).
• For state q4, there is only one incident edge with output
1. So, we don't need to split this state in Moore machine.
• Transition table for Moore machine will be:

Engineering (CSE)

Engineering
Transition diagram for Moore (CSE)
machine will be:

Conversion from Moore machine to Mealy Machine
• Example 1:
• Convert the following Moore machine into its equivalent
Mealy machine.

Engineering (CSE)
Solution:
The transition table of given Moore machine is as follows:
Q A b Output(λ)
q0 q0 q1 0
q1 q0 q1 1

• The equivalent Mealy machine can be obtained as

follows:
• λ' (q0, a) = λ(δ(q0, a))
• = λ(q0)
• =0
• λ' (q0, b) = λ(δ(q0, b))

• = λ(q1)
• =1

• The λ for state q1 is as follows:

• λ' (q1, a) = λ(δ(q1, a))
• = λ(q0)
• =0
• λ' (q1, b) = λ(δ(q1, b))

• = λ(q1)
• =1

Engineering
Hence the transition (CSE)
table for the Mealy machine can be
drawn as follows:

Engineering (CSE)
The equivalent Mealy machine will be,

FAQ
:
1. Construct a finite automata that accepts {0,1}+.
2. Eliminate the e production from the CFG given below
• A-> 0 B 1 | 1 B 1
• B-> 0 B | 1 B | e
3. Find the derivation tree for the grammar, G = {{s , A , B },
{a , b }, P , s } ,whereP = {S / bA / bB , A / ab , B / aBb
/
a}

FAQ
:
1. Construct a finite automata that accepts {0,1}+.
2. Eliminate the e production from the CFG given below
• A-> 0 B 1 | 1 B 1
• B-> 0 B | 1 B | e
3. Find the derivation tree for the grammar, G = {{s , A , B },
{a , b }, P , s } ,whereP = {S / bA / bB , A / ab , B / aBb
/
a}

IMPORTANT TOPICS

Chomsky's Normal Form (CNF)

CNF stands for Chomsky normal form. A CFG(context free grammar) is
in CNF(Chomsky normal form) if all production rules satisfy one of the
following conditions:
• Start symbol generating ε. For example, A → ε.

• A non-terminal generating two non-terminals. For example, S → AB.
• A non-terminal generating a terminal. For example, S → a.

For example:
• G1 = {S → AB, S → c, A → a, B → b}
• G2 = {S → aA, A → a, B → c}
The production rules of Grammar G1 satisfy the rules specified for CNF,
so the grammar G1 is in CNF. However, the production rule of Grammar
G2 does not satisfy the rules specified for CNF as S → aZ contains
terminal followed by non-terminal. So the grammar G2 is not in CNF.

Context-Free Grammar
Context Free Grammar is formal grammar, the syntax or structure of a
formal language can be described using context-free grammar (CFG), a
type of formal grammar. The grammar has four tuples: (V,T,P,S).
• V - It is the collection of variables or nonterminal symbols.

• T - It is a set of terminals.
• P - It is the production rules that consist of both terminals and non-
terminals.
• S - It is the Starting symbol.

A grammar is said to be the Context-free grammar if every production is

in the form of :
• G -> (V∪T)*, where G ∊ V

And the left-hand side of the G, here in the example can only be a Variable, it
cannot be a terminal.
But on the right-hand side here it can be a Variable or Terminal or both
combination of Variable and Terminal.
Above equation states that every production which contains any
combination of the ‘V’ variable or ‘T’ terminal is said to be a context-free
grammar.
Pushdown Automata
Pushdown Automata is a finite automata with extra memory called stack which helps Pushdown
automata to recognize Context Free Languages.
A Pushdown Automata (PDA) can be defined as :
• Q is the set of states

• ∑is the set of input symbols
• Γ is the set of pushdown symbols (which can be pushed and popped from stack)
• q0 is the initial state
• Z is the initial pushdown symbol (which is initially present in stack)
• F is the set of final states
• δ is a transition function which maps Q x {Σ ∪ ∈} x Γ into Q x Γ*. In a given state, PDA will read input
symbol and stack symbol (top of the stack) and move to a new state and change the symbol of stack.

Context-sensitive grammar
Context-sensitive grammar has 4-tuples. G = {N, Σ, P, S}, Where
• N = Set of non-terminal symbols

• Σ = Set of terminal symbols
• S = Start symbol of the production
• P = Finite set of productions

The context sensitive grammar follows the following rules:
• The context sensitive grammar may have more than one symbol on the left hand side of their
production rules.
• The number of symbols on the left-hand side must not exceed the number of symbols on the
right-hand side.
• The rule of the form A → ε is not allowed unless A is a start symbol. It does not occur on the right-
hand side of any rule.
• Production is in the form of V → T
Where the count of symbol in V is less than or equal to T.

Turing machine
• Turing machine was invented in 1936 by Alan Turing. It is an accepting device which
accepts Recursive Enumerable Language generated by type 0 grammar.
There are various features of the Turing machine:
• It has an external memory which remembers arbitrary long sequence of input.

• It has unlimited memory capability.
• The model has a facility by which the input at left or right on the tape can be read easily.
• The machine can produce a certain output based on its input. Sometimes it may be
required that the same input has to be used to generate the output. So in this machine,
the distinction between input and output has been removed. Thus a common set of
alphabets can be used for the Turing machine.

A Turing machine can be defined as a collection of 7 components:
• Q: the finite set of states

• ∑: the finite set of input symbols
• T: the tape symbol
• q0: the initial state
• F: a set of final states
• B: a blank symbol used as a end marker for input
• δ: a transition or mapping function.
REFERENCES :
• Martin J.C., “Introduction to Languages and Theory of
Computation”, Tata McGraw-Hill Publising Company
Limited, 3rd Edition.
• Hopcroft J.E. and Ullman J.D., “Introduction to Automata
Theory Languages and Computation”, Narosa
Publications.
• https://www.javatpoint.com/conversion-from-mealy-
machine-to-moore-machine
• https://www.javatpoint.com/conversion-from-moore-
machine-to-mealy-machine

TOC - Unit-1

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

TOC - Unit-1

Uploaded by

Copyright:

Available Formats

Department of Computer and Science Engineering (CSE)

Learning Objectives & Outcomes

Course Co-ordinator: Ms. Tanvi

What is Automata Theory?

Course Co-ordinator: Ms. Tanvi

(A pioneer of automata theory)

• Father of Modern Computer

• Studied abstract machines called

• Heard of the Turing test?

Course Co-ordinator: Ms. Tanvi

Theory of Computation: A Historical Perspective

1930s • Alan Turing studies Turing machines

1940-1950s • “Finite automata” machines studied

• N. Chomsky, Information and

Course Co-ordinator: Ms. Tanvi

Regular Context- Recursively-

Course Co-ordinator: Ms. Tanvi

2.It takes the string of symbol as input and changes its

3.At the time of transition, the automata can either move

4. Finite automata have two states, Accept

Finite Automata : Examples

• Modeling recognition of the word “then”

Start state Transition Intermediate Final state

Q: finite set of states

∑: finite set of the input symbol

q0: initial state

Course Co-ordinator: Ms. Tanvi

Finite automata can be represented by input tape and finite

Input tape: It is a linear tape having some number of cells.

Finite control: The finite control decides the next state on

Course Co-ordinator: Ms. Tanvi

There are two types of finite automata:

DFA(deterministic finite automata)

NFA(non-deterministic finite automata)

Course Co-ordinator: Ms. Tanvi

Every DFA is NFA, but NFA is not DFA.

There can be multiple final states in both NFA and DFA.

DFA is used in Lexical Analysis in Compiler.

NFA is more of a theoretical concept.

Course Co-ordinator: Ms. Tanvi

An alphabet is a finite, non-empty set of symbols

Course Co-ordinator: Ms. Tanvi

– ∑k = the set of all strings of length k

Course Co-ordinator: Ms. Tanvi

• Length of a string w, denoted by “|w|”, is equal to

– xy = concatentation of two strings x and y

Course Co-ordinator: Ms. Tanvi

2. Explain model of finite automata.

3. Explain tuples of finite automata.

Course Co-ordinator: Ms. Tanvi

Definition: Ø denotes the Empty language

• Let L = {}; Is L=Ø? NO

Course Co-ordinator: Ms. Tanvi

Course Co-ordinator: Ms. Tanvi

Course Co-ordinator: Ms. Tanvi

Example of Formal Language:

• A language is a collection of appropriate string. A language

Course Co-ordinator: Ms. Tanvi

= {a, aa, aaa, abb, abbb, ababb} Infinite

Course Co-ordinator: Ms. Tanvi

Course Co-ordinator: Ms. Tanvi

Equivalence of DFA & NFA

2. Only-if part is trivial:

Course Co-ordinator: Ms. Tanvi

(1) 0 (0) 1 (0 + 1)*