1

Equivalence of NFAs and DFAs

 2

Equivalence of Machines
 For DFAs or NFAs:

 Machine M1 M2
is equivalent to machine

 if

LM1   LM 2 
 3

Example NFA
M1
LM1   {10} * 0

q0 q1 0, 1 q2
1


DFA
M2 0,1
0
LM 2   {10} *
q0 q1 1 q2
1
0
Since
LM1   LM 2   10* 4

machines M1 and M 2are equivalent

0
NFA
M1 q0 q1 0, 1 q2
1


0,1
0
DFA
M2 q0 q1 1 q2
1
0
 5

Equivalence of NFAs and DFAs

Question: NFAs = DFAs ?

Same power?
Accept the same languages?
Equivalence of NFAs and DFAs 6

YES!
Question: NFAs = DFAs ?

Same power?
Accept the same languages?
 7
We will prove:


Languages Languages
accepted accepted
by NFAs by DFAs
 8
We will prove:


Languages Languages
accepted accepted
by NFAs by DFAs

NFAs and DFAs have the same

computation power
 9
Step 1


Languages Languages
accepted accepted
by NFAs by DFAs
 10
Step 1


Languages Languages
accepted accepted
by NFAs by DFAs

Proof: Every DFA is also an NFA

 11
Step 1


Languages Languages
accepted accepted
by NFAs by DFAs

Proof: Every DFA is also an NFA

A language accepted by a DFA

is also accepted by an NFA
 12
Step 2


Languages Languages
accepted accepted
by NFAs by DFAs
 13
Step 2


Languages Languages
accepted accepted
by NFAs by DFAs

Proof: Any NFA can be converted to an

equivalent DFA
 14
Step 2


Languages Languages
accepted accepted
by NFAs by DFAs

Proof: Any NFA can be converted to an

equivalent DFA

A language accepted by an NFA

is also accepted by a DFA
 15

NFA
NFA to DFA a
q0 a q1  q2

b

DFA

q0 
 16

NFA
NFA to DFA a
q0 a q1  q2

b

DFA

q0  a
q1, q2 
 17

NFA
NFA to DFA a
q0 a q1  q2

b

DFA

q0  a
q1, q2 
b


 18

NFA
NFA to DFA a
q0 a q1  q2

b

DFA
a

q0  a
q1, q2 
b


 19

NFA
NFA to DFA a
q0 a q1  q2

b

DFA b a

q0  a
q1, q2 
b


 20

NFA
NFA to DFA a
q0 a q1  q2

b

DFA b a

q0  a
q1, q2 
b

 a, b
 21

NFA
NFA to DFA a
q0 a q1  q2

b
a
DFA b

q0  a
q1, q2 
b

 a, b
 22

NFA to DFA: Remarks

 We are given an NFA

M
 We want to convert it

 to an equivalent DFA

M
 With LM   L(M )
 23

If the NFA has states

q0 , q1, q2 ,...

the DFA has states in the powerset

, q0 , q1, q1, q2 , q3 , q4 , q7 ,....

 24

Single Final State

for NFAs and DFAs
 25

Observation

Any Finite Automaton (NFA or DFA)

can be converted to an equivalent NFA

with a single final state

 Example 26

a NFA

a b

Equivalent NFA
a
a b

b 

 In General 27

NFA

Equivalent NFA


 Single
final state

 Extreme Case 28

NFA without final state

Add a final state

Without transitions
 29


Languages Languages
accepted accepted
by NFAs by DFAs

Regular Languages Regular Languages

Three Views
Three equivalent formal ways to look at what we’re up
to:

Regular Expressions
(a way to code the automata)

Finite State Automata Regular Languages

(machinery for accepting) (the accepted strings)
 Grammars
 A grammar is a set of rules which governs
what is and isn’t syntactically correct.
 English example: “I dog love.”
-- violates English Grammar
 Java example:
public class Wrong{
private int x
}
-- violates Java’s specification (given by a
grammar linked to from links page
www.cs.columbia.edu/~zeph/3261/links.html)
 Grammars
GOAL: Can we automate the process of finding syntactic
errors by magically transforming any grammatical
specification to a computer or computer program? If
so, which of the computer models studied in Automata
Theory is suitable for the purpose?
 Formal Languages
Automata theory: Computers are defined to
act purely on strings and therefore define
certain languages.
Grammars: Grammars generate languages by
admitting a subset of all possible strings.
(most natural )
Algorithms: Instances of algorithmic problems
are viewed as strings by identifying them with
their description. (least natural )
 Formal Languages
Formal languages will act as the glue that
unifies the treatment of all three perspectives.
Pre-definition: A formal language is a set
consisting of strings.
In some sense, one can think of English as consisting
of all valid English words
{a, aback, abacus, abaft, abandon, … }
Probably more accurate to think of English as
consisting of valid English sentences.
Java is the set of all compiling Java programs.
 Chronolgical Overview
Automata, Grammars and Algorithms are
explored simultaneously through four
computational models, progressively more
complicated (though #3 is more powerful
than #4)
1. Finite Automata
2. Pushdown Stack Automata
3. Turing Machines
4. Polynomial-Time Turing Machines
Computation
38

CPU memory
 39

temporary memory

input memory
CPU
output memory

Program memory
 3
Example: f ( x)  x 40

temporary memory

input memory
CPU
output memory
Program memory
compute xx
2
compute x x
 3
f ( x)  x 41

temporary memory
input memory
x2
CPU
output memory
Program memory
compute xx
2
compute x x
 3
temporary memory f ( x)  x 42

z  2*2  4
f ( x)  z * 2  8
input memory
x2
CPU
output memory
Program memory
compute xx
2
compute x x
 3
temporary memory f ( x)  x 43

z  2*2  4
f ( x)  z * 2  8
input memory
x2
CPU
f ( x)  8
Program memory output memory
compute xx
2
compute x x
 Automaton 44

temporary memory

Automaton
input memory
CPU
output memory

Program memory
 Different Kinds of Automata 45

Automata are distinguished by the temporary memory

• Finite Automata: no temporary memory

• Pushdown Automata: stack

• Turing Machines: random access memory

 Finite Automaton 46

temporary memory

input memory
Finite
Automaton
output memory

Vending Machines (small computing power)

 Pushdown Automaton 47

Stack Push, Pop

input memory
Pushdown
Automaton
output memory

Programming Languages (medium computing power)

 Turing Machine 48

Random Access Memory

input memory
Turing
Machine
output memory

Algorithms (highest computing power)

 Power of Automata
49

Finite Pushdown Turing

Automata Automata Machine
Noam Chomsky
1928 –

Chomsky Hierarchy
Generative Grammar

Most Cited person alive!

According to a 1992 tabulation

of sources from the previous 12
years in the Arts and Humanities
Citation Index, Chomsky was the
most frequently-cited person
alive, and one of the eight most
frequently-cited authors of all
time.
 Chomsky Hierarchy

All

REC = computed by TM

Context Free = computed by PDA

Regular = computed by FA
 52

Languages
A language is a set of strings

String: A sequence of letters

Examples: “cat”, “dog”, “house”, …

Defined over an alphabet:

  a, b, c, , z
 53

Alphabets and Strings   a, b

We will use small alphabets:
u  ab
Strings a v  bbbaaa
ab w  abba
abba
baba
aaabbbaabab
 54

String Operations
w  a1a2  an abba
v  b1b2 bm bbbaaa

Concatenation

wv  a1a2  anb1b2 bm abbabbbaaa

 55

w  a1a2  an ababaaabbb

Reverse

R
w  an  a2 a1 bbbaaababa
 56

String Length
w  a1a2  an
w n
Length:

abba  4
Examples: aa  2
a 1
 57

Empty String 
A string with no letters:

 0
Observations:

w  w  w

abba  abba  abba

 58

Substring
Substring of string:
a subsequence of consecutive
characters
abbab ab
 String Substring
abbab abba
abbab b
abbab bbab
 59

Prefix and Suffix

Prefixes Suffixes
abbab
 abbab w  uv
a bbab prefix

ab bab suffix

abb ab
abba b
abbab 
 60

Another Operation
n
w  ww
 
w
n
abba   abbaabba
2

Example:

0
w 

abba   
Definition: 0

 61

The * Operation
 *: the set of all possible strings from
  alphabet

  a, b
*   , a, b, aa, ab, ba, bb, aaa, aab,


 The + Operation 62


 : the set of all possible strings from
alphabet except



  a, b
*   , a, b, aa, ab, ba, bb, aaa, aab,


   * 

  a, b, aa, ab, ba, bb, aaa, aab,
 63
A language is any subset of
Language *
Example:
  a, b
*   , a, b, aa, ab, ba, bb, aaa,

Languages:


a, aa, aab
{ , abba, baba, aa, ab, aaaaaa}
 64

Another Example
n n
L  {a b : n  0}
An infinite language


ab
L abb  L
aabb
aaaaabbbbb
 65

Operations on Languages
The usual set operations

a, ab, aaaabb, ab  {a, ab, bb, aaaa}

a, ab, aaaabb, ab  {ab}
a, ab, aaaa bb, ab  a, aaaa
L   * L

Complement:

a, ba   , b, aa, ab, bb, aaa,

 66

Reverse
R R
L  {w : w  L}
Definition:
ab, aab, baba  ba, baa, abab
R

Examples:
n n
L  {a b : n  0}

R n n
L  {b a : n  0}
 67

Concatenation
L1L2  xy : x  L1, y  L2 
Definition:

a, ab, bab, aa

Example:

 ab, aaa, abb, abaa, bab, baaa

 68

Another Operation
n
L  LL L
Definition:
n
a, b  a, ba, ba, b 
3

aaa, aab, aba, abb, baa, bab, bba, bbb

0
L  
Special case:
a , bba , aaa 0  
 69

More Examples
n n
 L  {a b : n  0}

2 n n m m
L  {a b a b : n, m  0}

2
aabbaaabbb  L
 70

Star-Closure (Kleene *)

0 1 2
 Definition: L*  L  L  L 

 Example: , 
a, bb, 
 
a, bb*   
 aa , abb , bba , bbbb , 

aaa, aabb, abba, abbbb,
 71

Positive Closure

 1 2
Definition: L  L  L 
 L *  

a, bb, 
  
a, bb  aa, abb, bba, bbbb, 
aaa, aabb, abba, abbbb,
 
 72

Regular Language
 The set of regular languages over an alphabet Σ is defined
recursively as below. Any language belonging to this set is a
regular language over .
 Definition of Set of Regular Languages :
 Basis Clause: , {} and {a} for any symbol a Σ are regular languages.
 Inductive Clause: If Lr and Ls are regular languages, then Lr Ls , LrLs
and Lr* are regular languages.
 Extremal Clause: Nothing is a regular language unless it is obtained
from the above two clauses.

 For example, let Σ = {a, b}. Then since {a} and {b} are regular
languages, {a, b} ( = {a} {b} ) and {ab} ( = {a}{b} ) are regular
languages. Also since {a} is regular, {a}* is a regular language
which is the set of strings consisting of a's such as , a, aa, aaa,
aaaa etc. Note also that Σ*, which is the set of strings consisting
of a's and b's, is a regular language because {a, b} is regular.
 73

Example
LM   abba M

a, b

q5
b a a, b
a b
q0 a q1 b q2 b q3 a q4
accept
 74

Another Example
LM    , ab, abba M

a, b

q5
b a a a, b
b
q0 a q1 b q2 b q3 a q4
accept accept accept
 75

Formally
For a DFA M  Q, ,  , q , F 
0

Language accepted by M

 LM   w  * :  * q0 , w F 

alphabet
transition initial final
function state states
Observation
76

 Language accepted by : M

LM   w  * :  * q0 , w F 

Language rejected by :M

LM   w  * :  * q0 , w F 

 77


More Examples
LM   {a b : n  0}
n

a a, b

q0 b q1 a, b q2
accept trap state
 78

LM  = { all substrings with prefix ab }

 a, b

q0 a q1 b q2

b a accept

q3 a, b
 79

L M  = { all strings without

001
substring }


1 0 0,1
1

 0 1
0 00 001

0
 80

Regular Languages

L is regular if there is
 A language

 a DFAM such that L  LM 

 All regular languages form a language family

 81

Example
The language L  awa : w  a, b*
is regular:
a
b

b
q0 a q2 q3

b a
q4

a, b
 82

Some Properties of
Regular Languages
 83

Properties
L1 L2
For regular languages
we will prove that:
and

Union:
L1  L2
Are regular
Concatenation:
L1L2 Languages

Star:
L1 *
 84

We Say:
Regular languages are closed under

Union:
L1  L2

Concatenation:
L1L2

Star:
L1 *
 85

Regular language
L1 Regular language
L2

LM1   L1 LM 2   L2
NFA
M1 NFA
M2

Single final state Single final state

 86

Example M1
a
n
L1  {a b} b

M2
a
L2  ba b
 87

UnionL1  L2
NFA for
M1

 

 M2 
 88

Example n
NFA for
L1  L2  {a b}  {ba}
 n
L1  {a b}
a
b
 

 L2  {ba} 
b a
 89

Concatenation
L1L2
NFA for

M1 M2
 
 90

Example
n n

L1L2  {a b}{ba}  {a bba}
NFA for

n
L1  {a b}
a L2  {ba}
b  b a 
 91

Star Operation
L1 *

NFA for
  L1 *
M1
 


 92

Example
n
L1*  {a b} *
NFA for 
n
L1  {a b}
a
 b 


Regular Expressions
 94

Example
r  a  b * a  bb 
Regular expression

Lr   a, bb, aa, abb, ba, bbb,...

 95

Example
r  aa * bb * b
Regular expression

Lr   {a b 2n 2m
b : n, m  0}
 96

Example
r  (0  1) * 00 (0  1) *
Regular expression

L(r ) = { all strings with at least

two consecutive 0 }
 97

Example
r  (1  01) * (0   )
Regular expression

L(r ) = { all strings without

two consecutive 0 }
 98

Equivalent Regular Expressions

Definition:

 Regular expressions r1 and r2

 equivalent if
are
L(r1 )  L(r2 )
 99

Example
L = { all strings with at least
two consecutive 0 }


r1  (1  01) * (0   )
r2  (1* 011*) * (0   )  1* (0   )

r1 and
r2
L(r1 )  L(r2 )  L are equivalent
regular expr.
 100

Regular Expressions
and
Regular Languages
 101

Theorem


Languages
Generated by Regular
Regular Expressions Languages
 Theorem - Part 1 102


Languages
Generated by Regular
Regular Expressions Languages

r
1. For any regular expression
the language is regular
L(r )
 Theorem - Part 2 103


Languages
Generated by Regular
Regular Expressions Languages

2. For any regular language L

there is
a regular expression
r
with
L(r )  L
 104

Proof - Part 1
1. For any regular expression r
the language is regular
L(r )

Proof by induction on the size of

r
 105

Induction Basis ,  , 
NFAs
Primitive Regular Expressions:

L( M1 )    L()
regular
L( M 2 )  {}  L( ) languages

L( M 3 )  {a}  L(a )
 106

Inductive Hypothesis

r1 r2
Assume

for regular expressions and

L(r1 ) L(r2 )
that

 and are regular languages


 107

Inductive Step
Lr1  r2 
We will prove:

Lr1  r2 
Are regular
Languages

Lr1 *

Lr1 
By definition of regular expressions: 108

Lr1  r2   Lr1   Lr2 

Lr1  r2   Lr1  Lr2 

Lr1 *  Lr1 *

Lr1   Lr1 
 By inductive hypothesis we know: 109

and are regular languages

L(r1 ) L(r2 )

We also know:

Regular languages are closed under

union
Lr1   Lr2 
concatenation
Lr1  Lr2 
star
Lr1 *
 110

 Therefore:
Lr1  r2   Lr1   Lr2 

Are regular
Lr1  r2   Lr1  Lr2  languages

Lr1 *  Lr1 *

 111

And trivially:

L((r1 )) is a regular language


 112

Proof – Part 2
2. For any regular language L
there is
a regular expression
r
with
L(r )  L

Proof by construction of regular expression

 113

L is regular take the

Since

NFA
M that accepts it

L( M )  L

Single final state

 114

From M construct the equivalent

Generalized Transition Graph
 transition labels
 are regular expressions
Example:

M
a c a c
a, b ab
 115

b b
a
Another Example: q0 q1 a, b q2
b

b b
a
q0 q1 a  b q2
b
 116

b b
a
Reducing the states: q0 q1 a  b q2
b

bb * a b

q0 bb * (a  b) q2
 117

Resulting Regular Expression:

bb * a b

q0 bb * (a  b) q2

r  (bb * a ) * bb * (a  b)b *

L(r )  L( M )  L
 118

In General e
d c
Removing states:
qi q qj
a b

ae * d ce * b
ce * d
qi qj
ae * b
 119

Obtaining the final regular expression:

r1 r4
r3
q0 qf
r2

r  r1 * r2 (r4  r3r1 * r2 ) *

L(r )  L( M )  L
 120

More Properties
of
Regular Languages
 121

We have proven
Regular languages are closed under:

Union

Concatenation

Star operation

Reverse
 122

Namely, for regular languages and

L1 : L2
Union
L1  L2
Concatenation

Star operation
L1L2 Regular
Reverse Languages


L1
R
L1
 123

We will proveare closed under:

Regular languages

Complement

Intersection
 124

Namely, for regular languages and

L1: L2

Complement L1
Regular
Intersection Languages

L1  L2
 125

ComplementFor regular language

Theorem:
the complement
L
is regular

Proof: Take DFA that accepts

• nonfinal states final L
and make

• final states nonfinal

Resulting DFA accepts

L
 L  L ( a * b)
Example: 126

a a, b

q0 b q1 a, b q2

L  L(a *  a * b(a  b)(a  b)*)

a a, b

q0 b q1 a, b q2
 127

Intersection For regular languages

Theorem:
the intersection
L1
and
is regular
L2
L1  L2

Proof: Apply DeMorgan’s Law:

L1  L2  L1  L2
L1 , L2
128
regular

L1 , L2 regular

L1  L2 regular

L1  L2 regular

L1  L2 regular
 129

Standard Representations
of
Regular Languages
Standard Representations 130

of Regular Languages
Regular Languages

DFAs
Regular
Grammars

NFAs
Regular
Expressions
 131

Elementary Questions

about

Regular Languages
 132

Membership Question
Question:
L
Given regular language
w
and string
how can we check if ?

w L

Answer: L
Take the DFA that accepts
w
and check if is accepted
 133
DFA

w
w L

DFA

w
w L
 134
Question: Given regular language
how can we check L
if is empty: ?

L ( L  )

Answer: Take the DFA that accepts

L
Check if there is a path from
the initial state to a final state
 135
DFA

L

DFA

L
 136
Question: Given regular language
how can we check L
if is finite?

Answer: Take the DFA that accepts

L
Check if there is a walk with cycle
from the initial state to a final state
DFA 137

L is infinite

DFA

L is finite
 138
Question: Given regular languages
how can we check if
and
?
L1 L2
L1  L2

Answer: Find if
( L1  L2 )  ( L1  L2 )  
 ( L1  L2 )  ( L1  L2 )   139

L1  L2   and
L1  L2  

L1 L2 L L2 L1 L1
2
L1  L2 L2  L1

L1  L2
 ( L1  L2 )  ( L1  L2 )   140

L1  L2   or
L1  L2  

L1 L2 L2 L1

L1  L2 L2  L1

L1  L2
 141

Elementary Questions

about

Regular Languages
 142

Non-regular languages
 143
n n
Non-regular languages
{a b : n  0}
R
{ww : w {a, b}*}

Regular languages

a *b b*c  a
b  c ( a  b) *
etc...
 144

Theorem: The language

regular or not?

Proof: Use a DFA

 is not regular?
L
How can we prove that a language 145

L
Prove that there is no DFA that accepts

Problem: this is not easy to prove

Solution: the Pumping Lemma !!!

 146

The Pigeonhole Principle

 pigeons
4 147

3 pigeonholes
 148

A pigeonhole must
contain at least two pigeons

 149

n pigeons


...........

m pigeonholes
nm
...........
 150

The Pigeonhole Principle

n pigeons


m pigeonholes
There is a pigeonhole
with at least 2 pigeons
nm

...........
The Pigeonhole Principle 151

and

DFAs
 152

 4DFA with states

b
b b

a b b
q1 q2 q3 q4
a a
 In walks of strings:
a no state
is repeated
153

aa
 aab

b
b b

a a b
q1 q2 q3 q4
a a
 aabb
154
In walks of strings: a state
is repeated

bbaa

abbabb
abbbabbabb...
b
b b

a a b
q1 q2 q3 q4
a a
 w
If the walk of string has length
155

| w|  4
then a state is repeated

b
b b

a a b
q1 q2 q3 q4
a a
 156

Pigeonhole principle for any DFA:

w
If in a walk of a string
 transitions states of DFA
then a state is repeated

b
b b

a a b
q1 q2 q3 q4
a a
 w
In other words for a string : 157

transitions are pigeons

states are pigeonholes

a
q

b
b b

a a b
q1 q2 q3 q4
a a
 158
In general:

w 
A string has length number of states

q A state w
must be repeated in the walk

walk of
w
...... ......
q
 159

The Pumping Lemma

 160

L
Take an infinite regular language

DFA that accepts L

m
states
 161

w
Take string
w
with
L

w
There is a walk with label :

.........

walk
w
 162

wIf string | w|  m
has length number
of states

then, from the pigeonhole principle:

q a state w
is repeated in the walk

......
q ......

walk
w
 163

Write w x y z

......
q ......

x z
 length
|x y|  m number 164
of states
Observations:

length
| y | 1

......
q ......

x z
 165

Observation:The string xz is accepted

......
q ......

x z
 166

Observation:
xyyz
The string
is accepted

......
q ......

x z
 167

Observation: xyyyz
The string
is accepted

......
q ......

x z
 168

i
xy z
The string
In General: is accepted
i  0, 1, 2, ...
y

......
q ......

x z
 169

In other words, we described:

The Pumping Lemma !!!

 The Pumping Lemma:
170

• Given a infinite regular language

L
• there exists an integer
m
w L | w| m
• for any string with length

• we can write
w x y z
h
|x y|  m and
| y | 1
i i  0, 1, 2, ...
• such that:
xy z  L
 171

Applications

of

the Pumping Lemma

 172

n n
Theorem: L  {a b : n  0}
The language

is not regular

Proof: Use the Pumping Lemma

 173

n n
L  {a b : n  0}
Assume for contradiction
that is a regular language
L

SinceL is infinite
we can apply the Pumping Lemma
 174
n n
L  {a b : n  0}
m Let be the integer in the Pumping Lemma

w
Pick a string such that: w L
length

| w| m

m m
Example: pick
wa b
 m m
a b xyz
175

Write:

From the Pumping Lemma

| x y |  m, | y | 1
it must be that: length

m m
m m
Therefore:
a b  a...aa...a...ab...b
x y z
k
y  a , k 1
 176

m m
x y za b
We have:
k
y  a , k 1
From the Pumping Lemma:
i
xy z  L
i  0, 1, 2, ...

Thus:
2
xy z  L
2 m k m
xy z  xyyz  a b  L
 177

Therefore: m k m
a b L
n n
BUT: L  {a b : n  0}

m k m
a b L

CONTRADICTION!!!
 178

Therefore: L
Our assumption that
is a regular language is not true

Conclusion:

L is not a regular language

 179

n n
Non-regular languages
{a b : n  0}

Regular languages

a *b b*c  a
b  c ( a  b) *
etc...
 180

Is the language

regular or not?

Use the Pumping Lemma

 181

Theorem: The language

regular or not?

Proof: Use a DFA

 182

Theorem: n!
L  {a : n  0}
The language

is not regular

n!  1  2  (n  1)  n

Proof: Use the Pumping Lemma

 183

The language
Theorem:
  {a, b}
is not regular

Proof:
Use the Pumping Lemma
 184

The language
Theorem:
R
L  {ww : w  *}   {a, b}
is not regular

Proof:
Use the Pumping Lemma
 185

The language

Theorem:
is not regular

Proof: Use the Pumping Lemma

 186

The language

n l n l
Theorem: L  {a b c : n, l  0}
is not regular

Proof: Use the Pumping Lemma

 187

Is the language

regular or not?

Use the Pumping Lemma

 188

Is the language

regular or not?

Use the Pumping Lemma

 189

Theorem: The language

regular or not?

Proof: Use a DFA

 190

Theorem: The language

is not regular

Proof: Use the Pumping Lemma

 191

R
n n
{a b } {ww }

Regular Languages
 192

 Context-Free Languages

R
n n
{a b } {ww }

Regular Languages
 193

Context-Free Languages

Context-Free Pushdown
Grammars Automata

stack

automaton