النظرية الاحتسابية

Models of Computation
Copyright 2011@Xu Dezhi 1

Computation
CPU memory

temporary memory
input memory
CPU
output memory
Program memory

Example: f ( x)  x 3
temporary memory
input memory
CPU
output memory
Program memory
compute xx
compute x x
2
4
Copyright 2011@Xu Dezhi
f ( x)  x 3
temporary memory
input memory
x2
CPU
output memory
Program memory
compute xx
compute x x
2
temporary memory f ( x)  x 3
z  2*2  4
f ( x)  z * 2  8
input memory
x2
CPU
output memory
Program memory
compute xx
compute x x
2

temporary memory f ( x)  x 3
z  2*2  4
f ( x)  z * 2  8
input memory
x2
CPU
f ( x)  8
Program memory output memory
compute xx
compute x x
2

Automaton
temporary memory
Automaton
input memory
CPU
output memory
Program memory

Different Kinds of Automata
Automata are distinguished by the temporary memory
• Finite Automata: no temporary memory
• Pushdown Automata: stack
• Turing Machines: random access memory

Finite Automaton
temporary memory
input memory
Finite
Automaton
output memory
Example: Vending Machines

(small computing power)
Pushdown Automaton
Stack Push, Pop
input memory
Pushdown
Automaton
output memory
Example: Compilers for Programming Languages

(medium computing power)
Turing Machine
Random Access Memory
input memory
Turing
Machine
output memory
Examples: Any Algorithm

(highest computing power)
Power of Automata
Finite Pushdown Turing

Automata Automata Machine
Less power More power

Solve more
computational problems
The End

Mathematical Preliminaries
1
Mathematical Preliminaries
• Sets
• Functions
• Relations
• Graphs
• Proof Techniques
2
SETS
A set is a collection of elements
A  {1, 2, 3}
B  {train, bus, bicycle, airplane}
We write
1 A
ship  B
3
Set Representations
C = { a, b, c, d, e, f, g, h, i, j, k }
C = { a, b, …, k } finite set
S = { 2, 4, 6, … } infinite set
S = { j : j > 0, and j = 2k for some k>0 }
S = { j : j is nonnegative and even }
4
A = { 1, 2, 3, 4, 5 }
U
6 A
2 3 8
1
7 4 5
9
10
Universal Set: all possible elements
U = { 1 , … , 10 }
5
Set Operations
A = { 1, 2, 3 } B = { 2, 3, 4, 5}
A B
• Union
2 4
1
A U B = { 1, 2, 3, 4, 5 } 3 5
• Intersection
U
A B = { 2, 3 } 2
3
• Difference
A-B={1}
1
B - A = { 4, 5 }
Venn diagrams
6
• Complement
Universal set = {1, …, 7}
A = { 1, 2, 3 } A = { 4, 5, 6, 7}
4
A
A 3 6
1
2
5 7
A=A
7
{ even integers } = { odd integers }
Integers
1 odd
even
6 5
2
0
4
3 7
8
DeMorgan‟s Laws
AUB=A B
U
A B=AUB
U
9
Empty, Null Set:
={}
SU =S
S =
U
= Universal Set
S- =S
-S=
10
Subset
A = { 1, 2, 3} B = { 1, 2, 3, 4, 5 }
A B
U
Proper Subset: A B
U
B
A
11
Disjoint Sets
A = { 1, 2, 3 } B = { 5, 6}
A B=
U
A B
12
Set Cardinality
• For finite sets
A = { 2, 5, 7 }
|A| = 3
(set size)
13
Powersets
A powerset is a set of sets
S = { a, b, c }
Powerset of S = the set of all the subsets of S
2S = { , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} }
Observation: | 2S | = 2|S| ( 8 = 23 )
14
Cartesian Product
A = { 2, 4 } B = { 2, 3, 5 }
A X B = { (2, 2), (2, 3), (2, 5), ( 4, 2), (4, 3), (4, 5) }
|A X B| = |A| |B|
Generalizes to more than two sets
AXBX…XZ
15
Functions
domain range
4 A B
f(1) = a a
1
2 b
3 c
5
f : A -> B
If A = domain
then f is a total function
otherwise f is a partial function
16
Relations
R = {(x1, y1), (x2, y2), (x3, y3), …}
xi R yi
e. g. if R = „>‟: 2 > 1, 3 > 2, 3 > 1
17
Equivalence Relations
• Reflexive: xRx
• Symmetric: xRy yRx
• Transitive: x R y and y R z xRz
Example: R = „=„
•x=x
•x=y y=x
• x = y and y = z x=z
18
Equivalence Classes
For equivalence relation R
equivalence class of x = {y : x R y}
Example:
R = { (1, 1), (2, 2), (1, 2), (2, 1),
(3, 3), (4, 4), (3, 4), (4, 3) }
Equivalence class of 1 = {1, 2}

Equivalence class of 3 = {3, 4}
19
Graphs
A directed graph
e
b
node
a d
c
• Nodes (Vertices)
V = { a, b, c, d, e }
• Edges
E = { (a,b), (b,c), (b,e),(c,a), (c,e), (d,c), (e,b), (e,d) }
20
Labeled Graph
2
6 e
b 2
1 3
a 6 d
5
c
21
Walk
e
b
a d
Walk is a sequence of adjacent edges

(e, d), (d, c), (c, a)
22
Path
e
b
a d
Path is a walk where no edge is repeated
Simple path: no node is repeated

23
Cycle
base e
b
3
a 1 d
2
c
Cycle: a walk from a node (base) to itself
Simple cycle: only the base node is repeated
24
Euler Tour
8 base
7 e
b 1
4 6
a 5 2 d
3
c
A cycle that contains each edge once
25
Hamiltonian Cycle
5 base
e
b 1
4
a 2 d
3
c
A simple cycle that contains all nodes
26
Finding All Simple Paths
e
b
a d
c
origin
27
Step 1
e
b
a d
c
origin
(c, a)
(c, e)
28
Step 2
e
b
a d
(c, a) c
origin
(c, a), (a, b)
(c, e)
(c, e), (e, b)
(c, e), (e, d)
29
Step 3
e
b
a d
(c, a) c
origin
(c, a), (a, b)
(c, a), (a, b), (b, e)
(c, e)
(c, e), (e, b)
(c, e), (e, d)
30
Step 4
e
b
a d
(c, a)
(c, a), (a, b) c
origin
(c, a), (a, b), (b, e)
(c, a), (a, b), (b, e), (e,d)
(c, e)
(c, e), (e, b)
(c, e), (e, d)
31
Trees
root
parent
leaf
child
Trees have no cycles
32
root
Level 0
Level 1
leaf Height 3
Level 2
Level 3
33
Binary Trees
34
Proof Techniques
• Proof by induction
• Proof by contradiction
•Proof by construction
35
Induction
We have statements P1, P2, P3, …
If we know
• for some b that P1, P2, …, Pb are true
• for any k >= b that
P1, P2, …, Pk imply Pk+1
Then
Every Pi is true
36
Proof by Induction
• Inductive basis
Find P1, P2, …, Pb which are true
• Inductive hypothesis
Let‟s assume P1, P2, …, Pk are true,
for any k >= b
• Inductive step
Show that Pk+1 is true
37
Example
Theorem: A binary tree of height n
has at most 2n leaves.
Proof by induction:
let L(i) be the maximum number of
leaves of any subtree at height i
38
We want to show: L(i) <= 2i
• Inductive basis
L(0) = 1 (the root node)
• Inductive hypothesis
Let‟s assume L(i) <= 2i for all i = 0, 1, …, k
• Induction step
we need to show that L(k + 1) <= 2k+1
39
Induction Step
height
k
k+1
From Inductive hypothesis: L(k) <= 2k
40
Induction Step
height
k L(k) <= 2k
k+1
L(k+1) <= 2 * L(k) <= 2 * 2k = 2k+1
(we add at most two nodes for every leaf of level k)
41
Remark
Recursion is another thing
Example of recursive function:
f(n) = f(n-1) + f(n-2)
f(0) = 1, f(1) = 1
42
Proof by Contradiction
We want to prove that a statement P is true
• we assume that P is false

• then we arrive at an incorrect conclusion
• therefore, statement P must be true
43
Example
Theorem: 2 is not rational
Proof:
Assume by contradiction that it is rational
2 = n/m
n and m have no common factors
We will show that this is impossible
44
2 = n/m 2 m2 = n2
n is even
Therefore, n2 is even
n=2k
m is even
2 m2 = 4k2 m2 = 2k2
m=2p
Thus, m and n have common factor 2
Contradiction!
45
Proof by Construction
We want to prove that a statement
about something with a property is
true
– constructing a concrete example with a
property to show that something having
that property exists.
– constructive proof is in contrast to a
non-constructive proof which does not
provide a means of constructing an
example.
46
Example 1
16 can be exactly divided.
Proof
– A concrete example is 16/2. Therefore,

the statement is true.
End
47
Example 2
There exist two irrational numbers which make ab
rational.
Proof
Let a=b= 2
case 1: is rational. done, otherwise

2
2
2
case 2: let a= 2 , then ( 2 )  2  2 , done.
2 2 2
End
48
Question
Is example 2 the constructive proof?
Why if yes? Why if no?
49
Example 3
Show that there is no "largest integer“.
Proof
Let n be any integer.
Let m = n+1
m is an integer
m>n
Therefore m is an integer that is larger than n
Therefore, for any integer there exists an
integer m = n + 1 that is larger than it.
End
50
Question
51
Example 4
Show that there is no "largest" prime number.
Proof
Let n be any prime number
Let m = n! + 1, then m>n
Case 1:
m = n! + 1 is a prime number, then we have
constructed a prime number that is larger than
the previous prime number.
Case 2:
m = n! + 1 is not a prime number, then it has at
least one prime factor
52
Example 4 (Cont.)
Explanation:
If you divide m by any of the prime numbers that

are smaller than or equal to n, you will always get
a remainder of 1,
because each prime number less than or equal to

n divides evenly into n!.
Therefore any prime factors of m must be

greater than n.
End
53
Question
54
Languages
55
A language is a set of strings
String: A sequence of letters
Examples: “cat”, “dog”, “house”, …
Defined over an alphabet:

  a, b, c,, z
56
Alphabets and Strings
We will use small alphabets:   a, b
Strings
a
ab u  ab
abba v  bbbaaa
baba w  abba
aaabbbaabab
57
String Operations
w  a1a2  an abba
bbbaaa
v  b1b2 bm
Concatenation
wv  a1a2 anb1b2 bm abbabbbaaa
58
w  a1a2 an ababaaabbb
Reverse
w  an a2a1
R
bbbaaababa
59
String Length
w  a1a2 an
Length: w n
Examples: abba  4
aa  2
a 1
60
Length of Concatenation
uv  u  v
Example: u  aab, u  3
v  abaab, v  5
uv  aababaab  8
uv  u  v  3  5  8
61
Empty String
A string with no letters: 
Observations:  0
w  w  w
abba  abba  abba

62
Substring
Substring of string:
a subsequence of consecutive characters
String Substring
abbab ab
abbab abba
abbab b
abbab bbab
63
Prefix and Suffix
abbab
Prefixes Suffixes
 abbab w  uv
a bbab
prefix
ab bab
suffix
abb ab
abba b
abbab 
64
Another Operation
w  ww

n
 
w
n
Example: abba   abbaabba

2
Definition: w 
0
abba   
0
65
The * Operation
 * : the set of all possible strings from
alphabet 
  a, b
*   , a, b, aa, ab, ba, bb, aaa, aab,
66
The + Operation
 : the set of all possible strings from

alphabet  except 
  a, b
*   , a, b, aa, ab, ba, bb, aaa, aab,

   * 

  a, b, aa, ab, ba, bb, aaa, aab,
67
Languages
A language is any subset of *
Example:   a, b
*   , a, b, aa, ab, ba, bb, aaa,
Languages: 
a, aa, aab
{ , abba, baba, aa, ab, aaaaaa}
68
Note that:
Sets   { }  {}
Set size {}    0
Set size {}  1

String length  0
69
Another Example
An infinite language L  {a b : n  0}
n n

ab
L abb  L
aabb
aaaaabbbbb
70
Operations on Languages
The usual set operations
a, ab, aaaa  bb, ab  {a, ab, bb, aaaa}

a, ab, aaaa  bb, ab  {ab}
a, ab, aaaa  bb, ab  a, aaaa
Complement: L   * L
a, ba   , b, aa, ab, bb, aaa,

71
Reverse
Definition: L  {w : w  L}
R R
Examples: ab, aab, baba  ba, baa, abab

R
L  {a b : n  0}
n n
L  {b a : n  0}
R n n
72
Concatenation
Definition: L1L2  xy : x  L1, y  L2 
Example: a, ab, bab, aa
 ab, aaa, abb, abaa, bab, baaa
73
Another Operation
Definition: L 
n

LL L
n
a, b  a, ba, ba, b 

3
aaa, aab, aba, abb, baa, bab, bba, bbb

0
Special case: L  
a , bba , aaa 0  

74
More Examples
L  {a b : n  0}
n n
L  {a b a b : n, m  0}
2 n n m m
aabbaaabbb  L 2
75
Star-Closure (Kleene *)
Definition: L*  L  L  L 
0 1 2
Example:
 , 
a, bb, 
 
a, bb*   
 aa , abb , bba , bbbb, 
aaa, aabb, abba, abbbb,
76
Positive Closure
Definition: 
L  L  L 
1 2
 L * 
a, bb, 
  
a, bb  aa, abb, bba, bbbb, 
aaa, aabb, abba, abbbb,
 
77
The End
78
Finite Automata
1
Finite Automaton
Input
String
Output
Finite String
Automaton
2
Finite Accepter
Input
String
Output
“Accept”
Finite
or
Automaton
“Reject”
3
Transition Graph
Abba -Finite Accepter a, b
q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4
initial final
state state
transition
state “accept”
4
Initial Configuration
Input String
a b b a
a, b
q5
a a, b
b a b
5
Reading the Input
a b b a
a, b
q5
a a, b
b a b
6
a b b a
a, b
q5
a a, b
b a b
7
a b b a
a, b
q5
a a, b
b a b
8
a b b a
a, b
q5
a a, b
b a b
9
Input finished
a b b a
a, b
q5
a a, b
b a b
Output: “accept”
10
Rejection
a b a
a, b
q5
a a, b
b a b
11
a b a
a, b
q5
a a, b
b a b
12
a b a
a, b
q5
a a, b
b a b
13
a b a
a, b
q5
a a, b
b a b
14
Input finished
a b a
a, b
Output:
q5 “reject”
a a, b
b a b
15
Another Rejection
a, b
q5
a a, b
b a b
16

a, b
q5
a a, b
b a b
Output:
“reject” 17
Another Example
a a b
a a, b
q0 b q1 a, b q2
18
a a b
a a, b
q0 b q1 a, b q2
19
a a b
a a, b
q0 b q1 a, b q2
20
a a b
a a, b
q0 b q1 a, b q2
21
Input finished
a a b
a a, b
Output: “accept”
q0 b q1 a, b q2
22
Rejection
b a b
a a, b
q0 b q1 a, b q2
23
b a b
a a, b
q0 b q1 a, b q2
24
b a b
a a, b
q0 b q1 a, b q2
25
b a b
a a, b
q0 b q1 a, b q2
26
Input finished
b a b
a a, b
q0 b q1 a, b q2
Output: “reject”
27
Formalities
Deterministic Finite Accepter (DFA)
M  Q, , , q0 , F 
Q : set of states
 : input alphabet
 : transition function
q0 : initial state
F : set of final states

28
Input Alphabet 
  a, b
a, b
q5
b a a, b
a b
29
Set of States Q
Q  q0 , q1, q2 , q3 , q4 , q5
a, b
q5
a a, b
b a b
30
Initial State q0
a, b
q5
a a, b
b a b
31
Set of Final States F
F  q4 
a, b
q5
a a, b
b a b
32
Transition Function 
 :Q  Q
a, b
q5
b a a, b
a b
33
 q0 , a   q1
a, b
q5
b a a, b
a b
34
 q0 , b   q5
a, b
q5
b a a, b
a b
35
 q2 , b   q3
a, b
q5
b a a, b
a b
36
 a b
q0 q1 q5
q1 q5 q2
q2 q5 q3
q3 q4 q5 a, b
q4 q5 q5
q5 q5 q5 q5
b a a, b
a b
37
Extended Transition Function  *
 * : Q  *  Q
a, b
q5
b a a, b
a b
38
 * q0 , ab   q2
a, b
q5
b a a, b
a b
39
 * q0 , abba   q4
a, b
q5
b a a, b
a b
40
 * q0 , abbbaa   q5
a, b
q5
b a a, b
a b
41
Observation: There is a walk from q to q
with label w
 * q, w  q
q w q
w  1 2  k
1 2 k
q q
42
Example: There is a walk from q0 to q5
with label abbbaa
 * q0 , abbbaa   q5
a, b
q5
b a a, b
a b
43
Recursive Definition
 * q ,    q
 * q, w    ( * (q, w), )
q w q1  q
 *  q , w   q
 * q, w    (q1, )
 (q1, )  q
 * q, w    ( * (q, w), )
 * q, w  q1
44
 * q0 , ab  
  * (q0 , a ), b  
   * q0 ,  , a , b  
  q0 , a , b  
 q1 , b  
q2 a, b
q5
b a a, b
a b
45
Languages Accepted by DFAs
Take DFA M
Definition:
The language LM  contains
all input strings accepted by M
LM  = { strings that drive M to a final state}
46
Example
LM   abba M
a, b
q5
b a a, b
a b
accept
47
Another Example
LM    , ab, abba M
a, b
q5
b a a a, b
b
accept accept accept
48
Formally
For a DFA M  Q, , , q0 , F 
Language accepted by M :
LM   w  * :  * q0 , w  F 
q0 w q q  F
49
Observation
Language rejected by M :
LM   w  * :  * q0 , w  F 
q0 w q q  F
50
More Examples
LM   {a b : n  0}
n
a a, b
q0 b q1 a, b q2
accept trap state
51
LM  = { all strings with prefix ab }
a, b
q0 a q1 b q2
b a accept
q3 a, b
52
LM  = { all strings without
substring 001 }
1 0 0,1
1
 0 1
0 00 001
0
53
Regular Languages
A language L is regular if there is

a DFA M such that L  LM 
All regular languages form a language family
54
Examples of regular languages:
abba  , ab, abba {a b : n  0}

n
{ all strings with prefix ab }
{ all strings without substring 001 }
There exist automata that accept these

Languages (see previous slides).
55
Another Example
The language L  awa : w  a, b*
is regular:
a
b
L  L M  b
q0 a q2 q3
b a
q4
a, b
56
There exist languages which are not Regular:
Example: L {a b : n  0}
n n
There is no DFA that accepts such a language
(we will prove this later in the class)
57
The End
58
Non Deterministic Automata
1
Nondeterministic Finite Accepter (NFA)
Alphabet = {a}
q1 a q2
a
q0
a
q3
2
Alphabet = {a}
Two choices q1 a q2
a
q0
a
q3
3
Alphabet = {a}
Two choices q1 a q2 No transition

a
q0
a
q3 No transition
4
First Choice
a a
q1 a q2
a
q0
a
q3
5
First Choice
a a
q1 a q2
a
q0
a
q3
6
First Choice
a a
q1 a q2
a
q0
a
q3
7
First Choice
a a
All input is consumed
q1 a q2 “accept”
a
q0
a
q3
8
Second Choice
a a
q1 a q2
a
q0
a
q3
9
Second Choice
a a
q1 a q2
a
q0
a
q3
10
Second Choice
a a
q1 a q2
a
q0
a
No transition:
q3
the automaton hangs
11
Second Choice
a a
Input cannot be consumed
q1 a q2
a
q0
a
q3 “reject”
12
An NFA accepts a string:
when there is a computation of the NFA
that accepts the string
AND
all the input is consumed and the automaton
is in a final state
13
Example
aa is accepted by the NFA:
“accept”
q1 a q2 q1 a q2
a a
q0 q0
a a
q3 q3 “reject”
because this
computation
accepts aa
14
Rejection example
q1 a q2
a
q0
a
q3
15
First Choice
q1 a q2
a
q0
a
q3
16
First Choice
a
“reject”
q1 a q2
a
q0
a
q3
17
Second Choice
q1 a q2
a
q0
a
q3
18
Second Choice
q1 a q2
a
q0
a
q3
19
Second Choice
q1 a q2
a
q0
a
q3 “reject”
20
An NFA rejects a string:
when there is no computation of the NFA
that accepts the string:
• All the input is consumed and the

automaton is in a non final state
OR
• The input cannot be consumed

21
Example
a is rejected by the NFA:
“reject”
q1 a q2 q1 a q2
a a
q0 q0
a a
q3
“reject” q3
All possible computations lead to rejection

22
Rejection example
a a a
q1 a q2
a
q0
a
q3
23
First Choice
a a a
q1 a q2
a
q0
a
q3
24
First Choice
a a a
q1 a q2
a
q0
a No transition:
the automaton hangs
q3
25
First Choice
a a a
q1 a q2 “reject”
a
q0
a
q3
26
Second Choice
a a a
q1 a q2
a
q0
a
q3
27
Second Choice
a a a
q1 a q2
a
q0
a
q3
28
Second Choice
a a a
q1 a q2
a
q0
a
No transition:
q3
the automaton hangs
29
Second Choice
a a a
q1 a q2
a
q0
a
q3 “reject”
30
aaa is rejected by the NFA:
“reject”
q1 a q2 q1 a q2
a a
q0 q0
a a
q3 q3 “reject”
All possible computations lead to rejection

31
Language accepted: L  {aa}
q1 a q2
a
q0
a
q3
32
Lambda Transitions
q0 a q1  q2 a q3
33
a a
q0 a q1  q2 a q3
34
a a
q0 a q1  q2 a q3
35
(read head does not move)
a a
q0 a q1  q2 a q3
36
a a
q0 a q1  q2 a q3
37
all input is consumed
a a
“accept”
q0 a q1  q2 a q3
String aa is accepted
38
Rejection Example
a a a
q0 a q1  q2 a q3
39
a a a
q0 a q1  q2 a q3
40
(read head doesn’t move)
a a a
q0 a q1  q2 a q3
41
a a a
q0 a q1  q2 a q3
No transition:
the automaton hangs
42
a a a
“reject”
q0 a q1  q2 a q3
String aaa is rejected

43
Language accepted: L  {aa}
q0 a q1  q2 a q3
44
Another NFA Example
q0 a q1 b q2  q3

45
a b
q0 a q1 b q2  q3

46
a b
q0 a q1 b q2  q3

47
a b
q0 a q1 b q2  q3

48
a b
“accept”
q0 a q1 b q2  q3

49
Another String
a b a b
q0 a q1 b q2  q3

50
a b a b
q0 a q1 b q2  q3

51
a b a b
q0 a q1 b q2  q3

52
a b a b
q0 a q1 b q2  q3

53
a b a b
q0 a q1 b q2  q3

54
a b a b
q0 a q1 b q2  q3

55
a b a b
q0 a q1 b q2  q3

56
a b a b
“accept”
q0 a q1 b q2  q3

57
Language accepted
L  ab, abab, ababab, ...


 ab
q0 a q1 b q2  q3

58
Another NFA Example
0
q0 q1 0, 1 q2
1

59
Language accepted
L(M ) = {λ, 10, 1010, 101010, ...}

= {10}*
0
q0 q1 0, 1 q2
1 (redundant
state)

60
Remarks:
•The  symbol never appears on the
input tape
•Simple automata:
M1 M2
q0 q0
L(M1 ) = {} L(M 2 ) = {λ}

61
•NFAs are interesting because we can
express languages easier than DFAs
NFA M1 DFA M2 a
q2
q0 a q1
a
q0 a q1
L( M1 ) = {a} L( M 2 ) = {a}
62
Example
NFA M1
LM1   {10} * 0
q0 q1
1
DFA M2 0,1
LM 2   {10} * 0
q0 q1 1 q2
1
0
63
Formal Definition of NFAs
M  Q, ,  , q0 , F 
Q: Set of states, i.e. q0 , q1, q2 

: Input aplhabet, i.e. a, b
: Transition function
q0 : Initial state
F: Final states
64
 q0 , 1  q1
0
q0 q1 0, 1 q
2
1

65
 (q1,0)  {q0 , q2}
0
q0 q1 0, 1 q
2
1

66
 (q0 ,  )  {q0 , q2}
0
q0 q1 0, 1 q
2
1

67
 (q2 ,1)  
0
q0 q1 0, 1 q
2
1

68
Extended Transition Function  *
 * q0 , a   q1
q4 q5
a a
q0 a q1 b q2  q3

69
 * q0 , aa   q4 , q5
q4 q5
a a
q0 a q1 b q2  q3

70
 * q0 , ab   q2 , q3 , q0 
q4 q5
a a
q0 a q1 b q2  q3

71
Formally
q j   * qi , w : there is a walk from qi to q j
with label w
qi w qj
w  1 2  k
1 2 k
qi qj
72
The Language of an NFA M
F  q0 ,q5
q4 q5
a a
q0 a q1 b q2  q3

 * q0 , aa   q4 , q5 aa  L(M )

F
73
F  q0 ,q5
q4 q5
a a
q0 a q1 b q2  q3

 * q0 , ab   q2 , q3 , q0  ab  LM 

F
74
F  q0 ,q5
q4 q5
a a
q0 a q1 b q2  q3

 * q0 , abaa   q4 , q5 aaba  L(M )

F
75
F  q0 ,q5
q4 q5
a a
q0 a q1 b q2  q3

 * q0 , aba   q1 aba  LM 

F
76
q4 q5
a a
q0 a q1 b q2  q3

LM     ab* {aa}
77
Formally
The language accepted by NFA M is:
LM   w1, w2 , w3 ,...
where  * (q0 , wm )  {qi , q j ,..., qk ,}
and there is some qk  F (final state)
78
w LM   * (q0 , w)
qi
w
q0
qk qk  F
w
w qj
79
NFAs accept the Regular
Languages
80
Equivalence of Machines
Definition for Automata:
Machine M1 is equivalent to machine M 2
if L  M1   L  M 2 
81
Example of equivalent machines
NFA M1
LM1   {10} * 0
q0 q1
1
DFA M2 0,1
LM 2   {10} * 0
q0 q1 1 q2
1
0
82
We will prove:

Languages
Regular
accepted
Languages
by NFAs
Languages
accepted
by DFAs
NFAs and DFAs have the
same computation power
83
Step 1
Languages
accepted  Regular
Languages
by NFAs
Proof: Every DFA is trivially an NFA
Any language L accepted by a DFA

is also accepted by an NFA
84
Step 2
Languages
accepted  Regular
Languages
by NFAs
Proof: Any NFA can be converted to an

equivalent DFA
Any language L accepted by an NFA

is also accepted by a DFA
85
Convert NFA to DFA
NFA M
a
q a
0 q 1
 q 2
b
DFA M
q0 
86
Convert NFA to DFA
NFA M
a
q a
0 q  q1 2
b
DFA M
q0  a
q1, q2 
87
Convert NFA to DFA
NFA M
a
q a
0 q  q 1 2
b
DFA M
q0  a
q1, q2 
b

88
Convert NFA to DFA
NFA M
a
q a
0 q  q 1 2
b
a
DFA M
q0  a
q1, q2 
b

89
Convert NFA to DFA
NFA M
a
q a
0 q  q 1 2
b
a
DFA M b
q0  a
q1, q2 
b

90
Convert NFA to DFA
NFA M
a
q a
0 q  q 1 2
b
a
DFA M b
q0  a
q1, q2 
b
 a, b
91
Convert NFA to DFA
NFA M
a
q a
0 q  q 1 2
LM   L(M )
b
a
DFA M b
q0  a
q1, q2 
b
 a, b
92
More example: Converting NFA to DFA
• NFA N4 = (Q, {a,b},δ,1, {1}), the set of

states Q is {1,2,3} as shown in the
following figure.
b a

a
2 3
a, b
The NFA N 4
93
Step1: Determine DFA’s states
• N 4 has three states {1,2,3}, so we

construct DFA D with eight.
• We label each of D’s states with the

corresponding subset. Thus D’s state set
is {, {1}, {2}, {3}, {1,2}, {1,3}, {2,3}, {1,2,3}}
94
Step2:Determine the start and accept states
 {1} {2} {1,2}
{3} {1,3} {2,3} {1,2,3}
95
Step3: Determine transition function
a,a,bb a b
 {1} {2} {1,2}
a, b
b b a
b a
a a
{3} {1,3} {2,3} {1,2,3}
a
b
b
96
After removing unnecessary states
a,a,bb
 {2}
b b b a
a
a a
{3} a {1,3} {2,3} {1,2,3}
b
b
97
Rearranging states
a a, b
b
{1,3} {3} 
a
b b
b a
a
{2} {2,3} {1,2,3}
a
b
98
Renaming states
a a, b
b
q0 a q2 q5
b b
b a
a
q1 q3 q4
a
b
99
More simplified
q0 a q2
b b
b a
a
q1 q3 q4
a
b
100
NFA to DFA: Remarks
We are given an NFA M
We want to convert it
to an equivalent DFA M 
With LM   L(M )

101
If the NFA has states
q0 , q1, q2 ,...
the DFA has states in the powerset
,q0 ,q1,q1, q2 ,q3 , q4 , q7 ,....
102
Procedure NFA to DFA
1. Initial state of NFA: q0
Initial state of DFA: q0 
103
Example
NFA M a
q0 a q1  q2
b
DFA M
q0 
104
2. For every DFA’s state {qi , q j ,..., qm }
Compute in the NFA

 * qi , a ,
 * q j , a ,  {qi , qj ,..., qm
}
...
Add transition to DFA
 {qi , q j ,..., qm }, a   {qi , qj ,..., qm
}
105
Exampe
NFA M a
q0 a q1  q2
b
 * (q0 , a)  {q1, q2}
DFA M 
q0  a
q1, q2 
 q0 , a   q1, q2 
106
δ *(2, b) = {3}, δ *(3, b) = φ
δ({2, 3), b) = {3}  φ = {3}
{3} {2,3}
107
Repeat Step 2 for all letters in alphabet,

until
no more transitions can be added.
108
Example
NFA M a
q0 a q1  q2
b
a
DFA M b
q0  a
q1, q2 
b
 a, b
109
3. For any DFA state {qi , q j ,..., qm }
If some q j is a final state in the NFA
Then, {qi , q j ,..., qm }

is a final state in the DFA
110
Example
NFA M a
q0 a q1  q2 q1  F
b
a
DFA M b
q0  a
q1, q2 
b q1, q2  F 
 a, b
111
Theorem
Take NFA M
Apply procedure to obtain DFA M
Then M and M are equivalent :
L M   L M  
112
Proof
L M   L M  
L M   L M   AND L M   L M  
113
First we show: L M   L M  
Take arbitrary: w L(M )
We will prove: w  L(M )
114
w L(M )
M: q0 w qf
w  1 2  k
1 2 k
M: q0 qf
115
We will show that if w L(M )
w  1 2  k
1 2 k
M: q0 qf
1 2 k
M:
{q0 } {q f ,}
w  L(M )
116
More generally, we will show that if in M:
(arbitrary string) v  a1a2 an
a1 a2 an
M: q0 qi qj ql qm
an
M:
a1 a2
{q0 } {qi ,} {q j ,} {ql ,} {qm ,}
117
Proof by induction on |v|
Induction Basis: v  a1
a1
M: q0 qi
M:
a1
{q0 } {qi ,}
118
Induction hypothesis: 1 | v | k
v  a1a2 ak
a1 a2 ak
M: q0 qi qj qc qd
M:
a1 a2 ak
{q0 } {qi ,} {q j ,} {qc ,} {qd ,}
119
Induction Step: | v | k  1
v  a1a2 ak ak 1  vak 1
 
v
a1 a2 ak
M: q0 qi qj qc qd
v
M:
a1 a2 ak
{q0 } {qi ,} {q j ,} {qc ,} {qd ,}
v 120
Induction Step: | v | k  1
v  a1a2 ak ak 1  vak 1
 
v
a1 a2 ak ak 1
M: q0 qi qj qc qd qe
v
M:
a1 a2 ak ak 1
{q0 } {qi ,} {q j ,} {qc ,} {qd ,} {qe ,}
v 121
Therefore if w L(M )
w  1 2  k
1 2 k
M: q0 qf
1 2 k
M:
{q0 } {q f ,}
w  L(M )
122
We have shown: L M   L M  
We also need to show: L M   L M  
(proof is similar)
123
The End
124
Single Final State for NFAs
1
Any NFA can be converted
to an equivalent NFA
with a single final state
2
Example
a NFA
a b
a Equivalent NFA
a b
b 

3
In General
NFA
Equivalent NFA

 Single
 final state
4
Extreme Case
NFA without final state
Add a final state

Without transitions
5
Properties of
Regular Languages
6
For regular languages L1 and L2
we will prove that:
Union: L1  L2
Concatenation: L1L2
Star: Are regular
L1 *
Languages
Reversal: L1R
Complement: L1
Intersection: L1  L2
7
We say: Regular languages are closed under
Union: L1  L2
Concatenation: L1L2
Star: L1 *
Reversal: L1R
Complement: L1
Intersection: L1  L2
8
Regular language L1 Regular language L2
LM1   L1 LM 2   L2
NFA M1 NFA M2
Single final state Single final state

9
Example
M1
n0
a
L1  {a b}
n
b
M2
L2  ba b a
10
Union
NFA for L1  L2
M1
 
 M2 
11
Example
NFA for L1  L2  {a b}  {ba}
n
L1  {a b}n
a
b
 
 L2  {ba} 
b a
12
Concatenation
NFA for L1L2
M1 M2
 
13
Example
NFA for L1L2  {a b}{ba}  {a bba}

n n
L1  {a b}n
a L2  {ba}
b  b a 
14
Star Operation
NFA for L1 *

  L1 *
M1
 

15
Example
w  w1w2  wk
NFA for L1*  {a b} *
n
wi  L1

L1  {a b} n
a
 b 

16
Reverse
R
NFA for L1
L1 M1 M1
1. Reverse all transitions
2. Make initial state final state

and vice versa
17
Example
M1
a
L1  {a b}
n
b
M1
a
R
L1  {ba }
n b
18
Complement
L1 M1 L1 M1
1. Take the DFA that accepts L1
2. Make final states non-final,

and vice-versa
19
Example
M1
a a, b
L1  {a b}
n b a, b
M1
a a, b
L1  {a, b} * {a b}
n
b a, b
20
Intersection
DeMorgan’s Law: L1  L2  L1  L2
L1 , L2 regular
L1 , L2 regular
L1  L2 regular
L1  L2 regular
L1  L2 regular
21
Example
L1  {a b} regular
n
L1  L2  {ab}
L2  {ab, ba} regular regular
22
Regular Expressions
23
Regular Expressions
Regular expressions
describe regular languages
Example: (a  b  c) *
describes the language

a, bc*   , a, bc, aa, abc, bca,...
24
Recursive Definition
Primitive regular expressions: ,  , 
Given regular expressions r1 and r2
r1  r2
r1  r2
Are regular expressions
r1 *
r1 
25
Examples
A regular expression: a  b  c  * (c  )
Not a regular expression: a  b  
26
Languages of Regular Expressions
Lr  : language of regular expression r
Example
L(a  b  c) *   , a, bc, aa, abc, bca,...
27
Definition
For primitive regular expressions:
L   
L    
La   a
28
Definition (continued)
For regular expressions r1 and r2
Lr1  r2   Lr1   Lr2 
Lr1  r2   Lr1  Lr2 
Lr1 *   Lr1  *
Lr1   Lr1 
29
Example
Regular expression: a  b   a *
La  b   a *  La  b  La *
 La  b  La *
  La   Lb   La  *
 a  b a *
 a, b , a, aa, aaa,...
 a, aa, aaa,..., b, ba, baa,...
30
Example
Regular expression r  a  b  * a  bb 
Lr   a, bb, aa, abb, ba, bbb,...
31
Example
Regular expression r  aa  * bb  * b
Lr   {a b
2n 2m
b : n, m  0}
32
Example
Regular expression r  (0  1) * 00 (0  1) *
L(r ) = { all strings with at least

two consecutive 0 }
33
Example
Regular expression r  (1  01) * (0   )
L(r ) = { all strings without

two consecutive 0 }
34
Equivalent Regular Expressions
Definition:
Regular expressions r1 and r2
are equivalent if L(r1)  L(r2 )
35
Example
L = { all strings without
two consecutive 0 }
r1  (1  01) * (0   )
r2  (1* 011*) * (0   )  1* (0   )
r1 and r2
L(r1)  L(r2 )  L
are equivalent
regular expr.
36
Regular Expressions
and
Regular Languages
37
Theorem
Languages
Generated by
Regular Expressions
 Regular
Languages
38
Theorem - Part 1
Languages
Generated by  Regular
Languages
Regular Expressions
1. For any regular expression r

the language L(r ) is regular
39
Theorem - Part 2
Languages
Generated by  Regular
Languages
Regular Expressions
2. For any regular language L there is

a regular expression r with L(r )  L
40
Proof - Part 1
1. For any regular expression r

the language L(r ) is regular
Proof by induction on the size of r
41
Induction Basis
Primitive Regular Expressions: ,  , 
NFAs
L( M1)    L()
regular
L( M 2 )  {}  L( )
languages
a
L( M 3 )  {a}  L(a)
42
Inductive Hypothesis
Assume
for regular expressions r1 and r2
that
L(r1) and L(r2 ) are regular languages
43
Inductive Step
We will prove:
Lr1  r2 
Lr1  r2 
Are regular
Languages
Lr1 *
Lr1 
44
By definition of regular expressions:
Lr1  r2   Lr1   Lr2 
Lr1  r2   Lr1  Lr2 
Lr1 *   Lr1  *
Lr1   Lr1 
45
By inductive hypothesis we know:
L(r1) and L(r2 ) are regular languages
We also know:
Regular languages are closed under:
Union Lr1   Lr2 
Concatenation Lr1  Lr2 
Star  Lr1  *
46
Therefore:
Lr1  r2   Lr1   Lr2 
Are regular
Lr1  r2   Lr1  Lr2 
languages
Lr1 *   Lr1  *
47
And trivially:
L((r1)) is a regular language
48
Proof – Part 2
2. For any regular language L there is

a regular expression r with L(r )  L
Proof by construction of regular expression
49
Since L is regular take the
NFA M that accepts it
L( M )  L
Single final state

50
From M construct the equivalent
Generalized Transition Graph
in which transition labels are regular expressions
Example:
M
a c a c
a, b ab
51
b b
Another Example:
a
q0 q1 a, b q2
b
b b
a
q0 q1 a  b q2
b
52
b b
Reducing the states:
a
q0 q1 a  b q2
b
bb * a b
q0 bb * (a  b) q2
53
Resulting Regular Expression:
bb * a b
q0 bb * (a  b) q2
r  (bb * a) * bb * (a  b)b *
L( r )  L( M )  L
54
In General
Removing states: e
d c
qi q qj
a b
ae * d ce* b
ce* d
qi qj
ae * b
55
The final transition graph:
r1 r4
r3
q0 qf
r2
The resulting regular expression:
r  r1 * r2 (r4  r3r1 * r2 ) *
L( r )  L( M )  L
56
Example
a
1
2
a
b
b a
57
Example (Cont.)
b
a
1 2
a
ε ε
b a
b a
s
3 ε
58
Example (Cont.)
aa∪b
2
a
ε
s ab ba∪a a
b ε
bb
59
Example (Cont.)
a(aa∪b)*
s
a
a(aa∪b)*ab∪b (ba∪a)(aa∪b)*∪ε
(ba∪a)(aa∪b)*ab∪bb
60
Example (Cont.)
s a
(a(aa∪b)*ab ∪b)((ba ∪a)(aa ∪b)*ab ∪bb) *((ba ∪ a)(aa ∪b)* ∪ ε) ∪ a(aa ∪b)*
61
The End
62
Grammars
1
Grammars
Grammars express languages
Example: the English language
sentence  noun _ phrase predicate
noun _ phrase  article noun
predicate  verb
2
article  a
article  the
noun  cat
noun  dog
verb  runs
verb  walks
3
A derivation of “the dog walks”:
sentence  noun _ phrase predicate

 noun _ phrase verb
 article noun verb
 the noun verb
 the dog verb
 the dog walks
4
A derivation of “a cat runs”:
sentence  noun _ phrase predicate

 noun _ phrase verb
 article noun verb
 a noun verb
 a cat verb
 a cat runs
5
Language of the grammar:
L = { “a cat runs”,
“a cat walks”,
“the cat runs”,
“the cat walks”,
“a dog runs”,
“a dog walks”,
“the dog runs”,
“the dog walks” }
6
Notation
Production Rules
noun  cat
noun  dog
Variable Terminal
7
Another Example
Grammar: S  aSb
S 
Derivation of sentence ab :
S  aSb  ab
S  aSb S 
8
Grammar: S  aSb
S 
Derivation of sentence aabb :

S  aSb  aaSbb  aabb
S  aSb S 
9
Other derivations:
S  aSb  aaSbb  aaaSbbb  aaabbb
S  aSb  aaSbb  aaaSbbb

 aaaaSbbbb  aaaabbbb
10
Language of the grammar
S  aSb
S 
L  {a b : n  0}
n n
11
More Notation
Grammar G  V ,T , S , P 
V: Set of variables
T : Set of terminal symbols

S : Start variable
P: Set of Production rules

12
Example
Grammar G : S  aSb
S 
G  V ,T , S , P 
V  {S} T  {a, b}
P  {S  aSb, S  }
13
More Notation
Sentential Form:
A sentence that contains
variables and terminals
Example:
Sentential Forms sentence

14
*
We write: S  aaabbb
Instead of:
15
*
In general we write: w1  wn
If: w1  w2  w3    wn
16
*
By default: w  w
17
Example
Grammar Derivations
S  aSb *
S 
S 
*
S  ab
*
S  aabb
*
S  aaabbb
18
Example
Grammar Derivations
S  aSb 
S  aaSbb
S 

aaSbb  aaaaaSbbbbb
19
Another Grammar Example
Grammar G : S  Ab
A  aAb
A
Derivations:
S ⇒Ab ⇒b
S ⇒Ab ⇒aAbb ⇒abb
S ⇒Ab ⇒aAbb ⇒aaAbbb ⇒aabbb
20
More Derivations
S  Ab  aAbb  aaAbbb  aaaAbbbb
 aaaaAbbbbb  aaaabbbbb

S  aaaabbbbb

S  aaaaaabbbbbbb

S a b b
n n
21
Language of a Grammar
For a grammar G
with start variable S :

L(G )  {w : S  w}
String of terminals
22
Example
For grammar G: S  Ab
A  aAb
A
L(G )  {a b b : n  0}
n n

Since: S a b b n n
23
A Convenient Notation
A  aAb
A  aAb | 
A
article  a
article  a | the
article  the
24
Linear Grammars
25
Linear Grammars
Grammars with
at most one variable at the right side
of a production
Examples: S  aSb S  Ab
S  A  aAb
A
26
A Non-Linear Grammar
Grammar G: S  SS
S 
S  aSb
S  bSa
L(G )  {w : na ( w)  nb ( w)}
Number of a in string w
27
Another Linear Grammar
Grammar G : SA
A  aB | 
B  Ab
L(G )  {a b : n  0}
n n
28
Right-Linear Grammars
All productions have form: A  xB
or
A x
Example: S  abS string of

S a terminals
29
Left-Linear Grammars
All productions have form: A  Bx
or
A x
Example: S  Aab string of

A  Aab | B terminals
Ba
30
Regular Grammars
31
Regular Grammars
A regular grammar is any
right-linear or left-linear grammar
Examples:
G1 G2
S  abS S  Aab
S a A  Aab | B
Ba
32
Observation
Regular grammars generate regular languages
Examples: G2
G1 S  Aab
S  abS A  Aab | B
S a Ba
L(G1)  (ab) * a L(G2 )  aab(ab) *

33
Regular Grammars
Generate
Regular Languages
34
Theorem
Languages
Generated by
Regular Grammars
 Regular
Languages
35
Theorem - Part 1
Languages
Languages
Regular Grammars
Any regular grammar generates

a regular language
36
Theorem - Part 2
Languages
Languages
Regular Grammars
Any regular language is generated

by a regular grammar
37
Proof – Part 1
Languages
Languages
Regular Grammars
The language L(G ) generated by

any regular grammar G is regular
38
The case of Right-Linear Grammars
Let G be a right-linear grammar
We will prove: L(G ) is regular
Proof idea: We will construct NFA M

with L( M )  L(G )
39
Grammar G is right-linear
Example: S  aA | B
A  aa B
Bb B|a
40
Construct NFA M such that
every state is a grammar variable:
A
special
S VF
final state
B
S  aA | B
A  aa B
Bb B|a 41
Add edges for each production:
a A
S VF
B
S  aA
42
a A
S VF

B
S  aA | B
43
A
a a
S a VF

B
S  aA | B
A  aa B
44
A
a a
S a VF

B
S  aA | B
b
A  aa B
B  bB 45
A
a a
S a VF
 a
B
S  aA | B
b
A  aa B
B  bB | a 46
A
a a
S a VF
 a
B
S  aA  aaaB  aaabB  aaaba

47
NFA M Grammar
A G
a S  aA | B
a
A  aa B
S a B  bB | a
 VF
a
B
L( M )  L(G ) 
b
aaab * a  b * a
48
In General
A right-linear grammar G
has variables: V0 ,V1,V2 ,
and productions: Vi  a1a2 amV j

or
Vi  a1a2 am
49
We construct the NFA M such that:
each variable Vi corresponds to a node:
V1 V3
V0
VF
V2 V4 special
final state
50
For each production: Vi  a1a2 amV j
we add transitions and intermediate nodes
Vi a1 a2 ………
am V
j
51
For each production: Vi  a1a2 am
we add transitions and intermediate nodes
Vi a1 a2 ………
am
VF
52
Resulting NFA M looks like this:
a9
a2 a4
a1 V1 V3
a3 a5
V0
a3 a4
VF
a8 a9
V2 a5
V4
It holds that: L(G )  L( M )

53
The case of Left-Linear Grammars
Let G be a left-linear grammar
We will prove: L(G ) is regular
Proof idea:
We will construct a right-linear

grammar G with L(G )  L(G) R
54
Since G is left-linear grammar
the productions look like:
A  Ba1a2 ak
A  a1a2 ak
55
Construct right-linear grammar G
Left A  Ba1a2 ak
G
linear
A → Bv
Right
G A  ak a2a1B
linear
Av B R
56
Construct right-linear grammar G
Left
G A  a1a2 ak
linear
Av
Right
G A  ak a2a1
linear
Av R
57
It is easy to see that: L(G )  L(G) R
Since G is right-linear, we have:
L(G) L(G) R
L(G )
Regular Regular Regular
Language Language Language
58
Proof - Part 2
Languages
Languages
Regular Grammars
Any regular language L is generated

by some regular grammar G
59
Any regular language L is generated
by some regular grammar G
Proof idea:
Let M be the NFA with L  L(M ).
Construct from M a regular grammar G

such that L( M )  L(G )
60
Since L is regular
there is an NFA M such that L  L(M )
b
Example:
M a
a
q0 q1 q2
 b
L  ab * ab(b * ab) * q3
L  L(M )
61
Convert M to a right-linear grammar
b
M a
a
q0 q1 q2
q0  aq1  b
q3
62
b
M a
a
q0 q1 q2
q0  aq1
 b
q1  bq1 q3
q1  aq2
63
b
M a
a
q0 q1 q2
q0  aq1
q1  bq1  b
q1  aq2 q3
q2  bq3
64
L(G )  L( M )  L
G b
q0  aq1 M
a a
q0 q1 q2
q1  bq1
q1  aq2  b
q2  bq3 q3
q3  q1
q3  
65
In General
a
For any transition: q p
Add production: q  ap
variable terminal variable

66
For any final state: qf
Add production: qf 
67
Since G is right-linear grammar
G is also a regular grammar
with L(G )  L( M )  L
68
The End
69
Standard Representations
of Regular Languages
Regular Languages
DFAs
Regular
Grammars
NFAs
Regular
Expressions
1
When we say: We are given
a Regular Language L
We mean: Language L is in a standard

representation
2
What are the differences among NFA/DFA,
regular expression and regular grammar?
NFA/DFA accepts languages
Regular expresses operate languages
Grammar generates language
3
Elementary Questions
about
Regular Languages
4
Membership Question
Question: Given regular language L
and string w
how can we check if w  L?
Answer: Take the DFA that accepts L

and check if w is accepted
5
DFA
w
w L
DFA
w
w L
6
how can we check
if L is empty: ( L  ) ?
Check if there is any path from

the initial state to a final state
7
DFA
L
DFA
L
8
how can we check
if L is finite?
Check if there is a walk with cycle

from the initial state to a final state
9
DFA
L is infinite
DFA
L is finite
10
Question: Given regular languages L1 and L2
how can we check if L1  L2 ?
Answer: Find if ( L1  L2 )  ( L1  L2 )  
11
( L1  L2 )  ( L1  L2 )  
L1  L2   and L1  L2  
L1 L2 L L2 L1 L1
2
L1  L2 L2  L1
L1  L2
12
( L1  L2 )  ( L1  L2 )  
L1  L2   or L1  L2  
L1 L2 L2 L1
L1  L2 L2  L1
L1  L2
13
Non-regular languages
14
{a b : n  0}
n n
{vv : v {a, b}*}
R
Regular languages
a *b b*c  a
b  c ( a  b) *
etc...
Finite languages are regular 15
How can we prove that a language L
is not regular?
Prove that there is no DFA that accepts L
Ha Ha Ha……^-^
Problem: this is not easy to prove
Solution: the Pumping Lemma !!!
16
The Pigeonhole Principle
17
4 pigeons
3 pigeonholes
18
A pigeonhole must
contain at least two pigeons
19
n pigeons
...........
m pigeonholes nm
...........
20
n pigeons
m pigeonholes
There is a pigeonhole
nm with at least 2 pigeons
...........
21
Ex 1: Show that if any five numbers from 1
to 8 are chosen, then two of them will add
up to 9.
Solution: {1,8},{2,7},{3,6},{4,5}
22
Ex 2: show that if any 11 numbers are chosen
from the set {1,2,…,20}, then one of them
will be a multiple of another.
Solution: {1,2,…,20}={2km|k could be any

positive integer including 0, m=some odd
number}, odd number={k=0,m=1,3,…,19},
2={k=1,m=1},…,12={k=2,m=3},…,18={k=1,m=9}
…
2k1m, 2k2m
23
Ex 3: Consider the region shown in the
figure. It is bounded by a regular hexagon
whose sides are of length 1 unit. Show that
if any seven points are chosen in this
region, then two of them must be no
farther apart than 1 unit.
24
Ex 4: shirts numbered consecutively from 1
to 20 are worn by the 20 members of a
bowling league. When any 3 of these
members are chosen to be a team, the sum
of their shirt numbers is used as a code
number for the team. Show that if any 8 of
the 20 members are selected, then from
theses 8 we may form at least two
different teams having the same code
number. (one member can be in several
different teams simultaneously)
Solution: C(8,3)=56, 1+2+3=6,…,18+19+20=57,

57-6+1=52
25
and
DFAs
26
DFA with 4 states
b
b b
q1 a q2 b q3 b q4
a a
27
In walks of strings: a no state
aa is repeated
aab
b
b b
q1 a q2 a q3 b q4
a a
28
In walks of strings: aabb a state
is repeated
bbaa
abbabb
abbbabbabb...
b
b b
q1 a q2 a q3 b q4
a a
29
If string w has length | w |  4 :
Then the transitions of string w
are more than the states of the DFA
Thus, a state must be repeated

b
b b
q1 a q2 a q3 b q4
a a
30
In general, for any DFA:
String w has length  number of states
A state q must be repeated in the walk of w
walk of w
...... q ......
Repeated state 31
In other words for a string w:
a transitions are pigeons
q states are pigeonholes
walk of w
...... q ......
Repeated state
32
The Pumping Lemma
33
The Pumping Lemma:
• Given a infinite regular language L
• there exists an integer m
• for any string w L with length | w| m
• we can write w x y z
• with | x y |  m and | y |  1
• such that: xy z  L
i i  0, 1, 2, ...
34
Applications
of
the Pumping Lemma
35
Theorem: The language L  {a nb n : n  0}
is not regular
Proof: Use the Pumping Lemma
36
L  {a b : n  0}
n n
Assume for contradiction

that L is a regular language
Since L is infinite
we can apply the Pumping Lemma
37
L  {a b : n  0}
n n
Let m be the integer in the Pumping Lemma
Pick a string w such that: w  L

length | w| m
We pick wa b
m m
38
Write: a b xyz
m m
From the Pumping Lemma

it must be that length | x y |  m, | y | 1
m m
xyz  a b
m m
 a...aa...aa...ab...b
x y z
Thus: y  a , k 1
k
39
x y za bm m
y  a , k 1
k
From the Pumping Lemma: xy z  L

i
i  0, 1, 2, ...
Thus: xy z  L
2
40
x y za b m m
y  a , k 1
k

2
mk m
xy z  a...aa...aa...aa...ab...b  L
2
x y y z
m k m
Thus: a b L
41
m k m k ≥1
a b L
BUT: L  {a b : n  0}
n n
m k m
a b L
CONTRADICTION!!!
42
Therefore: Our assumption that L
is a regular language is not true
Conclusion: L is not a regular language
43
Non-regular languages {a b : n  0}
n n
Regular languages
44
Understanding pumping lemma more
45
Take an infinite regular language L
There exists a DFA that accepts L
m
states
46
Take string w with w L
There is a walk with label w:
.........
walk w
47
If string w has length | w |  m (number
of states
of DFA)
then, from the pigeonhole principle:
a state is repeated in the walk w
...... q ......
walk w
48
Let q be the first state repeated in the
walk of w
...... q ......
walk w
49
Write w x y z
...... q ......
x z 50
Observations: length | x y |  m number
of states
length | y | 1 of DFA
...... q ......
x z 51
Observation: The string xz
is accepted
...... q ......
x z 52
Observation: The string xyyz
is accepted
...... q ......
x z 53
Observation: The string xyyyz
is accepted
...... q ......
x z 54
i
In General: The string xy z
is accepted i  0, 1, 2, ...
...... q ......
x z 55
In General: i
x y z ∈L i  0, 1, 2, ...
Language accepted by the DFA
...... q ......
x z 56
In other words, we described:
The Pumping Lemma !!!
57
More Applications
of
the Pumping Lemma
1
The Pumping Lemma:
• Given a infinite regular language L
• there exists an integer m
• for any string w L with length | w| m
• we can write w x y z
• with | x y |  m and | y |  1
• such that: xy z  L
i i  0, 1, 2, ...
2
Non-regular languages L  {vv : v  *}
R
Regular languages
3
Theorem: The language
L  {vv : v  *}
R
  {a, b}
is not regular
4
L  {vv : v  *}
R

Since L is infinite
5
L  {vv : v  *}
R
Pick a string w such that: w  L and
length | w| m
We pick wa b b a
m m m m
6
Write a b b a xyz
m m m m

m m m m
xyz  a...aa...a...ab...bb...ba...a
x y z
Thus: y  a , k 1
k
7
x y za b b a
m m m m
y  a , k 1
k

i
i  0, 1, 2, ...
Thus: xy z  L
2
8
x y za b b a
m m m m
y  a , k 1
k

2
m+k m m m
2
xy z = a...aa...aa...a...ab...bb...ba...a ∈L
x y y z
m k m m m
Thus: a b b a L
9
m k m m m
a b b a L k 1
BUT: L  {vv : v  *}

R
m k m m m
a b b a L
CONTRADICTION!!!
10
11
n l n l
L  {a b c : n, l  0}
Regular languages
12
Theorem: The language
n l n l
L  {a b c : n, l  0}
is not regular
13
n l n l
L  {a b c : n, l  0}

Since L is infinite
14
n l n l
L  {a b c : n, l  0}
Pick a string w such that: w  L and
length | w| m
We pick wa b c m m 2m
15
Write
m m 2m
a b c xyz

m m 2m
xyz  a...aa...aa...ab...bc...cc...c
x y z
Thus: y  a , k 1
k
16
x y za b c
m m 2m
y  a , k 1
k

i
i  0, 1, 2, ...
0
Thus: x y z = xz ∈ L
17
x y za b c
m m 2m
y  a , k 1
k
From the Pumping Lemma: xz  L

mk m 2m
xz  a...aa...ab...bc...cc...c  L
x z
mk m 2 m
Thus: a b c L
18
mk m 2 m
a b c L k 1
BUT: n l n l
L  {a b c : n, l  0}
mk m 2 m
a b c L
CONTRADICTION!!!
19
20
Non-regular languages L  {a : n  0}
n!
Regular languages
21
Theorem: The language L  {a : n  0}
n!
is not regular
n!  1 2  (n  1)  n
22
L  {a : n  0}
n!

Since L is infinite
23
L  {a : n  0}
n!
Pick a string w such that: w  L

length | w| m
We pick wa m!
24
Write a m!
xyz

m m!m
xyz  a m!
 a...aa...aa...aa...aa...a
x y z
Thus: y  a , 1 k  m
k
25
x y za m!
y  a , 1 k  m
k

i
i  0, 1, 2, ...
Thus: xy z  L
2
26
x y za m!
y  a , 1 k  m
k

2
mk m!m
xy z  a...aa...aa...aa...aa...aa...a  L
2
x y y z
Thus: m! k
a L
27
m! k
a L 1 k  m
Since: L  {a : n  0}
n!
There must exist p such that:
m! k  p!
28
However: m! k  m! m for m 1
 m! m!
 m!m  m!
 m!(m  1)
 (m  1)!
m! k  (m  1)!
m! k  p! for any p

29
m! k 1 k  m
a L
BUT: L  {a : n  0}
n!
m! k
a L
CONTRADICTION!!!
30
31
Context-Free Languages
1
n n R
{a b : n  0} {ww }
Regular Languages
a *b * ( a  b) *
2
n n R
{a b } {ww }
Regular Languages
3
Context-Free Pushdown
Grammars Automata
stack
automaton
4
Context-Free Grammars
5
Example
A context-free grammar G: S  aSb

S 
A derivation:
S  aSb  aaSbb  aabb

6
S 
Another derivation:

7
S  aSb
S 
L(G )  {a b : n  0}
n n
8
Example
A context-free grammar G: S  aSa

S  bSb
S 
A derivation:
S  aSa  abSba  abba

9
A context-free grammar G: S  aSa
S  bSb
S 
Another derivation:
S  aSa  abSba  abaSaba  abaaba

10
S  aSa
S  bSb
S 
L(G )  {ww : w {a, b}*}

R
11
Example

S  SS
S 
A derivation:
S  SS  aSbS  abS  ab
12
S  SS
S 
A derivation:
S  SS  aSbS  abS  abaSb  abab

13
S  aSb
S  SS
S 
L(G )  {w : na ( w)  nb ( w),
and na (v)  nb (v)
in any prefix v}
14
Definition: Context-Free Grammars
Grammar G  (V ,T , S , P)
Variables Terminal Start

symbols variable
Productions of the form:

A x
Variable String of variables
and terminals
15
G  (V ,T , S , P)
*
L(G )  {w : S  w, w  T *}
16
Definition: Context-Free Languages
A language L is context-free
if and only if
there is a context-free grammar G

with L  L(G )
17
Derivation Order
1. S  AB 2. A  aaA 4. B  Bb
3. A   5. B  
Leftmost derivation:
1 2 3 4 5
S  AB  aaAB  aaB  aaBb  aab
Rightmost derivation:
1 4 5 2 3
S  AB  ABb  Ab  aaAb  aab
18
S  aAB
A  bBb
B  A|
Leftmost derivation:
S  aAB  abBbB  abAbB  abbBbbB
 abbbbB  abbbb
Rightmost derivation:
S  aAB  aA  abBb  abAb
 abbBbb  abbbb
19
Derivation Trees
20
S  AB A  aaA |  B  Bb | 
S  AB
S
A B
21
S  AB A  aaA |  B  Bb | 
S  AB  aaAB
S
A B
a a A
22
S  AB A  aaA |  B  Bb | 
S  AB  aaAB  aaABb
S
A B
a a A B b
23
S  AB A  aaA |  B  Bb | 
S  AB  aaAB  aaABb  aaBb

S
A B
a a A B b

24
S  AB A  aaA |  B  Bb | 
S  AB  aaAB  aaABb  aaBb  aab

Derivation Tree S
A B
a a A B b
 
25
S  AB A  aaA |  B  Bb | 
S  AB  aaAB  aaABb  aaBb  aab

Derivation Tree S
A B
yield
a a A B b aab
 aab
 
26
Partial Derivation Trees
S  AB A  aaA |  B  Bb | 
S  AB
Partial derivation tree S
A B
27
S  AB  aaAB
A B
a a A
28
sentential
S  AB  aaAB
form
A B
yield
a a A
aaAB
29
Sometimes, derivation order doesn’t matter
Leftmost:
S  AB  aaAB  aaB  aaBb  aab
Rightmost:
S  AB  ABb  Ab  aaAb  aab
S
Same derivation tree
A B
a a A B b
  30
Ambiguity
31
E  E  E | E  E | (E) | a
a  aa
E E  E  E  a E  a EE
 a  a E  a  a*a
E  E
leftmost derivation
a E  E
a a
32
E  E  E | E  E | (E) | a
a  aa
E  EE  E  EE  a EE E

 a  aE  a  aa
E  E
leftmost derivation
E  E a
a a
33
E  E  E | E  E | (E) | a
a  aa
Two derivation trees
E E
E  E E  E
a E  E E  E a
a a a a
34
The grammarE  E  E | E  E | (E) | a
is ambiguous:
string a  a  a has two derivation trees
E E
E  E E  E
a E  E E  E a
a a a a
35
The grammarE  E  E | E  E | (E) | a
is ambiguous:
string a  a  a has two leftmost derivations
E  E  E  a E  a EE
 a  a E  a  a*a
E  EE  E  EE  a EE

 a  aE  a  aa 36
Definition:
A context-free grammar G is ambiguous
if some string w L(G ) has:
two or more derivation trees
37
In other words:
A context-free grammar G is ambiguous
if some string w L(G ) has:
two or more leftmost derivations

(or rightmost)
38
Why do we care about ambiguity?
a  aa
take a2
E E
E  E E  E
a E  E E  E a
a a a a
39
2  22
E E
E  E E  E
2 E  E E  E 2
2 2 2 2
40
2  22  6 2  22  8
6 8
E E
2 4 4 2
E  E E  E
2 2 2 2
2 E  E E  E 2
2 2 2 2
41
Correct result: 2  22  6
6
E
2 4
E  E
2 2
2 E  E
2 2
42
• Ambiguity is bad for programming languages
• We want to remove ambiguity
43
We fix the ambiguous grammar:
E  E  E | E  E | (E) | a
New non-ambiguous grammar: E  E T

E T
T T F
T F
F  (E)
F a 44
E  E T T T  F T  a T  a T F
 a  F F  a  aF  a  aa
E a  aa
E  E T
E  T
E T
T T F T T  F
T F
F F a
F  (E)
F a a a
45
Unique derivation tree
E a  aa
E  T
T T  F
F F a
a a
46
The grammar G: E  E T
E T
T T F
T F
F  (E)
F a
is non-ambiguous:
Every string w L(G ) has
a unique derivation tree
47
Another Ambiguous Grammar
IF_STMT  if EXPR then STMT

| if EXPR then STMT else STMT
48
If expr1 then if expr2 then stmt1 else stmt2
IF_STMT
if expr1 then STMT
if expr2 then stmt1 else stmt2
IF_STMT
if expr1 then STMT else stmt2
if expr2 then stmt1

49
Inherent Ambiguity
Some context free languages

have only ambiguous grammars
Example: L  {a b c }  {a b c }
n n m n m m
S  S1 | S2 S1  S1c | A S2  aS2 | B
A  aAb |  B  bBc | 
50
n n n
The string a b c
has two derivation trees
S S
S1 S2
S1 c a S2
51
Simplifications
of
Context-Free Grammars
1
A Substitution Rule
Equivalent
grammar
S  aB
S  aB | ab
A  aaA
Substitute A  aaA
A  abBc B b A  abBc | abbc
B  aA
B  aA
Bb
2
A Substitution Rule
S  aB | ab
A  aaA
A  abBc | abbc
B  aA
Substitute
B  aA
S  aB | ab | aaA
Equivalent
A  aaA
A  abBc | abbc | abaAc
grammar
3
In general:
A  xBz
B  y1
Substitute
B  y1
equivalent
A  xBz | xy1z grammar
4
Nullable Variables
  production : A
Nullable Variable: A  
5
Removing Nullable Variables
Example Grammar:
S  aMb
M  aMb
M 
Nullable variable
6
Final Grammar
S  aMb
S  aMb
Substitute S  ab
M  aMb M 
M  aMb
M 
M  ab
7
Unit-Productions
Unit Production: A B
(a single variable in both sides)
8
Removing Unit Productions
Observation:
A A
Is removed immediately
9
Example Grammar:
S  aA
Aa
A B
BA
B  bb
10
S  aA
S  aA | aB
Aa
Substitute Aa
A B A B B  A| B
BA
B  bb
B  bb
11
S  aA | aB S  aA | aB
Aa Remove Aa
B  A| B BB BA
B  bb B  bb
12
S  aA | aB
S  aA | aB | aA
Aa Substitute
BA Aa
BA
B  bb
B  bb
13
Remove repeated productions
Final grammar
S  aA | aB | aA S  aA | aB
Aa Aa
B  bb B  bb
14
Useless Productions
S  aSb
S 
SA
A  aA Useless Production
Some derivations never terminate...
S  A  aA  aaA    aaaA  
15
Another grammar:
SA
A  aA
A
B  bA Useless Production
Not reachable from S
16
In general: contains only
terminals
if S    xAy    w
w L(G )
then variable A is useful
otherwise, variable A is useless
17
A production A  x is useless
if any of its variables is useless
S  aSb
S  Productions
Variables S  A useless
useless A  aA useless
useless B  C useless
useless CD useless

18
Removing Useless Productions
Example Grammar:
S  aS | A | C
Aa
B  aa
C  aCb
19
First: find all variables that can produce
strings with only terminals
S  aS | A | C Round 1: { A, B}
Aa SA
B  aa
C  aCb Round 2: { A, B, S}
20
Keep only the variables
that produce terminal symbols: { A, B, S}
(the rest variables are useless)
S  aS | A | C
Aa S  aS | A
B  aa Aa
C  aCb B  aa
Remove useless productions
21
Second: Find all variables
reachable from S
Use a Dependency Graph
S  aS | A
Aa S A B
B  aa not
reachable
22
Keep only the variables
reachable from S
(the rest variables are useless)
Final Grammar
S  aS | A
S  aS | A
Aa
Aa
B  aa
Remove useless productions
23
Removing All
Step 1: Remove Nullable Variables
Step 2: Remove Unit-Productions
Step 3: Remove Useless Variables
24
Normal Forms
for
Context-free Grammars
25
Chomsky Normal Form
Each productions has form:
A  BC or Aa
variable variable terminal
26
Examples:
S  AS S  AS
S a S  AAS
A  SA A  SA
Ab A  aa
Chomsky Not Chomsky
Normal Form Normal Form
27
Convertion to Chomsky Normal Form
Example: S  ABa
A  aab
B  Ac
Not Chomsky
Normal Form
28
Introduce variables for terminals: Ta , Tb , Tc
S  ABTa
S  ABa A  TaTaTb
A  aab B  ATc
B  Ac Ta  a
Tb  b
Tc  c
29
Introduce intermediate variable: V1
S  AV1
S  ABTa
V1  BTa
A  TaTaTb
A  TaTaTb
B  ATc
B  ATc
Ta  a
Ta  a
Tb  b
Tb  b
Tc  c
Tc  c
30
Introduce intermediate variable: V2
S  AV1
S  AV1
V1  BTa
V1  BTa
A  TaV2
A  TaTaTb
V2  TaTb
B  ATc
B  ATc
Ta  a
Ta  a
Tb  b
Tb  b
Tc  c
Tc  c 31
Final grammar in Chomsky Normal Form:
S  AV1
V1  BTa
A  TaV2
Initial grammar
V2  TaTb
S  ABa B  ATc
A  aab Ta  a
B  Ac Tb  b
Tc  c 32
In general:
From any context-free grammar

(which doesn’t produce  )
not in Chomsky Normal Form
we can obtain:
An equivalent grammar
in Chomsky Normal Form
33
The Procedure
First remove:
Nullable variables
Unit productions
34
Then, for every symbol a:
Add production Ta  a
In productions: replace a with Ta
New variable: Ta
35
Replace any production A  C1C2 Cn
with A  C1V1
V1  C2V2

Vn2  Cn1Cn
New intermediate variables: V1, V2 , ,Vn2

36
Theorem: For any context-free grammar
(which doesn’t produce  )
there is an equivalent grammar
in Chomsky Normal Form
37
Observations
• Chomsky normal forms are good

for parsing and proving theorems
• It is very easy to find the Chomsky normal

form for any context-free grammar
38
Pushdown Automata
PDAs
1
Pushdown Automaton -- PDA
Input String
Stack
States
2
Initial Stack Symbol
Stack
stack
$
head
bottom
special symbol
3
The States
Input Pop Push

symbol symbol symbol
q1 a, b  c q2
4
a, b  c
q1 q2
input
 a   a 
stack
b top c
h Replace h
e e
$ $
5
a,   c
q1 q2
input
 a   a 
stack c
b top b
h Push h
e e
$ $
6
a, b  
q1 q2
input
 a   a 
stack
b top
h Pop h
e e
$ $
7
a,   
q1 q2
input
 a   a 
stack
b top b
h No Change h
e e
$ $
8
A Possible Transition
a, $  
q1 q2
input
 a   a 
stack empty
$ top Pop
9
A Bad Transition
a, b  c
q1 q2
input
 a 
Empty stack
HALT
The automaton Halts in state q1

and Rejects the input string
10
A Bad Transition
a,   c
q1 q2
input
 a 
Empty stack
HALT
The automaton Halts in state q1

and Rejects the input string
11
No transition is allowed to be followed
When the stack is empty
x, y  z
q1 q2
Empty stack
12
A Good Transition
a, $  b
q1 q2
input
 a   a 
stack
$ top Pop b
13
Non-Determinism
a, b  c q2
q1 , b  c
q1 q2
a, b  c   transition
q3
These are allowed transitions in a

Non-deterministic PDA (NPDA)
14
NPDA: Non-Deterministic PDA
Example:
a,   a b, a  
q0 ,    q b, a   q , $  $ q
1 2 3
15
Execution Example: Time 0
Input
a a a b b b
$
Stack
current a,   a b, a  
state
q0  ,    q b, a   q , $  $ q
1 2 3
16
Time 1
Input
a a a b b b
$
Stack
a,   a b, a  
q0  ,    q1 b, a   q2  , $  $ q3
17
Time 2
Input
a a a b b b a
$
Stack
a,   a b, a  
q0  ,    q1 b, a   q2  , $  $ q3
18
Time 3
Input a
a a a b b b a
$
Stack
a,   a b, a  
q0  ,    q1 b, a   q2  , $  $ q3
19
Time 4
a
Input
a
a a a b b b a
$
Stack
a,   a b, a  
q0  ,    q1 b, a   q2  , $  $ q3
20
Time 5
a
Input
a
a a a b b b a
$
Stack
a,   a b, a  
q0  ,    q1 b, a   q2  , $  $ q3
21
Time 6
Input a
a a a b b b a
$
Stack
a,   a b, a  
q0  ,    q1 b, a   q2  , $  $ q3
22
Time 7
Input
a a a b b b a
$
Stack
a,   a b, a  
q0  ,    q1 b, a   q2  , $  $ q3
23
Time 8
Input
a a a b b b
$
Stack
a,   a b, a  
accept
q0  ,    q1 b, a   q2  , $  $ q3
24
A string is accepted if there is
a computation such that:
All the input is consumed

AND
The last state is a final state
At the end of the computation,

we do not care about the stack contents
25
The input string aaabbb
is accepted by the NPDA:
a,   a b, a  
q0  ,    q1 b, a   q2  , $  $ q3
26
In general,
L  {a b : n  0}
n n
is the language accepted by the NPDA:
a,   a b, a  
q0  ,    q1 b, a   q2  , $  $ q3
27
Another NPDA example
NPDA M
L( M )  {ww }R
a,   a a, a  
b,   b b, b  
q0 ,    q1 , $  $ q2
28
Input
a b b a
$
Stack
a,   a a, a  
b,   b b, b  
q0 ,    q1 , $  $ q2
29
Time 1
Input
a b b a a
$
Stack
a,   a a, a  
b,   b b, b  
q0 ,    q1 , $  $ q2
30
Time 2
Input
b
a b b a a
$
Stack
a,   a a, a  
b,   b b, b  
q0 ,    q1 , $  $ q2
31
Time 3
Input
b
a b b a
Guess the middle a
of string $
Stack
a,   a a, a  
b,   b b, b  
q0 ,    q1 , $  $ q2
32
Time 4
Input
b
a b b a a
$
Stack
a,   a a, a  
b,   b b, b  
q0 ,    q1 , $  $ q2
33
Time 5
Input
a b b a a
$
Stack
a,   a a, a  
b,   b b, b  
q0 ,    q1 , $  $ q2
34
Time 6
Input
a b b a
$
Stack
a,   a a, a  
b,   b b, b  
accept
q0 ,    q1 , $  $ q2
35
Rejection Example: Time 0
Input
a b b b
$
Stack
a,   a a, a  
b,   b b, b  
q0 ,    q1 , $  $ q2
36
Time 1
Input
a b b b a
$
Stack
a,   a a, a  
b,   b b, b  
q0 ,    q1 , $  $ q2
37
Time 2
Input
b
a b b b a
$
Stack
a,   a a, a  
b,   b b, b  
q0 ,    q1 , $  $ q2
38
Time 3
Input
b
a b b b
Guess the middle a
of string $
Stack
a,   a a, a  
b,   b b, b  
q0 ,    q1 , $  $ q2
39
Time 4
Input
b
a b b b a
$
Stack
a,   a a, a  
b,   b b, b  
q0 ,    q1 , $  $ q2
40
Time 5
Input There is no possible transition.
a b b b Input is not a
consumed
$
Stack
a,   a a, a  
b,   b b, b  
q0 ,    q1 , $  $ q2
41
Another computation on same string:
Input Time 0
a b b b
$
Stack
a,   a a, a  
b,   b b, b  
q0 ,    q1 , $  $ q2
42
Time 1
Input
a b b b a
$
Stack
a,   a a, a  
b,   b b, b  
q0 ,    q1 , $  $ q2
43
Time 2
Input
b
a b b b a
$
Stack
a,   a a, a  
b,   b b, b  
q0 ,    q1 , $  $ q2
44
Time 3
Input b
b
a b b b a
$
Stack
a,   a a, a  
b,   b b, b  
q0 ,    q1 , $  $ q2
45
Time 4 b
Input b
b
a b b b a
$
Stack
a,   a a, a  
b,   b b, b  
q0 ,    q1 , $  $ q2
46
Time 5 b
Input b
No final state b
a b b b is reached a
$
Stack
a,   a a, a  
b,   b b, b  
q0 ,    q1 , $  $ q2
47
There is no computation
that accepts string abbb
abbb  L(M )
a,   a a, a  
b,   b b, b  
q0 ,    q1 , $  $ q2
48
A string is rejected if there is
no computation such that:
All the input is consumed

AND
The last state is a final state
At the end of the computation,

we do not care about the stack contents
49
In other words, a string is rejected
if in every computation with this string:
The input cannot be consumed

OR
The input is consumed and the last
state is not a final state
OR
The stack head moves below the
bottom of the stack
50
NPDA M
L(M )  {w : w {a, b}* , na  nb for any prefix of w}
a,   a
b, a  
b, $  
q0
51
Input
a a b
a,   a $
b, a   Stack
b, $  
q0
52
Time 1
Input
a a b a
a,   a $
b, a   Stack
b, $  
q0
53
Time 2
Input
a
a a b a
a,   a $
b, a   Stack
b, $  
q0
54
Time 3
Input
a a b a
a,   a $
b, a   Stack
b, $  
accept
q0
55
Rejection example: Time 0
Input
a b b b
$
Stack
q0
56
Time 1
Input
a b b b a
a,   a $
b, a   Stack
b, $  
q0
57
Time 2
Input
a b b b
a,   a $
b, a   Stack
b, $  
q0
58
Time 3
Input
a b b b
a,   a
b, a   Stack
b, $  
q0
59
Time 4
Input
a b b b
a,   a
b, a   Stack
b, $  
Halt and Reject

q0
60
Pushing Strings
Input Pop Push

symbol symbol string
q1 a, b  w q2
61
Example:
a, b  cdf
q1 q2
input
a a
c pushed
stack d
top string
b f
h Push h
e e
$ $
62
NPDA M
L( M )  {w : na  nb }
a, $  0$ b, $  1$
a, 0  00 b, 1  11
a, 1   b, 0  
q1 , $  $ q2
63
Input
a b b a a b
$
a, $  0$ b, $  1$
Stack
a, 0  00 b, 1  11
a, 1   b, 0  
current
state
q1 , $  $ q2
64
Time 1
Input
a b b a a b
0
$
a, $  0$ b, $  1$ Stack
a, 0  00 b, 1  11
a, 1   b, 0  
q1 , $  $ q2
65
Time 3
Input
a b b b a a
0
$
a, $  0$ b, $  1$ Stack
a, 0  00 b, 1  11
a, 1   b, 0  
q1 , $  $ q2
66
Time 4
Input
a b b b a a
1
$
a, $  0$ b, $  1$ Stack
a, 0  00 b, 1  11
a, 1   b, 0  
q1 , $  $ q2
67
Time 5
Input
a b b b a a 1
1
$
a, $  0$ b, $  1$ Stack
a, 0  00 b, 1  11
a, 1   b, 0  
q1 , $  $ q2
68
Time 6
Input
a b b b a a 1
1
$
a, $  0$ b, $  1$ Stack
a, 0  00 b, 1  11
a, 1   b, 0  
q1 , $  $ q2
69
Time 7
Input
a b b b a a
1
$
a, $  0$ b, $  1$ Stack
a, 0  00 b, 1  11
a, 1   b, 0  
q1 , $  $ q2
70
Time 8
Input
a b b b a a
$
a, $  0$ b, $  1$
Stack
a, 0  00 b, 1  11
a, 1   b, 0  
accept
q1 , $  $ q2
71
Formalities for NPDAs
72
q1 a, b  w q2
Transition function:
 (q1, a, b)  {(q2 , w)}
73
q2
a, b  w
q1
a, b  w q3
Transition function:
 (q1, a, b)  {(q2 , w), (q3 , w)}

74
Formal Definition
Non-Deterministic Pushdown Automaton
NPDA
M  (Q, Σ, Γ, δ, q0 , z, F ) Final
States states
Input Stack
alphabet Transition Initial start
Stack
function state symbol
alphabet
75
Instantaneous Description
( q, u , s )
Current Current
Remaining
state stack
input
contents
76
Example: Instantaneous Description
(q1, bbb, aaa$)
a
Time 4: Input a
a a a b b b a
$
Stack
a,   a b, a  
q0  ,    q1 b, a   q2  , $  $ q3
77
Example: Instantaneous Description
(q2 , bb, aa$)
a
Time 5: Input a
a a a b b b a
$
Stack
a,   a b, a  
q0  ,    q1 b, a   q2  , $  $ q3
78
We write:
(q1, bbb, aaa$)  (q2 , bb, aa$)

Time 4 Time 5
79
A computation:
(q0 , aaabbb,$)  (q1, aaabbb,$) 

(q1, aabbb, a$)  (q1, abbb, aa$)  (q1, bbb, aaa$) 
(q2 , bb, aa$)  (q2 , b, a$)  (q2 ,  ,$)  (q3 ,  ,$)
a,   a b, a  
q0  ,    q1 b, a   q2  , $  $ q3
80
(q0 , aaabbb,$)  (q1, aaabbb,$) 
(q1, aabbb, a$)  (q1, abbb, aa$)  (q1, bbb, aaa$) 
(q2 , bb, aa$)  (q2 , b, a$)  (q2 ,  ,$)  (q3 ,  ,$)
For convenience we write:

(q0 , aaabbb,$)  (q3 ,  ,$)
81
Formal Definition
Language L(M ) of NPDA M :

L( M )  {w : (q0 , w, s )  (q f ,  , s' )}
Initial state Final state
82
Example: 
(q0 , aaabbb,$)  (q3 ,  ,$)
aaabbb  L(M )
NPDA M :
a,   a b, a  
q0  ,    q1 b, a   q2  , $  $ q3
83

(q0 , a b ,$)  (q3 ,  ,$)
n n
a b  L(M )
n n
NPDA M :
a,   a b, a  
q0  ,    q1 b, a   q2  , $  $ q3
84
Therefore: L( M )  {a b : n  0}
n n
NPDA M :
a,   a b, a  
q0  ,    q1 b, a   q2  , $  $ q3
85

النظرية الاحتسابية

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

النظرية الاحتسابية

Uploaded by

Copyright:

Available Formats

Models of Computation

Copyright 2011@Xu Dezhi 1

Copyright 2011@Xu Dezhi 2

Copyright 2011@Xu Dezhi 3

Copyright 2011@Xu Dezhi 6

Copyright 2011@Xu Dezhi 7

Copyright 2011@Xu Dezhi 8

• Finite Automata: no temporary memory

• Pushdown Automata: stack

• Turing Machines: random access memory

Copyright 2011@Xu Dezhi 9

Example: Vending Machines

Example: Compilers for Programming Languages

Random Access Memory

Examples: Any Algorithm

Finite Pushdown Turing

Less power More power

Copyright 2011@Xu Dezhi 14

S = { j : j > 0, and j = 2k for some k>0 }

S = { j : j is nonnegative and even }

Universal Set: all possible elements

Powerset of S = the set of all the subsets of S

Generalizes to more than two sets

e. g. if R = „>‟: 2 > 1, 3 > 2, 3 > 1

Equivalence class of 1 = {1, 2}

Walk is a sequence of adjacent edges

Path is a walk where no edge is repeated

Simple path: no node is repeated

Cycle: a walk from a node (base) to itself

Simple cycle: only the base node is repeated

A cycle that contains each edge once

A simple cycle that contains all nodes

Trees have no cycles

We have statements P1, P2, P3, …

From Inductive hypothesis: L(k) <= 2k

L(k+1) <= 2 * L(k) <= 2 * 2k = 2k+1

(we add at most two nodes for every leaf of level k)

Example of recursive function:

f(n) = f(n-1) + f(n-2)

We want to prove that a statement P is true

• we assume that P is false

We will show that this is impossible

Thus, m and n have common factor 2

– A concrete example is 16/2. Therefore,

case 1: is rational. done, otherwise

Is example 2 the constructive proof?

Why if yes? Why if no?

Show that there is no "largest integer“.

Is example 3 the constructive proof?

Why if yes? Why if no?

If you divide m by any of the prime numbers that

because each prime number less than or equal to

Therefore any prime factors of m must be

Is example 4 the constructive proof?

Why if yes? Why if no?

String: A sequence of letters

Examples: “cat”, “dog”, “house”, …

Defined over an alphabet:

wv  a1a2 anb1b2 bm abbabbbaaa

abba  abba  abba

Example: abba   abbaabba

Set size {}  1

a, ab, aaaa  bb, ab  {a, ab, bb, aaaa}