Theoretical Computer Science I:

Automata and Formal Languages

Prof. Dr. Sjouke Mauw

Slides for Chapters 1, 2 and 3.

Version 2024.1
‚ Instructors: Sjouke Mauw, Reynaldo Gil-Pons and Manel
Jerbi.
‚ Format: 14 lectures, 14 practical work sessions.
‚ Lecture notes & slides: Informal, for internal UL use only.
‚ Book: Michael Sipser, Introduction to the Theory of
Computation, 3rd Ed. (EMEA adaptation), Cengage Learning,
2013.
Evaluation

‚ Take-home Assignments (20% of final grade).

‚ Mid-term Exam on Chapters 1 to 7 of these lecture notes
(40% of final grade).
‚ Reduced/Full Exam
‚ If you pass the Mid-term Exam then you can do the Reduced
End-term Exam on Chapters 8 to 9 of these lecture notes
(40% of final grade).
‚ If you fail the Mid-term Exam or you participate in the Retake
Exam in later semesters, then you have to do the Full Exam
on Chapters 1 to 9 of these lecture notes (80% of final grade).
‚ In case of a Retake Exam, the grades for the Take-home
Assignments will be kept.

The mid-term exam will take place on Wednesday 10 April 2024

and will cover chapters 1 to 7 of the lecture notes.
Chapter 1. Introduction
Theoretical foundation of computer science

‚ What is a computer?
‚ What is a computation?
‚ What can be computed?

‚ Turing machine (1936).

What is a language?

‚ tred, green, yellow , blue, black, whiteu

‚ All even numbers t2n | n ě 0u
‚ All bit strings having an equal number of zeroes and ones
tw P t0, 1u˚ | #0 pw q “ #1 pw qu
‚ All palindromes (e.g. madam) tw P ta, b, . . . , zu˚ | w “ w R u
‚ All prime numbers t2, 3, 5, . . .u
‚ All English words ta, aback, abacus, . . . , zuluu
‚ All English jokes.
Applications
‚ user-interfaces (e.g. coffee machine)
‚ communication protocols (e.g. TCP/IP)
‚ distributed systems (e.g. process theory)
‚ compilers (e.g. parsing and compiling a program)
‚ algorithmics (e.g. substring matching)
‚ semantics (e.g. meaning of a specification language)
‚ cryptography
‚ ...

“There is nothing more practical than a good theory.”

(Kurt Lewin)
Chapter 2. Preliminaries
Sets

‚ t1, 2, 3u X t2, 3, 4u “ t2, 3u intersection

Sets

‚ t1, 2, 3u X t2, 3, 4u “ t2, 3u intersection

‚ t1, 2, 3u Y t2, 3, 4u “ t1, 2, 3, 4u union
Sets

‚ t1, 2, 3u X t2, 3, 4u “ t2, 3u intersection

‚ t1, 2, 3u Y t2, 3, 4u “ t1, 2, 3, 4u union
‚ t1, 2, 3uzt2, 3, 4u “ t1u difference
Sets

‚ t1, 2, 3u X t2, 3, 4u “ t2, 3u intersection

‚ t1, 2, 3u Y t2, 3, 4u “ t1, 2, 3, 4u union
‚ t1, 2, 3uzt2, 3, 4u “ t1u difference
‚ If the universe is N, then t1, 2u “ t0, 3, 4, 5, . . .u complement
Sets

‚ t1, 2, 3u X t2, 3, 4u “ t2, 3u intersection

‚ t1, 2, 3u Y t2, 3, 4u “ t1, 2, 3, 4u union
‚ t1, 2, 3uzt2, 3, 4u “ t1u difference
‚ If the universe is N, then t1, 2u “ t0, 3, 4, 5, . . .u complement
‚ Ppt1, 2, 3uq “ power set
tH, t1u, t2u, t3u, t1, 2u, t1, 3u, t2, 3u, t1, 2, 3uu
Sets

‚ t1, 2, 3u X t2, 3, 4u “ t2, 3u intersection

‚ t1, 2, 3u Y t2, 3, 4u “ t1, 2, 3, 4u union
‚ t1, 2, 3uzt2, 3, 4u “ t1u difference
‚ If the universe is N, then t1, 2u “ t0, 3, 4, 5, . . .u complement
‚ Ppt1, 2, 3uq “ power set
tH, t1u, t2u, t3u, t1, 2u, t1, 3u, t2, 3u, t1, 2, 3uu
‚ |t5, 6, 7u| “ 3 norm
Sets

‚ t1, 2, 3u X t2, 3, 4u “ t2, 3u intersection

‚ t1, 2, 3u Y t2, 3, 4u “ t1, 2, 3, 4u union
‚ t1, 2, 3uzt2, 3, 4u “ t1u difference
‚ If the universe is N, then t1, 2u “ t0, 3, 4, 5, . . .u complement
‚ Ppt1, 2, 3uq “ power set
tH, t1u, t2u, t3u, t1, 2u, t1, 3u, t2, 3u, t1, 2, 3uu
‚ |t5, 6, 7u| “ 3 norm
‚ t1, 2, 3u ˆ ta, bu “ tp1, aq, p2, aq, p3, aq, p1, bq, p2, bq, p3, bqu
product
Sets

‚ t1, 2, 3u X t2, 3, 4u “ t2, 3u intersection

‚ t1, 2, 3u Y t2, 3, 4u “ t1, 2, 3, 4u union
‚ t1, 2, 3uzt2, 3, 4u “ t1u difference
‚ If the universe is N, then t1, 2u “ t0, 3, 4, 5, . . .u complement
‚ Ppt1, 2, 3uq “ power set
tH, t1u, t2u, t3u, t1, 2u, t1, 3u, t2, 3u, t1, 2, 3uu
‚ |t5, 6, 7u| “ 3 norm
‚ t1, 2, 3u ˆ ta, bu “ tp1, aq, p2, aq, p3, aq, p1, bq, p2, bq, p3, bqu
product
‚ The following predicates are true:
a P ta, bu element of
tbu Ď ta, bu, H Ď ta, bu, ta, bu Ď ta, bu subset of
Graphs

‚ Example:
a b c
Graphs

‚ Example:
a b c
‚ A graph is defined by the structure G “ pV , E q, where
‚ V is a set of vertices, and
‚ E Ď V ˆ V is a set of edges.
This graph is represented by the structure pV , E q, where
‚ V “ ta, b, cu, and
‚ E “ tpa, bq, pb, aq, pb, bq, pb, cqu
Graphs

‚ Example:
a b c
‚ A graph is defined by the structure G “ pV , E q, where
‚ V is a set of vertices, and
‚ E Ď V ˆ V is a set of edges.
This graph is represented by the structure pV , E q, where
‚ V “ ta, b, cu, and
‚ E “ tpa, bq, pb, aq, pb, bq, pb, cqu
‚ Edges can be labeled.
‚ A path is a sequence of nodes connected by edges.
Example: the sequence ababbbc is a path in the previous
graph. This path starts at node a and ends at node c.
Strings

‚ An alphabet Σ is a nonempty, finite set of symbols.

Examples:
‚ ta, . . . , zu
‚ t0, 1u
‚ tapple, pear , orangeu
‚ A string is a finite sequence of symbols over the alphabet.
Examples:
‚ abracadabra
‚ 0111001
‚ apple apple orange
‚ ε (the empty string)
Operations on strings

‚ Concatenation of strings:
if x “ x1 x2 . . . xn and y “ y1 y2 . . . ym then

xy “ x1 x2 . . . xn y1 y2 . . . ym

We have xε “ εx “ x.
Example: The concatenation of 110 and 011 is 110011.
Inductive structure of strings

‚ A string is either the empty string ε or constructed by adding

a symbol a P Σ to a string x, yielding xa.
‚ Formulated differently: Every string can be constructed by
starting from the empty string and then appending a number
of symbols.
Example: The string abc is constructed by starting from ε and
then adding a, b, and c in this order.
This yields εabc, which is abc.
Inductive definition of length

Two ways to define length of a string:

1 informal: |x1 x2 . . . xn | “ n
Inductive definition of length

Two ways to define length of a string:

1 informal: |x1 x2 . . . xn | “ n
2 inductive:
|ε| “ 0
|xa| “ |x| ` 1
Inductive definition of length

Two ways to define length of a string:

1 informal: |x1 x2 . . . xn | “ n
2 inductive:
|ε| “ 0
|xa| “ |x| ` 1
Example:
|abc|
“ |εabc|
“ |εab| ` 1
“ p|εa| ` 1q ` 1
“ pp|ε| ` 1q ` 1q ` 1
“ pp0 ` 1q ` 1q ` 1
“3
Inductive definition of string reversal

Two ways to define reversing a string:

1 informal: px1 x2 . . . xn qR “ xn . . . x2 x1
Inductive definition of string reversal

Two ways to define reversing a string:

1 informal: px1 x2 . . . xn qR “ xn . . . x2 x1
2 inductive:
εR “ ε
pxaqR “ ax R
Inductive definition of string reversal

Two ways to define reversing a string:

1 informal: px1 x2 . . . xn qR “ xn . . . x2 x1
2 inductive:
εR “ ε
pxaqR “ ax R
Example:
pabcqR
“ pεabcqR
“ ppεabqcqR
“ cpεabqR
“ cbpεaqR
“ cbapεqR
“ cbaε
“ cba
Repeated concatenation

Two ways to define repeated concatenation of a string:

1 informal: x k “ xx . . . x (k times).
Note that x 0 “ ε.
Repeated concatenation

Two ways to define repeated concatenation of a string:

1 informal: x k “ xx . . . x (k times).
Note that x 0 “ ε.
2 inductive:
‚ x0 “ ε
‚ x n`1 “ xpxqn
Repeated concatenation

Two ways to define repeated concatenation of a string:

1 informal: x k “ xx . . . x (k times).
Note that x 0 “ ε.
2 inductive:
‚ x0 “ ε
‚ x n`1 “ xpxqn
Example:
pabq3
“ ab pabq2
“ ab ab pabq1
“ ab ab ab pabq0
“ ab ab ab ε
“ ab ab ab
Kleene star and language

‚ Kleene star Σ˚ on alphabet Σ denotes all finite strings over Σ.

Example:
ta, bu˚ “ tε, a, b, aa, ab, ba, bb, aaa, aab, aba, abb, baa, . . .u
tau˚ “ tε, a, aa, aaa, aaaa, . . .u
tu˚ “ tεu
Kleene star and language

‚ Kleene star Σ˚ on alphabet Σ denotes all finite strings over Σ.

Example:
ta, bu˚ “ tε, a, b, aa, ab, ba, bb, aaa, aab, aba, abb, baa, . . .u
tau˚ “ tε, a, aa, aaa, aaaa, . . .u
tu˚ “ tεu
‚ A language is a set of strings, L Ď Σ˚ .
Examples of languages over the alphabet Σ “ ta, bu:
‚ The empty set H.
‚ The set containing the empty string tεu.
‚ The singleton set tbu.
‚ The finite set taa, ab, ba, bbu.
‚ The infinite set tε, a, aa, aaa, aaaa, . . .u.
‚ The full set of all strings ta, bu˚ .
Chapter 3. Finite automata
Example automaton

0 1
1 0
a b c
0, 1
Example automaton

0 1
1 0
a b c
0, 1
‚ States (a, b, c).
‚ Start state (a, indicated with incoming arrow).
‚ End state (b, indicated with double border).
‚ Transitions (labeled edges).
Note that the edge labeled 0, 1 stands for two edges: one with
label 0 and one with label 1.
Behaviour of an automaton

0 1
1 0
a b c
0, 1
‚ Start in start state.
‚ Follow a transition, while remembering its label.
‚ Repeat this; you are allowed to stop if you are in an end state.
‚ If you stop, the sequence of labels encountered forms a string,
accepted by the automaton.
Behaviour of an automaton

0 1
1 0
a b c
0, 1
‚ Start in start state.
‚ Follow a transition, while remembering its label.
‚ Repeat this; you are allowed to stop if you are in an end state.
‚ If you stop, the sequence of labels encountered forms a string,
accepted by the automaton.

Example:
1101 is accepted
0110 is not accepted
Formal definition of a Finite Automaton

Definition
A finite automaton is a 5-tuple pQ, Σ, δ, q0 , F q, where
1 Q is a finite set called the states,
Formal definition of a Finite Automaton

Definition
A finite automaton is a 5-tuple pQ, Σ, δ, q0 , F q, where
1 Q is a finite set called the states,
2 Σ is a finite set called the alphabet,
Formal definition of a Finite Automaton

Definition
A finite automaton is a 5-tuple pQ, Σ, δ, q0 , F q, where
1 Q is a finite set called the states,
2 Σ is a finite set called the alphabet,
3 δ : Q ˆ Σ Ñ Q is a total function, which is called the
transition function,
Formal definition of a Finite Automaton

Definition
A finite automaton is a 5-tuple pQ, Σ, δ, q0 , F q, where
1 Q is a finite set called the states,
2 Σ is a finite set called the alphabet,
3 δ : Q ˆ Σ Ñ Q is a total function, which is called the
transition function,
4 q0 P Q is the start state, and
Formal definition of a Finite Automaton

Definition
A finite automaton is a 5-tuple pQ, Σ, δ, q0 , F q, where
1 Q is a finite set called the states,
2 Σ is a finite set called the alphabet,
3 δ : Q ˆ Σ Ñ Q is a total function, which is called the
transition function,
4 q0 P Q is the start state, and
5 F Ď Q is the set of accept states.
Example of formal definition

0 1
1 0
a b c
0, 1
1 States Q “ ta, b, cu
2 Alphabet Σ “ t0, 1u
δpa, 0q “ a
δpa, 1q “ b δ 0 1
δpb, 0q “ c a a b
3 Transition function δ or shorter
δpb, 1q “ b b c b
δpc, 0q “ b c b b
δpc, 1q “ b
4 Start state q0 “ a
5 Accept states F “ tbu
Language recognized by an automaton

LpMq “ The language of strings accepted by finite automaton M.

We say that M recognizes language LpMq.
Language recognized by an automaton

LpMq “ The language of strings accepted by finite automaton M.

We say that M recognizes language LpMq.

Example: Let M be defined by

0 1
1 0
a b c
0, 1
Language recognized by an automaton

LpMq “ The language of strings accepted by finite automaton M.

We say that M recognizes language LpMq.

Example: Let M be defined by

0 1
1 0
a b c
0, 1

LpMq “ tw P t0, 1u˚ |

w contains at least one 1 and an even number of 0s follow the last 1u
Example M2

Which language is recognized by the following automaton M2 ?

0 1
1
q1 q2
0
Example M2

Which language is recognized by the following automaton M2 ?

0 1
1
q1 q2
0

LpM2 q “ tw | w ends in a 1u
Example M3

Which language is recognized by the following automaton M3 ?

0 1
1
q1 q2
0
Example M3

Which language is recognized by the following automaton M3 ?

0 1
1
q1 q2
0

LpM3 q “ tw | w is the empty string or ends in a 0u

Example trap node
Which language is recognized by the automata A and A1 ?
A:
a
b
q1 q2
Example trap node
Which language is recognized by the automata A and A1 ?
A:
a
b
q1 q2

A1 :
a
b a, b a, b
q1 q2 q3
Example trap node
Which language is recognized by the automata A and A1 ?
A:
a
b
q1 q2

A1 :
a
b a, b a, b
q1 q2 q3

LpAq “ LpA1 q “ tan b | n ě 0u

A is a shorthand notation for A1 .
Formal definition of string accepted by an automaton

Definition
Let M be a finite automaton pQ, Σ, δ, q0 , F q.
String w “ w1 w2 . . . wn P Σ˚ is accepted by M if there exists a
sequence of states r “ r0 r1 r2 . . . rn P Q ˚ such that
1 r0 “ q0 ,
2 δpri , wi`1 q “ ri`1 , (for i “ 0, . . . , n ´ 1)
3 rn P F .
Formal definition of string accepted by an automaton

Definition
Let M be a finite automaton pQ, Σ, δ, q0 , F q.
String w “ w1 w2 . . . wn P Σ˚ is accepted by M if there exists a
sequence of states r “ r0 r1 r2 . . . rn P Q ˚ such that
1 r0 “ q0 ,
2 δpri , wi`1 q “ ri`1 , (for i “ 0, . . . , n ´ 1)
3 rn P F .

r0 r1 r2 r3 . . . rn´1 rn
Formal definition of string accepted by an automaton

Definition
Let M be a finite automaton pQ, Σ, δ, q0 , F q.
String w “ w1 w2 . . . wn P Σ˚ is accepted by M if there exists a
sequence of states r “ r0 r1 r2 . . . rn P Q ˚ such that
1 r0 “ q0 ,
2 δpri , wi`1 q “ ri`1 , (for i “ 0, . . . , n ´ 1)
3 rn P F .

w
1 2 w 3 w n w
r0 ÝÑ r1 ÝÑ r2 ÝÑ r3 . . . rn´1 ÝÑ rn
Formal definition of string accepted by an automaton

Definition
Let M be a finite automaton pQ, Σ, δ, q0 , F q.
String w “ w1 w2 . . . wn P Σ˚ is accepted by M if there exists a
sequence of states r “ r0 r1 r2 . . . rn P Q ˚ such that
1 r0 “ q0 ,
2 δpri , wi`1 q “ ri`1 , (for i “ 0, . . . , n ´ 1)
3 rn P F .

1w 2 w 3 w n w
q0 “ r0 ÝÑ r1 ÝÑ r2 ÝÑ r3 . . . rn´1 ÝÑ rn P F
Example accepted string

0 1
1
p q
0
This automaton accepts the string w1 w2 w3 w4 w5 “ 11010.
Example accepted string

0 1
1
p q
0
This automaton accepts the string w1 w2 w3 w4 w5 “ 11010.

Proof: Find sequence of states r0 r1 r2 r3 r4 r5 “ pqqpqp, such that

‚ r0 “ p is the start state,
‚ δpr0 , w1 q “ r1 , meaning δpp, 1q “ q, and
δpr1 , w2 q “ r2 , meaning δpq, 1q “ q, and
δpr2 , w3 q “ r3 , meaning δpq, 0q “ p, etc.
‚ r5 “ p is a final state.
Example accepted string

0 1
1
p q
0
This automaton accepts the string w1 w2 w3 w4 w5 “ 11010.

Proof: Find sequence of states r0 r1 r2 r3 r4 r5 “ pqqpqp, such that

‚ r0 “ p is the start state,
‚ δpr0 , w1 q “ r1 , meaning δpp, 1q “ q, and
δpr1 , w2 q “ r2 , meaning δpq, 1q “ q, and
δpr2 , w3 q “ r3 , meaning δpq, 0q “ p, etc.
‚ r5 “ p is a final state.

1 1 0 1 0
Formulated differently, we have p Ý
ÑqÝ
ÑqÝ
ÑpÝ
ÑqÝ
Ñ p.
Language recognized by an automaton

Definition
Let M “ pQ, Σ, δ, q0 , F q be a finite automaton.
A language A is recognized by M if

A “ tw P Σ˚ | M accepts w u

The language recognized by M is denoted by LpMq.

Language recognized by an automaton

Definition
Let M “ pQ, Σ, δ, q0 , F q be a finite automaton.
A language A is recognized by M if

A “ tw P Σ˚ | M accepts w u

The language recognized by M is denoted by LpMq.

Denote the set of finite automata by DFA.

Language recognized by an automaton

Definition
Let M “ pQ, Σ, δ, q0 , F q be a finite automaton.
A language A is recognized by M if

A “ tw P Σ˚ | M accepts w u

The language recognized by M is denoted by LpMq.

Denote the set of finite automata by DFA.

Definition
A language is called a regular language if some finite automaton
recognizes it.

L P Reg ðñ DMPDFA L “ LpMq

Example regular language L1

Prove that the following language is regular:

L1 “ tabw | w P ta, bu˚ u

Example regular language L1

Prove that the following language is regular:

L1 “ tabw | w P ta, bu˚ u

Proof: Construct a finite automaton that recognizes L1 .

Example regular language L1

Prove that the following language is regular:

L1 “ tabw | w P ta, bu˚ u

Proof: Construct a finite automaton that recognizes L1 .

q0 a q2 b q3

1. Make backbone for ab.

Example regular language L1

Prove that the following language is regular:

L1 “ tabw | w P ta, bu˚ u

Proof: Construct a finite automaton that recognizes L1 .

q0 a q2 b q3

2. Determine final state(s).

Example regular language L1

Prove that the following language is regular:

L1 “ tabw | w P ta, bu˚ u

Proof: Construct a finite automaton that recognizes L1 .

a, b

q0 a q2 b q3

3. Finish transitions from q3 .

Example regular language L1

Prove that the following language is regular:

L1 “ tabw | w P ta, bu˚ u

Proof: Construct a finite automaton that recognizes L1 .

q1
b a, b

q0 a q2 b q3

4. Finish transitions from q0 to trap node.

Example regular language L1

Prove that the following language is regular:

L1 “ tabw | w P ta, bu˚ u

Proof: Construct a finite automaton that recognizes L1 .

q1
b a a, b

q0 a q2 b q3

5. Finish transitions from q2 to trap node.

Example regular language L1

Prove that the following language is regular:

L1 “ tabw | w P ta, bu˚ u

Proof: Construct a finite automaton that recognizes L1 .

a, b
q1
b a a, b

q0 a q2 b q3

6. Finish transitions from trap node.

Example regular language L2
Prove that the following language is regular:

L2 “ tw P t0, 1u˚ | w does not have a substring 001u

Example regular language L2
Prove that the following language is regular:

L2 “ tw P t0, 1u˚ | w does not have a substring 001u

Proof: Construct a finite automaton that recognizes L2 .

Example regular language L2
Prove that the following language is regular:

L2 “ tw P t0, 1u˚ | w does not have a substring 001u

Proof: Construct a finite automaton that recognizes L2 .

q0 0 q1 0 q2 1 q3

1. Make backbone for 001.

Example regular language L2
Prove that the following language is regular:

L2 “ tw P t0, 1u˚ | w does not have a substring 001u

Proof: Construct a finite automaton that recognizes L2 .

q0 0 q1 0 q2 1 q3

2. Determine final state(s).

Example regular language L2
Prove that the following language is regular:

L2 “ tw P t0, 1u˚ | w does not have a substring 001u

Proof: Construct a finite automaton that recognizes L2 .

0, 1
q0 0 q1 0 q2 1 q3

3. Finish transitions from q3 (trap node).

Example regular language L2
Prove that the following language is regular:

L2 “ tw P t0, 1u˚ | w does not have a substring 001u

Proof: Construct a finite automaton that recognizes L2 .

0
0, 1
q0 0 q1 0 q2 1 q3

4. Finish transitions from q2 .

Example regular language L2
Prove that the following language is regular:

L2 “ tw P t0, 1u˚ | w does not have a substring 001u

Proof: Construct a finite automaton that recognizes L2 .

0
0 0, 1
q0 q1 0 q2 1 q3
1
5. Finish transitions from q1 .
Example regular language L2
Prove that the following language is regular:

L2 “ tw P t0, 1u˚ | w does not have a substring 001u

Proof: Construct a finite automaton that recognizes L2 .

1 0
0 0, 1
q0 q1 0 q2 1 q3
1
6. Finish transitions from q0 .
Example regular language L2
Prove that the following language is regular:

L2 “ tw P t0, 1u˚ | w does not have a substring 001u

Proof: Construct a finite automaton that recognizes L2 .

1 0
0 0, 1
q0 q1 0 q2 1 q3
1
Intuitive meaning of the nodes:
‚ q0 : no problem;
‚ q1 : we have read one 0;
‚ q2 : we have read two 0’s;
‚ q3 : we have read 001 and fail;
Example regular language L3

Prove that the following language is regular:

L3 “ tawa | w P ta, bu˚ u

Example regular language L3

Prove that the following language is regular:

L3 “ tawa | w P ta, bu˚ u

Proof: Construct a finite automaton that recognizes L3 .

Example regular language L3

Prove that the following language is regular:

L3 “ tawa | w P ta, bu˚ u

Proof: Construct a finite automaton that recognizes L3 .

a, b
q1
b
b a a
q0 a q2 q3
b

a a
The backbone is q0 Ý
Ñ q2 Ý
Ñ q3