Formal Language and Automata

Theory

Chapter 1:Introduction
 Theory of Computation
• It is a branch of computer science that deals with how efficiently a problem can be solved
on a model of computation, using an algorithm.
• Theory of computation is divided into 3 fields:
1. Automata Theory and Formal Languages
2. Computability Theory
3. Complexity Theory
• The goal is to ascertain the power and limits of computation.
• In order to study these aspects, it is necessary to define precisely what constitutes a model of
computation as well as what constitutes a computational problem.
• This is the purpose of automata theory.
• The computational models are automata, while the computational problems are formulated as
formal languages. 2
 Cont’d...
1. Automata Theory and Formal
Languages
• It deals with the definition and properties of various mathematical model of computation.
• Examples:
i. Finite Automata(FA)
ii. Context Free Grammar (CFG)
iii. Turing Machine (TM)
2. Computability Theory
• It deals with what can/cannot be computed by the model.
3. Complexity Theory
• It groups the computable problems based on their hardness.
Main purpose of Theory of Computation
• Develop formal mathematical model of computation that reflect real world computers.
3
 Cont'd….

2. Computability
Theory
•It deals with what can/cannot be computed by the model.
3. Complexity Theory
• It groups the computable problems based on their hardness.

4
 Alphabets and Strings
1. Symbol
• Symbol(often also called character) is the smallest building block,
which can be any alphabet, letter, or picture.
• Example
• a, b, c, …, z
• 0,1,…, 9
• +,*, #, … special characters
2. Alphabet
• An alphabet is a finite, non-empty set of symbols.
• It is denoted by ∑.
• Examples: ∑ = {0,1} set of binary alphabets. 5
 Cont'd….
• Examples: ∑ = {a, b, c, …, z} set of all lowercase letters.
∑ = {A, B, C, …Z} is an alphabet of uppercase letters.
∑ = {+, #, &,$, …} set of all special characters
3. String or Word
• It is a finite collection of symbols from the alphabet.
• The string is denoted by w.
• Example: w = 01110 is string from binary alphabet ∑ = {0,1}
w = aabbaacaa is string from alphabet ∑ = {a, b, c}

Symbol Alphabet String/

Word 6
 Cont'd….
4. Empty String
• Empty string is a string with zero occurrences of symbol(no symbol)..
• It is denoted by ' ε ' → no symbol {}
5. Length of String
• It is the number of symbols in the string.
• It is denoted by |w|.
• Example 1: w = 010110101 from binary alphabet ∑ = {0,1}
Length of the string |w| =9.
Example 2: w = 010
Length of Sting |w| = 3 7
 Cont'd….
6. Concatenation of String : joining of 2 or more strings.
• Let x = a1, a2, a3, …,
an y= b1,b2,b3, …, bn
• Concatenation of string
xy = a1, a2, a3, …,
anb1,b2,b3, …, bn
• Example: S = ababa and T = cdcddc
Concatenation of string ST = ababacdcddc
7. Power of an Alphabet
• If ' ∑ ' is an alphabet, we can express set of all string of certain length
8
 Cont'd….
• Example: ∑ = {0,1} has 2 symbols.
(i) ∑1 = {0,1} therefore 21 = 2 where k = 1
(ii) ∑2 = {00,01, 10,11} therefore 22 = 4 where k = 2
(ii) ∑3 = {000, 001, 010, 011, 100, 101, 110, 111 } therefore 23 = 8
where k = 3
• The set of strings over an alphabet ∑ is usually denoted by is
∑* kleen
closure.
• For instance ∑* = {0,1}*
= {ε,0,1,00,01,10,11,…}
9
• Therefore, kleen closure ∑* = ∑0 U ∑1 U ∑2 U ∑3 …
 Cont'd….
• The set of strings over an alphabet ∑ excluding ε is denoted by ∑+ is called
kleen plus.
• For instance ∑+ = {0,1}+
= {0,1,00,01,10,11,…}
• Therefore, kleen plus ∑+ = ∑* - {ε } or
∑+ = ∑* - {ε }
8. Language:
• A language is a collection of appropriate string.
• A language which is formed over Σ can be Finite or Infinite.
10
Example: 1 Cont'd….

• L1 = {Set of string of length 2} = {aa, bb, ba, bb} Finite Language

• L2 = {Set of all strings starts with 'a'} = {a, aa, aaa, abb, abbb, ababb}
Infinite Language
Grammar:
• A grammar is finite set of rules defining a language.

11
 Formal Definition of Grammar
Definition: set of rules used to describe the string of the language.
Grammar G is a collection of four tuples.
G = {V, T, P, S }
V is finite set of variables
T is finite set terminal symbols
P is finite number of production rules
S is starting symbol
S→
Example 1: S → aS Example 2: AB
S → A→e
bS S B→ 12
 Cont’d …
From Example 1 above: From Example 2 above:
S is starting symbol S is starting symbol
{a,b} are terminal symbols A,B are set of variables
Number of variables {-} e,b are set of terminals
Number of production =3 Number of production
=3
Language: set of strings
generated by any grammar.
Example 1: S → Example 2: S → aS|bS|a
AB • This grammar can be also
A→a written as:
S → aS
L=
B→ S →
{ab} bS S
b 13
→a
 Cont’d …
• From Example 2 above: S → aS|bS|a
S
S S S
aS
a aS bS
a
{a} bS
aS a
aS
a {ba}
• L= (a+b)*a a
{aaa}
• Strings of the language are: {abaa}
{a, aaa, aba, aaba} = ending with a.

14
 Cont’d …
• Problems: Identify languages generated by grammars
• Example 1: S → aS|bS|ε
• Minimum string is ε.
• The grammar can generate any combination of a and b with including empty string.
• Strings generated by the above grammar are:
• Strings are: {ε, a, b, aa, bb, ab, ba, abb, aab, aba, abaa, …}
• Language L = (a+b)*
• Example 2: S → aS|bS|a|b
• Minimum string is a and b without including ε.
• String generated by the above grammar are:
• {a, b, aa, bb, ab, ba, abb, aab, …} Language L
=(a+b)+ 15
 Automata
• Theory of automata is a theoretical branch of computer science and mathematical.
• It is the study of abstract machines and the computation problems that can be solved
using these machines.
• The abstract machine is called the automata.
• The main motivation behind developing the automata theory was to
develop methods to describe and analyze the dynamic behavior of discrete systems.
• This automaton consists of states and transitions.
• The State is represented by circles, and the Transitions is represented by arrows.
• Automata is the kind of machine which takes some string as input and this input
goes through a finite number of states and may enter in the final state.

16
 Finite Automata
• Finite automata are used to recognize patterns.
• It takes the string of symbol as input and changes its state accordingly.
• When the desired symbol is found, then the transition occurs.
• At the time of transition, the automata can either move to the next state or stay in
the same state.
• Finite automata have two states:
• Accept state or
• Reject state.
• When the input string is processed successfully, and the automata reached
its final state, then it will accept, otherwise it will reject the input.
17
 Cont’d …

• Finite automata can be represented by:

• Graphical (Transition diagrams or transition tables)
• Tabular (transition tables)
• Mathematical (Transition function or Mapping function)

18
 Formal Definition of FA

• A finite automaton is a collection of 5-tuples (Q, ∑, δ, q0, F), where:

• Q is finite set of states
• ∑ is finite set of the input alphabets
• q0 ϵ Q is start state also called initial state
• F is accept state or final state
• δ: Q x ∑ → Q is transition function

19
 Transition Diagrams
• Transition diagram is a directed graph associated with vertices of the
graph corresponds to the state of finite automata.
1
0 1
• Example:
1 q2
q0 q1
Final or
Start or Accept State
Initial State
• {0, 1} are inputs
• qo initial state
• q1 intermediate state
• q2 is final state 20
 Transition Table
• Transition Table is basically a tabular representation, of the transition function that
takes two arguments( a state and a symbol ) and returns a value (next state).

• Rows corresponds to states,

• Columns corresponds to input symbols
• Entries corresponds to next state
• The start state is marked with an arrow (→)
• Accept state are marked with a star(*)
• q1 intermediate state
• q2 is final state
21
 Cont’d …

• Example:
0, 1
0
0 1
1 q1
q0 →qo q0 q1
* q1 q1 q1
Transition diagram

• qo initial state Transition table

• q2 is final state
• {0, 1} are inputs

22
 Transition Function
• The mapping function or transition function denoted by δ.
 Two parameters are passed to this transition function
1. current state
2. input symbols
• Transition function always returns a state which can be called as next state.
δ(current state, current input symbol) = next state
Qx∑→Q

• Example: δ(q0, a) = q1
δ(q0, 1) =
23
 Types of Automata

Finite Automata

Finite Automata without output Finite Automata with output

Deterministic Non-deterministic Non-deterministic Moore Mealy

Finite Finite Automata Finite Automata Machine Machine
Automata without output with ε - Moves
without output (NFA)
(DFA)
24
 Deterministic Finite Automata (DFA)
• Finite automata is called deterministic finite automata if the machine read an
input string one symbol at a time.
• Deterministic refers to the uniqueness the computation.
• In DFA, there is only one path for specific input symbol from current state
to the next state.
• DFA does not accept the null move.
• DFA cannot change the state without any input character.

• DFA can contains multiple final state.

• DFA is used in lexical analysis in compiler.
25
 Cont’d …

a, b

q0 b q1

a
qo is initial state
q2
q1 is final state
{a, b} input
symbols

26
 Formal Definition of DFA

• A DFA is a collection of 5 tuples (same as FA)

 Q is finite set of states
∑ is finite set of input symbols
 q0 is initial state
 F is final state
 δ is transition function
• Transition function can be defined as δ : Q x ∑ → Q

27
 Cont’d …

• The state is represented by vertices.

• The arc labeled with an input character show the transition
• The initial state is marked with an arrow.
• The final state is denoted by a double circle.
Acceptance of a Language
• A language acceptance is defined by “if a string w is accepted by a machine m”
• If it is reaching the final state F by taking the string w.
• Not accepted if not reaching the final state.
• Example: L = → q0 no string
{Ø}
→ q0 .
L={∑} 28
 Cont’d …

• DFA to accept
'a'
q0 a q1

• DFA to accept zero or more q0

'a'
• L = { ∑, a, aa, aaa, aaaa, …. }

• Note : states depends on the length of the string.

29
 Cont’d …

• Example 1: Let DFA Q = {q0, q1,q2}

be
∑ = {0, 1}
initial state ={q0}
final state ={q2}
Transition table Transition Diagram
Present State 0 1
→q0 q0 q1
q1 q2 q0
*q2 q1 q2

30
 Cont’d …

• Example 2: Design DFA with given ∑ = {0,1} that accepts those string which starts
with 1 and ends with 0.
• Solution: minimum length of string is 2.
L = { 10, 100, 1010, 11001100, …. }
Number of states = |minimum length| + 1

DFA 0
1
Diagram:

q0 1 q1 0 q2
1
31
 Cont’d …

• Example 3: Design DFA with given ∑ = {0,1} that accepts even number of 0’s and
even number of 1’s.
• Solution: minimum length of string is 3.
L = { 001, 110, 100, …. }

DFA Diagram:

32
 Cont’d …

• Example 4: Construct DFA that accepting all strings over ∑ = {a,b} ending with
ab.
• Solution: minimum length of string is 2.
number of states = 2+1 =3
L = {ab, aab, bab,
bbab…. } b
DFA b a
Diagram:
q0 a q1 b q2
a

b 33
 Non Deterministic Finite Automata (DFA)

• The finite automata are called NFA when there exist many paths for specific input
from the current state to the next state.
• It is easy to construct an NFA than DFA for a given regular language.
• Every NFA is not DFA, but each NFA can be translated into DFA.
• NFA is defined in the same way as DFA but with the following two exceptions:
1. it contains multiple next states, and
2. it contains ε transition.

34
 Cont’d …

• In the following image,

• we can see that from state q0 for input a, there are two next states q1 and q2
• similarly, from q0 for input b, the next states are q0 and q1.
• Thus it is not fixed or determined that with a particular input where to go next.
• Hence this FA is called non-deterministic finite automata.

35
 Cont’d …
Formal definition of NFA
• NFA also has five tuples same as DFA, but with different transition function,
as shown follows:
δ: Q x ∑ →2Q
• where,
• Q finite set of States
• ∑ finite set of input symbol
• q0 initial state
• F final state
•δ transition function

36
 Cont’d …
Graphical Representation of an NFA
• An NFA can be represented by digraphs called state diagram. In which:
 The state is represented by vertices.
 The arc labeled with an input character show the transitions.
 The initial state is marked with an arrow.
 The final state is denoted by the double circle.
• Example 1: Construct transition diagram of NFA from the below given
information.

Q = {q0, q1, q2}

∑ = {0, 1}
q0 =
{q0} F = 37
 Cont’d …
• Transition Table for the above NFA
diagram
Present State Next state for Input 0 Next State of Input 1
→q0 q0, q1 q1
q1 q2 q0
*q2 q2 q1, q2
• In the above diagram, we can see that when the current state is q0, on input 0, the next state
will be q0 or q1, and on 1 input the next state will be q1.
• When the current state is q1, on input 0 the next state will be q2 and on 1 input, the next
state will be q0.
• When the current state is q2, on 0 input the next state is q2, and on 1 input the next state
will be q1 or q2.
38
 Cont’d …
• Example 2: NFA with ∑ = {0, 1} accepts all strings starts with 01.
• Solution:

• Transition Table:

Present State Next state for Input 0 Next State of Input 1

→q0 q1 ε
q1 ε q2
*q2 q2 q2
39
 Cont’d …
• Example 3: NFA with ∑ = {0, 1} and accept all string of length at least 2.
• Solution:

• Transition Table:
Present State Next state for Input 0 Next State of Input 1

→q0 q1 q1
q1 q2 q2
*q2 ε ε

40
 Cont’d …

• Example 4: Design a NFA for the transition table as given below:

Present State 0 1
→q0 q0, q1 q0, q2
q1 q3 ε
q2 q2, q3 q3
*q3 q3 q3

41
 Cont’d …
• The transition diagram can be drawn by using the mapping function
as:

42
 Cont’d …

• Example 5: Design an NFA with ∑ = {0, 1} accepts all string ending with 01.
• Solution:

43
 Cont’d …

• Example 6: Construct NFA for set of strings of all that starts with 0 over {0,
1}
Solution:

0,1

q0 0 q1

44
 NFA with ε
• ε is empty symbol( no character to accept)
• Regular NFA with 5 tuples {Q, ∑, q0, F,
δ}
where δ : Q x ∑ → 2Q
• ε -NFA with 5 tuples {Q, ∑, q0, F,
δ} where δ : Q x ∑ U ε → 2 Q
1 0

• Example: 0 ε
A B C

B on seeing nothing, it can go to ‘C’

Note: Every state on ε goes to itself
45
 Difference between DFA and NFA
DFA NFA
The transition from a state is to a single The transition from a state can be to
particular next state for each i/p symbol. multiple next state for each i/p symbol.
Hence it is called deterministic. Hence, it is called non-deterministic.
Empty string transitions are not seen in NFA permits empty string transitions.
DFA.
DFA is a subset of NFA. Need to convert NFA to DFA in
the design of compiler.
δ: Q x ∑ →Q δ: Q x ∑ →2Q
Example: δ(q0, a) = q1 Example: δ(q0,a) = {q1, q2}
Requires more space Requires less space
A string is accepted by a DFA, if it A string is accepted by a NFA, if at least
transits to a final state. one of all possible transitions ends in a
final state. 46
 Conversion of NFA to DFA

DFA: δ: Q x ∑ →Q a, b
NFA: δ: Q x ∑
→2Q
Example 1: b b q2
q0 q1

Step 1:construct NFA transition

table a b
q0 q0 q0,q1
q1 - q2
q2 - -

47
 Cont’d …
Step 1: Construct NFA transition a b
table a b q0 q0 [q0,q1]
NF DFA
A q0 q0 q0,q1 : [q0,q1] [q0] [q0q1q2]
q1 - q2 [q0q1q2] [q0] q0q1q2]
q2 - - q2 - -
• [q0,q1] new single state

a b

b b q0q1q2
q0 q0q1

a
a 48
 Cont’d …
Example 2: Convert the given NFA to
DFA
a, b b

a a q2 a, b q3
q0 q1
Step 1:construct NFA transition b
table
δ a b
→q0 {q0,q1} q0
q1 q2 q1
*q2 q3 q3
*q3 - q2

49
 Cont’d …
Step 2: Construct DFA transition table and create new states
NFA DF
δ a b δ a A b
→q0 {q0,q1} q0 →q0 [q0,q1] q0
q1 q2 q1 [q0q1] [q0q1q2] q0q1
*q2 q3 q3 *[q0q1q2] [q0q1q2q3] [q0q1q3]
*q3 - q2 *[q0q1q2q3] [q0q1q2q3] [q0q1q3q2]
*[q0q1q3] [q0q1q2] [q0q1q2]
b b

a a a, b a, b
q0 q0,q1 q0,q1,q2 q0,q1,q2,q3
b a, b
q0,q1,q3 50
 Cont’d …
Problem:
Convert the following Non-Deterministic Finite Automata (NFA) to
Deterministic Finite Automata (DFA):

51
 Equivalence of Two Finite Automata

• For any pair of states(qi, qj) the transition for input a ε ∑ is defined by
{qa, qb} where δ{qi,a}=qa and δ{qj,a}=qb.
• The two automata are not equivalent if for a pair {qa, qb} one
is intermediate state and the other is final state.
• Ifinitial state is final state of one automata, then second
automata also initial state must be final state for them to be
equivalent.
52
 Cont’d …
Example1: find out whether the following two finite automata are equivalent or not.
c
c d
d
A q1 q2 q3

c
d
c
d
q7 • Pair of states both are
q4
B
c c final state or initial state
d
d
are equivalent.
q5 c q6 d
53
 Cont’d …
State/ Input c d
{q1, q4} {q1, q4} {q2, q5}
FS, FS IS, IS
{q2, q5} {q3, q6} {q1, q4}
IS, IS FS, FS
{q3, q6} {q2, q7} {q3, q6}
IS, IS IS, IS
{q2, q7} {q3, q6} {q1, q4}
IS, IS FS, FS

• Based on the above table , the given finite automata A and B are
equivalent.
54
 Cont’d …
Example2: find out whether the following two finite automata are equivalent or not.
c
c d
d
A q1 q2 q3

c
d
c
c
q7 • Pair of states both are
q4
B c
c final state or initial state
d c
or intermediate state are
q5 q6 d equivalent.
d 55
 Cont’d …

State/ Input c d
{q1, q4} {q1, q4} {q2, q5}
FS, FS IS, IS
{q2, q5} {q3, q7} {q1, q6}
IS, IS FS, IS

• Based on the above table , the given finite automata A and B are not
equivalent.

56
 Minimization of DFA
•The minimization of DFA means reducing the number of states from a given FA.
Step1: Remove all the states that are unreachable from initial state via any set
of transition of DFA.
Step2: Draw the transition table for all pair of states.
Step3: Now split the transition table into two tables T1 and T2.
T1 contains all final states
T2 contains non-final states
• Step4: Find similar rows from T1,
such that:
δ(q, a) =p • Find the two states which have same value of a and b and

δ(r, a) =p remove one of them.

57
 Cont’d …
Step5: Repeat step 3 until we find no similar rows available in T1
Step6: Repeat step 3 and step 4 for table T2 also.
Step7: Now combine the reduced T1 and T2 tables.
i.e.. the final transition table of minimized DFA.
• Example 1: Minimization of DFA

q0 0 q1 0
q2 1
0 1 q4
1 0, 1
q3
0, 1
q5 0, 1
58
 Cont’d …
Step1: Remove q2 and q4 in FA because they are unreachable
states.
0 q1
q0

0 1
1
0, 1
q3 q5 0, 1
Step2: Draw transition table for the rest of the states.
States 0 1
→q0 q1 q3
T1
q1 q0 q3
*q3 q5 q5
*q5 q5 q5 59
 Cont’d …
Step3: (i) tables which starts from non final states.
States 0 1
→q0 q1 q3
q1 q0 q3
T1

(ii) tables which starts from final

states. States 0 1
*q3 q5 q5
*q5 q5 q5
T2

• Step4: (i) set 1 has no similar rows, they will be same.

(ii) set 2 has similar rows, so skip q5 and then replace q5 by q3.
States 0 1
*q3 q3 q3 60
 Cont’d …
Step7: Combine set 1 and set 2
States 0 1
→q0 q1 q3
q1 q0 q3
*q3 q3 q3

0 q1
• The minimized DFA q0
0
1
1
q3 0, 1

61
 Application of Finite Automata
• Finite Automata (FA) used:
1. For the designing of lexical analysis of a compiler.
2. For recognizing the pattern using regular expressions.
3. For the designing of the combination and sequential
circuits using Mealy and Moore Machines.
4. Used in text editors.
5. For the implementation of spell checkers.

62
 Activity1
1. Draw a DFA for the language accepting strings ending with '01'
over input alphabets ∑ = {0, 1}.
2. Draw a DFA for the language accepting strings ending with 'abba'
over input alphabets ∑ = {a, b.
3. Draw a DFA for the language accepting strings ending with '0011' over
input alphabets ∑ = {0, 1}.
4. Draw a DFA for the language accepting strings ending with ‘0011’ over
input alphabets ∑ = {0, 1}.

63
 Activity2
1. Design an NFA in which all the string contain a substring 1110.
2. Design an NFA with ∑ = {0, 1} accepts all string in which the
third symbol from the right end is always 0.
3. Design an NFA with ∑ = {0, 1} in which double '1' is followed by double
'0’.
4. Design an NFA with ∑ = {a, b,c} in which a is followed by b
and followed by c.

64
 Activity3
1. Find out whether the following two finite automata are equivalent or not.

A.

B.

65
 Activity4
1. Minimize the given DFA:

2. Minimize the given DFA:

66
 End!!!
THANK
YOU!!!
3/28/2022 Fikru Tafesse(MSc. in Computer Science) 67