Introduction to Automata Theory

LECTURER: SILAS MAIYO

1
 “All computer science students must learn to
integrate theory
and practice, to recognize the importance of
abstraction, and
to appreciate the value of good engineering design”

2
 Computer Theory?
“Science whose main objective is to provide an
answer to a set of questions summarized as”:
 How to develop a program to solve a specific
problem?
 Does the program actually solve the problem?
 How long and how many resources are needed for
its resolution? (temporal and spatial complexities)
 Which computing model to use (Imperative, OOP,
Logic Prog., etc.)
 Which problems can and solved using a computer?
3
Definitions of Automata Theory?
Alan Turing in 1930’s
Study of abstract computing devices, or “machines”
Automaton = an abstract computing device
Note: A “device” need not even be a physical hardware!
A fundamental question in computer science:
Find out what different models of machines can do and cannot do
The theory of computation
In 1940’s and 1950’s – Finite Automata was studied (simpler machines)
Machines that modeled brain functions…and other purposes
N. Chomsky – studied formal “grammars” – to abstract automata (compilers)
Computability vs. Complexity

4
 (A pioneer of automata theory)

Alan Turing (1912-1954)

Father of Modern Computer Science
English mathematician
Studied abstract machines called Turing
machines even before computers existed
Heard of the Turing test?

5
Theory of Computation: A
historical perspective
Dates / Milestones / Generations
Duration
1930’s • Alan Turing studied Abstract Machines / capable of computing…i.e. Turing
machines
• Decidability & Halting problem
•Turing’s goal - What computing machine could do & what it could not do
1940’s – • “Finite automata” – simpler kinds of machines studied by researchers (to model
1950’s brain functions and other useful purppses)
• Noam Chomsky studied “formal grammars” (abstract automata application) and
formal languages – resulted to “Chomsky Hierarchy” languages – compilers…
1969 • S. Cook extended Turing’s study and separated problems solved efficiently by
computers from those that can be solved BUT in practice take much time - later
problems are “intractable” problems or “NP-Hard” problems
• Moore’s Law –computing speed exponential improvement in solving large
instances of intractable problems
1970 - Modern computer scientists do today heavily relied on Automata: compilers,
later computational & complexity theory evolve

6
 Why Finite Automata?
 Software for designing and checking the behaviour of digital
circuit
The “Lexical Analyzer” of a compiler…component that breaks the
input text into logical units such as identifiers, keywords, and
punctuation
 Software for scanning large bodies of text, such as collections of
Web pages, to find occurrences of words, phrases or other
patterns
 Software for verifying systems of all types that have a finite
number of distinct states such as communication protocols

7
Application fields
Automata theory: application in diverse fields that handle concepts as:
• – Control,
• – Action,
• – Memory
• – Objects are controlled with symbols, words or phrases.

Related to:
• – Communication Theory,
• – Logic of Sequential Circuits,
• – Computer networks and Coders,
• – Evolutionary and Reproductive Systems,
• – Pattern recognition,
• – Physiology of the nervous system structure,
• – Analysis of programming languages,
• – Automatic language translation,
• – Algebra Theory.

8
Finite Automata (FA)
 ON / OFF Switch

FA for recognition of “then”

9
 Structural Representation -
FA
 There are two important notations are not automata-like but play
important role in study of automata & their applications
 Grammars – useful models –processes data with a recursive structure. e.g.
Parser – component of compiler that deals with recursively nested features
of programming languages like expressions – arithmetic, conditional etc…
grammatical rule such as E =>E + E ( refer later topic on CFGs)
Regular Expressions – also denote structure of data where patterns of
strings described is same as what can be described by FA. Style of these
expressions differ from that of grammars . Example below– where we need
to know / interpret expressions with range of characters / symbols…

10
 Automata & Complexity
 Automata are essential for the study of limits of
computation. These are:
 What can a computer do at all? This study is called “Decidability”,
and problems that can be solved by computer are called
“decidable”…refer next topic
 What can a computer do efficiently? This study is called
“Intractability” , and a problem can be solved by a computer
using no more time than some slowly growing function of the size
of the input are called “tractable”…we take all polynomial
functions to be “slowly growing”…while those growing faster are
deemed to grow fast.

11
 Introduction to Formal
PROOF
 To develop useful models, Automata Theory covers
the methodologies of formal proof. It lends itself to
natural and interesting proofs, both -
 Deductive kind (a sequence of justified steps) and
 Inductive kind ( recursive proofs of a parameterized
statement that use the statement itself with “lower” values
of the parameter)

12
 (1). DEDUCTIVE PROOFS
 Consists of a sequence of statements whose truth leads us to some
initial statement, called the hypothesis or the given statement(s), to a
conclusion statement.
Each step in the proof must follow some acceptable logical principle.
From either some facts or previous statements in the deductive proof
or combination of these.
Hypothesis maybe TRUE or FALSE, depending of values of its
parameters.
 The theorem that is proved when we go from a hypothesis H to a
conclusion C is a statement “if H then C”. We say C is deduced from H
Example: if x >= 4, then 2x >= x2 .
13
 Example: if x >= 4, then 2x >= x2
.
 Not hard to convince informally that above THEOREM is TRUE, although a
formal proof requires induction.
First, note hypothesis H is “X >= 4” , has a parameter X, which is either
TRUE or FALSE.
Its truth depends on value of parameter X; e.g. H is true for x=6 and false
for x=2
Likewise, the conclusion C is “2x >=x2 ”, statement also uses parameter x and
is true for certain values and not others. e.g. C is false for X=3 and is true for
X=4
Intuitive argument tells, conclusion “2x >=x2 ” will be true whenever
hypothesis H is “X >= 4”
As X grows larger than 4, the left side, 2x doubles each time X increases by
1 ----starting at x=4. 14
THEOREM:

If X is the sum of the squares of four positive integers, then 2X

>=X2
 Proof is that if the hypothesis is true for X, that is, X is the
sum of the squares of four positive integers, then X must be at
least 4.
The reasoning can be expressed as a sequence of steps. Each
step is either the hypothesis to be proved or a statement that
follows from one or more previous statements.
Logical rule is called modus ponens i.e. if we know H is true,
and we know “if H then C” is true, we may conclude that C is
true. We allow other logical steps to be used in creating a
statement that follows from one or more previous
statements…e.g if A and B are previous statements, then we
deduce the statement “A and B”
15
 Reduction to Definitions
 If you are not sure how to start a proof, convert all terms in the
hypothesis to their definitions. Example of a theorem that is simple to
prove once we have expressed its statement in elementary terms. We
use below definitions:
 (1) A set S is finite if there exist an integer n such that S has exactly n
elements. We write |S| = n, where |S| denotes the number of elements
in a set S. If the set S in not finite, we say S is infinite. An infinite set is a
set that contains more than any integer number of elements.
(2) If S and T are both subsets of some set U, then T is the complement
of S (with respect to U) if S U T = U and S T = Ø. Meaning each element
of U is in exactly one of S and T OR we say; T consists of exactly those
elements of U that are not in S.

16
 Proof of Contradiction
 3) THEOREM: Let S be a finite subset of some infinite set U.
Let T be the complement of S with respect to U. Then T is
infinite.
PROOF: Theorem says, if you have infinite supply of
something (U), and you take a finite amount away (S), then
you still have an infinite amount left.

17
 Proof of Contradiction
 We need to use PROOF of CONTRADICTION, where we assume that
CONCLUSION is FALSE.
Using such assumption, together with parts of hypothesis, to prove the
opposite of one of the given statements of the hypothesis. We show that
its impossible for all parts of hypothesis to be true and for the conclusion
to be false at the same time. The only possibility remaining is for
conclusion to be true whenever the hypothesis is true i.e. the theorem is
true.
For theorem (3), contradiction of the conclusion is “T is finite”. Let’s
assume T is finite, along with hypothesis that says S is finite; i.e. |S| = n for
some integer n. Similarly, assumption that T is finite as |T| = m for some
integer m.
Given statements are S U T = U and S T = Ø ; there must be n+m elements
of U since elements of U are exactly elements of S and T. Therefore, |U| =
n+m, it follows that U is finite, since we showed elements of U are some
integers 18
BUT the statement that U is finite contradicts the given
statement that U is infinite?? We used contradiction of
our conclusion to prove the contradiction of one of the
given statements of the hypothesis, and by principle of
“PROOF by CONTRADICTION” we may conclude the
theorem is true.

19
Summarised statements -
PROOF

20
Proof by Counterexample
Exercise

21
 (2) INDUCTIVE PROOFS
 Special form of proofs called “inductive” essential when
dealing with recursively defined objects.
Given statement S(n), about an integer n, to prove. Common
approach is to prove 2 things:
(1) the BASIS, where we show S(i) for a particular integer i.
usually i = 0 or i = 1, but we also can start at some higher i, if
statement S is false for a few small integers.
(2) the INDUCTIVE step, where we assume n >= i, where i is
the basis integer and we show that “if S(n) then S(n+1)”.

22
The two parts should convince that S(n) is true for every
integer n that is equal to or greater than the basis integer i.
we argue as follows:
Suppose S(n) were false for one or more of those integers.
Then there would have smallest value of n, say j, for which
S(j) is false and yet j >= i. Now j could not be i, because we
prove in the basis part that S(i) is true. This j must be greater
than i. we now know j-1 >= i, and S(j-1) is true.
Inductive part, we prove, if n >= i, then S(n) implies S(n+1).
Suppose we let n = j-1, then from inductive step that S(j-1)
implies S(j). Since we also know, S(j-1) , we conclude S(j).
23
THE INDUCTION PRINCIPAL:
If we prove S(i) and we prove that for all n >= i, S(n)
implies S(n +1), then we may conclude S(n) for all n >=i.

Example: Use induction principal to prove theorem

For all n >=0:

24
PROOF: The proof is in two parts: the basis and the inductive
step; we prove each in turn
BASIS: For each basis, we pick n=0. It might seem surprising
that theorem makes sense for n=0, since left side is when n=0.
However, general principal is that when upper limit of a sum
(here is 0) is less than lower limit (1 here), the sum is over no
terms and therefore sum is 0. That is, = 0. The right side is also
0, since 0x(0+1)x(2x0+1)/6=0. Thus it is true when n=0.
INDUCTION: Now, assume n>=0. we must prove inductive
step, that the equation implies the same formula with n+1
substituted for n. the latter formula is

25
Simplify equations by expanding the sums and products
on right sides. They become

26
 We prove equations since in induction principal, these
are statements S(n+1) and S(n) respectively.

 Final verification that last equation is true requires only simple

polynomial algebra on the left side to show it is identical to right side.

27
 Structural Inductions
In Automata Theory, there are several recursively defined
structures about which we need to prove statements. Good
example is notions of trees and expressions.
Like inductions, all recursive definitions have a basis case,
where one or more elementary structures are defined, and
an inductive step, where more complex structures are
defined in terms of previously defined structures.
EXAMPLE: THE RECURSIVE DEFINITION OF A TREE

28
29
30
The Central Concepts
of Automata Theory

31
 Alphabet
An alphabet is any finite, non-empty, set of symbols e.g.
ASCII, Unicode, {0,1} (binary alphabet ), {a,b,c}.
We use the symbol ∑ (sigma) to denote an alphabet
Common alphabet Examples:
◦ ∑ = {0,1} , the binary alphabet
◦ ∑ = {a,b,c,..z} , the set of all lower-case letters
◦ ∑ = {a-z, A-Z, 0-9} , the set of all alphanumeric/ ASCII
characters
◦ ∑ = {a,c,g,t} , DNA molecule letters
◦…
32
 Strings
A string (or sometimes word) is a finite sequence of symbols
chosen from some alphabet (∑) . Empty string is denoted  (or
“epsilon”), has zero occurrences of symbols- string of length 0.
The set of strings over an alphabet Σ is the set of lists, each
element of which is a member of alphabet Σ.
- fire, truck are both strings over alphabet Σ= {a,……………z}
- 01101 is a string from the binary alphabet Σ = {0,1}.
Length of a string w, denoted by “|w|”, is equal to the number
of (non- ) characters in the string
◦ E.g., x = 010100 |x| = 6
◦ x = 01  0  1  00  |x| = ?
◦ |  | = 0 empty string of 0 characters

33
 Example: Strings
{0,1}* = {, 0, 1, 00, 01, 10, 11, 000, 001, . . . }
Subtlety: 0 as a string, 0 as a symbol look the same.
◦ Context determines the type.
Concatenation= ◦ is operator for concatenation
◦ w1 = fire, w2 = truck
◦ w1 ◦ w2 = firetruck
◦ w2 ◦ w1 = truckfire
◦ w2 ◦ w2 = trucktruck
◦ Often you can drop ◦ so as to be w1w2 = firetruck
◦ For any string w, w  = w
34
 Concatenation & Reversal
Let x and y be strings, then xy denotes concatenation of x
and y, that is string formed by copy of x and followed by
copy of y.
We can concatenate a string with itself e.g.
◦ w1 = w
◦ w2 = ww
◦ w3 = www
By definition, w0 =  e,g, w = w = w
Can Reverse a string : wR
◦ E.g. truckR = kcurt

35
 Powers of an alphabet
Let ∑ be an alphabet. Set of all strings of certain length from that
alphabet are expressed using exponential notation.
◦ We define ∑k = the set of all strings of length k , each of whose
symbols is in ∑.
◦ ∑* set of all strings over alphabet ∑ .
◦ ∑0 = {} , regardless of what alphabet ∑ is. That is,  is the only
string whose length is 0.
◦ ∑* = ∑0 U ∑1 U ∑2 U … e.g. {0,1}* = {, 0, 1, 00, 01, 10, 11, 000,…}
◦ ∑+ = ∑1 U ∑2 U ∑3 U …
◦ ∑* = ∑+ U {}

36
 Languages (or Formal Language)
 A Language L is a subset of ∑* for some Alphabet ∑ .
 A set of strings all of which are chosen from some ∑*, where ∑ is a
particular alphabet.
 L is said to be a language over alphabet ∑, only if L  ∑*
 this is because ∑* is the set of all strings (of all possible length
including 0) over the given alphabet ∑
Examples:
1. Let L be the language of all strings consisting of n 0’s followed by n
1’s: L = {, 01, 0011, 000111,…}
2. Let L be the language of all strings of with equal number of 0’s and
1’s: L = {, 01, 10, 0011, 1100, 0101, 1010, 1001,…}
Definition: Ø denotes the Empty language Let L = {}; Is L=Ø?

37
 Examples
L1 = {a, aa, abba, bbba}
L2 = {car, truck, goose}
L3 = {1, 11, 111, 1111,….}
A Language can either be finite or infinite
Example: The set of strings of 0’s and 1’s with no two
consecutive 1’s.
L = {ε, 0, 1, 00, 01, 10, 000, 001, 010, 100, 101, 0000,
0001, 0010, 0100, 0101, 1000, 1001, 1010, . . . }

38
More Examples
 The language of all strings consisting of n 0’s followed by n 1’s, for
some n>=0: L = {, 01, 0011, 000111, … }
 The set of strings of 0’s and 1’s with an equal number of each: L =
{, 01, 0011, 000111, 0101, 1001, 1100, ….}
 The set of binary numbers whose value is a prime:
{10,11,101,111,1011,…..}
 Ø, the empty language, is a language over any alphabet ∑
{} , the language consisting of only the empty string, is also
language over any alphabet.
NOTE: Ø {} ; the former has no strings and the latter has one string

39
 Language Concatenation
We can concatenate Languages as well as Strings
◦ L1L2 = {wv : w ε L1 Λ v ε L2}
◦ {a, ab} {bb, b} = ??

40
 Language Concatenation
We can concatenate Languages as well as Strings
◦ L1L2 = {wv : w ε L1 Λ v ε L2}
◦ {a, ab} {bb, b} = {abb, ab, abbb}
◦ {a, ab} {a, ab} = ??

41
 Language Concatenation
We can concatenate Languages as well as Strings
◦ L1L2 = {wv : w ε L1 Λ v ε L2}
◦ {a, ab} {bb, b} = {abb, ab, abbb}
◦ {a, ab} {a, ab} = {aa, aab, aba, abab}
◦ {a, aa} {a, aa} = ??

42
 Language Concatenation
We can concatenate Languages as well as Strings
◦ L1L2 = {wv : w ε L1 Λ v ε L2}
◦ {a, ab} {bb, b} = {abb, ab, abbb}
◦ {a, ab} {a, ab} = {aa, aab, aba, abab}
◦ {a, aa} {a, aa} = {aa, aaa, aaaa}

What can we say about | L1L2 | if we know | L1| = m and | L2 | = n ?

43
We can Concatenate Language with itself just like
strings
◦ L1 = L L2 = LL L3 = LLL , etc..
◦ What should L0 be, and why???

◦ L0 = {ε}
◦ Where { } is the empty Language
◦ {ε} is the trivial language

44
 Problems
 In Automata Theory, a Problem is the question of deciding
whether a given string is a member of a particular language.
 If ∑ is an alphabet, and L is a language over ∑, then the
problem L is:
 Given a string in ∑* , decide whether or not is in L.
Languages and problems are really the same thing.

45
 Set – Formers to define
Language
 It is common to describe a language using a “Set-formers”:
 { | something about }
This expression is read “the set of words such that (whatever is
said about to the right of vertical bar )” e.g.
 { | consists of an equal number of 0’s and 1’s}
 { | is a binary integer that is prime}
We can replace by expressions with parameters e.g
 {0n1n | n>=1} – the set of 0 to n and 1 to n such that n is greater than
or equal to 1.
 {0i1j | 0 <= i <= j} – the language consist of strings with some 0’s
followed by at least as many 1’s.
46
 The Membership Problem
Given a string w ∑*and a language L over ∑, decide
whether or not w L.
Example:
Let w = 100011
Qz). Is w  the language of strings with equal number of 0s
and 1s?

47
 Kleene Closure (L*)
L* = L0 U L1 U L2 U L3 U …..

48