Introduction to Automata

Theory
Reading: Chapter 1

1
What is Automata Theory?
◼ Study of abstract computing devices, or
“machines”
◼ Automaton = an abstract computing device
◼ Note: A “device” need not even be a physical
hardware!
◼ A fundamental question in computer science:
◼ Find out what different models of machines can do
and cannot do
◼ The theory of computation
◼ Computability vs. Complexity
2
 abstract computing device
◼ A theoretical model or conceptual framework that is used to describe
computation and the behavior of computer systems without specifying
the physical details of their implementation. It provides a high-level,
mathematical or logical representation of how computations are
performed, focusing on the essential properties and capabilities of a
computing system.

◼ Often used in computer science and theoretical computer science to

analyze algorithms, study computability theory, and explore the limits
of computation. They help researchers and theorists' reason about the
fundamental properties of computation, such as time complexity,
space complexity, and decidability(can be resolved or not).

3
 Examples of abstract computing devices

◼ Turing Machines: A Turing machine is a theoretical device that

consists of a tape of infinite length, a read/write head, and a set of rules
for manipulating the symbols on the tape.
◼ Lambda Calculus: The lambda calculus is a formal system developed
by mathematician Alonzo Church. It is a mathematical notation for
representing and manipulating functions. The lambda calculus is a
fundamental model of computation in the field of computability theory
and has heavily influenced the design of programming languages.
◼ Cellular Automata: are abstract computing devices that operate on a
grid of cells, where each cell can be in a finite number of states. The
behavior of the automaton is determined by a set of rules that specify
how the states of the cells evolve over time based on the states of their
neighboring cells. Cellular automata have been used to explore
emergent behaviors and simulate complex systems.
4
5
(A pioneer of automata theory)

Alan Turing (1912-1954)

◼ Father of Modern Computer
Science
◼ English mathematician
◼ Studied abstract machines called
Turing machines even before
computers existed
◼ Heard of the Turing test?

6
 The Turing test
◼ is a test of a machine's ability to exhibit intelligent behavior
that is indistinguishable from that of a human. It was
proposed by the British mathematician and computer
scientist Alan Turing in 1950 as a way to assess a machine's
ability to exhibit intelligent behavior. The test involves a
human judge engaging in a conversation with a machine
and a human, without knowing which is which. If the judge
cannot reliably distinguish the machine from the human
based on their responses, then the machine is said to have
passed the Turing test.

7
Theory of Computation: A
Historical Perspective
1930s • Alan Turing studies Turing machines
• Decidability
• Halting problem
1940-1950s • “Finite automata” machines studied
• Noam Chomsky proposes the
“Chomsky Hierarchy” for formal
languages
1969 Cook introduces “intractable” problems
or “NP-Hard” problems
1970- Modern computer science: compilers,
computational & complexity theory evolve
8
 Languages & Grammars
◼ Languages: “A language is a
Or “words” collection of sentences of
finite length all constructed
from a finite alphabet of
symbols”
◼ Grammars: “A grammar can
be regarded as a device that
enumerates the sentences of
a language” - nothing more,
nothing less

◼ N. Chomsky, Information
and Control, Vol 2, 1959

Image source: Nowak et al. Nature, vol 417, 2002

9
The Chomsky Hierachy
• A containment hierarchy of classes of formal languages

Regular Context-
(DFA) Context-
free Recursively-
sensitive
(PDA) enumerable
(LBA) (TM)

10
 Deterministic Finite Automata
◼ DFA stands for Deterministic Finite Automaton. It is a mathematical model
used to describe and analyze the behavior of finite-state machines (FSMs) or
finite-state automata.
◼ A DFA consists of a finite set of states, a set of input symbols, a transition
function that specifies how the machine transitions from one state to another
based on the current state and the input symbol, a designated start state, and
a set of accepting or final states.
◼ The behavior of a DFA is determined by its current state and the input symbol
it receives. Upon receiving an input symbol, the DFA transitions to a new state
according to the transition function. This process continues until the input is
exhausted. If the DFA ends in an accepting state, it accepts the input;
otherwise, it rejects the input.

11
 Pushdown Automaton
A PDA, or Pushdown Automaton, is a mathematical model used
to describe machines with a stack or pushdown store. It consists
of states, input and stack symbols, a transition function, a start
state, accepting states, and an initial stack symbol. Similar to a
DFA, a PDA can read input symbols, push symbols onto the
stack, and pop symbols from the stack. The stack enables the
PDA to store and manipulate information during computation.
PDAs are commonly used to recognize and process context-free
languages, especially in parsing programming language syntax.
While more powerful than DFAs, PDAs still have limitations and
cannot recognize languages requiring unbounded memory or
arbitrary computation, for which Turing machines are used.
12
 Linear Bounded Automaton
◼ LBA is an extension of a Pushdown Automaton (PDA) that adds a
tape with limited size and one-way movement. It consists of
states, input and tape symbols, a transition function, a start
state, accepting states, and a tape initially filled with input. The
tape serves as a work area, allowing reading and writing of
symbols while performing computations. LBAs can recognize and
process context-sensitive languages. They are more powerful
than PDAs but have bounded memory and computational
resources. LBAs are used in formal language theory and compiler
design. However, LBAs are still less powerful than Turing
machines, which can perform arbitrary computations with an
unbounded tape.
13
 Turing machine
TM is a theoretical model of computation that consists of a tape
with an infinite length divided into cells, a read/write head that can
move along the tape, a finite set of states, a transition function, and
an initial state. The TM can read the symbol on the tape under its
head, write a new symbol, move the head left or right, and change
its state based on the current state and the symbol read. It can
perform arbitrary computations and is considered a universal model
of computation. Turing machines are used to study computability,
complexity theory, and the limits of what can be computed. They
have been instrumental in the development of theoretical
computer science and the understanding of algorithmic processes.

14
The Central Concepts of
Automata Theory

15
Alphabet
An alphabet is a finite, non-empty set of
symbols
◼ We use the symbol ∑ (sigma) to denote an
alphabet
◼ Examples:
◼ Binary: ∑ = {0,1}
◼ All lower case letters: ∑ = {a,b,c,..z}
◼ Alphanumeric: ∑ = {a-z, A-Z, 0-9}
◼ DNA molecule letters: ∑ = {a,c,g,t}
◼ …
16
Strings
A string or word is a finite sequence of symbols
chosen from ∑
◼ Empty string is  (or “epsilon”)

◼ Length of a string w, denoted by “|w|”, is

equal to the number of (non- ) characters in the
string
◼ E.g., x = 010100 |x| = 6
◼ x = 01  0  1  00  |x| = ?

◼ xy = concatentation of two strings x and y

17
Powers of an alphabet
Let ∑ be an alphabet.

◼ ∑k = the set of all strings of length k

◼ ∑* = ∑0 U ∑1 U ∑2 U …

◼ ∑+ = ∑1 U ∑2 U ∑ 3 U …

18
Languages
L is a said to be a language over alphabet ∑, only if L  ∑*
➔ this is because ∑* is the set of all strings (of all possible
length including 0) over the given alphabet ∑
Examples:
1. Let L be the language of all strings consisting of n 0’s
followed by n 1’s:
L = {, 01, 0011, 000111,…}
2. Let L be the language of all strings of with equal number of
0’s and 1’s:
L = {, 01, 10, 0011, 1100, 0101, 1010, 1001,…}
Canonical ordering of strings in the language

Definition: Ø denotes the Empty language

◼ Let L = {}; Is L=Ø? NO
19
The Membership Problem
Given a string w ∑*and a language L
over ∑, decide whether or not w L.

Example:
Let w = 100011
Q) Is w  the language of strings with
equal number of 0s and 1s?

20
Finite Automata
◼ Some Applications
◼ Software for designing and checking the behavior
of digital circuits
◼ Lexical analyzer of a typical compiler
◼ Software for scanning large bodies of text (e.g.,
web pages) for pattern finding
◼ Software for verifying systems of all types that
have a finite number of states (e.g., stock market
transaction, communication/network protocol)

21
Finite Automata : Examples
action
◼ On/Off switch state

◼ Modeling recognition of the word “then”

Start state Transition Intermediate Final state

state
22
Structural expressions
◼ Grammars
◼ Regular expressions
◼ E.g., unix style to capture city names such
as “Palo Alto CA”:
◼ [A-Z][a-z]*([ ][A-Z][a-z]*)*[ ][A-Z][A-Z]

Start with a letter

A string of other
letters (possibly Should end w/ 2-letter state code
empty)

Other space delimited words

(part of city name) 23
Formal Proofs

24
Deductive Proofs
From the given statement(s) to a conclusion
statement (what we want to prove)
◼ Logical progression by direct implications

Example for parsing a statement:

◼ “If y≥4, then 2y≥y2.”
given conclusion

(there are other ways of writing this).

25
 Deductive proofs
◼ Deductive proofs are a fundamental tool in logic and
mathematics for establishing the truth of a statement based on
a set of axioms or previously proven statements using logical
rules and reasoning. These proofs follow a systematic and
rigorous process to derive conclusions from given premises.
◼ In a deductive proof, the goal is to demonstrate that a specific
statement, known as the theorem, necessarily follows from a
set of assumptions or premises. The proof typically proceeds by
applying logical rules, such as modus ponens, modus tollens, or
the law of syllogism, to manipulate and combine statements in
a valid manner.

26
 Deductive proofs
◼ The proof begins with the assumptions or premises, which are considered to
be true. Then, using logical inference steps, the proof progresses by
establishing intermediate statements until the desired theorem is derived.
Each step of the proof must be justified by logical rules or previously proven
statements.
◼ Deductive proofs are characterized by their validity and soundness. A proof is
valid if the conclusion necessarily follows from the premises and the logical
rules used are correct. A proof is sound if it is valid and all the premises are
true.
◼ Deductive proofs play a crucial role in various areas of mathematics, including
algebra, geometry, number theory, and formal logic. They provide a rigorous
foundation for establishing the truth of mathematical statements and form the
basis for mathematical reasoning and problem-solving.

27
 Example: Deductive proof
Let Claim 1: If y≥4, then 2y≥y2.

Let x be any number which is obtained by adding the squares

of 4 positive integers.
Claim 2:
Given x and assuming that Claim 1 is true, prove that 2x≥x2
◼ Proof:
1) Given: x = a2 + b2 + c2 + d2
2) Given: a≥1, b≥1, c≥1, d≥1
3) ➔ a2≥1, b2≥1, c2≥1, d2≥1 (by 2)
4) ➔x≥4 (by 1 & 3)
5) ➔ 2x ≥ x2 (by 4 and Claim 1)
“implies” or “follows”
28
 On Theorems, Lemmas and Corollaries
We typically refer to:
◼ A major result as a “theorem”
◼ An intermediate result that we show to prove a larger result as a
“lemma”
◼ A result that follows from an already proven result as a
“corollary”

An example:
Theorem: The height of an n-node binary
tree is at least floor(lg n)
Lemma: Level i of a perfect binary tree has
2i nodes.
Corollary: A perfect binary tree of height h
has 2h+1-1 nodes.
29
Quantifiers
“For all” or “For every”
◼ Universal proofs
◼ Notation=
“There exists”
◼ Used in existential proofs
◼ Notation=
Implication is denoted by =>
◼ E.g., “IF A THEN B” can also be written as “A=>B”

30
 Contradiction Vs Induction Vs Contrapositive
1. Contradiction: In a proof by contradiction, the goal is to establish the truth of a statement
by assuming the opposite (negation) of the statement and then demonstrating that it
leads to a contradiction or an inconsistency. If assuming the negation of the statement
leads to a logical contradiction, it implies that the original statement must be true. This
method relies on the principle of non-contradiction, which states that a statement and its
negation cannot both be true simultaneously.
2. Induction: Induction is a proof technique used to establish the truth of a statement for an
infinite or arbitrarily large set of cases. It typically involves two steps: a base case and an
inductive step. In the base case, the statement is shown to be true for a specific starting
point or small case. In the inductive step, it is demonstrated that if the statement holds for
a particular case, it also holds for the next case. By combining these two steps, the truth of
the statement is extended to all cases within the infinite or arbitrarily large set.
3. Contrapositive: The contrapositive is a logical inference that states if a conditional
statement is true, then its contrapositive (formed by negating and switching the
hypothesis and conclusion) is also true. A proof by contrapositive involves proving the
contrapositive of a statement instead of directly proving the original statement. If the
contrapositive is shown to be true, it implies the truth of the original statement. 31
Proving techniques
◼ By contradiction
◼ Start with the statement contradictory to the given
statement
◼ E.g., To prove (A => B), we start with:
◼ (A and ~B)
◼ … and then show that could never happen

What if you want to prove that “(A and B => C or D)”?

◼ By induction
◼ (3 steps) Basis, inductive hypothesis, inductive step
◼ By contrapositive statement
◼ If A then B ≡ If ~B then ~A
32
Proving techniques…
◼ By counter-example
◼ Show an example that disproves the claim

◼ Note: There is no such thing called a

“proof by example”!
◼ So when asked to prove a claim, an example that
satisfied that claim is not a proof

33
Different ways of saying the same
thing
◼ “If H then C”:
i. H implies C
ii. H => C
iii. C if H
iv. H only if C
v. Whenever H holds, C follows

34
“If-and-Only-If” statements
◼ “A if and only if B” (A <==> B)
◼ (if part) if B then A ( <= )
◼ (only if part) A only if B ( => )
(same as “if A then B”)
◼ “If and only if” is abbreviated as “iff”
◼ i.e., “A iff B”
◼ Example:
◼ Theorem: Let x be a real number. Then floor of x =
ceiling of x if and only if x is an integer.
◼ Proofs for iff have two parts
◼ One for the “if part” & another for the “only if part”
35
Summary
◼ Automata theory & a historical perspective
◼ Chomsky hierarchy
◼ Finite automata
◼ Alphabets, strings/words/sentences, languages
◼ Membership problem
◼ Proofs:
◼ Deductive, induction, contrapositive, contradiction,
counterexample
◼ If and only if

◼ Read chapter 1 for more examples and exercises

36