Main Reference

• Michael Sipser, Introduction to the Theory of Computation, 3rdEdition.

2
Chapter Zero

Introduction

3
 Models of Computation

• Outline of the course contents

Computation

CPU memory

4
temporary memory

input
CPU
output

Program memory

5
 temporary memory Example: f ( x) = x 3

input
CPU
output
Program memory
compute xx
compute x x
2
6
 temporary memory f ( x) = x 3
input
x=2
CPU
output
Program memory
compute xx
compute x x
2
7
 temporary memory
z = 2*2 = 4
f ( x) = z * 2 = 8 f ( x) = x 3

input
x=2
CPU
output
Program memory
compute xx
compute x 2
x
8
 temporary memory
z = 2*2 = 4
f ( x) = z * 2 = 8 f ( x) = x 3

input
x=2
CPU
f ( x) = 8
Program memory output
compute xx
compute x x
2
9
 Automaton

temporary memory

Automaton
input
CPU
output

Program memory

10
 Automaton

temporary memory

Automaton
input

output
transition

state
11
 Different Kinds of Automata

Automata are distinguished by the temporary memory:

•Finite Automata: no temporary memory

•Pushdown Automata: stack

•Turing Machines: random access memory

12
 Finite Automaton

temporary memory

input
Finite
Automaton
output

Example: Elevators, Vending Machines

(small computing power)
13
 Pushdown Automaton
Temp.
memory
Stack Push, Pop

Pushdown input

Automaton
output

Example: Compilers for Programming Languages

(medium computing power)
14
 Turing Machine

Temp.
memory Random Access Memory

input
Turing
Machine
output

Examples: Any Algorithm

(highest computing power)
15
 Power of Automata
Simple More complex Hardest
problems problems problems

Finite Pushdown Turing

Automata Automata Machine

Less power More power

Solve more
computational problems
16
Turing Machine is the most powerful
computational model known

Question: Are there computational

problems that a Turing Machine
cannot solve?

Answer: Yes (unsolvable problems)

17
Time Complexity of Computational Problems:

NP-complete problems
Believed to take exponential
time to be solved

P problems
Solved in polynomial time
18
 AUTOMATA, COMPUTABILITY, AND COMPLEXITY

• So, there are three traditionally central areas of the theory of

computation: automata, computability, and complexity.

• They are linked by the question:

What are the fundamental capabilities and limitations of computers?

• This question goes back to the 1930s when mathematical

logicians first began to explore the meaning of computation.

• Technological advances since that time have greatly

increased our ability to compute and have brought this
question out of the realm of theory into the world of practical
concern.
19
 AUTOMATA, COMPUTABILITY, AND COMPLEXITY

• In each of the three areas—automata, computability, and

complexity—this question is interpreted differently, and the
answers vary according to the interpretation.

20
 COMPLEXITY THEORY

• Computer problems come in different varieties; some are easy,

and some are hard. For example, the sorting problem is an
easy one. Say that you need to arrange a list of numbers in
ascending order. Even a small computer can sort a million
numbers rather quickly.

• Compare that to a scheduling problem. Say that you must find

a schedule of classes for the entire university to satisfy some
reasonable constraints, such as that no two classes take place
in the same room at the same time. The scheduling problem
seems to be much harder than the sorting problem. If you
have just a thousand classes, finding the best schedule may
require centuries, even with a supercomputer.
21
 COMPLEXITY THEORY

What makes some problems computationally hard and others

easy?

• This is the central question of complexity theory. Remarkably,

there’s no answer to it, though it has been intensively
researched for over 40 years.

22
 COMPLEXITY THEORY

• In one important achievement of complexity theory thus far,

researchers have discovered an elegant scheme for classifying
problems according to their computational difficulty.
• It is analogous to the periodic table for classifying elements
according to their chemical properties.
• Using this scheme, we can demonstrate a method for giving
evidence that certain problems are computationally hard,
even if we are unable to prove that they are.

23
 COMPLEXITY THEORY
• There are several options when you confront a problem that
appears to be computationally hard.
• First, by understanding which aspect of the problem is at the root of
the difficulty, you may be able to alter it so that the problem is more
easily solvable.
• Second, you may be able to settle for less than a perfect solution to
the problem. In certain cases, finding solutions that only
approximate the perfect one is relatively easy.
• Third, some problems are hard only in the worst case situation, but
easy most of the time.
• Depending on the application, you may be satisfied with a
procedure that occasionally is slow but usually runs quickly.
• Finally, you may consider alternative types of computation, such as
randomized computation, that can speed up certain tasks.
24
 COMPLEXITY THEORY
• There are several options when you confront a problem that
appears to be computationally hard.
• First, by understanding which aspect of the problem is at the root of
the difficulty, you may be able to alter it so that the problem is more
easily solvable.
• Second, you may be able to settle for less than a perfect solution to
the problem. In certain cases, finding solutions that only
approximate the perfect one is relatively easy.
• Third, some problems are hard only in the worst case situation, but
easy most of the time.
• Depending on the application, you may be satisfied with a
procedure that occasionally is slow but usually runs quickly.
• Finally, you may consider alternative types of computation, such as
randomized computation, that can speed up certain tasks.
25
 COMPLEXITY THEORY

• One applied area that has been affected directly by complexity

theory is the ancient field of cryptography.

• In most fields, an easy computational problem is preferable to a

hard one because easy ones are cheaper to solve.

• Cryptography is unusual because it specifically requires

computational problems that are hard, rather than easy. Secret
codes should be hard to break without the secret key or password.

• Complexity theory has pointed cryptographers in the direction of

computationally hard problems around which they have designed
revolutionary new codes.
26
 COMPUTABILITY THEORY

• During the first half of the twentieth century, mathematicians such

as Kurt G¨odel, Alan Turing, and Alonzo Church discovered that
certain basic problems cannot be solved by computers.

• One example of this phenomenon is the problem of determining

whether a mathematical statement is true or false.

• This task is the bread and butter of mathematicians.

• No computer algorithm can perform this task.

27
 COMPUTABILITY THEORY

• The theories of computability and complexity are closely related.

• In complexity theory, the objective is to classify problems as easy

ones and hard ones;
• Whereas in computability theory, the classification of problems is by
those that are solvable and those that are not.

• Computability theory introduces several of the concepts used in

complexity theory.

28
 AUTOMATA THEORY

• Automata theory deals with the definitions and properties of

mathematical models of computation.

• These models play a role in several applied areas of computer

science.

• One model, called the finite automaton, is used in text processing,

compilers, and hardware design.

• Another model, called the context-free grammar, is used in

programming languages and artificial intelligence.

29
 Languages
• Language: a set of strings

• String: a sequence of symbols

• from some alphabet

• Example:
• Strings: cat, dog, house
• Language: {cat, dog, house}

Alphabet:  = a, b, c,, z

30
 Languages

Languages are used to describe

computation problems:

PRIMES = {2,3,5,7,11,13,17, }

EVEN = { 0,2,4,6, }

Alphabet:  = {0,1,2, ,9}

31
 Alphabets and Strings

An alphabet is a set of symbols

Example Alphabet:  = a, b
A string is a sequence of
symbols from the alphabet
a
Example Strings ab u = ab
abba v = bbbaaa
aaabbbaaba b w = abba

32
Decimal numbers alphabet  = { 0,1,2, ,9}

102345 567463386

Binary numbers alphabet  = { 0,1}

100010001 101101111
33
 Unary numbers alphabet  = {1}

Unary number: 1 11 111 1111 11111

Decimal number: 1 2 3 4 5

34
 String Operations

w = a1a2  an abba
v = b1b2 bm bbbaaa

Concatenation

wv = a1a2 anb1b2 bm abbabbbaaa

35
 String Operations

w = a1a2  an ababaaabbb

Reverse

w = an  a2 a1
R
bbbaaababa
36
 String Length

w = a1a2  an
• Length: w =n

• Examples: abba = 4
aa = 2
a =1

37
 Length of Concatenation

uv = u + v

• Example: u = aab, u = 3
v = abaab, v = 5

uv = aababaab = 8
uv = u + v = 3 + 5 = 8
38
 Empty String

• A string with no letters is denoted:  or 

• Observations:  =0

w = w  = w

abba = abba = abba = abba

39
 Substring
• Substring of string:
• a subsequence of consecutive characters

• String Substring

abbab ab
abbab abba
abbab b
abbab bbab
40
 Prefix and Suffix
abbab
• Prefixes Suffixes

 abbab w = uv
a bbab
prefix
ab bab
suffix
abb ab
abba b
abbab 
41
 Another Operation

w = ww
n


w
n

• Example: (abba ) = abbaabba

2

• Definition: w =0

(abba ) =  0

42
 The * Operation

 * the set of all possible strings from

alphabet 

 = a, b
* =  , a, b, aa, ab, ba, bb, aaa, aab,

43
 The + Operation
+ : the set of all possible strings from

alphabet  except 
 = a, b
* =  , a, b, aa, ab, ba, bb, aaa, aab,

+
 =  * −
+
 = a, b, aa, ab, ba, bb, aaa, aab,
44
 Languages

• A language over alphabet 

• is any subset of *
• Examples:
 = a, b
* =  , a, b, aa, ab, ba, bb, aaa,
Language:  
Language: a, aa, aab
Language: { , abba, baba, aa, ab, aaaaaa}
45
 More Language Examples

Alphabet  = {a , b }
• An infinite language L = {a b : n  0}
n n


ab abb  L
L
aabb
aaaaabbbbb
46
 Prime numbers

Alphabet  = { 0,1,2, ,9}

Language:

PRIMES = {x : x   and x is prime}

*

PRIMES = {2,3,5,7,11,13,17,}
47
 Even and odd numbers

Alphabet  = { 0,1,2, ,9}

EVEN = {x : x   and x is even}

*

EVEN = { 0,2,4,6, }

ODD = {x : x   and x is odd}

*

ODD = {1,3,5,7, }
48
 Unary Addition

Alphabet:  = {1,+, =}
Language:

ADDITION = {x + y = z : x = 1 , y = 1 , z = 1 ,
n m k

n+m =k}

11 + 111 = 11111  ADDITION

111 + 111 = 111  ADDITION
49
 Squares

Alphabet:  = {1, # }
Language:

SQUARES = {x # y : x = 1 , y = 1 , m = n }
n m 2

11#1111  SQUARES
111#1111  SQUARES
50
 Note that:

Sets  = { }  {}

Set size {} =  = 0

Set size {} = 1

String length  =0
51
 Operations on Languages

• The usual set operations

a, ab, aaaa  bb, ab = {a, ab, bb, aaaa}

a, ab, aaaa  bb, ab = {ab}
a, ab, aaaa − bb, ab = a, aaaa

• Complement: L =  * −L

a, ba =  , b, aa, ab, bb, aaa,

52
 Reverse

• Definition: LR = {w R : w  L}

• Examples: ab, aab, baba = ba, baa, abab

R

L = {a b : n  0}
n n

LR = {b n a n : n  0}
53
 Concatenation

• Definition: L1L2 = xy : x  L1, y  L2 

• Example: a, ab, bab, aa

= ab, aaa, abb, abaa, bab, baaa

54
 Another Operation

• Definition: Ln =  
LL L
n

a, b = a, ba, ba, b =

3

aaa, aab, aba, abb, baa, bab, bba, bbb

L0 = 
• Special case:

0
a , bba , aaa  = 
55
 Another Operation

L = {a b : n  0}
n n

L = {a b a b : n, m  0}
2 n n m m

aabbaaabbb  L 2

56
 Star-Closure (Kleene *)

• All strings that can be constructed from L

• Definition: L* = L0  L1  L2 
• Example:
 , 
a, bb, 
 
a, bb* =  
aa, abb, bba, bbbb, 

aaa, aabb, abba, abbbb,

57
 Positive Closure

• Definition: L = L  L 
+ 1 2

Same with L but without the 

*

a, bb, 
+  
a, bb = aa, abb, bba, bbbb, 
aaa, aabb, abba, abbbb,
 
58
 Important Links:

• In The following links you can find an explanation for the entire
chapter:

▪ https://youtu.be/VpGe7c8EIT0

▪ https://youtu.be/nJWpANAClcI

▪ https://youtu.be/hh3qftQA6Ew

59
 ‫وصلّى اهلل وبارك على نبيّنا محمد‬

The End Summary of Chapter Zero

T.Mariah Khayat

Adham University College

mskhayat@uqu.edu.sa

60