1

Other Models of Computation

2
Models of computation:
Turing Machines
Recursive Functions
Post Systems
Rewriting Systems
3
Churchs Thesis:
All models of computation are equivalent
Turings Thesis:
A computation is mechanical if and only if
it can be performed by a Turing Machine
4
Churchs and Turings Thesis are similar:
Church-Turing Thesis
5
Recursive Functions
An example function:
1 ) (
2
! n n f
Domain Range
3 10
10 ) 3 ( ! f
6
We need a way to define functions
We need a set of basic functions
7
Zero function:
0 ) ( ! x z
Successor function:
1 ) ( ! x x s
Projection functions:
1 2 1 1
) , ( x x x p !
2 2 1 2
) , ( x x x p !
Basic Primitive Recursive Functions
8
Building complicated functions:
Composition: )) , ( ), , ( ( ) , (
2 1
y x g y x g h y x f !
Primitive Recursion:
)) , ( ), , ( ( ) 1 , (
2
y x f y x g h y x f !
) ( ) 0 , (
1
x g x f !
9
Any function built from
the basic primitive recursive functions
is called:
Primitive Recursive Function
10
A Primitive Recursive Function: ) , ( y x add
x x add ! ) 0 , ( (projection)
)) , ( ( ) 1 , ( y x add s y x add !
(successor function)
11
5
) 4 (
)) 3 ( (
))) 0 , 3 ( ( (
)) 1 , 3 ( ( ) 2 , 3 (
!
!
!
!
!
s
s s
add s s
add s add
12
) , ( y x mult
0 ) 0 , ( ! x mult
)) , ( , ( ) 1 , ( y x mult x add y x mult !
Another Primitive Recursive Function:
13
Theorem:
The set of primitive recursive functions
is countable
Proof:
Each primitive recursive function
can be encoded as a string
Enumerate all strings in proper order
Check if a string is a function
14
Theorem
there is a function that
is not primitive recursive
Proof:
Enumerate the primitive recursive functions
- , , ,
3 2 1
f f f
15
Define function
1 ) ( ) ( ! i f i g
i
g differs from every
i
f
g
is not primitive recursive
16
A specific function that is not
Primitive Recursive:
Ackermanns function:
)) , ( , 1 ( ) 1 , (
) 1 , 1 ( ) 0 , (
1 ) , 0 (
y x A x A y x A
x A x A
y y A
!
!
!
Grows very fast,
faster than any primitive recursive function
17
Q
Recursive Functions
0 ) , ( such that smallest )) , ( ( ! ! y x g y y x g y Q
Accermans function is a
Q
Recursive Function
18
Primitive recursive functions
Q
Recursive Functions
19
Post Systems
Have Axioms
Have Productions
Very similar with unrestricted grammars
20
Example: Unary Addition
Axiom:
11 1 1 !
Productions:
1 1
1 1
3 2 1 3 2 1
3 2 1 3 2 1
V V V V V V
V V V V V V
! p !
! p !
21
1111 11 11 111 1 11 11 1 1 ! ! !
1 1
3 2 1 3 2 1
V V V V V V ! p !
1 1
3 2 1 3 2 1
V V V V V V ! p !
A production:
22
Post systems are good for
proving mathematical statements
from a set of Axioms
23
Theorem:
A language is recursively enumerable
if and only if
a Post system generates it
24
Rewriting Systems
Matrix Grammars
Markov Algorithms
Lindenmayer-Systems
They convert one string to another
Very similar to unrestricted grammars
25
Matrix Grammars
Example:
P P p p
p p
p
2 1 3
2 2 1 1 2
2 1 1
, :
, :
:
S S P
c bS S aS S P
S S S P
A set of productions is applied simultaneously
Derivation:
aabbcc cc bbS aaS c bS aS S S S
2 1 2 1 2 1
26
p p
p p
p
2 1 3
2 2 1 1 2
2 1 1
, :
, :
:
S S
c bS S aS S
S S S
} 0 : { > ! n c b a L
n n n
Theorem:
A language is recursively enumerable
if and only if
a Matrix grammar generates it
27
Markov Algorithms
Grammars that produce
P
Example:
P . p
p
p
S
S aSb
S ab
Derivation:
P S aSb aaSbb aaabbb
28
P . p
p
p
S
S aSb
S ab
} 0 : { u ! n b a L
n n
29
In general: } : {
*
P ! w w L
Theorem:
A language is recursively enumerable
if and only if
a Markov algorithm generates it
30
Lindenmayer-Systems
They are parallel rewriting systems
Example:
aa a p
aaaaaaaa aaaa aa a
Derivation:
} 0 : {
2
u ! n a L
n
31
Lindenmayer-Systems are not general
As recursively enumerable languages
Extended Lindenmayer-Systems:
u y a x p ) , , (
Theorem:
A language is recursively enumerable
if and only if an
Extended Lindenmayer-System generates it
32
Computational Complexity
33
Time Complexity:
Space Complexity:
The number of steps
during a computation
Space used
during a computation
34
Time Complexity
We use a multitape Turing machine
We count the number of steps until
a string is accepted
We use the O(k) notation
35
Example:
} 0 : { u ! n b a L
n n
Algorithm to accept a string : w
Use a two-tape Turing machine
Copy the on the second tape
Compare the and
a
a
b
36
|) (| w O
Copy the on the second tape
Compare the and
a
a
b
Time needed:
|) (| w O
Total time:
|) (| w O
} 0 : { u ! n b a L
n n
37
For string of length
time needed for acceptance:
} 0 : { u ! n b a L
n n
n
) (n O
38
Language class:
) (n DTIME
A Deterministic Turing Machine
accepts each string of length
in time
) (n DTIME
1
L
2
L
3
L
) (n O
n
39
) (n DTIME
} 0 : { u n b a
n n
} {ww
40
In a similar way we define the class
)) ( ( n T DTIME
) (n T
For any time function:
Examples: ),... ( ), (
3 2
n I n I
41
Example: The membership problem
for context free languages
} grammar by generated is : { G w w L !
) (
3
n DTIME L
(CYK - algorithm)
Polynomial time
42
Theorem:
) (
k
n I
) (
1 k
n I
) ( ) (
1 k k
n I n I

43
Polynomial time algorithms:
) (
k
n I
Tractable algorithms
For small we can compute the
result fast
k
44
) (
k
n I P !
for all
k
The class
P
All tractable problems
Polynomial time
45
P
CYK-algorithm
} {
n n
b a
-
} {ww
46
Exponential time algorithms:
) 2 (
n
I
Intractable algorithms
Some problem instances
may take centuries to solve
47
Example: the Traveling Salesperson Problem
problem} TSP in the paths shortest { ! L
) 2 ( ) ! (
n
DTIME n DTIME L <
Exponential time
Intractable problem
48
Example: The satisfiability Problem
Boolean expressions in
Conjunctive Normal Form:
k
t t t t .
3 2 1
p i
x x x x t v v v v ! .
3 2 1
Variables
Question: is expression satisfiable?
49
) ( ) (
3 1 2 1
x x x x v v
Satisfiable:
1 , 1 , 0
3 2 1
! ! ! x x x
1 ) ( ) (
3 1 2 1
! x x x x
Example
50
2 1 2 1
) ( x x x x
Not satisfiable
Example
51
e} satisfiabl is expression : { w w L !
) 2 (
n
I L
For variables:
n
Algorithm:
search exhaustively all the possible
binary values of the variables
exponential
52
Non-Determinism
Language class: ) (n NTIME
A Non-Deterministic Turing Machine
accepts each string of length
in time
) (n NTIME
1
L
2
L
3
L
) (n O
n
53
Example: } {ww L !
Non-Deterministic Algorithm
to accept a string : ww
Use a two-tape Turing machine
Guess the middle of the string
and copy on the second tape
Compare the two tapes
w
54
|) (| w O
Time needed:
|) (| w O
Total time: |) (| w O
} {ww L !
Use a two-tape Turing machine
Guess the middle of the string
and copy on the second tape
Compare the two tapes
w
55
) (n I
} {ww L !
56
In a similar way we define the class
)) ( ( n T NTIME
) (n T
For any time function:
Examples: ),... ( ), (
3 2
n I n I
57
Non-Deterministic Polynomial time algorithms:
) (
k
n I L
58
) (
k
n I P !
for all
k
The class NP
Non-Deterministic Polynomial time
59
Example:
The satisfiability problem
Non-Deterministic algorithm:
Guess an assignment of the variables
e} satisfiabl is expression : { w w L !
Check if this is a satisfying assignment
60
Time for variables: n
) (n O
e} satis iabl is expression : { w w L !
Total time:
Guess an assignment of the variables
Check if this is a satisfying assignment ) (n O
) (n O
61
e} satisfiabl is expression : { w w L !
NP L
The satisfiability problem is an -Problem NP
62
Observation:
NP P _
Deterministic
Polynomial
Non-Deterministic
Polynomial
63
Open Problem: ? NP P !
WE DO NOT KNOW THE ANSWER
64
Example: Does the Satisfiability problem
have a polynomial time
deterministic algorithm?
Open Problem: ? NP P !
WE DO NOT KNOW THE ANSWER
65
NP-Completeness
A problem is NP-complete if:
It is in NP
Every NP problem is reduced to it
(in polynomial time)
66
Observation:
If we can solve an NP-complete problem
in Deterministic Polynomial Time (P time)
then we know:
NP P !
67
Observation:
If prove that
we cannot solve an NP-complete problem
in Deterministic Polynomial Time (P time)
then we know:
NP P =
68
Cooks Theorem:
The satisfiability problem is NP-complete
69
Cooks Theorem:
The satisfiability problem is NP-complete
Proof:
Convert a Non-Deterministic Turing Machine
to a Boolean expression
in conjunctive normal form
70
Other NP-Complete Problems:
The Traveling Salesperson Problem
Vertex cover
Hamiltonian Path
All the above are reduced
to the satisfiability problem
71
Observations:
It is unlikely that NP-complete
problems are in P
The NP-complete problems have
exponential time algorithms
Approximations of these problems
are in P