You are on page 1of 43

Theory of Computation

Instructor: E-mail: yen@cc.ee.ntu.edu.tw

Web: http://www.ee.ntu.edu.tw/~yen
Time: 9:10-12:10 PM, Monday Place: BL 103 Office hours: by appointment Class web page:
http://www.ee.ntu.edu.tw/~yen/courses/TOC-2009.html
2

TEXTBOOK
Introduction to Automata Theory, Languages, and Computation 3rd Edition
John E. Hopcroft, Rajeev Motwani, Jeffrey D. Ullman,
(Addison-Wesley, 2006)
3

2nd Edition
Introduction to Automata Theory, Languages, and Computation
John E. Hopcroft, Rajeev Motwani, Jeffrey D. Ullman,
(2nd Ed. Addison-Wesley, 2001)

1st Edition
Introduction to Automata Theory, Languages, and Computation
John E. Hopcroft, Jeffrey D. Ullman,

(Addison-Wesley, 1979)

Grading
HW/Project : 20% Midterm exam.: 40%
Final exam.: 40%

A simple computer
BATTERY

input: switch
output: light bulb actions: flip switch

states: on, off


7

A simple computer
f
BATTERY

start

off
f

on

input: switch
output: light bulb actions: f for flip switch bulb is on if and only if there was an odd number of flips

states: on, off


8

Another computer 1
1 start off

1 2
1 2

off

BATTERY

2 2

off

on

inputs: switches 1 and 2


actions: 1 for flip switch 1 actions: 2 for flip switch 2 bulb is on if and only if both switches were flipped an odd number of times
9

states: on, off

A design problem 4 1
BATTERY

2 3

Can you design a circuit where the light is on if and only if all the switches were flipped exactly the same number of times?
10

A design problem
Such devices are difficult to reason about, because they can be designed in an infinite number of ways By representing them as abstract computational devices, or automata, we will learn how to answer such questions

11

These devices can model many things


They can describe the operation of any small computer, like the control component of an alarm clock or a microwave They are also used in lexical analyzers to recognize well formed expressions in programming languages:
ab1 is a legal name of a variable in C 5u= is not
12

Different kinds of automata


This was only one example of a computational device, and there are others We will look at different devices, and look at the following questions:
What can a given type of device compute, and what are its limitations? Is one type of device more powerful than another?
13

Some devices we will see


finite automata
push-down automata

Devices with a finite amount of memory. Used to model small computers.


Devices with infinite memory that can be accessed in a restricted way. Used to model parsers, etc.

Turing Machines Devices with infinite memory. Used to model any computer.

time-bounded Infinite memory, but bounded running Turing Machines time. Used to model any computer program that runs in a reasonable amount of 14 time.

Why Study Automata Theory?


Finite automata are a useful model for important kinds of hardware and software:
Software for designing and checking digital circuits. Lexical analyzer of compilers. Finding words and patterns in large bodies of text, e.g. in web pages. Verification of systems with finite number of states, e.g. communication protocols.
15

Why Study Automata Theory? (2)


The study of Finite Automata and Formal Languages are intimately connected. Methods for specifying formal languages are very important in many areas of CS, e.g.:
Context Free Grammars are very useful when designing software that processes data with recursive structure, like the parser in a compiler. Regular Expressions are very useful for specifying lexical aspects of programming languages and search patterns.

16

Why Study Automata Theory? (3)


Automata are essential for the study of the limits of computation. Two issues:
What can a computer do at all? (Decidability) What can a computer do efficiently? (Intractability)
17

...

Applications

Quantum computing

Supervisory control
Pattern recognition

circuits
Comm. protocols Prog. languages

Compiler

Automata Theory, Formal Languages, Computability, Complexity

Computer-Aided Verification

Theoretical Computer Science

18

Aims of the Course

To familiarize you with key Computer Science concepts in central areas like
- Automata Theory

- Formal Languages - Models of Computation - Complexity Theory

To equip you with tools with wide applicability in the fields of CS and EE, e.g. for
- Complier Construction

- Text Processing - XML


19

Fundamental Theme
What are the capabilities and limitations of computers and computer programs?
What can we do with computers/programs? Are there things we cannot do with computers/programs?

20

Studying the Theme


How do we prove something CAN be done by SOME program?
How do we prove something CANNOT be done by ANY program?

21

Example: The Halting Problem (1)


Consider the following program. Does it terminate for all values of n 1? while (n > 1) { if even(n) { n = n / 2; } else { n = n * 3 + 1; } }

22

Example: The Halting Problem (2)


Not as easy to answer as it might first seem. Say we start with n = 7, for example: 7, 22, 11, 34, 17, 52, 26, 13, 40, 20, 10, 5, 16, 8, 4, 2, 1 In fact, for all numbers that have been tried (a lot!), it does terminate . . . . . . but in general?
23

Example: The Halting Problem (3)


Then the following important undecidability result should perhaps not come as a total surprise: It is impossible to write a program that decides if another, arbitrary, program terminates (halts) or not. What might be surprising is that it is possible to prove such a result. This was first done by the British mathematician Alan Turing.
24

Our focus
Computability

Automata

Complexity
25

Topics
1.

Finite automata, Regular languages, Regular grammars: deterministic vs. nondeterministic, one-way vs. two-way finite automata, minimization, pumping lemma for regular sets, closure properties.
Context-free grammars: deterministic vs. nondeterministic, one-way vs. two-way PDAs, reversal bounded PDAs, linear grammars, counter machines, pumping lemma for CFLs, Chomsky normal form, Greibach normal form, closure 26 properties.

2. Pushdown automata, Context-free languages,

Topics (contd)
3. Linear

bounded automata, Contextsensitive languages, Context-sensitive grammars.

4. Turing machines, Recursively enumerable sets, Type 0 grammars: variants of Turing machines, halting problem, undecidability, Post correspondence problem, valid and invalid computations of TMs.

27

Topics (contd)
5. Basic recursive function theory 6. Basic complexity theory: Various
resource bounded complexity classes, including NLOGSPACE, P, NP, PSPACE, EXPTIME, and many more. reducibility, completeness. 7. Advanced topics: Tree Automata, quantum automata, probabilistic automata, interactive proof systems, oracle computations, cryptography.

28

Who should take this course?

YOU
29

Languages
The terms language and word are used in a strict technical sense in this course: A language is a set of words. A word is a sequence (or string) of symbols. (or ) denotes the empty word, the sequence of zero symbols.
30

Symbols and Alphabets


What is a symbol, then? Anything, but it has to come from an alphabet which is a finite set. A common (and important) instance is = {0, 1}. , the empty word, is never an symbol of an alphabet.
31

Computation

CPU

memory

32

temporary memory input memory

CPU
output memory

Program memory

33

Example:
temporary memory

f ( x) x

input memory

CPU
Program memory compute compute output memory

xx

x x
2

34

f ( x) x
temporary memory

input memory

x2
CPU
Program memory compute compute output memory

xx

x x
2

35

temporary memory

z 2*2 4 f ( x) z * 2 8

f ( x) x

input memory

x2
CPU
Program memory compute compute output memory

xx

x x
2

36

temporary memory

z 2*2 4 f ( x) z * 2 8

f ( x) x

input memory

x2
CPU
Program memory compute compute

f ( x) 8
output memory

xx

x x
2

37

Automaton
temporary memory Automaton

input memory

CPU
output memory

Program memory

38

Different Kinds of Automata


Automata are distinguished by the temporary memory Finite Automata: Pushdown Automata: Turing Machines: no temporary memory stack random access memory

39

Finite Automaton
temporary memory

Finite Automaton

input memory output memory

Example: Vending Machines (small computing power)


40

Pushdown Automaton
Stack Push, Pop

Pushdown Automaton

input memory output memory

Example: Compilers for Programming Languages (medium computing power)


41

Turing Machine
Random Access Memory

Turing Machine

input memory output memory

Examples: Any Algorithm (highest computing power)


42

Power of Automata

Finite Automata

Pushdown Automata

Turing Machine

Less power

More power Solve more computational problems


43