Legend
Language, Automata: , \alpha : usually denotes a string in this course. , \beta : usually denotes a string in this course. , \delta : usually denotes a transition function in this course. , \sigma : usually denotes a symbol in an alphabet in this course. , \Delta : usually denotes a blank space in this course. , \Gamma : usually denotes a set of stack symbols in this course. , \Lambda : usually denotes an empty string in this course. , \Pi : usually denotes a partition in this course. , \Sigma : usually denotes an alphabet in this course. , \goto : usually denotes a (one step) transition in this course. Logic: , ~ : logical not , ^ : logical and , V : logical or , > : logical imply , <> : logical if and only if (equivalent) , => : logical tautologically imply , <=> : logical tautologically equivalent , \A : logical for all , \E : logical for some (there exists)
Sets: , \in : belongs to , \not\in : does not belong to , @ : empty set U, : universal set , \subset : proper subset , \not\subset : not a proper subset , \subseteq : subset , \not\subseteq : not a subset , \cup : set union Ai , \cup(i=1 to n) A_i : union of n sets , \cap : set intersection Ai , \cap(i=1 to n) A_i : intersection of n sets , \bar A : complement of set A (A) , P(A) : power set of set A , X : Cartesian product Ai , X(i=1 to n) A_i : cartesian product of n sets Relation: < a, b > : ordered pair < a1, a2, ..., an > : ordered ntuple , <= : precedes (partial order) Functions: xi , Sum(i=1 to n) x_i : sum of n xi's O(f) , O(f) : of order smaller than or equal to f
o(f) , o(f) : of order smaller than f (f) , Omega : of order greater than or equal to f (f) , omega : of order greater than f (f) , Theta : of the same order as f f(x) , lim(x > inf) f(x) : limit of f as x goes to infinity
Introduction to Theoretical Computer Science
Today computers are used everywhere: banks, hospitals, schools, airline companies, gas stations, grocery stores, in our cars, in home appliances, PCs, etc., etc. Some are used to crunch numbers, some are used to process images, some are used to process other nonnumeric data and some are used to control operations of various devices. They can reason, they can prove many mathematical theorems, they can beat chess experts in their game, they can solve some very complex problems, they can understand our languages, they can answer our questions and of course they can crunch numbers much much faster than us. Let us for a moment call what computers do computation for convenience, though some of the things computers do such as controling appliances, answering our questions etc. don't fall into our traditional sense of computation. Then these computers seem to be able to compute an awfully lot of things if not everything. But are they capable of computing anything ? Are there things computers can not do ? If there are things computers can not do, what are they ? And why ? If there aren't things computers can not do, then how can we tell ? What do we exactly mean by computation ? Unfortunately there are many things computers can not do. Computers can not solve certain types of problems. For example no computer can tell in general whether or not a given computer program stops after a finite amount of time on a given input. They can not solve some other types of problems fast enough even though they can solve them in some finite amount of time. For example take the traveling salesman problem: a salesman is given a road map with distances between cities and wants to find a shortest round trip route that visits all the cities on the map exactly once. At the moment the so called traveling salesman problem requires an extremely large amount of time to solve. No one has been able to find a reasonably fast algorithm to solve it and the consensus is that it is not likely that anyone can find such an algorithm. I have just given you an example of the problems that computers could not solve. How do we know that that is the case ? Are there other problems like that ? How can we tell whther or not a given problem can be solved and solved fast enough ?
We are going to investigate limitations of computers and computations by studying the essence of compuers and computations rather than all the variations of computer and computation. Since so many systems in practice can be described by regular languages. We say finite automata recognize languages. Thus by studying Turing machines we can learn capabilities hence limitatgions of computers. It turns out that solving a problem can be viewed as recognizing a language. we are also going to study regular languages in detail as well as finite automata. In particular we are going to see a few problems that can not be solved by Turing machines hence by computers and how we can tell that they are unsolvable. every task modern computers perform can also be accomplished by Turing machines. which is the key to the unsolvability of problem by computers. Thus when a finite automaton is processing strings. So they provide a good introduction to our study of Turing machines. it is generally believed (Church's thesis) that any "computation" humans do can be done by Turing machines and that "computation" is the computation performed by Turing machines. The other two are contextfree languages and contextsensitive languages. After briefly studying contextfree languges. we go to Turing machines. In addition finite automata can model a large number of systems used in practice. ways to describe them and how to use them to model many of the real life systems. We are going to learn their properties. We call a set of strings (of symbols) a language. There we learn how computers can be simulated by Turing machines and what it means that a Turing machine recognizes (decides) a language. The languages Turing machines recognize are called Type 0 (or phrase structure) languages (regular languages are Type 3) and they are more complex than regular languages. Finite automata process strings. Before proceeding to the study of Turing machines and their computations in this course. The languages that are recognized by finite automata are called regular languages. Turing machines also recognize languages. Thus they are a powerful tool to design and study those systems with. These two type of languages belong to a hierarchy of four languages called Homsky hierarchy. It was first conceived of by Alan Turing in early 20th century. that is to study limitations of computers and computation. it can actually be solving a problem.
. Though it has not been proven. This essence is a device called Turing machine. Those four languages are together called formal languages. Then with Turing machines we investigate limitations of computers and computations. Consequently they are less capable than Turing machines but then their operations are simpler. we study a simpler type of computing device called finite automata.The main objective of this course is to answer those questions. which are also heavily used in practice. It is a very simple device but remarkably. Finite automata are very similar to Turing machines but a few restrictions are imposed on them. More specifically they answer the question whether or not a given string belongs to a language.
• •
2 is a odd number. If the computation time is 2n where n is the size of the problem. respectively.
Basic Mathematical Objects
Back to Table of Contents
The following are the contents of this introductory chapter. Increasing the processor speed does not help much for such problems.000 times it can handle only ten or so more larger problem sizes. Unfortunately there is nothing we can do to speed them up. true or false. Among the solvable problems there are problems that can be solved within a reasonable amount of time and there are problems that are known to require a finite but very large amount of time to solve.000 times 110 cities would already be too many. The time complexity issues are investigated using Turing machines so that the results apply to all computers.
• • • •
Logic Sets Relations Functions
Logic
Proposition and Logical Connectives "Proposition" can be defined as a declarative statement having a specific truthvalue. then even if the processor speed increased 1.
"Connective": Two or more propositions can be combined together to make compound propositions with the help of logical connectives.Our last topic is time complexities of various problems. Example: The following statements are propositions as they have precise truth values. The time needed to solve such a problem increases at least exponentially with the size of the problem as long as we use Turing machines (hence computers). 4 is a perfect square. then with the increase in the processor speed of 1. Their truth values are false and true. Let us start with review of mathematics.
. We are going to see some of those which take a large amount of time. For example for the traveling salesman problem if 100 cities were too many to solve fast enough.
For the first compound proposition to be true both the propositions have to be true as the connective is AND and as OR is the connective for the second one if either of the propositions is true the truth value of the compound proposition is true. Truth table is given below p T T F F q T F T F pVq T T T F
c.Example: Above two propositions can be used to make a compound proposition using any of the logical connectives. The following are the logical connectives used commonly: a.
. 2 is an odd number OR 4 is a perfect square. Conjunction The logical conjunction is understood in the same way as commonly used ôandö. Disjunction This is logical "or" read as either true value of the individual propositions.
Their truth vales are false and true respectively. The compound proposition truthvalue is true iff all the constituent propositions hold true. Negation This is the logical "negation" and it is expressed by Truth table is given below
as
p for "not p". It is represented as " ^ ".
• •
2 is an odd number AND 4 is a perfect square. Truth table for two individual propositions p and q with conjunction is given below p T T F F q T F T F p^q T F F F
b.
Biconditional A proposition (p q) ^ (q p) can be abbreviated using biconditional conjunction as p q and is read as "if p then q.: p ^ p Logical implication and equivalence If the value of p > q is true in every case. e. "q is a necessity/consequence of p" and "q whenever p" are all differnt ways of saying "if p then q". Contradiction This is the opposite of tautology. E. which is false in every case. E. Tautology A compound proposition.: p V p g. Conditional This is used to define as "a proposition holds true if another proposition is true" i. then p is said to logically imply q. For example. "if not q then not p" . which is true in every case. then q" Truth table is given below p T T F F q T F T F p T F T T q
p > q is also expressed in a number of different (but equivalent) ways in English.g. Following are some of the useful identities and implications from propositional logic: Identities
.e. f.g. "q is necessary for p". and if q then p". "p is sufficient for q" . It is represented as p => q. "p only if q" .p T F
p F T
d. If p and q have the same truthvalue in every case then they are said to be logically equivalent and it is represented as p <=> q. p q is read as "if p.
"The sky is blue".
. [(P 2. [(P 3.contrapositive
For explanations. the sentences "The car Tom is driving is blue". examples and proofs of these implications go to Implications Predicate and Predicate Logic The propositional logic is not powerful enough to represent all types of assertions that are used in computer science and mathematics.1.implication Q) R] [P (Q R)] . [(P Q) Q] P . (P
Q) ( P Q) . (P 4. Central to the predicate logic are the concepts of predicate and quantifier. (P 2. examples and proofs of these identities go to Identities Implications 1.DeMorgan's Law Q) ( P Q) . and "The cover of this book is blue" come from the template "is blue" by placing an appropriate noun/noun phrase in front of it. or a relationship among objects represented by the variables.exportation Q) ( Q P) . The phrase "is blue" is a predicate and it describes the property of being blue.modus tollens Q) (R S)] [(P R) (Q S)] Q) (Q R)] (P R)
For explanations. For example. (P 3. or to express certain types of relationship between propositions such as equivalence ( for more detail click here for example for example ).DeMorgan's Law Q) ( P Q) . For more complex reasoning we need more powerful logic capable of expressing complicated propositions and reasoning. The predicate logic is one of the extensions of propositional logic and it is fundamental to most other types of logic. [(P 5. A predicate is a template involving a verb that describes a property of objects.
"Blue" or "B" can be used to represent the predicate "is blue" among others. such as x > 1 or P(x). x > 1". This new statement is true or false in the universe of discourse. For example any of "is_blue". Hence it is a proposition once the universe is specified. It can be the set of real numbers. the statement x > 1 to "for some object x in the universe. which is expressed as " x x > 1". and it becomes a true statement. assign a value to the variable 2.Predicates are often given a name. The propositions in the predicate logic are statements on objects of a universe. The universe is thus the domain of the (individual) variables. B(x) reads as "x is blue". by using quantifiers on variables . Universe of Discourse The universe of discourse. also called universe . x > 1 becomes 3 > 1 if 3 is assigned to x. A predicate with variables. it is true or false in the universe of discourse. for example. If we adopt B as the name for the predicate "is_blue". In general. x > 1". The universal quantifier turns.
. is the set of objects of interest. a quantification is performed on formulas of predicate logic (called wff ). and hence it is a proposition once the universe is specified. sentences that assert an object is blue can be represented as "B(x)". the statemen t x > 1 to "for every object x in the universe. where x represents an arbitrary object." Again. hence a proposition. which is expressed as " x x > 1. Similarly the existential quantifier turns. There are two types of quantifiers: universal quantifier and existential quantifier. For example. quantify the variable using a quantifier (see below). called atomic formula. can be made a proposition by applying one of the following two operations to each of its variables: 1. for example.
It allows one to reason about properties and relationships of individual objects. some of which are given below. In predicate logic. x [ P(x) Q(x) ] [ x P(x) x Q(x) ] 4. one can use some additional inference rules. Predicate logic is more powerful than propositional logic. the set of all cars on a parking lot. x [ P(x) Q(x) ] [ x P(x) x Q(x) ] For more discussions and examples on these rules and others. implications and inference rules. as well as those for propositional logic such as the equivalences. But it should be obvious from the context. see Reasoning(with predicate logic) and Quantifiers and Connectives in Discrete Structures course.
Sets
. the set of all students in a classroom etc. [ x P(x) x Q(x) ] x [ P(x) Q(x) ] 3. Also for proof and proof techniques see Mathematical Reasoning. The universe is often left implicit in practice.the set of integers. x [ P(x) Q(x) ] [ x P(x) x Q(x) ] 2. Important Inference Rules of Predicate Logic: First there is the following rule concerning the negation of quantified statement which is very useful: x P(x) x P(x) Next there is the following set of rules on quantifiers and connvectives: 1.
How to specify a Set? One way is to enumerate the elements completely.3.5} Alternate way is to give the properties that characterize the elements of the set.1.
Universal Set The set U of all the elements we might ever consider in the discourse is called the universal set. A is a subset of B is represented as A B.
Note: If A is a subset of B and B is a subset of A then A=B. Subset Let A and B be two sets. if A is a subset of.4} call it Z by writing Z = {x  x N  x 5} where N represents the set of natural numbers. if every element of A is an element of B.2. Also. Using this notation we can specify the set {0.
.What is a set? Set is a group of elements. It is read as "the set of natural numbers that are less than or equal to 5". but not equal to B represented as A B. Example: A = {1.3.4. Example: B = {x  x is a positive integer less than or equal to 5} Some sets can also be defined recursively.2. A is a subset of B. having a property that characterizes those elements. All the elements belonging to the set are explicitly given. Set terminology Belongs To x B means that x is an element of set B.
then A' = { x  x U ^ x > 3}.2.4.4. A B=
ø.4.A .5.A = {4.3.3. It is denoted by A' or .3.
Example: A = { 1. Disjoint sets A and B are said to be disjoint if they contain no elements in common i.3 } .e. Set Operations The operations that can be performed on sets are: 1. then the intersection of A and B is the set that consists of the elements in both A and B .8}. where means " is not an element of ".B = {1.5} then A B = {1.5} .B. Example: If A = {1.5 }
.4. It is denoted by A B.4. where ø
is the Empty set. Intersection If A and B are two sets.2. It is denoted by A . 3.8. Difference If A and B are two sets.9 } are disjoint.2} Note that in general A .2.5} then A .B B .2. then the difference of A from B is the set that consists of the elements of A that are not in B..8} B = {3. Example: If U is the set of natural numbers and A = { 1. then the complement of A is the set consisting of all elements of the universal set that are not in A. then the union of A and B is the set that contains all the elements that are in A and B including the ones in both A and B.2.2.5} 2.8} then A B = {3.3} B = {3. Union If A and B are two sets. Example: If A = {1. Example: If A = {1.Complement If A is a set. For A and B of the above example B . Thus A' = { x  x U ^ x A } . and B = { 6. It is denoted by A B.3} and B = {3.
It is a very good tool to get a general idea. The Commutative laws: A B=B A A B=B A The Associative laws: A (B C) = (A A (B C) = (A The Distributive laws: A (B C) = (A A (B C) = (A The Idempotent laws: A A=A A A=A The Absorptive laws: A (A B) = A A (A B) = A The De Morgan laws: (A B)' = A' B' (A B)' = A' B' Other laws involving Complements: ( A' )' = A A A A' = ø A' = U B) B) B) B) C C (A (A C) C)
Other laws involving the empty set A A
ø=A ø=ø
Other laws involving the Universal Set: A U=U A U=A Venn Diagrams A common technique in working with Set Operations is to illustrate them by drawing Venn Diagrams.Following is a list of some standard Set Identities A.
. C represent arbitrary sets and ø is the empty set and U is the Universal Set. B.
however.4 } can be represented as shown below using Venn Diagrams:
Set A
U represents the Universal set in which A is one of the Set.2.3.2. The idea of Venn Diagram is to draw a region representing the universe and within that to draw the regions representing the component sets we are starting with so that the resulting diagram describes their interrelationships.8. because they can represent only very limited situations and miss many other possibilities.
. that Venn Diagrams must NOT be used for rigorous discussions. For example sets A = { 1.4 } and B = { 6.Note.
Set B
The following Venn Diagram is used to illustrate A B
.
A
B
.
The following Venn Diagram is used to illustrate A U B
A A
B
B is the set consisting of all the different elements in A and B.
.
4.5.7.3.2.(A
B)' is the yellow region in the Venn diagram given below.6.6.4.4 } A B = { 1.4.(A B) = { 5.3.
For example:
U = { 1.6.8 }
.2. 7 }
B = { 2.8 } A = { 1.3.2.8 } (A B)' = U .
. For example expressions we often use A holds. A2 .A .B is the yellow shaded region and B .A is the blue shaded region in the Venn Diagram shown below
Generalized Set Operations Union.
This can be generalized for the union of any finite number of sets as A1 An . intersection and Cartesian product of sets are associative.. which we write as Ai This generalized union of sets can be rigorously defined as follows: Definition ( Ai) :
.. To denote either of these B C.
Inductive Clause:
Ai = (
Ai)
An+1 Ai and generalized Cartesian product
Similarly the generalized intersection Ai can be defined. De Morgan's law on set union and intersection can also be generalized as follows: Theorem (Generalized De Morgan)
=
. This part of the definition specifies the "seeds" of the set from which the elements of the set are generated using the methods given in the inductive clause. The set of elements specified here is called basis of the set being defined.
.Basis Clause: For n = 1 .
Based on these definitions. The basis clause (or simply basis) of the definition establishes that certain objects are in the set.
Ai = A1.
and
=
Recursive Definition
Recursive Definition
Subjects to be Learned
• • • • •
recursive/inductive definition basis clause basis inductive clause extremal clause
A recursive definition of a set always consists of three distinct clauses: 1.
and the new object is their child . Example 2. Extremal Clause: Nothing is in unless it is obtained from the Basis and Inductive Clauses. can be included in N. 0. the set of natural numbers N can be obtained as follows: First by (1). and 2 is the child of 1. 0 is the parent of 1. the object is not a member of the set. since 0 is in N. Note that if we don't have (3).5. 1. which is not what we want as the set of natural numbers. Extremal Clause: Nothing is in unless it is obtained from the Basis and Inductive Clauses. The x + 1 in the Inductive Clause is the parent of x. Proceeding in this manner all the natural numbers are put into N. The inductive clause (or simply induction) of the definition establishes the ways in which elements of the set can be combined to produce new elements of the set.. The Set of Natural Numbers Basis Clause: Inductive Clause: For any element x in . The basis for this set N is { 0 } . x + 1 is in . Let us call the objects used to create a new object the parents of the new object. The Set of Nonnegative Even Numbers Basis Clause: Inductive Clause: For any element x in . 1 + 1 (= 2) is in N. 0 is put into N. 1 is the parent of 2. .5.
Examples of Recursive Definition of Set
Example 1. x + 2 is in .5. and 1 is the child of 0.2. Example 3. 0 + 1 (= 1) is in N.. Then by (2) again. The Set of Even Integers Basis Clause:
. 3. then they can be combined in certain specified ways to create other objects. The extremal clause asserts that unless an object can be shown to be a member of the set by applying the basis and inductive clauses a finite number of times. and x is the child of x + 1. 2. The inductive clause always asserts that if objects are elements of the set. Following this definition. Then by (2).
Example 4. There are two sets of questions. say x.
Inductive Clause: For any element x in . etc. bbabaa. Then the value of the function at an element.
. try simplest elements in the set such as smallest numbers (0.
Recursive Definition of Function
Some functions can also be defined recursively. . or shortest strings. Extremal Clause: Nothing is in unless it is obtained from the Basis and Inductive Clauses. To see how it is defined click here. Click Yes or No . Condition: The domain of the function you wish to define recursively must be a set defined recursively. then Submit. Then see how other elements can be obtained from them. Basis Clause: . The set of propositions (propositional forms) can also be defined recursively. Here ax means the concatenation of a with x. and x . and . or 1).Inductive Clause: For any element x in .
Test Your Understanding of Recursive Definition
Indicate which of the following statements are correct and which are not. How to define function recursively: First the values of the function for the basis elements of the domain are specified. The Set of Strings over the alphabet excepting empty string This is the set of strings consisting of a's and b's such as abbab. A few examples are given below. and generalize that generation process for the "Inductive Clause". of the domain is defined using its value at the parent(s) of the element x. simplest expressions. They are all on functions from integer to integer except the last one. Tips for recursively defining a set: For the "Basis Clause". and . x + 2. Extremal Clause: Nothing is in unless it is obtained from the Basis and Inductive Clauses.2 are in .
Hence 2 ! = 2 * 1 ! = 2 * 1 = 2 . Using this definition. L(a) = 1 and L(b) = 1. See Example 5 for the extremal clause.Example 5: The function f(n) = n! for natural numbers n can be defined recursively as follows: Basis Clause: f(0) = 0! = 1 Inductive Clause: For all natural number n. Example 8: The function L from the set S of strings over {a. L(xy) = L(x) + L(y) . 1 ! = 1 * 0 ! = 1 * 1 = 1 . where xy is the concatenation of strings x and y. Inductive Clause: For any string x and y of S. See Example 5 for the extremal clause. Hence 3 ! = 3 * 2 ! = 3 * 2 * 1 = 6 . b} to the set of natural numbers that gives the length of a string can be defined recursively as follows: Basis Clause: For symbols a and b of the alphabet. This function L gives the number of a's and b's L(x) for a string x is also often denoted by  x . Example 7: The function f(n) = 2n for natural numbers n can be defined recursively as follows: Basis Clause: f(0) = 1 Inductive Clause: For all natural number n. f(n+1) = f(n) + 2 . 3! can be found as follows: Since 0 ! = 1. f(n+1) = (n+1) f(n). f(n+1) = 2 f(n) . because the set of natural numbers can be defined recursively and that has the extremal clause in it. So there is no chance of other elements to come into the function being defined. See above for the extremal clause. Note that here Extremal Clause is not necessary. Example 6: The function f(n) = 2n + 1 for natural numbers n can be defined recursively as follows: Basis Clause: f(0) = 1 Inductive Clause: For all natural number n.
Induction
Mathematical Induction
.
Subjects to be Learned
• • • • •
first principle of mathematical induction basis step induction hypothesis induction second principle of mathematical induction
Contents First Priciple of Mathematical Induction
As we have seen in recursion . Taking advantage of this. The first principle of mathematical induction states that if the basis step and the inductive step are proven. To prove that a statement P(n) is true for all natural number natural number.. . and the process of generating an element from a known element in the set. which is 1. if P(k) is true (called induction hypothesis). natural numbers can be proven to have certain properties as follows: First it is proven that the basis element. then the next element. Then it is proven that if an arbitrary natural number. has the same property by the inductive step. then P(n) is true for all natural number . then P(k+1) is true. When these two are proven. where is a
Induction: Prove that for any integer . This process is somewhat analogous to the knocking over a row of dominos with knocking over the first domino corresponding to the basis step.
As a first step for proof by induction. has that property (inductive step). denote it by n. More generally mathematical statements involving a natural number n such as 1 + 2 + . has the property in question (basis step). and its elements can be generated one by one starting with 0 by adding 1. the set of natural numbers can be defined recursively. that is n + 1. any natural number can be shown to have the property. Then since 1 has the property. Proceeding likewise.. we proceed as follows: Basis Step: Prove that P( ) is true. then it follows that all the natural numbers have that property. has the property in question. the element next to it. it is often a good idea to restate P(k+1) in terms of
. Thus the set of natural numbers can be described completely by specifying the basis element (0). that is 0. + n = n( n + 1 )/2 can be proven by mathematical induction by the same token. For since 0 has the property by the basis step. has the same property again by the inductive step. which is 2. the element next to it.
Here let us try LHS for n + 1 = 0 + 1 + . Factoring (n + 1) out. Using the induction hypothesis. Proof: Basis Step: If n = 0. + n) + (n + 1) . 0 + 1 + . we get (n + 1)(n + 2) / 2 .
Second Priciple of Mathematical Induction
There is another form of induction over the natural numbers based on the second principle of induction to prove assertions of the form x P(x) .. then LHS = 0. Here k [ k < n P(k) ] is the induction hypothesis. 0 + 1 + . Example 1: Let us prove the following equality using the second principle: For any natural number n .Induction Hypothesis To prove this for n+1. Certain problems can be proven more easily by using the second principle than the first principle because P(k) for all k < n can be used rather than just P(n .1) to prove P(n)... The reason that this principle holds is going to be explained later after a few examples of proof. then n P(n) can be concluded. Example: Prove that for any natural number n.. first try to express LHS for n+1 in terms of LHS for n. Thus LHS = RHS for n+1. This form of induction does not require the basis step.. + ( 2n + 1 ) = ( n + 1 )2. and RHS = 0 * (0 + 1) = 0 . 1 + 3 + ..P(k) so that P(k). Induction: Assume that for an arbitrary natural number n.. which is assumed to be true. + n = n( n + 1 )/2 . and in the inductive step P(n) is proved assuming P(k) holds for all k < n . + n + (n + 1) = (0 + 1 + . Hence LHS = RHS. which is equal to the RHS for n+1. can be used.
.. the last expression can be rewritten as n( n + 1 )/2 + (n + 1) . End of Proof.. + n = n( n + 1 )/2 . Formally the second principle of induction states that if n [ k [ k < n P(k) ] P(n) ] .. . and somehow use the induction hypothesis.
+ ( 2n + 1 ) = ( 1 + 3 + .. A binary relation from A into B is any subset of the Cartesian product A x B. i ( i! ) = ( n + 1 )! .... tan slacks}. say p and q. the individual can wear. + ( 2n + 1 ) = ( n + 1 )2 holds for all natural numbers. However.
Relations
Definition Relation Let A and B be sets.. k can be written as the product of prime numbers. + ( 2n . Since both p and q are smaller than n. by the induction hypothesis they can be written as the product of prime numbers (Note that this is not possible if the First Principle is being used). Let's assume that a person owns three shirts and two pairs of slacks.1 ) ) + ( 2n + 1 ) = n2 + ( 2n + 1 ) = ( n + 1 )2 by the induction hypothesis. Since n is an integer. If n is a prime number. If n is not a prime number. the individual may wish to restrict
.. Then certainly A x B is the set of all possible combinations (six) of shirts and slacks that nbsp.. Then 1 * 1! + 2 * 2! + . We are going to prove that n can be written as the product of prime numbers. More precisely. Example 2: Prove that for all positive integer n. k < n. then it is the product of 1. Hence by the second principle of induction 1 + 3 + . + k * k! = ( k + 1 )! . mint green shirt} and B = {gray slacks.1 holds for all
Example 3: Prove that any positive integer n > 1. = ( n + 1 )n! . let A = {blue shirt..Proof: Assume that 1 + 3 + . it is either a prime number or not a prime number... Therefore the statement holds true. and itself.1 Proof: Assume that 1 * 1! + 2 * 2! + . + ( 2k + 1 ) = ( k + 1 )2 holds for all k.. then it is a product of two positive integers. Hence n can also be written as the product of prime numbers. i ( i! ) = ( n + 1 )! . + ( n .1 ) * ( n ..1 Hence by the second principle of induction positive integers. Example1: nbsp.1 for all k.1 )! + n * n! = n! . n > k > 1. Then 1 + 3 + . k < n.1 + n * n! by the induction hypothesis. Proof: Assume that for all positive integers k. can be written as the product of prime numbers. which is a prime number.
written as RS. or R(x) =y where y = x2 . For example PP. b) Rand (b. R could be more naturally expressed as R(x) = x2 . Other notations are used depending on the past practice. The composition of R and S. (5. 5. b)  a A ^ b A ^ a is a child of b } . 6) and define a relation R from A into A by (a. (mint green shirt. R and S of Example 2 above are relations on A = {2. PropertiesOf Relations
. b) R if and only if a divides evenly into b. 6). tan slacks) }. 5.
Relation on a Set A relation from a set A into itself is called a relation on A. c) A x C. gray slack). 3. In some cases R can be described by actually listing the pairs which are in R. Consider the following relation on real numbers. is the composition of P with itself and it is a relation which we know as grandparentgrandchild relation.himself to combinations which are color coordinated. and S be a relation from set B into set C. 6)}. y)  y is the square of x} and S = { (x. This may not be all possible pairs in A x B but will certainly be a subset of A x B. Composition Let R be a relation from a set A into set B. A typical element in R is an ordered pair (x. 6}. 3. c) RS if and only if there exists b B such that (a. R = {(2. 2). (3. R = { (x. Then P is a relation on A which we might call a parentchild relation. So. (6. where P is the parentchild relation given above. y)  x <= y}.6). Example2: Let A = {2. or "related". where (a. This may not be convenient if R is relatively large. (3. For example. tan slacks). c) S. (black shirt. Let A be a set of people and let P = {(a. as in the previous example. 3). y). one such subset may be { (blue shirt. (2. 5). is the set of pairs of the form(a.
from a set A to a set B is a relation from A to B that satisfies 1. b> and <a.
. if aRb and bRc. in other words. then aRc. c> are in the relation.Assume R is a relation on set A. 1. Equivalence: R is an equivalence relation on A if R is reflexive. there is an element b in B such that <a. if <a. denote it by f. The set A in the above definition is called the domain of the function and B its codomain. codomain image image of set range sum of functions product of functions onetoone function (injection) onto function (surjection) onetoone onto function (bijection) inverse function composite function
Definition (function): A function. 3. Thus. f is a function if it covers the domain (maps every element of the domain) and it is single valued. a R a. 2. b> is in the relation. if aRb. and 2. b) R .
A x A. then b = c . Function
Functions
Subjects to be Reviewed
• • • • • • • • • • • •
function domain.
4. symmetric and transitive. Transitive: R is transitive if for every a. for each element a in A. then bRa. b and c in A. Reflexive: R is reflexive if for every a A. R R b to denote (a. Let us write a
Symmetric: R is symmetric if for every a and b in A.
for example.. 9. where f(x)*g(x) is the product of two real numbers f(x) and g(x). Definition (sum and product): Let f and g be functions from a set A to the set of real numbers R. where S is a subset of the domain A of f .
. Example: Let f(x) = 3x + 1 and g(x) = x2 . Example: The function f(x) = 2x from the set of natural numbers N to the set of nonnegative even numbers E is onetoone and onto. 16. 4. nothing in N can be mapped to 3 by this function. i. that is. Then the sum and the product of f and g are defined as follows: For all x. and ( f*g )(x) = 3x3 + x2 Definition (onetoone): A function f is said to be onetoone (injective) .The relation given by f between a and b represented by the ordered pair <a. { 0. Definition (bijection): A function is called a bijection . and is denoted by f(S) . Note that f(x) = x2 is not onetoone if it is from the set of integers(negative as well as nonnegative) to N . under this function is 9. f(x) = 2x from the set of natural numbers N to N is not onto.e. Example: The function f(x) = 2x from the set of natural numbers N to the set of nonnegative even numbers E is an onto function. there is an element x in A such that f(x) = y . The image of the domain under f is called the range of f . .} . However. ( f*g )(x) = f(x)*g(x) . Example: The function f(x) = x2 from the set of natural numbers N to N is a onetoone function.. and for all x. Definition (onto): A function f from a set A to a set B is said to be onto(surjective) . ( f + g )(x) = f(x) + g(x) . because. f is onto if and only if f( A ) = B . 1. that is. Then the domain and codomain of this f are N.. Thus it is a bijection. b> is denoted as f(a) = b . because for example f(1) = f(1) = 1 . and b is called the image of a under f . The set of images of the elements of a set S under a function f is called the image of the set S under f. if it is onto and onetoone. say 3. Every bijection has a function called the inverse function. x = y . Then ( f + g )(x) = x2 + 3x + 1 . f(S) = { f(a)  a S }. if and only if whenever f(x) = f(y) . and its range is the set of squares. Example: Let f be the function from the set of natural numbers N to N that maps each natural number x to x2 . the image of. if and only if for every element y of B .
Example: The inverse function of f(x) = 2x from the set of natural numbers N to the set of nonnegative even numbers E is f 1(x) = 1/2 x from E to N . Then the composition of functions f and g . denoted by fg .These concepts are illustrated in the figure below. It is also a bijection. Others such as languages of logics. are all languages in that sense. A function is a relation. g(y) = x .
Introduction to Language
A language is. Example: Let f(x) = x2 . Therefore one can also talk about composition of functions.
Definition (inverse): Let f be a bijection from a set A to a set B. and arrows show < x. and let f be a function from B to a set C .
. languages of mathematics. f(x) > relation. For example. where f(x) = y . a set of strings of symbols. and g(x) = x + 1 . In each figure below. Note that such an x is unique for each y because f is a bijection. Then the function g is called the inverse function of f. Then f( g(x) ) = ( x + 1 )2 . Programming langauges we use are a language in that sense. natural languages etc. is the function from A to C that satisfies fg(x) = f( g(x) ) for all x in A . the points on the left are in the domain and the ones on the right are in the codomain. the rightmost function in the above figure is a bijection and its inverse is obtained by reversing the direction of each arrow. in this course. Definition (composite function): Let g be a function from a set A to a set B . if for every element y of B. and it is denoted by f 1 .
are quite useful for modeling systems used in practice such as co9mputer network communication protocols. On the other hand DFAs are suited for writing a simulator program because there is no nondeterminism such as going to two or more states from a state upon reading one input symbol. The four classes are regular (or type 3) languages.). in general there are more than one NFAs and DFAs that reconize one language. In asddition two of the formal languages.What we are going to study on languages in this course are four classes of languages called (Chomsky) formal languages and their properties. In the following chapters we first learn about languages. the simplest of the four formal languages. then the resulting DFA is unique up to the state names for a given regular language. regular and contextfree languages. contextsensitive (or type 1) languages and phrase structure (or type 0) languages.e. if the number of states of DFA is minimized. Then after seeing yet another way of representing regular laguages. Also for some important classes of problems. we are going to learn modeling of systems finite automata. Type 3 is a subset of type 2 which is a subset of type 1 and type 0 is the most general including the other three as a subset. However. regular grammars. These formal languages are characterized by grammars which are essentially a set of rewrite rules for generating strings belonging to a language as we see later. Using automata and formal languages we can study limitations of computer and computation. contextfree (or type 2) languages. checking whether or not a string is in a language. nondeterministic finite automata (NFA) and nondeterministic finite automata with transitions (NFA. Then we investigate various kinds of finite automata: deterministic finite automata (DFA). together with regular expressions which are a method of representing regular languages. We are going to see an algorithm for converting NFAto NFA which recognizes the same language and another for NFA to DFA conversion. Our last topic on regular language is testing of languages for nonregularity. Then we study regular languages. As we are going to learn next. solving them can be seen as recognizing languages i. NFA and NFAare conceptually simpler and easier to use when modeling a system because there are no restrictions on transitions for them unlike for DFA. Also there are various kinds of computing devices called automata which process these types of languages Thus formal languages can also be characterized by the computing devices which process them.
Definitions on Language
Subjects to be Learned
. It can be rigorously shown that some problems can not be solved by computers in any finite amount of time and that some others are practically unsolvable because of the time it takes to solve them. These formal languages and automata capture the essense of various computing devices and computation in a very simple way. lexical analyzers and parser for compilers for programming languages. They are devices that recognize regular languages.
1. Thus {a.• • • •
alphabet string (word) language operations on languages: concatenation of strings. operations on languages and some of their properties. Though has no symbols. Then uv denotes the string obtained by concatenating u with v. We are going to use first few symbols of English alphabet such as a and b to denote symbols of an alphabet and those toward the end such as u and v for strings. b.b}) and {0. Some special languages
The empty set is a language which has no strings. namely . So a string is a substring of itself. intersection. that is. Kleene star
Contents
Here we are going to learn the concept of language in very abstract and general sense. Thus   = 0. The set { } is a language which has one string. baa} is a language (over alphabert {a. So it is not empty.1}). intersetion and difference of two languages over an alphabet are languages over
. Note that u and v may be an empty string. For example if u = aab and v = bbab. ab. union. 2. it has no symbols. 3. uv is the string obtained by appending the sequence of symbols of v to that of u. The empty string (also called null string) is the string with length 0. the set of all strings over (including the empty string) is denoted by . Thus a language over alphabet is a subset of . It can be defined more formally by recursive definition. Operations on languages Since languages are sets. A language is a set of strings over an alphabet. an alphabet is a finite set of symbols. For a string w its length is represented by w. this set has an object in it. b} and 0. 10 and 001 are examples of string over alphabet {0. b} is another alphabet with two symbols and English alphabet is also an alphabet. 111} is a language (over alphabet {0. Thus the union. a and aabab are examples of string over alphabet {a. all the set operations can be applied to languages. 1} is an alphabet with two symbols. A string (also called a word) is a finite sequence of symbols of an alphabet. 1}. then uv = aabbbab. Note that vu = bbabaab uv. For example {0. Let u and v be strings. The empty string is denoted by (capital lambda). The number of symbols in a string is called the length of the string. That is. A string x is a prefix of another string y if there is a string v such that y = xv. A string x is called a substring of another string y if there are strings u and v such that y = uxv. {a. v is called a suffix of y. For any alphabet . Basic concepts First.
Let L1 and L2 be languages. bbbb. For example if L = { aba.
*
The following two types of languages are generalizations of them quite often in this course. Here a0 = and u0 = . Hence Lk is the set of strings that can be obtained by concatenating k strings of L. Recursive definition of L*: Basis Clause: L*
and we are going to see
Inductive Clause: For any x L* and any w L. uk denotes the concatenation of k u's. For example {ab. bb }. bb. baaba}.. For a string u and a natural number k. ak and uk can be defined similarly. baaa. ak represents the concatenation of
k a's. For example Lk can be defined recursively as follows. Extremal Clause: Nothing is in L* unless it is obtained from the above two clauses.
Recursive definition of L+: Basis Clause: L L+
. . Recursive definition of Lk: Basis Clause: L0 = { } Inductive Clause: L(k+1) = Lk L. bbaba. aaba} = {abaaa. aba. Powers : For a symbol a and a natural number k.. L* is the set of strings obtained by concatenating zero or more strings of L as we are going to see in Theorem 1. then L* = { . abb. ababb. abaaba. the extremal clause is not necessary. That is L1L2 is the set of strings obtained by concatenating strings of L1 with those of L2. b} {aaa.L and it is also a
Another operation onlanguages is concatenation. Lk means the concatenation of k L's. } The * in
*
is also the same Kleene star defined above. Then the concatenation of L1 with L2 is denoted as L1L2 and it is defined as L1L2 = { uv  u L1
and v L2 }.
is
.. Similarly for a language L. abaaba. These powers can be formally defined recursively. xw L*. This * is called Kleene star. babb. ababb. The complement of a language L over an alphabet language. Since Lk is defined for natural numbers k.
. wk are strings of L*. Thus L+ is the set of strings obtained by concatenating one or more strings of L.. Other proofs are omitted.. bb }.wm1.. L* can be proven as follows: by Theorem 2. Proof: Because we can see that L* (L*)*.. abaaba. For example if L = { aba.. Hence x is in L* .. i. Since w1.w1m1w21.e..wmmk . bb.. .
(i. ) as
={xx
Lk for some
Then the following relationships hold on L* and L+. any nonempty string in L* or L+ can be expresssed as the concatenation of strings of L. w1w2..e... bbaba..w2m2..
. Theorems 1 and 2 are proven in "General Induction" which you study in the next unit. xw L+. Theorem 5: L* = (L*)*.. Let us list one of them as a theorem and prove it. ababb. Then there are nonempty strings w1.wk . . .... . L0
L
L2 .. bbbb. Extremal Clause: Nothing is in L+ unless it is obtained from the above two clauses.. where wi's are strings of L.Inductive Clause: For any x L+ and any w L.... Theorem 1: Ln Theorem 2: Theorem 3: Theorem 4: L+ = L L* = L*L Note: According to Theorems 2 and 3.. w2. then L+ = { aba. by applying Theorem 2 to the language L* L*
Conversely ( L* )*
Let x be an arbitrary nonempty string of ( L* )*. w2. wi2. }
Let us also define natural number k } . wimi in L such that wi = wi1wi2.wimi Hence x = w11 . .. L* and L* have a number of interesting properties.. for each wi there are strings wi1. wk in L* such that x = w1w2.wk for some k.
If x is an empty string, then it is obviously in L* . Hence ( L* )* Since L* L* . L* , L* = ( L* )* .
(L*)* and ( L* )*
Problem Solving as Language Recognition
Subjects to be Learned
• • •
problem instance problem as language problem solving as language recognition
Contents
In the previous section the concept of language was introduced and its properties have been briefly studied. You might be wondering why we study language. The main reason for studying language is that solving problems can be viewed as a language recognition problem as explained below, that is, the problem of checking whether or not a string belongs to a language. Thus instead of studying what kind of problems can be solved by what kind of computational devices and how, we can study languages and devices to recognize them which are simpler to deal with uncluttered with variations in actual devices, programming languages etc. Below an example is given to illustrate how solving a problem can be viewed as recognizing a language. Consider the following problem: Is the longest of the distances between two nodes(i.e. the diameter) of a given graph less than a given integer k ? Here the distance is the smallest number of edges (or hops) between the nodes. Some of the instances of this problem are as shown below:
Instance 1 asks whether or not the diameter of the given graph with one edge and two nodes is less than 1. Instance 2 asks whether or not the diameter of the given graph with four edges and four nodes is less than 2. Simiarlyt for Instance 3. These problem instances can be represented by a string as follows: Instance 1: 1,2;(1,2);1 Instance 2: 1,2,3,4;(1,2)(1,3)(1,4)(3,4);2 Instance 3: 1,2,3,4;(1,2)(1,3)(1,4)(2,3)(2,4)(3,4);3 Here the set of nodes, the set of edges and k are separated by ; in that order in the strings. The solutions to these instances are: Instance 1: No Instance 2: No Instance 3: Yes There are infinitely many 'Yes' instances and 'No' instances for this problem. The set of 'Yes' instances is a language and so is the set of 'No' instances as well as the set of all
instances and many others for this problem. We can thus see that solving the problem for a given instance is equivalent to checking whether or not the string representing the given instance belongs to the language of 'Yes' instances of the problem. That is, the problem solving is the same as the language recognition. A problem can be solved if and only if the language of its 'Yes' instances is recognizable or decidable by a Turing machine. It is not solvable if the language is merely accecptable but not recognizable, or even worse if it is not even acceptable.
(a.k.a Structural Induction)
Mathematical statements involving an element of a recursively defined set can be proven by induction. To prove by induction that a statement P(x) is true for all the elements x of a recursively defined set S, proceed as follows: Basis Step: Prove that P(x) is true for all the elements x in the basis of S. Induction: Prove that for any element(s) x of S if P(x) is true, then P(y) is true for any element y obtained from x by the induction step of the recursive definition of S. Note 1 : In the Induction we try to prove that if a parent has the property then all of its children also have that property. In the process we need the relationship between the parent and the children. That relationship is found in the Inductive Clause of the recursive definition of the set in question. Note 2 : As a first step for general induction proof, it is often a good idea to express y in terms of x so that P(x) can be used. Example 1 (Theorem 1 in "Language") : Prove that Ln L* for any natural number n and any language L. Let us first review the definitions. Recursive definition of Lk: Basis Clause: L0 = { } Inductive Clause: L(k+1) = LkL. Since Lk is defined for natural numbers k, the extremal clause is not necessary. Recursive definition of L*:
L0 L* . Lk
Next let us prove L*
. Basis Step: Since by the definitions L0 = { Inductive Step: Assume that Lk Hypothesis We are going to show that Lk+1 }. xy Hence w Thus Lk+1 L* . Lk . L* .Basis Clause:
L*
Inductive Clause: For any string x L* and any string w L. . L* . Then there exist strings x and y that satisfy x L and w = xy by the definition of Lk+1.Induction L* . Note in the proof below that Basis and Inductive Steps mirror the Basis and Inductive Clauses of the definition of Ln .
L* by theInduction Hypothesis. and L* . y
Let w be an arbitrary string in Lk+1 . xw L*. L* since y L. . Hence
.
.
L* for an arbitrary natural numer k. Extremal Clause: Nothing is in L* unless it is obtained from the above two clauses.
number k. x L* . Then by the definition of L*.
Proof: Let us first prove Suppose that x
Lk for some natural L* . Since Lk L* . Hence x . x
Example 2 (Theorem 2 in "Language") Let us prove L* = Note that ={xx Lk for some natural number k } . Now let us prove that Ln L* by induction on Ln. Then by the definition of L* . By Example 1 above .
It produces the reversal of a given string x (i. then for some natural number k .
Prove that for arbitrary strings x and y of . Basis Step: L0 since L0 = { }. Basis Clause: REV( )= .e.
Hence by the definition of
Inductive Step: Assume that for an arbitrary x in L*. x spelled backward). . So we show that the property for x is inherited by its children xy.
If x
. of L* has the property. REV(xa) =
Inductive Clause: For any string aREV(x). also have the property.
Note that each step mirror the recursive definition of
. and any symbol . REV(xy) = REV(y) REV(x) holds. So we first prove that * the element of the basis of L has the propertyy. . Let us prove the inheritance. Hence xy End of Inductive Step and Proof Hence we have proven Example 3 . by concatenating y) a child of x in is obtained. holds.e. x
We are going to show that for an arbitrary element y L . x
Lk . Then we show that if any element. . where y is an arbitrary elememt of L. The function REV(x) on strings x over the alphabet is defined as follows.Note that L* is defined recursively and that below we are trying to prove that the elements of L* have the property that they also belong to . by Example 1 above. xy holds. then its children xy. Note here that x is a parent and by applying an operation (i. say x.
Hence xy Lk+1 by the definition of Ln .
Omitted. LrLs and Lr* are regular languages. Regular language The set of regular languages over an alphabet is defined recursively as below. REV(xy) = REV(y) REV(x) holds. Extremal Clause: Nothing is a regular language unless it is obtained from the above two clauses. Induction: Assume that for an arbitrary string y of . The proof mirrors the recursive definition of .Proof First let us note that
*
can be defined recursively as follows: . which is what we needed.
The proof of the equality in question is going to be proven for an arbitrary fixed x by induction on y. But by induction hypothesis a REV(xy) = a REV(y)REV(x). Inductive Clause: If Lr and Ls are regular languages. xa is also in . Any language belonging to this set is a regular language over . REV(xya) = REV((xy)a) = a REV(xy). . Thus the statement to be proven is for an arbitrary fixed string x.
Definitions of Regular Language and Regular Expression
Subjects to be Learned
• •
regular language regular expression
1. { } and {a} for any symbol a are regular languages.
* Basis Clause: where is an empty string. Definition of Set of Regular Languages : Basis Clause: . End of Proof. Since a REV(y) = REV(ya). REV(xy) = REV(y) REV(x) holds. Inductive Clause: For arbitrary strings x of and a of ExtremalClause: As usual.
.Induction Hypothesis Then for an arbitrary symbol a of . and an arbitrary string y of . REV(xya) = REV(ya)REV(x). then Lr Ls . Basis Step: REV(x ) = REV( x ) = REV( )REV( x ) .
Extremal Clause: Nothing is a regular expression unless it is obtained from the above two clauses. respectively. {a. Conventions on regular expressions (1) When there is no danger of confusion. LrLs and Lr* . ( r + s ) is used in stead of ( r + s ). The language corresponding to rk is Lrk. is a regular language because {a. bold face may not be used for regular expressions. b} ( = {a} {b} ) and {ab} ( = {a}{b} ) are regular languages. Thus the regular expression ( a + ( b( c*) ) ) is written as a + bc*. is written as rk. then ( r + s ) . Note also that *. Inductive Clause: If r and s are regular expressions corresponding to languages Lr and Ls . (4) We use ( r+) as a regular expression to represent Lr+ . and a are regular expressions corresponding to languages . where a is an element of . a. b}. aaaa etc. b} is regular.For example. Also since {a} is regular. let = {a. where Lr is the language corresponding to the regular expression r.
Basis Clause: . aaa. For a recursive definition of Lrk click here. Then since {a} and {b} are regular languages. (2) The operation * has precedence over concatenation. Regular expression Regular expressions are used to denote regular languages. { } and {a}. The set of regular expressions over an alphabet is defined recursively as below. Thus for example rr = r2 . (3) The concatenation of k r's . So for example. They can represent regular languages and operations on them succinctly. respectively. which has precedence over union ( + ). aa. {a}* is a regular language which is the set of strings consisting of a's such as . which is the set of strings consisting of a's and b's. Any element of that set is a regular expression.
. 2. where r is a regular expression. ( rs ) and ( r*) are regular expressions corresponding to languages Lr Ls .
Definition of Equality of Regular Expressions Regular expressions are equal if and only if they correspond to the same language. a regular language. a*b+a* corresponds to the set of strings consisting of zero or more a's followed by one or more b's followed by zero or more a's.
Note:A regular expression is not unique for a language. b}. In general.Examples of regular expression and regular languages corresponding to them
•
• • •
( a + b )2 corresponds to the language {aa. b}. ababab.
.. That is. . it is not easy to see by inspection whether or not two regular expressions are equal. the set of strings of repeated ab's. abab. bb}. b}. b}. a*b* corresponds to the set of strings consisting of zero or more a's followed by zero or more b's. that is the set of strings of length 2 over the alphabet {a. in general.. corresponds to more than one regular expressions. ( a + b )* corresponds to the set of all strings over the alphabet {a. For example ( a + b )* and ( a*b* )* correspond to the set of all strings over the alphabet {a. }. ba. ( ab )+ corresponds to the language {ab. Thus for example ( a + b )* = ( a*b* )* . b}. In general ( a + b )k corresponds to the set of strings of length k over the alphabet {a. because they both represent the language of all strings over the alphabet {a. ab. that is.
b. 3: Let r1 and r2 be arbitrary regular expressions over some alphabet. (b) A string corresponding to r1 consists of only a's or only b's or the empty string. However. (a) (r1 + r2 + r1r2 + r2r1)* (b) (r1(r1 + r2)*)+ Solution: One general strategy to approach this type of question is to try to see whether or not they are equal to simple regular expressions that are familiar to us such as a. they do not produce any strings that are not represented by (r1 + r2)*. Thus anything that comes after the first r1 in (r1(r1 + r2)*)+ is represented by (r1 + r2)*. Hence (r1(r1 + r2)*)
. a+. The only strings corresponding to r2 which consist of only a's or b's are a. (a) Since (r1 + r2)* represents all strings consisting of strings of r1 and/or r2 . (b) (r1(r1 + r2)*)+ means that all the strings represented by it must consist of one or more strings of (r1(r1 + r2)*). which are strings in the language with length 1 or less. Of the strings wiht length 2 aa.Exercise Questions on Regular Language and Regular Expression
Ex. (a) find a string corresponding to r2 but not to r1 and (b) find a string corresponding to both r1 and r2. the strings of (r1(r1 + r2)*) start with a string of r1 followed by any number of strings taken arbitrarily from r1 and/or r2. For example ab and ba are such strings. Find a simple (the shortest and with the smallest nesting of * and +) regular expression which is equal to each of the following regular expressions. bb and ab are in the language. Thus the answer is ba. (a + b)*. r1 = a* + b* r2 = ab* + ba* + b*a + (a*b)*
Solution: (a) Any string consisting of only a's or only b's and the empty string are in r1. a. (a + b)+ etc. b and the strings consiting of only b's (from (a*b)*). So we need to find strings of r2 which contain at least one a and at least one b. 2: For the two regular expressions given below. r1r2 + r2r1 in the given regular expression is redundant. Ex. However. Solution: It can easily be seen that . that is. ba is not in it. 1: Find the shortest string that is not in the language represented by the regular expression a*(ab)*b*. a*. Ex. Thus (r1 + r2 + r1r2 + r2r1)* is reduced to (r1 + r2)*.
strings of L are generated one by one by prepending aab or bb to any of the already generated strings. Solution: Let us see what kind of strings are in L. and since an arbitrasry string of b's can be represented by the regular expression b*. 5: Find a regular expression corresponding to the language L defined recursively as follows: Basis Clause: L and a L. Thus (aab)*(bb)* is a regular expression for L. then aabx L and xbb L . b } defined recursively as follows: Basis Clause: L
Inductive Clause: If x L .
Inductive Clause: If x L . 7: Find a regular expression corresponding to the language of all strings over the
. Then starting with . Ex. First of all and a are in L . then aabx L and bbx L . behind the second a and between the two a's. strings of L are generated one by one by prepending aab or appending bb to any of the already generated strings. Solution: Let us see what kind of strings are in L. Since any string of b's can be placed in front of the first a. b*a b*a b* is a regular expression for this language. Solution: A string in this language must have at least two a's. Ex.also represents the strings of (r1(r1 + r2)*)+. Then starting with or a. Extremal Clause: Nothing is in L unless it can be obtained from the above two clauses. Hence (r1(r1 + r2)*)+ is reduced to (r1(r1 + r2)*). Extremal Clause: Nothing is in L unless it can be obtained from the above two clauses. 4: Find a regular expression corresponding to the language L over the alphabet { a . Ex. Thus (aab + bb)*(a + ) is a regular expression for L. b } that contain exactly two a's. Ex. Hence a string of L consists of zero or more aab's in front and zero or more bb's following them. First of all L . Hence a string of L has zero or more of aab's and bb's in front possibly followed by a at the end. and conversely (r1(r1 + r2)*)+ represents the strings represented by (r1(r1 + r2)*). 6: Find a regular expression corresponding to the language of all strings over the alphabet { a.
Hence if a string does not end with ab then it ends with a or if it ends with b the last b must be preceded by a symbol b. Ex. 11: Describe as simply as possible in English the language corresponding to the
.alphabet { a. Hence the string is in this language. Hence a string preceding the aa can be represented by ( b + ab )*. 9: Find a regular expression corresponding to the language of strings of even lengths over the alphabet of { a. Since it can have any string in front of the last a or bb. Thus simply put. Hence any string that follows aa is represented by ( b + ba )*. b } that do not end with ab. On the other hand if an a precedes the aa. b }. Ex. 8: Find a regular expression corresponding to the language of all strings over the alphabet { a. Hence if a string of the language contains aa then it corresponds to the regular expression ( b + ab )*aa( b + ba )* . then applying the same argument as for aa to a. it is the set of strings over the alphabet { a. If an a comes after that aa. 10: Describe as simply as possible in English the language corresponding to the regular expression a*b(a*ba*b)*a* . b } that contain an odd number of b's Ex. ( b + ab )*a( b + ba )* is obtained as a regular expression corresponding to such strings. Note that 0 is an even number. ( b + ab )*( b + ba )* is obtained as a regular expression corresponding to such strings. Solution: Any string in a language over { a . Solution: If there is one substring aa in a string of the language. Solution: A string in the language can start and end with a or b. then applying the same argument as for aa to . Solution: Since any string of even length can be expressed as the concatenation of strings of length 2 and since the strings of length 2 are aa. Any numbe of a's can appear any place in the string. b } that contain no more than one occurence of the string aa. ( a + b )*( a + bb ) is a regular expression for the language. then that a must be preceded by b because otherwise there are two occurences of aa. Ex. If there may not be any a in a string of the language. Altogether ( b + ab )*( + a + aa )( b + ba )* is a regular expression for the language. a regular expression corresponding to the language is ( aa + ab + ba + bb )*. b } must end in a or b. then it must be followed by b. ab. bb. it has at least one b. and after the first b all the b's in the string appear in pairs. If there is no aa but at least one a exists in a string of the language. then that aa can be followed by any number of b. ba.
regular expression (( a + b )3)*(
+a+b). is not regular as we shall see later. a finite language is a set of n
. the given regular expression represents the strings of length 3n and 3n + 1. 12: Describe as simply as possible in English the language corresponding to the regular expression ( b + ab )*( a + ab )*. concatenation and Kleene star operations. The following theorem shows that any finite language is regular. Note 1: Later we shall see that the complement of a regular language and the intersection of regular laguages are also regular. Note 2: The union of infinitely many regular languages is not necessarily regular. Thus the set of regular languages is closed under those operations. We say a language is finite if it consists of a finite number of strings. For example while { akbk } is regular for any natural number k . Proof: Let Lr and Ls be regular languages over an alphabet . Since (( a + b )3)*( a + b ) represents the strings of length 3n + 1. Ex. Then by the definition of the set of regular languages .
Solution: (( a + b )3) represents the strings of length 3. that is. and ( a + ab )* represents strings which do not contain any substring bb. Lr Ls . Hence (( a + b )3)* represents the strings of length a multiple of 3. where n is a natural number. Regularity of finite languages
Theorem 1: The set of regular languages over an alphabet is closed under operations union. Hence altogether it represents any string consisting of a substring with no aa followed by one b followed by a substring with no bb. LrLs and Lr* are regular languages and they are obviously over the alphabet . Solution: ( b + ab )* represents strings which do not contain any substring aa and which end in b. concatenation and Kleene star. { anbn  n is a natural number } which is the union of all the languages { akbk } .
Properties of Regular Language
Subjects to be Learned
• •
Closure of the set of regular languages under union. where n is a natural number.
L { w } is a regular language by the definition of regular language. Hence { aw } is regular. We then prove that a language consisting of a single string is regular. } and { a } are
Inductive Step: Assume that { w } is a regular language for an arbitrary string w over .strings for some natural number n. { regular languages for any arbitrary symbol a of .
Basis Step: (corresponding to n = 0) is a regular language by the Basis Clause of the definition of regular language. Inductive Step: Assume that a language L consisting of n strings is a regular language (induction hypothesis). Then since { w } is a regular language as proven below. then we have proven the theorem. Claim 2: Let w be a string over an alphabet . Hence by the Inductive Clause of the definition of regular language { a }{ w } is regular. Theorem 2: A finite language is regular. { a } is a regular language from the Basis Step. Claim 1: A language consisting of n strings is regular for any natural number n (that is.
. a finite language is regular) if { w } is regular for any string w. Then for any symbol a of . Proof of the Claim 1: Proof by induction on the number of strings. Then { w } is a regular language. Basis Step: By the Basis Clause of the definition of regular language. End of proof of Theorem 2. Proof: Let us first assume that a language consisting of a single string is regular and prove the theorem by induction. End of proof for Claim 2 Note that Claim 2 can also be proven by induction on the length of string. End of proof of Claim 1 Thus if we can show that { w } is a regular language for any string w.
Proof of Claim 2: Proof by induction on strings.
Introduction to Finite Automata
In this chapter we are going to study a class of machines called finite automata. We are going to learn languages which are not regular and ways to test languages for nonregularity. There are languages which are not regular and which.
In this example you as a vending machine have gone through (transitions between) a number of states responding to the inputs from the customer (coins in this case). Pretend that you are the machine. and equivalence and conversions between them. A vending machine looked at this way is an example of finite automaton. When the customer selects a soft drink. then you have now received 15 cents and you wait for the customer to select a soft drink.e. When a customer comes and puts in the first coin. After that you stay in that state until another coin is put in to start the process anew or you may terminate the operation and start all over from the initial state. The states and the transitions between them of this vending machine can be represented with the diagram below. You have received 10 cents and are waiting for more coins to come. Let us assume that only nickels and dimes are used for simplicity. Ds on arrows represent a dime and Ns a nickel. can not be recognized by finite automata. say 15cents state. circles represent states and arrows state transitions. Their operations can be simulated by a very simple computer program. Finite automata are computing devices that accept/recognize regular languages and are used to model operations of many systems we find in practice. therefore. Click "NICKEL" or "DIME" in the figure and see how it operates (see how arrows turn red). Initially you are waiting for a customer to come and put some coins. So you are in another state. We have learned that regular languages are represented by regular expressions and conversely. you are no longer in the waitingforcustomer state. In fact Turing machines are as
. Later we are going to learn an extension of finite automata called Turing machines. Unfortunately not all languages and systems are simple like regular languages or finite automata. A kind of systems finite automnata can model and a computer program to simulate their operations are discussed later. So we might say you are in the 10cents state. you must give the customer a can of soft drink. Then we are going to see that for every regular language a unique finite automaton can be constructed which can recognize the language (i. In the next few chapters first we are going to learn different kinds of finite automata. In the figure. say a dime. that is. It is assumed that the machine terminates its operation when it receives 15 cents or more. Though Turing machines are simple modification of finite automata. they are much more powerful computing devices than finite automata. you are in the waitingforcustomer state. tell whether or not a given string belongs to the regular language). If the customer puts in a nickel. We are then going to study how finite automata can be used to simulate operations of systems we see in practice. Let us consider the operation of a soft drink vending machine which charges 15 cents for a can.
Thus in the example of vending machine. then (q. Also let be a function from Q
to Q .
Definition of Deterministic Finite Automata
Subjects to be Learned
• • •
Finite automata State transition diagram State transition table
Definition of deterministic finite automaton Let Q be a finite set and let be a finite set of symbols. Thus for each state q of Q and for each symbol a of . Its elements can.powerful as computers and it is generally believed. that any computation human beings do (with or without computers) can be performed by Turing machines. the sequence of input symbols given to the finite automaton is "accepted". 2. a) is equal to "have received 5 cents". Thus in the example of vending machine. the transition function. however. Then a deterministic finite automaton is a 5tuple < Q . 3. Note that is a function. "Waiting for a customer to put a coin in" can be considered the initial state of this automaton and the state in which the machine gives out a soda can can be considered the accepting state. though not proven. the states of the machine such as "waiting for a customer to put a coin in". q0 the initial state and A the set of accepting states. (q. a) if it receives the input symbol a while in state q. be interpreted as a state that the system (automaton) is in. for example. The transition function is also called a next state function meaning that the automaton moves into the state (q. are the elements of Q. Otherwise it is not accepted. a) must be specified. . The set Q in the above definition is simply a set with a finite number of elements. If the finite automaton is in an accepting state when the input ceases to come. 4. let q0 be a state in Q and let A be a subset of Q. if q is the initial state and a nickel is put in. For example. in the Example 1 below. The accepting states are used to distinguish sequences of inputs given to the finite automaton. A > Notes on the definition 1.
. "have received 5 cents" etc. We call the elements of Q a state. . q0 .
the following table. = { a }. aaa. the initial state is 0 and is as shown in
State (q) Input (a) Next State ( 0 a 1 1 a 2 2 a 2
(q. a) )
A state transition diagram for this DFA is given below.
If the alphabet of the Example 1 is changed to { a. ) = q . 2 }. etc. They are called transition table. It is a little more complex DFA. then we need a DFA such as shown in the following examle to accept the same string a. The vertices (denoted by single circles) of a transition diagram represent the states of the DFA and the arcs labeled with an input symbol correspond to the transitions. A deterministic finite automaton is also called simply a "finite automaton". are not accepted. 1. Transition functions can also be represented by tables as seen below. But any other strings such as aa. q ) from vertex p to vertex q with label represents the transition (p. A = { 1 }. 5. An arc ( p .
.the string a is accepted by the finite automaton.
DFAs are often represented by digraphs called (state) transition diagram. The accepting states are indicated by double circles. Abbreviations such as FA and DFA are used to denote deterministic finite automaton. b } in stead of { a }. Examples of finite automaton Example 1: Q = { 0.
the initial state is 0 and is as shown in
State (q) Input (a) Next State ( 0 a 0 0 b 1 1 a 1 1 b 1
(q.
= { a. in the following table. b } is the next example. the following table. b }. A = { 1 }. a) )
. A = { 0 }. the initial state is 0 and
is as shown
State (q) Input (a) Next State ( 0 a 1 0 b 2 1 a 2 1 b 2 2 a 2 2 b 2
(q.
A DFA that accepts all strings consisting of only symbol a over the alphabet { a. 1. A state transition diagram for this DFA is given below. while in the Example 1 there is only one row for each state. a) )
Note that for each state there are two rows in the table for corresponding to the symbols a and b.Example 2: Q = { 0. = { a. 1 }. Example 3: Q = { 0. 2 }. b }.
A state transition diagram for this DFA is given below. 20 }. A = { 15. 5. 15. 10. its transition function is as shown in the following table. 20 }. If we make it a DFA.
State (q) Input (a) Next State ( 0 N 5 0 D 10 5 N 10 5 D 15 10 N 15 10 D 20 15 N 5 15 D 10 20 N 5 20 D 10
(q. the initial state q0 = 0.
Example 4: For the example of vending machine of the previous section. a) )
. Q = { 0. = { D. N }.
The tape is divide into squares in each of which a symbol can be written prior to the start of the operation of the automaton. it stops and the automaton terminates its operation. The head is always at the leftmost square at the beginning of the operation. 6. 5. When it sees no symbol.
. 4. There is a finite control which determines the state of the automaton and also controls the movement of the head. The tape has a read only head. It never moves to the left. 2. The tape has the left end and extends to the right without an end.A finite automaton as a machine A finite automaton can also be thought of as the device shown below consisting of a tape and a control circuit which satisfy the following conditions: 1. The head moves to the right one square every time it reads a symbol. 3.
are not accepted by this automaton. Since the state 0 is also the accepting state. Initially it is in state 0. b etc. this DFA never leaves state 1. when all the a's on the tape are read. If b is read while it is in state 0 (initially or after reading some a's). Once it gets to state 1. it goes into state 1 and the input string is not accepted by the DFA.
. Thus this automaton accepts any string of a's. aaaaaa etc.Operation of finite automata Let us see how an automaton operates when it is given some inputs. the DFA is in the accepting state. For example strings aaa. it stays in state 0 while it reads all the a's (without breaks) on the tape. are accepted but strings such as aaba. Hence when b appears anywhere in the input. it moves to state 1. As an example let us consider the DFA of Example 3 above. When zero or more a's are given as an input to it. then no matter what symbol is read.
*
of DFA and its Properties
Subjects to be Learned
• •
*
Language accepted by DFA
Contents
Here we are going to formally describe what is meant by applying a transition repeatedly, that is the concept of * For a state q and string w, *( q , w ) is the state the DFA goes into when it reads the string w starting at the state q. In general a DFA goes through a number of states from the state q responding to the symbols in the string w. Thus for a DFA < Q , , q0 , , A > , the function
* :Q > Q is defined recursively as follows: *
Definition of
*
:
*
Basis Clause: For any state q of Q ,
(q,
) = q , where
*
denotes the empty string. and any symbol a ,
Inducitve Clause: For any state q of Q, any string y * ( q , ya ) = ( *( q , y ) , a ) .
In the definition, the Basis Clause says that a DFA stays in state q when it reads an empty string at state q and the Inductive Clause says that the state DFA reaches after reading string ya starting at state q is the state it reaches by reading symbol a after reading string y from state q. Example For example suppose that a DFA contains the transitions shown below.
Then
*
*
( q , DNR ) can be calculated as follows:
( q , DNR ) = ( *( q , DN ) , R ) by the Inductive Clause. = ( ( *( q , D ) , N ) , R ) by applying the Inductive Clause to *( q , DN ). = ( ( *( q , D ) , N ) , R ) since D = D . = ( ( ( *( q , ) , D ) , N ) , R ) by applying the Inductive Clause to *( q , D ). = ( ( ( q , D ) , N ) , R ) , since ( q , ) = q . = ( ( q1 , N ) , R ) , since ( q , D ) = q1 as seen from the diagram. = ( q2 , R ) , since ( q1 , N ) = q2 as seen from the diagram. = q3 since ( q2 , R ) = q3 as seen from the diagram. Properties of
*
We can see the following two properties of
*
. for a DFA < Q , , q0 , ,A
Theorem 1: For any state q of Q and any symbol a of >,
*
(q,a)=
(q,a)
Proof : Since a = a , * ( q , a ) = *( q , a ) . By the definition of * , * ( q , a ) = ( *( q , ) , a ) But *( q , ) = q by the definition of Hence ( *( q , ) , a ) = ( q , a ) .
*
.
The next theorem states that the state reached from any state, say q , by reading a string, say w , is the same as the state reached by first reading a prefix of w, call it x, and then by reading the rest of the w, call it y. Theorem 2: For any state q of Q and any strings x and y over q0 , , A > , for a DFA < Q , ,
*
( q , xy ) =
*
(
*
(q,x),y).
Proof : This is going to be proven by induction on string y. That is the statement to be proven is the following: * For an arbitrary fixed string x, ( q , xy ) = *( *( q , x ) , y ) holds for any arbitrary string y. First let us review the recursive definition of *. Recursive definition of Basis Clause:
* *
:
.
* * Inductive Clause: If x and a , then xa . * Extremal Clause: Nothing is in unless it is obtained from the above two clauses.
Now the proof of the theorem. Basis Step: If y = , then *( q , xy ) = *( q , x ) = *( q , x ) . Also *( *( q , x ) , y ) = *( *( q , x ) , ) = *( q , x ) by the definition of * . Hence the theorem holds for y = . Inductive Step: Assume that *( q , xy ) = *( *( q , x ) , y ) holds for an arbitrary string y. This is the induction hypothesis. We are going to prove that *( q , xya ) = *( *( q , x ) , ya ) for any arbitrary symbol a of . ( q , xya ) = ( *( q , xy ) , a ) by the definition of * = ( * ( *( q , x ) , y ) , a ) by the induction hypothesis. = *( *( q , x ) , ya ) by the definition of * . Thus the theorem has been proven.
*
For the following DFA answer the questions given below.
*
. if and only if
( q0 .The following notations are used in the questions: : \delta
*
: \delta^* : \Lambda
Language Accepted by DFA
Subjects to be Learned
•
Language accepted by DFA A string w is accepted by a DFA < Q . . w ) A } . w ) A . That is a string is accepted by a DFA if and only if the DFA starting at the initial state ends in an accepting state after reading the string. A language L is accepted by a DFA < Q . A > .
. q0 . That is. q0 . Example 1 :
. . A > . the language accepted by a DFA is the set of strings accepted by the DFA. if and only if
L
= { w  *( q0 .
To find the language it accepts. first from the initial state go to state 1 by reading one a.2 . which is not an accepting state. Then from state 1 go through the cycle 1 .1 and it can go through this cycle any number of times by reading substring ab repeatedly.1 any number of times by reading substring ab any number of times to come back to state 1. Thus the language it accepts is the empty set Example 3 : DFA with one cycle . This is represented by (ab)*. by reading an empty string .e.2 . Thus a string that is accepted by this DFA can be represented by a(ab)*aa .
This DFA has a cycle: 1 .This DFA accepts { } because it can go from the initial state to the accepting state (also the initial state) without reading any symbol of the alphabet i.
. Then from state 1 go to state 2 and then to state 3 by reading aa. and it stays there. Example 2 :
This DFA does not accept any string because it has no accepting state. It accepts nothing else because any nonempty symbol would take it to state 1.
2 . To find the language accepted by this DFA.2 .1 and 1 . Example 5 : DFA with two interleaved cycles
This DFA has two cycles: 1 .3 .0 .Example 4 : DFA with two independent cycles
This DFA has two independent cycles: 0 .0 and 0 .2 .1 .0.1 .2 .2 .0 .1.0 and it can move through these cycles any number of times in any order to reach the accepting state from the initial state such as 0 . Thus a string that is accepted by this DFA can be represented by ( ab + bb )*. first from state 0 go to state 1 by
.0 .
Thus the language that is accepted by this DFA is the union of the language accepted at state 0 and the one accepted at state 1.2 . At this point a substring a( baa + bba )* will have been read. At state 1 go through the cycle 1 .2 . first at state 0 read any number of b's. Example 6 :
This DFA has two accepting states: 0 and 1.1 and 1 . Then from state 1 go through the two cycles 1 . Thus the language accepted at state 1 is b*a(ba)* . respectively. The language accepted at state 0 is b* .1 any number of times by reading substring ba repeatedly.0 . Then go to state 1 by reading one a. To find the language accepted at state 1.
Definition of Nondeterministic Finite Automata
.3 . Then go from state 1 to state 2 and then to state 3 by reading bb.2 . Thus altogether a( baa + bba )*bb will have been read when state 3 is reached from state 0.1 any number of times in any order by reading substrings baa and bba. So we are not going to go any further on this problem here. There is a systematic way of finding the language accepted by a DFA and we are going to learn it later.reading a ( any other state which is common to these cycles such as state 2 can also be used instead of state 1 ). At this point (b*a) will have been read.
1 }. in which case the NFA aborts its operation. As in the case of DFA the set Q in the above definition is simply a set with a finite number of elements. a) must be specified. Thus for each state q of Q and for each symbol a of (q. the sequence of input symbols given to the finite automaton is "accepted". a) if it receives the input symbol a while in state q. A = { 1 }. let q0 be a state in Q and let A be a subset of Q. We call the elements of Q a state. . q0 the initial state and A the set of accepting states.
Examples of NFA Example 1: Q = { 0. Also let be a function from Q
to 2Q . 2. As in the case of DFA the accepting states are used to distinguish sequences of inputs given to the finite automaton. But it can be the empty set. 5. 3. the initial state is 0 and is as shown in
State (q) Input (a) Next State ( 0 a {1} 1 a
(q. Note that is a function. the transition function. A > Notes on the definition 1. = { a }. a) to select is determined nondeterministically. Its elements can be interpreted as a state that the system (automaton) is in. If the finite automaton is in an accepting state when the input ends i. Unlike DFAs an NFA moves into one of the states given by (q. 4. Otherwise it is not accepted. The transition function is also called a next state function . Note that any DFA is also a NFA. a) )
. q0 . Which one of the states in (q. . the following table.Subjects to be Learned
• • •
Nondeterministic finite automata State transition diagram State transition table
Definition of nondeterministic finite automaton Let Q be a finite set and let be a finite set of symbols.e. ceases to come. Then a nondeterministic finite automaton is a 5tuple < Q .
while in the Example 1 there is only one row for each state. b }. the initial state is 0 and
is as shown
State (q) Input (a) Next State ( 0 a {1.
= { a.
. 2 }. b } in stead of { a }. a) )
Note that for each state there are two rows in the table for corresponding to the symbols a and b. A state transition diagram for this finite automaton is given below.
is changed to { a. in the following table.A state transition diagram for this finite automaton is given below. this is still an NFA that accepts {
Example 2: Q = { 0. A = { 2 }. 1.2} 0 1 1 2 2 b a b a b {2}
(q.
If the alphabet a}.
If any other strings are given to this NFA. Thus the string ab is also accepted by this NFA. If on the other hand it moves to state 1 after reading a.
*
and then formalize the concepts of acceptance of
. We say that this automaton accepts the string a. Since the state 2 is the accepting state. it moves to either state 1 or state 2. then it stays in the accepting state. Initially it is in state 0. As an example let us consider the automaton of Example 2 above. if it moves to state 2 and no more inputs are given. Let us now define the function strings and languages by NFA. then it goes to state 2 and remains there.Operation of NFA Let us see how an automaton operates when some inputs are applied to it. When it reads the symbol a. it does not accept any of them. if the next input is b and if no more inputs are given.
*( q . . where
*
denotes the empty . any string y
*
and any symbol a
( q . Example State (q) Input (a) Next State ( 0 a {0. A > . q0 .
*
(q. Thus for an NFA < Q . In general an NFA nondeterministically goes through a number of states from the state q as it reads the symbols in the string w.1. a) ) For example consider the NFA with the following transition table:
. ya ) =
In the definition. the function
* :Q > 2Q is defined recursively as follows: *
Definition of *: Basis Clause: For any state q of Q. w ) is the set of states that the NFA can reach when it reads the string w starting at the state q.
Inducitve Clause: For any state q of Q. . string.Language Accepted by NFA
Subjects to be Learned
• • •
for NFA Language accepted by NFA Properties of *
*
Definition of
*
For a state q and string w. the Basis Clause says that an NFA stays in state q when it reads an empty string at state q and the Inductive Clause says that the set of states NFA can reach after reading string ya starting at state q is the set of states it can reach by reading symbol a after reading string y starting at state q.
) = { q }.3} 0 b {2} 1 1 2 2 3 3 a b a b a b {1} {3} {3} (q.
3}. Then
* *
( 0 .2. *( 0 . Hence *( 0 . 3 } . By the Basis Clause of the definition of *. b ) for all p
*
( 0 .
. ) = { 0 } .b)={2} {3} {1}={1. ab ) can be calculated as follows: ( p. 1 . a ) by the Inductive Clause of
* Now *( 0 . a ) is the union of ( p. b ) (1. a ) = { 0 . Hence *( 0 . ab ) is the union of the definition of * .
Suppose that the state 3 is an accepting state of this NFA. ab ) = ( 0 . a ) = ( 0 . a ) for all p ( 0 .b) (3. ) again by the Inductive * Clause of the definition of .
( 0 .The transition diagram for this NFA is as given below.
. .
*
(q. .
*
( q .* We say that a string x is accepted by an NFA < Q. ab. q0. for an NFA < Q . xy ) =
These theorems can be proven in a manner similar to those for Theorems 1 and 2 for DFA.
Some of the strings accepted by the NFA given above are the language it accepts is a*( ab + a + ba )(bb)* .a)=
(q. q0 . A > . aaa. . A > is the set of strings that are accepted by the NFA. x ) A is not empty. and
for NFA has properties similar to that for DFA.A
Theorem 1: For any state q of Q and any symbol a of >.a) for an NFA < Q . a. that is. A > if and only if * ( q0 . . abbbb etc.
. . . . The language accepted by an NFA < Q. if and only if it can reach an accepting state by reading x starting at the initial state.
Theorem 2: For any state q of Q and any strings x and y over q0 . q0.
*
.
Basically an NFA with Transitions is an NFA but can respond to an empty string and move to the next state.) and see some examples. q0 . for any NFA. Note that any NFA is also a NFA. Thus the tape head does not move when is read. As we are going to see later. .there is a NFA (hence DFA) which accepts the same language and vice versa.Definition of Nondeterministic Finite Automata with Transitions
Subjects to be Learned
• • •

Nondeterministic finite automata with State transition diagram State transition table
Transitions
Contents
One of the objectives of this chapter is to show that there is a onetoone correspondence between regular languages and finite automata. A transition on reading means that the NFA.
Definition of nondeterministic finite automaton with Let Q be a finite set and let
Transitions be a function from Q
be a finite set of symbols. Here we are going to formally define NFA with Transitions (abbreviated as NFA.. Also let
{ } to 2Q .
.A> Notes on the definition 1. 2. q0 the initial state and A the set of accepting states. We call the elements of Q a state. Then a nondeterministic finite automaton with Transitions is a 5tuple < Q . .makes the transition without reading any symbol in the input. These operations on FAs can be described conveniently if Transitions are used. let q0 be a state in Q and let A be a subset of Q. We are going to do that by showing that a finite automaton can be constructed from a given regular expression by combining simpler FAs using union. the transition function. concatenation and Kleene star operations.
5 }. b }. the initial state is 0 and
is as shown in the
State (q) Input (a) Next State ( 0 a {1} 0 {4} 1 {2} 2 { 3. 4 and 5 by reading a. then you come to state 4. it can move to any of the states other than 0. 3. A state transition diagram for this finite automaton is given below.
When a symbol a is read at the initial state 0. following table. If you read string ab. 4. 3. 2. there are no transitions on reading b except from state 3.
= { a. For though you go to states 1. Thus 4 is the only state you can go to from the initial state
. 2. a) )
Here the transitions to are omitted from the table. 4 and 5 without reading any symbol on the tape. 3.Example of NFA
Q = { 0. A =
. For once you are in state 1. 4 } 3 {5} 3 b {4} 4 a {5}
(q. 1. for example. you can go to state 2. for example.
by reading ab.
Language Accepted by NFASubjects to be Learned
• • • •
closure for NFALanguage accepted by NFAProperties of *
*
Contents
To formally define * for NFA , we start with the concept of closure for a state which is the set of states reachable from the state without reading any symbol. Using that concept we define * and then strings and languqges accepted by NFA . Definition of closure
Let < Q , , q0 , , A > be an NFA . Let us denote the closure of a set S of states of Q by ( S ). Then ( S ) is defined recursively as follows: Basis Clause: S (S)
Inductive Clause: For any state q of Q, if q ( S ) , then ( q , ) (S). Extremal Clause: Nothing is in ( S ) unless it is obtained by the above two clauses.
For the NFAFirst { 2 }
of the above figure, ( { 2 } ) , that is, 2 (2, )
( { 2 } ) is obtained as follows: ( { 2 } ) . Then since 2 ( { 2 } ) , by the
Inductive Clause, Since (2,
({2}). ({2}).
) = { 3 , 4 }, we now have { 2 , 3 , 4 }
Since 3 and 4 have been added to
({2}),
(3,
) = { 5 } and
(4,
)=
must
be included in ( { 2 } ) . Thus now { 2 , 3 , 4 , 5 } ({2}). Though 5 has become a memeber of the closure, since ( 5 , ) is empty, no new members are added to ( { 2 } ) . Since ( q , ) has been examined for all the states currently in ( { 2 } ) and no more elements are added to it, this process of generating the closure terminates and ( { 2 } ) = { 2 , 3 , 4 , 5 } is obtained. As we can see from the example, ( S ) is the set of states that can be reached from the states of S by traversing any number of arcs. That is, it is the set of states that can be reached from the states of S without reading any symbols in . Now with this closure, we can define
*
recursively as follows:
As in the cases of DFA and NFA, * gives the result of applying the transition function repeatedly as dictated by the given string. Definition of
* *
is going to be defined recursively. Let < Q , , q0 , , A > be an NFA
.
Basis Clause: For any state q of Q,
*
(q,
)=
({q}).
*
Inductive Clause: For any state q, a string y in
and a symbol a in
,
*
( q , ya ) =
(
).
What the Inductive Clause means is that *( q , ya ) is obtained by first finding the states that can be reached from q by reading y ( *( q , y ) ), then from each of those states p by reading a (i.e. by finding ( p , a ) ), and then by reading 's ( i.e. by taking the closure of the ( p , a )'s ) . Example : For the NFAbelow: of the following figure,
*
( 0 , ab ) can be obtained as
First let us compute *( 0 , a ) . For that we need ( { 0 } ). Since it is the set of states reached by traversing the arcs from state 0, ( { 0 } ) = {0,3,4}. Next from each of the states in ( { 0 } ) we read symbol a and move to another state (i.e. apply ). They are ( 0 , a ) = { 1 } , ( 3 , a ) = ( 4 , a ) = { 5 }. Hence We then traverse the } ) = { 1 , 2 , 3 } and = { 1 , 5 } for q = 0 . arcs from { 1 , 5 } to get to the states in *( 0 , a ) . Since * ({5})={5}, (0,a)={1,2,3,5}.
({1
NFAs and NFAto NFA
1
Conversion of NFA
Let M1 = < Q1 . ( 3 . For example the NFAof the figure given above accepts the language { .< Q . 4 } .
Equivalence of DFAs. x ) contains at least one accepting state. and ( 2 . b ) . b ) = { 4 } . NFA M2 = < Q2. A1 > be an NFA.
. q0 . . . The language accepted by an NFA. q1.Then to find *( 0 . b ) are empty sets. Now ( 1 .to (equivalent) NFA Conversion of NFA to (equivalent) DFA Equivalence of DFAs. A string x is accepted by an NFA. q0 . a . 4 } .< Q . ab ) read b from each of the states in *( 0 .s
Subjects to be Learned
• • •
Conversion of NFA. *( 0 . . Then the 2 . A > is the set of strings accepted by the NFA. Thus Since ( { 4 } ) = { 3 .0 . ab } . NFAs and NFA. a ) and then take the arcs from there. A > if and only if *( q0 . b ) and ( 5 .0 . A2 > that satisfies the following conditions recognizes L:
. ab ) = { 3 . .. q2. . .that recognizes a language L.
Thus to obtain an NFA M2 = < Q2. q2.0 through arcs in M1 . .0 = q1. a ): First ( { 0 } ) = { 0 .0. that is all the states that can be reached from q by traversing arcs.0 are the accepting states of M2 .
The set of states Q2 of NFA is { 0. Then for each state q of Q2 and each symbol a of
find
2
( q .a)=
(
)
A2 = A1
{ q1. then all the accepting states of M1 plus state q1. A1 > does. that is if an accepting state can be reached from the initial state q1. . Then collect all the states that can be reached from each state of ( {q} ) by traversing one arc labeled with the symbol a. q1. 3 ). The closure of the set of those states is 2( q . the initial state is 0 and the accepting states are 1 and 0.0 } )
A1
= A1 otherwise . 2. as the given NFA. 1 } .Q2 = Q1. A2 > which accepts the same language 1 . a ) =
1
*
(q.
2
( q. a ) as follows:
Find ( {q} ).M1 = < Q1 .0 } if
( { q1.
. Then from the transition function of the NFA
. Otherwise.0 . The set of accepting states A2 is the same as A1 if no accepting states can be reached from the initial state q1. 1.0 .
2
first copy the states of Q1 into Q2. Example 1: Let us convert the following NFAto NFA. a ) . q2.0 through arcs in M1 . since 1 is in ( { 0 } ) . The transition function 2 is obtained as follows: 2( 0 .
2}.2})={1. 1 } and
1
(0.( 0 . )(= ( {1.
2
(0. and Hence 2( 0 . They are given in the table ( { q } ) and
2
below together with State q Input 0 0 1 1 2 2 3 3 a b a b a b a b
. a ) = .1} {1} {1} {2} {2} {1.
.b)=
1
( 1.2} {1.
1
For . b ) . 2 }. b ) =
.2} ))
({q}) {0.3} {3} {1.2}
{1.2} {1. a ) =
1
( 1 .b)=
Similarly
2
can be obtained for other states and symbols.2}
The NFA thus obtained is shown below.1} {0.
2
( 0 .2}
{1. a ) = { 1 . (q. since
( { 0 } ) = { 0 .3} {1.3} {1. ({1.
4} {1.3}
))
{1. ) and
State q Input 0 0 1 1 2 2 3 3 4 a b a b a b a b a
({q}) {0. the initial state is 0 and the accepting states are 1 and 0.
The set of states Q2 of NFA is { 0.Example 2: Let us convert the following NFA
to NFA. 4 ).2.3}
.2.1} {0. 1. since 1 is in ( { 0 } ) .2. The transition function 2 is obtained as for Example 1.2} {4} {4} {4} {1.4} {1.
)(=
( {1.2}
2
(q.3} {2. 2 is given in the table below together with ( { q } ) . 1 ( p .4} {1.1} {1} {1} {2. 3.4}
{1. 2.3}
{1.2} {1.3} {3} {3} {1.
4
b
{1.4}
The NFA thus obtained is shown below.
.
Proof of Equivalence of NFA
and NFA
We are going to prove that the NFA obtained from NFAby the conversion algorithm accepts the same language as the NFA.
the induction hypothesis and the construction of NFA from NFA.
Proof: This is going to be proven by induction on w.a)=
2
* 2
(q.a). Recall that the set of strings is defined recursively (click here for a quick review).
1 *
(q.(1)
.0 .a).a)=
2
( q .. Claim 1: For any nonempty string w and for any state q. (q. wa ) =
2
*
( q . .
a)=
by the definition of Since
* 2
*
for NFA (click here for a review) . q2.
. wa ) =
. Thus we first prove that it is true for any arbitrary symbol.
1
.NFA
that recognizes a language L is denoted by M1 = < Q1 .(2)
. a ) = 1*( q .
1 *
(q.
then
* 1
( q . w ) for any nonempty string w. w ) for an arbitrary string w (Induction Hypothesis). Basis Step: We need to show that for any symbol a in
1 *
(q.
We are going to prove it by showing that both of them are equal to
Firstly 2( q .w)=
2
*
(q.
.
for NFA. . wa ) holds for any arbitrary symbol a in
First we show that using the definition of
2 *
2
*
( q . wa ) . .
Secondly
2
*
(q.
Inductive Step: We need to show that if
1
*
(q. it implies that NFAM1 and NFA M2 accept the same nonempty strings.w)=
2
*
( q .(2)
Hence from (1) and (2).w). and the conversion of NFAto NFA click here.a)=
2 2
*
(q . A2 >
(q. a ) .a). q1.(1) because of the way NFA is constructed from NFA(click here for a review) . = =
2 * 2
(q . a ) . . that is wa for any symbol a in the alphabet.a)=
2
*
(q. then assuming it holds for any arbitrary string w we prove it for any of the children of w. A1 > and
2
NFA obtained by the conversion is denoted by M2 = < Q2.
)={q}. =
1 *
Then we show that
( q .a). First we are going to prove that To review the definition of
* 1 *
.
Hence
(q.
(q.0 . The case when w is an empty string is going to be proven separately. and NFA
When it is proven.w)=
2
*
( q .
as proven below in Claim 3. wa ) . =
Since
2
(q. Hence we have proven (2). because
= . that is By the definition of
1 *
=
1 *
1
*
( q .a)= Substituting this into the left hand side of (2) produces = . wa ) . wa ) . . wa ) . wa ) =
Since
2
*
(q.
1 *
Thus from (1) and (2)
( q . wa ) =
2
*
( q . that is (1) has been proven. The right hand side of this equality is equal to ( the first and have been swapped to get this) . w ) by the induction hypothesis. wa ) .basically using the definition of Then from (1) and (2) we can see that
1 *
1
*
. By the definition of
2 * 2 *
( q . wa ) =
. by the definition of
1 *
.
. a ) by the way NFA is constructed from NFA=
.
= 1*( q . . To see an explanation for this click here.
Hence On the other hand = Hence
1 *
= ( q .
Let us first prove (1).
( q . wa ) =
2
*
( q .
Let us next prove (2).
for NFA
(p.w)=
1
*
( q .a)=
1
*
( q .
This can be shown to be equal to
.
Hence
2
*
( q .
As a preparation for the proof of commutativity of union and closure operations. Extremal Clause: Nothng is in ( X ) unless it is obtained by the Basis and Inductive Clauses.
A2 . What Part 1 states is that all the elements of (S T ) have the property of being in the set (S) (T). Let X be the set of states of an NFA. Let us review the definition of the closure of the set of states of an NFA.End of Induction With this Claim 1 we can see that any nonempty string w is accepted by NFA if and only if it is accepted by the corresponding NFA.. then its childen also have it. For that let us restate the statement so that the induction becomes clearer. Claim 2: (S T)= (S) (T). if it is accepted by an NFA. Hence
is accepted by NFA. q20 Conversely if NFAis accepted by NFA. By the way NFA is constructed from .
We are going to prove this in two parts:
(S
T) (S)
(S) (T)
( T ) and (S T). Since (S T ) is defined recursively. then ( { q10 } ) A1 .
Hence by the way A2 is constructed. in the Basis Step of our proof we prove the property for the elements of the basis of (S T ) and in the Inductive Step we prove that if an arbitrary element of (S T ) has that property. Hence is accepted by NFA.
Inductive Clause: If q ( X ) . ) (X).
and the corresponding NFA accept the same language. As for the empty string . Then the closure of X is defined recursively as
Basis Clause: X
(X). let us prove the following claim. then ( q . then q20 this means that Thus NFA( { q10 } ) A1
A2 .
Part 1 : (S T) (S) (T) This is going to be proven by induction on (S T).
.
End of Proof for Part 1
Part 2 : Proof of Part 2: We are going to prove (S) (S
(S) T ) and (S)
(T) (T) (T)
(S (S (S
T). then (q. That would imply that
Proof of
(S) (S By induction on (S
T): (S).S (S T). T ) with (T).
) . Inductive Step: We need to prove that for an arbitrary element q in ( S ) . Hence (T). Since q If q ( S ) . then (q. T). (T)
Thus all the elements of
(S
T ) have the property of being in (S T)
which is to say that
(S) (T). then (q. if q is in
. ) (S) (S) (T). S and T are subsets of (S T)
Inductive Step: We need to prove that if q is an arbitrary element of the property of being in (S) (S ( T ) . then ( q . ( T ) . ) (S) (T). (S) (T). (S T ) with the property of being in (S) ( T ) . (q. (T). ) (S) (T).
Basis Step: We need to show that S
Since S (S T ) . )
Let q be an arbitrary element of
T ) with the property of being in
( S ) by the definition of
( T ) . Hence ) Similarly if q Hence if q is an arbitrary element of (S) (T). and ( S T) (S T).q ( S ) or q ( S ) .
( T ) . T T).Proof of Part 1: Basis Step: We need to prove that ( S Since S ( S ) and T T) (S) (S) (S (S) (S) (T).
( Si ) = ( Si ) = ( S1 ) and ( S1 ) . 
T ) is a
closure. ) (S
closure
T).Inducion Hypothesis
( Si ) = ( = = (( Si ) (
( Si ) ) Si )
( Sn+1 ) by the definition of union. . then
(q. End of Proof of Part 2 End of Proof of Claim 2
Claim 3:
( Si ) = ( Si ) . If n = 1. then
Hence Inductive Step: Assume that (
( Si ) =
Si ) =
( Si ) holds for n = 1. since = (
Equivalence of NFA and DFA
We are going to prove that the DFA obtained from NFA by the conversion algorithm
.
Similarly Hence (S)
(T) (T)
(S
T ) holds.
Thus
(S)
(S
T ) has been proven. Proof : Proof by induction on n. End of Proof for Claim 3
Sn+1 ) by Claim 2 above.
Basis Step: n = 1. ( Sn+1 ) by the induction hypothesis.
(S T ) holds. Si is a set as well as Sn+1.(S Since q is in (S T ) and since (S
T ) . ( Si ) holds for n. Si ) by the definition of union. by the definition of (q.
)
(S
T).
0 .0 .0 .
2 1
.0 } by the construction of DFA M2 .0 . w ) for any string w. w ) =
2
*
( q2. A1 > and DFA
. .0 .
Proof: This is going to be proven by induction on w. =
* 1
( q1.
) = q2. Basis Step: For w =
2 *
. w ) . w ) = 2*( q2.0 .
Inductive Step: Assume that Induction Hypothesis
1
*
( q1.0 .0 .0 . w ) for an arbitrary string w. = { q1. w ) .
( q2. NFA that recognizes a language L is denoted by M1 = < Q1 . 
For the string w and an arbitrry symbol a in
1 *
. obtained by the conversion is denoted by M2 = < Q2.
) by the definition of
1
*
. a ) ( q2. a )
2( * 2
( q2. q2. A2 >
First we are going to prove by induction on strings that 1*( q1. it obviously implies that NFA M1 and DFA M2 accept the same strings. wa )
* 1
Thus for any string w
( q1. .0 .
Kleene's Theorem . q1.0 .Part 1
Subjects to be Learned
• • • •
Union of FAs Concatenation of FAs Kleene Star of FAs Acceptance of regular languages by FAs
Contents
. w ) =
2
*
( q2. w ) =
2
*
( q2.0 .0 . w ) holds. When it is proven. w ) . wa ) = = = =
2
(
1 2
* *
( q1.
1 *
( q1. Theorem: For any string w.0 .0 .accepts the same language as the NFA.0 .
( q1.0 by the definition of 2* .
. qk. Basis Step: As shown below the languages . Theorem 1 (Part 1 of Kleene's theorem): Any regular language is accepted by a finite automaton.0 . .
.Kleene's theorem. concatenation and Kleene star operations. Ak > . .
1
. where qu. We assume that Q1 Q2 = without loss of generality since states can be renamed if necessary. which are given below.0 . L1L2 and L1* are accepted by FAs.
2
.
k
. q2. . Then L1 = < Qc .{ } and { a } for any symbol a in are accepted by an FA. Mc
. respectively. that together with the Basis Step would prove the theorem.0 .0 is a state which is neither in Q1 nor in Q2 . Proof: This is going to be proven by (general) induction following the recursive definition of regular language.0 } . q1.0 . . .
Inductive Step: We are going to show that for any languages L1 and L2 if they are accepted by FAs. qu. A1 > and M2 = < Q2
. Au > . It states that any regular language is accepted by an FA and conversely that any language accepted by an FA is regular. then L1 L2 . Ac > and Mk = < Q2 .0 . qc. Suppose that L1 and L2 are accepted by FAs M1 = < Q1 . qu.
u
Mu = < Qu . Qu = Q1 Q2
. Since any regular language is obtained from { } and { a } for any symbol a in by using union.
c
. Au > :
{ qu.
u
L2 . L1L2 and L1* are accepted by the FAs Mu = < Qu .0 . respectively. A2 > .
.
.0 } ) } .0. q2.
.0 .
u
) = { q1.
. Qk = Q1
k
.0 } )  q A1 } Ak = { qk. Ac > :
Qc = Q1 Q2 qc.0 } . { q1. { q2. . where qk.0 . .
c
. { qk.0 = q1.0 } )  q
A1 } Ac = A2
Mk = < Qk . that is
u
(qu.0 } .u
=
1
2
{ (qu.
These NFA
s are illustrated below. a ) = for all a in . qc. Au = A1 A2
Mc = < Qc .0 }
=
1
{ (qk.0 .0. q2.0. Note that (qu.0 } ) } { (q. qk.0.
k
. Ak > :
{ qk.0 is a state which is not in Q1 .0 .0
c
=
1
2
{ (q. { q1.
End of Proof Examples of Mu . Mu.
. though we omit proofs. in fact accept L1 L2 . respectively. L1L2 and L1*. Mc and Mk .s .It can be proven. Mc and Mk: Example 1: An NFAthat accepts the language represented by the regular expression (aa + b)* can be constructed as follows using the operations given above. that these NFA.
Example 2: An NFAthat accepts the language represented by the regular expression ((a + b)a*)* can be constructed as follows using the operations given above.
.
Part 2
.Kleene's Theorem .
k)L(k+1. L(p. k)L(k+1. q. Lemma 1: L(p. See the figure below for the illustration. Given a finite automaton. where n is the number of states of the finite automaton. k) the set of strings representing paths from state p to state q that go through only states numbered no higher than k.Subjects to be Learned
•
Languages accepted by FAs are regular
Contents
The converse of the part 1 of Kleene Theorem also holds true. q.
What this lemma says is that the set of strings representing paths from p to q passing through states labeled with k+1 or lower numbers consists of the following two sets: 1. q. q. L(p. k+1. Note that paths may go through arcs and vertices any number of times. k) . Before proceeding to a proof outline for the converse. k) : The set of strings representing paths from p to q passing through states labeled wiht k or lower numbers. Next denote by L(p. k+1. then from k+1 to k+1 any number of times. It states that any language accepted by a finite automaton is regular. all without passing through states labeled higher than k. k+1. k)*L(k+1. k)*L(k+1.
. k) : The set of strings going first from p to k+1. Then the following lemmas hold. q. first relabel its states with the integers 1 through n. k+1. let us study a method to compute the set of strings accepted by a finite automaton. q. then from k+1 to q. 2. k) L(p. k+1) = L(p.
Hence if p and q are different. 0) is regular. L(p. >From Lemmas 1 and 2 by induction the following lemma holds. 0) is regular. where n is the number of states of the finite automaton. n) over all accepting states q.Lemma 2: L(p. q. Since the number of symbols is finite and since any finite language is regular. Since the language accepted by a finite automaton is the union of L(q0. If p = q. q. we have the following converse of the part 1 of Kleene Theorem. then it consists of single symbols representing arcs from p to q. 0) is the set of strings representing paths from p to q without passing any states in between. q. then is in it as well as the strings representing any loops at p (they are all single symbols). q. q. k) is regular for any states p and q and any natural number k. Lemma 3: L(p. Theorem 2 (Part 2 of Kleene's Theorem): Any language accepted by a finite automaton is regular. Example : Let us find the language accepted by the following finite automaton using the
. Proof: L(p.
2. 1) = r(2. 3.1.2.0) = a* .1.lemmas. 3.0) = a + .0) = ba+ + . 1. 2) = a+(b a+ + = a+(b a+ )*a .1. r(1. 1) + r(1. 2. since r(1. 2) = a* + a+(b a+)*b a* .0) = ba+ . k). 1) = r(3. 2)*r(3. r(2.0)*r(1.0)*r(1. 2. 1)r(2. r(1. 1) r(3.1. 1)r(2. q.1.1. 1. r(1.1. since r(3. 2.0) = ba* .
Let us denote by r(p.2. 1) + r(1.0) + r(3.1.2. 1.0)r(1.0) + r(2. 3. 1) r(1.1. 2) = r(3.0) + r(1. 3. since r(2. 1. 3.0)*r(1. 2) + r(1.0) = a+ . By Lemma 1. 2. r(2.0) = and r(3. 2) = r(1. 1) = a Hence r(1. 2): r(1. 1) = r(1.0) = a . 2. 1) = r(2. 2): r(1. 1. 1)*r(2.
.0) = b . 2) .1. 1) = r(3. 1. Then the language accepted by this NFA is r(1.2. 1. since r(1.1. 3.0)r(1. 3.0)r(1.2. 1) + r(3. 1)*r(2. 1) = r(2. 3. 3.1. 1. 2) = r(1. 2. 3. 3.1.0)*r(1. 1) = r(1. r(1.1.0) = and r(2. Hence r(1.2.0) + r(1. 1.2. 3) = r(1.1. 3). 2): r(3.2. 1. 1)r(2. 3. 1)*r(2.0)r(1.0)*r(1. q. 2. 3. 2)r(3. 2.0) = b. 1) r(1. 3.1. )*a
r(3.2.0) + r(2. k) the regular expression for the set of strings L(p.1. 1.0)r(1.
Q . 2) = r(3.1. q. If there are more accepting states. A > be a DFA that accepts a language L. 2): r(3. 1. where p is the initial state and n is the number of states in the given finite automaton.1.1. 1. 2) = = + ( ba+)+a
+ ba+( ba+ +
)*a
r(3.Hence r(3. 3. 1)*r(2. 1. Then a DFA that accepts the complement of L. i. 1) = r(3.0) = ba* Hence r(3. q0 . 1. . and all the r(p.1.e. 2.0)r(1. q. then r(p. n) must be found for each accepting state q. . q0 .A > is a DFA that accepts * .0)<SUP*< SUP>r(1. * . 1)r(2.
Hence r(1. 3) = a* + a+(b a+)*ba* + ( a+( ba+ )*a )(
This can be further simplified to (a + ab + abb)*. 1. can be obtained by swapping its accepting states with its nonaccepting states. )*ba* + ( ba+)+a )*( ba+ )*ba*. then to (a + ab)*.0) + r(3.
Comlement and Intersection of Regular Language
Subjects to be Learned
• • •
Complement of Regular Language Complement of DFA Intersection of Regular Languages
Contents
Complement Let M = < Q . 1) r(3.L . 2) = ba* + ba+( ba+ + = ( ba+ )*ba* . 1. 1) + r(3. that is Mc = < Q . 2. . The detail is left as an exercise though it would be quite challenging. In this example there is only one accepting state.
.L. 1. . n)'s must be added together to get the regular expression for the language accepted by the automaton.
. the complement of a regular language is also regular. b }.For example the following DFA accepts the language a+ over
= { a . Remark 2: Since a language is regular if and only if it is accepted by some NFA.
Remark 1: If we have NFA rather than DFA.
A DFA that accepts its complement is obtained from the above DFA by changing all single circles to double circles and vice versa as shown below. we must first convert it to DFA before swapping states to get its complement.
Click True or Fals .Regular Grammar Back to Study Schedule Back to Table of Contents
Finite Automata
Regular Grammar
Subjects to be Learned
• • •
Production and Grammar Regular Grammar ContextFree. then Submit. difference.
Test Your Understanding of Complemnent and Intersection of FAs
Indicate which of the following statements are correct and which are not. intersection. then their complements are regular languages. concatenation and Kleene star operations. L1 L2 is regular. if L1 and L2 are regular languages. ContextSensitive and Phrase Structure Grammars
.
Next . Since L1 L2 = by De Morgan's law.
Thus summing all this up we can say that the set of regular languages over an alphabet is closed with respect to union. Therefore all the properties of sets are inherited by languages. By Remark 2 above.Intersection of Regular Languages Langauges are sets. In particular De Morgan's law also applies to languages.
to obtain aS. If we are not interested in the intermediate steps. we write =>*G is obtained from a string and say that by applying productions of . aa. and S > aS . . Then for every production X > aY. For example consider the language represented by a+. regular. if L . finite automata and construction from simple languages using simple operations. Rewrite S using one of the following two rules: S > a . S corresponds to the initial state. i. where is a string of terminals and nonterminals with at least one nonterminal in it and is a string of terminals and nonterminals. that is a production is of the form X > a or X > aY.e. In general if a string a grammar G. A production has in general the form > . aaa. which ia a nonterminal.Contents
We have learned three ways of characterising regular languages: regular expressions. which is { a. A grammar is regular if and only if is a single nonterminal and is a single terminal or a single terminal followed by a single nonterminal. = {a. the fact that aa is obtained from S is written as S =>* aa . start with S and apply the second rule to replace S with the right hand side of the rule. a ) = Z. where X and Y are nonterminals and a is a terminal. aS. a ) = Y and for every production X > a add the transition ( X.{ by a regular grammar.
. S > bS. Thus the process of obtaining aa from S is written as S => aS => aa . . a start symbol S. Add another state as the accepting state Z. One can generate the strings of this language by the following procedure: Let S be a symbol to start the process with. If there is no
is derived from
ambiguity about the grammar G that is referred to. A grammar is a set of rewrite rules which are used to generarte strings by successively rewriting symbols. V = { S } and P = { S > aS. The following theorem holds for regular grammars.e. For example. that is by something called grammar. add the transition ( X. Theorem 3: A language L is accepted by an FA i. b}. There is yet another way of characterizing them. . S > } is a regular grammar and it generates all the strings consisting of a's and b's including the empty string. To generate the string aa for example. Then apply the first rule to aS to rewrite S as a. } . then we simply write =>* Formally a grammar consists of a set of nonterminals (or variables) V. We write S => aS to express that aS is obtained from S by applying a single production. and a set of rewrite rules (productions) P. a set of terminals (the alphabet of the language). That gives us aa. These rules mean that S is rewritten as a or as aS. } can be generated
This can be proven by constructing an FA for the given grammar as follows: For each nonterminal create a state.
then since {
} is regular . a regular grammar corresponding to the NFA given below is < Q. and adding transitions ( S.e. P = { S > aS. V = { S } and P = { S > aS. The NFA thus obtained is shown below. . b}. X > bS. Theorem 4 : If L is regular i. L =
Conversely from any NFA < Q. . X > aY. Z } and ( S. P.For example = {a. Y > bS. X. S > b } form a regular grammar which generates the language ( a + b )+. .
Thus L . a) = Y for some accepting state Y. a) = Y . b ) = { S. q0 > is obtained as follows: for any a in . S > .{ regular grammar. } is generated by a
For example. and nonterminals X and Y. A > a regular grammar < Q. then L . An NFA that recognizes this language can be obtained by creating two states S and Z. Z } . Thus the following converse of Theorem 3 is obtained. q0. accepted by an NFA. If L contains ( L { } ) { } is also regular. S > bS.
as its member. S > a. X > aY is in P if and only if (X. Y } . S > a }.
. X > a is in P if and only if (X. a ) = { S. and for any a in and any nonterminal X. b }. { a. P. where S is the initial state and Z is the accepting state of the NFA.{ } is regular. where Q = { S. S > aX.
where is a string of terminals and nonterminals. S > ab } with = { a. S > XYZ. S. contextsensitive grammars and phrase structure grammars. For example P = { S > XYZS1. respectively. As we shall see later this is an example of contextfree language which is not regular. They are characterized by contextfree grammars. aY > ab. S1 > XYZ. that is the set of regular languages is a subset of the set of contextfree languages which is in turn a subset of the set of contextsensitive languages and the set of contextsensitive languages is a subset of the set of phrase structure languages. c } and V = { X. A grammar is a contextsensitive grammar if and only if its production is of the form
1
X
2
>
1
2
. where X is a nonterminal and .
1
. Z.
. that is. These grammars are distinguished by the kind of productions they have but they also form a hierarchy. aX > aa. BY > bb. ZY > YZ. S1 } is a contextsensitive grammar and it generates the language { anbncn  n is a positive integer } . possibly the empty string.
2
and
are strings of terminals
and nonterminals. Y.In addition to regular languages there are three other types of languages in Chomsky hierarchy : contextfree languages. It is an example of contextsensitive language which is not contextfree. contextsensitive languages and phrase structure languages. ZX > XZ. b } and V = { S } is a contexfree grammar and it generates the language { anbn  n is a positive integer } . S1 > XYZS1. b. A grammar is a contextfree grammar if and only if its production is of the form X > . for every production > . cZ > cc } with = { a. bZ > bc. possibly empty except
Thus the nonterminal X can be rewritten as only in the context of 1X 2 . X > a. YX > XY. Contextsensitive grammars are also characterized by productions whose left hand side is not longer than the right hand side. For example P = { S > aSb.   .
q0 . A > be a DFA that accepts a language L.Minimization of DFA Back to Study Schedule Back to Table of Contents
Finite Automata
Minimization of DFA
One important result on finite automata. . is that for any regular language there is a unique DFA having the smallest number of states that accepts it. . that is a production of a phrase structure grammar can take the form can be any string. both theoretically and practically. where and
Test Your Understanding of Regular Grammar
Indicate which of the following statements are correct and which are not.A } of the set of states Q . that has the smallest number of states amomg the DFAs that accept L.
Next . There are two sets of questions. Let M = < Q . Then the following algorithm produces the DFA. then Submit. there is no restriction on the form of production. Minimization Algorithm for DFA Construct a partition = { A.For a phrase structure grammar. > . }. denote it by M1. Q . Click True or Fals . new := new_partition(
.
new := new_partition( := . then the minimum DFA M1 has a transition from p to q on symbol a.while ( :=
new new
) )
. The subsets thus formed are sets of the output partition in place of S. The accepting states of M1 are representatives that are in A. S remains in the output partition. These representatives are states of minimum DFA M1. end Minimum DFA M1 is constructed from
• •
final
as follows:
• •
Select one state in each set of the partition final as the representative for the set.e. Let us also denote by p and q the sets of states of the original DFA M represented by p and q. Any transitions to a dead state become undefined. Example 1 : Let us try to minimize the number of states of the following DFA. A state is a dead state if it is not an accepting state and has no outgoing transitions except to itself. final
function new_partition( ) for each set S of do partition S into subsets such that two states p and q of S are in the same subset of S if and only if for each input symbol. respectively. Note that the sets of final are either a subset of A or disjoint from A.
. If a transition from s to t on symbol a exists in M. Let s be a state in p and t a state in q. p and q make a transition to (states of) the same set of .
Remove from M1 the dead states and the states not reachable from the start state. The start state of M1 is the representative which contains the start state of M. Let p and q be representatives i. if there are any. states of minimum DFA M1. If S is not partitioned in this process.
and 3 goes to 1 on a in the original DFA. This becomes the second iteration. Note here that state 4 is a dead state because the only transitionout of it is to itself. and from 3 to 1 on a. 2 and 4 are separated from each other in new. and to 2 on b in the original DFA. states 3 and 4 are going to be separated from each other in new. since 1 and 5 do the same transitions.
new_partition is applied to . 3 }. in the minimized DFA transitions are added from 1 to 3 on a. Also since 2 goes to 1 on b. On the other hand 1 and 5 make the same transitions. Thus final = { { 1 . 3 .Initially
= { { 1 . since 1 goes to 3 on a. For the transitions. Since the rest are singletons. and 1 to 2 on b. all transitions between them are inherited for the minimized DFA. So they are not going to be split. since on b 2 goes to 1. 5 } . Since the rest of the states are singletons. in the
When new_partition is applied to this new . 4 goes to 4 and 1 and 4 are in different sets in . Select 1 as the representative for { 1 . Further. state 3 goes to state 4 and 1 and 4 are in different sets in . { 3 } . 4 } }. { 2 . { 3 } . { 4 ] }. { 4 ] }. { 2 } . remains unchanged.
. states 2 and 3 are going to be separated from each other in new . state 3 goes to state 5 and 4 and 5 are in different sets in . { 2 } . they have the obvious representatives. 2 . 5 } . Also since on a sate 4 goes to sate 4. Thus the new partition is { { 1 . Thus the set of states for the minimized DFA is { 1 . 5 } . 5 }. Since on b state 2 goes to state 1. in the minimized DFA transitions are added from 2 to 1 on b.
.Thus the minimized DFA is as given in the following figure:
Example 2 : Let us try to minimize the number of states of the following DFA.
{ 1 . 6 } }. new = { { 3 } . { 1 . then Submit. { 3 } . 6 } } is obtained. 2 . { 5 } . { 2 } . 5 . { 6 } } is obtained. { 6 } } is obtained. Click True or Fals . Applyting new_partition to this . { 2 } . { 1 . Applyting new_partition again.Initially = { { 3 } .
Test Your Understanding of Minimization of DFA
Indicate which of the following statements are correct and which are not. 4 } .Application of FA Back to Study Schedule Back to Table of Contents
Finite Automata
Application of Finite Automata
Subjects to be Learned
•
Reactive system
. 4 . Thus the number of states of the given DFA is already minimum and it can not be reduced any further.
Next . { 2 . 4 . { 4 } . new = { { 1 } . { 5 } . 5 } . new = { { 3 } . By applying new_partition to this .
Then the transitions between the states triggered by events and conditions. on the other hand. To model a reactive system with finite automaton. These become the states of the finite automaton that models it.
. An adder does not respond unless the input i. first the states the system goes in or the modes of its operation are identified. are identified and they become arcs in the transition diagram of the finite automaton.•
Modeling reactive systems with FA
Contents
We have seen an example of use of finite automata in describing the operation of a simplified version of vending machine. A system such as an adder is called a transformational system. computer network communication protocols. a system must respond to each stimulus. even to a fragment of input such as each coin tossed in for a can of soda or every message received.e. Many of those systems fall into the class of systems called reactive system. A reactive system is a system that changes its actions. two numbers to be added are ready. outputs and conditions/status in response to stimuli from within or outside it. lexical analysers for compilers etc. It is an event driven or control driven system continuously having to react to external and/or internal stimuli. Let us assume for simplicity that this computer accepts a single user at a time. In addition actions that may take place in those states can also be added to the model. For example consider the following very simplified version of login process to a computer from the computer point of view. Finite automata are formal and rigorous and computer programs can be easily written to simulate their behaviors. The inputs for a reactive system are never ready unlike for example when two numbers are added together by an adder (Here we are considering an adder at a higher level of abstraction than physical devices level ignoring for example the transient states of the electronic circuit that realizes an adder). external or internal to the system. It is generally agreed that finite automata are a natural medium to describe dynamic behaviors of reactive systems. In the case of vending machine or communication protocol. Many other systems operating in practice can also be modeled by finite automata such as control circuits of computers.
If the user name typed in is not valid. it sends "Acknowledgement" (ACK) to the other computer. If the second password fails. then it asks for and then waits for the password. goes back to the initial state and waits for another login. sending message and waiting for ACK.
Initially the computer is in wait state waiting for "Request for Next Message" (RFNM) to come from another computer. it checks whether or not the name is valid. it goes back to the initial state and waits for another RFNM to come. it goes into another wait state waiting for an ACK to come from the other computer. which is another state. it resends the message. which is another state. When a name is typed in. receiving RFNM. Thus a finite automaton that models this protocol has the following five states: initial state (wait for RFNM). The next example is a protocol for a computer to follow in communicating with another computer. different states would be identified and transitions would have to be selected accrdingly. But let us make it simple. Depending on how much detail we are interested in. When the session terminates. When it is complete. it starts sending the requested message to the other party. If a negative ACK is received.
. When a password is typed in and it is correct. After sending the ACK. If a positive ACK is received.Initially the computer waits for a user name to be typed in. Upon completion of the RFNM. it goes into the state of receiving it (Our interpretation is that the computer is in a state of receiving an RFNM and it is taking the action of receiving the RFNM) . If it is valid. When a RFNM starts coming. We could make it go to a different state and count the number of login attempts for security purpose. That is another state though it could further be broken down into a number of more states. That is a fourth state. different states and transitions would have to be chosen. then it accepts the user and starts a session. If the password typed in is incorrect. This is one state of the system. it goes back to the initial state. it goes to the initial state and starts all over again. Again what we have seen is a model for one level of abstraction. sending ACK. Again it is a very simplified version. Again depending on the level of abstraction. then it informs the user of that and waits for the next try. it gets a signal.
then it goes to state P indicating that a decimal point has been read. On the other hand if a digit has been read before a decimal point. denote it by G. 9 } . Therefore from state P it goes to another state. that indicates a digit has been read before a decimal point.2. then it goes into a state. after reading a digit and stays there as long as digits are read. in state P).d+ ). If the first symbol is a sign. If a decimal point is received before a digit. then it can continue receiving digits and stay in D.8 etc. denote it by Q. it goes into a state.. they can be represented by the following regular expression: ( s+ + s. . . followed by one or more digits. then it must receive at least one digit after that.+ ) ( d+. Since these numbers are represented by strings consisting of a possible sign.378. followed by a possible decimal point. If a decimal point has been read (i. If the first digit is received before a decimal point.e. where s+ and s.1. that indicates that a sign has been received. that indicates that a decimal point has been read. denote it by D. i. respectively and d {0. then it goes into a state. This Q is an accepting state.e. denote it by P. it is in state D. +213. This system can also be described by a regular expression. If a decimal point is read while in D.represent the positive and negative signs. After one digit it can continue receiving digits. This system can be modeled by the following finite automaton:
. 15. followed by zero or more digits.Our third example is a system that recognizes numbers with or without a sign such as 5. One such system initially waits for the first symbol to come in. regardless of whether a sign has been read or not. .d+ + d+ + . D is another accepting state.
A third array. while ( state NO_of_STATES and not End of Input ) index := STATEX [state] . Another array. Algorithm FA Simulator state := INITIAL_STATE. It uses four arrays. Those indices are used to access the contents of the other arrays. keeps the index of the first symbol in the TOKEN array for each state.
while ( TOKEN [index]
. called NEXT_STATE. holds the next state for each input symbol for each state. called ACTION. input := read_input( ) . we can use a general purpose program to simulate its operation.Next . indicates the actions taken at each state and a fourth. One such simulation algorithm is given below. 0 and TOKEN [index] input ) index := index + 1. called TOKEN.Simulation of FA Back to Study Schedule Back to Table of Contents
Finite Automata
Simulation of Finite Automata
Subjects to be Learned
•
Simulation of FA
Contents
Once a finite automaton is constructed. One array. called STATEX. stores for each state the input symbols that trigger transitions from the state.
to input 3. end Here 0 in the TOKEN array is a marker between states. Then type in a number you want the FA to recognize. The ACTION array would contain pointers to actions to be taken corresponding to arcs traversed such as converting a digit in BCD form to the corresponding binary number.45. You must hit the "Tab" key to move to the next box. C and H. else error input := read_input( ) . The numbers below NEXT_STATE array show the correspondence between the indices of the STATEX array and the states A. Then every time you click "SHOW" the number is processed digit by digit. first click the box pointed by the red arrow in the figure below. At the moment it is empty. The corresponding transitions are going to be shown by red arrows in the transition diagram.
. Then type 3 Tab . For example. In the DFA below all the transitions to the empty state (i.if ( TOKEN [index] 0) perform the action specified by ACTION [index]. B. S corresponds to 1. empty transitions) are omitted. So no action is taken as a number is processed. Tab 4 Tab 5. first click the box under the red arrow. S is the initial state and B and H are accepting states.
To see how this algorithm works. state := NEXT_STATE [index].e.
.If you are also interested in how code is executed. click here It is extremely slow. So be patient.
a finite automaton must remember how many a's it has read when it starts reading b's. For example to recognize the language { anbn  n is a natural number} . For example. either xz and yz are both in L or they are both not in L. Nonregularity test based on MyhillNerode's theorem Indistinguishability of strings: Strings x and y in * are indistinguishable with respect to a language L if and only if for every string z in *. That is the main limitation of finite automata. their properties and their usefulness for describing various systems.NonRegular Languages Back to Study Schedule Back to Table of Contents
Regular Languages
NonRegular Languages
Subjects to be Learned
• • •
Existence of nonregular languages Myhill . a and aa are indistinguishable with respect to the language an over alphabet { a }. languages that are not regular and therefore require devices other than finite automata to recognize them. we can conclude that { anbn  n is a natural number} is not regular. Thus it must be in different states when it has read different number of a's and starts reading the first b. But any finite automaton has only finite number of states. In this section we are going to study some of the methods for testing given languages for regularity and see some of the languages that are not regular.Next . There are. The main idea behind these test methods is that finite automata have only finite amount of memory in the form of states and that they can not distinguish infinitely many strings. Thus there is no way for a finite automaton to remember how many a's it has read for all possible strings anbn . however.Nerode Theorem for nonregularity test Pumping Lemma
Contents
We have learned regular languages. This is the basis of two of the regularity test methods we are going to study below: MyhillNerode Theorem and Pumping Lemma. Since a regular language must be recognized by a finite automaton. because aak and aaak are in the language an for any
. where n is a positive integer.
Using this concept of indistinguishability. the following theorem by Myhill and Nerod gives a criterion for (non)regularity of a language. operations + and * and left and right parentheses. whose strings are pairwise distinguishable with respect to L. where k and m are positive integers and k m . (x*y) . We are going to show that its strings are pairwise distinguishable with respect to L1. Inductive Clause: If and are in L3 . Let ak and am be arbitrary two different members of the set. Since ak and am are arbitrary strings of S2. a and aa are not indistinguishable (hence distinguishable). Select bm as a string to be appended to ak and am . Select bakb as a string to be appended to ak and am . b }* } is nonregular. Hence ak and am are distinguishable with respect to L1 . Theorem : A language L over alphabet is nonregular if and only if there is an infinite subset of * . x . b } and it is infinite. Hence L2 is nonregular. Hence ak and am are distinguishable with respect to L2 . Since ak and am are arbitrary strings of S1. because ab is in the language anbn while aab is not in the language. Extremal Clause: Nothing is in L3 unless it is obtained from the above two clauses. then ( + ) and ( * ) are in L3 . ( ( x + y ) * x ) and (( (x*y) + x ) + (y*y) ) are algebraic expressions. It can be shown to be pairwise distinguishable with respect to L2 as follows. Hence L1 is nonregular. with respect to the language anbn . L3 can be defined recursively as follows: Basis Clause: x and y are in L3 . where k and m are positive integers and k m . S1 is over alphabet { a . This set is infinite
.positive integer k. For more on MyhillNerode theorem click here. S1 satisfies the conditions of MyhillNerode theorem. Example 1: L1 = { anbn  n is a positive integer } over alphabet { a . It is stated without a proof. Consider the set of strings S2 which is the same as S1 of Example 1 above. Then akbm is not in L1 while ambm is in L1 . S2 satisfies the conditions of MyhillNerode theorem. Example 2: L2 = { ww  w {a. Let ak and am be arbitrary two different members of the set S1. b } can be shown to be nonregular using MyhillNerode as follows: Consider the set of strings S1 = { an  n is a positive integer } . Then akbakb is in L2 while ambakb is not in L2 . that is. For example. Consider the set of strings S3 = { (k x  k is a positive integer } . However. the set of strings consisting of one or more right parentheses followed by identifier x. Example 3: Let L3 be the set of algebraic expressions involving identifiers x and y.
Those strings which are accepted by this NFA and whose length is greater than 5 have a substring which can be repeated any number of times without being rejected by the NFA. then there must be a cycle in the NFA along some path from the initial state to some accepting state (such as the cycle 2342 in the above example). Then (k x + [ + x ) ]k is in L3 but (m x + [ + x ) ]k is not in L3 because the number of ('s is not equal to the number of )'s in the latter string. Pumping Lemma Let us consider the NFA given below. abbabbabbb etc. Hence S3 is pairwise distinguishable with respect to L3 . Hence L3 is not regular. are also accepted by the NFA. For example [ + x ) ]3 is +x) +x)+x) . the resultant strings such as abbb (bba repeated 0 times). abbabbabbb.and it can be shown to be pairwise distinguishable with respect to L3 as follows: Let (k x and (m x be arbitrary two strings of S3 . abbabbabbabbb etc.
This NFA accepts among others some strings of length greater than 5 such as abbabbb. The following theorem which is called Pumping Lemma is based on this observation. where k and m are positive integers and k m . Select [ + x ) ]k as a string to be appended to (k and (m . Then the substring representing that cycle (bba in the example) can be repeated any number of times within the string w without being rejected by the NFA. It
. In general if a string w (such as abbabbb in the example above) is accepted by an NFA with n states and if its length is longer than n. For example the string abbabbb is accepted by the NFA and if one of its substrings bba is repeated any number of times in abbabbb.
It is stated without a proof here. Then uv2w = anpa2pbn = an+pbn . let us prove that the language L = akbk is nonregular. then any long enough string of the language has a substring which can be repeated any number of times with the resultant strings still in the language. Thus Pumping Lemma can not be used to prove the regularity of a language. uvmw L. Then there is an FA that accepts L. Hence L is not a regular
0. Example 4: As an example to illustrate how Pumping Lemma might be used to prove that a language is nonregular. v = ap. Since p > 0 . Suppose that L is regular and let n be the number of states of an FA that accepts L. that is. uvmw
Test Your Understanding of Nonregularity
. v and w which satisfy the following relationships: x = uvw uv n v > 0 and for every integer m 0. Also since uv Let us now consider the string uvmw for m = 2. and for every m 0. where k is a natural number. for some p > 0 . Consider a string x = anbn for that n.
n . even if there is an integer n that satisfies the conditions of Pumping Lemma. Let n be the number of states of that FA. It can only show that a language is nonregular. Hence an+pbn can not be in the L.
Note that Pumping Lemma gives a necessity for regular languages and that it is not a sufficiency. Then there must be strings u. This violates the condition that for every m language. v. v has at least one symbol. n. Pumping Lemma : Suppose that a language L is regular.
Since v > 0 . the language is not necessarily regular. and w such that x = uvw. Then for any string x in L with x n. uv n v > 0 . there are strings u. n + p language L represented by akbk .states that if a language is regular. uvmw L.
Proof of Theorem Necessity Suppose that a language L is regular and two strings. if and only if
}.ContextFree Grammar Back to Study Schedule Back to Table of Contents
MyhillNerode Theorem
The nonregularity test for languages by MyhillNerode is based on the following theorem which is in the contrapositive form of the theorem used for nonregularity test. are distinguishable with respect to L. then Submit. the DFA reaches different states. If there are three strings that are distinguished with respect
. that is.
if and only if they are indistinguishable with respect to . Also it is a corollary to MyhillNerode theorem: Let { be the followijg relation on }={ : For strings and of . Click True or Fals . This means that if x and y are read by an DFA that recognizes L. Then there is a string z such that xz is in L and yz is not in L (or xz is not in L and yz is in L). Then the theorem is is staed as follows: Theorem: A language L over alphabet is regular if and only if the set of equivalence classes of is finite.
Next . say x and y.Indicate which of the following statements are correct and which are not.
then the DFA must have infinitely many states. ]. making them belong to different [ Hence is a function. ) as follows: ]} ]. then the language is not regular. namely ] and ]. Note that "indistinguishable with respect to L" is an equivalence relation over the set of strings (denote it by ) and [x]'s are equivalence classes. We will show that a DFA that accepts L can be constructed using these equivalence classes.. Then we construct a . be the number of distinct equivalence classes (i. let [x] denote a class of strings that are indistinguishable from a string x with respect to L.
be representatives of those distinct equivalence classes..
]. if the number of classes of strings that are pairwise indistinguishable with respect to a language L is finite. Sufficiency Conversely. Hence if there is an infinite set of strings which are pairwise distinguishable with respect to a language.to L. if and in [ ]. .e. [ .
are in different classes for
and are distinguishable with respect to L. which it can not because a DFA must have a finite number of states.... Let . .
. DFA ( = {[ =[ = {[ ( . then the language L is regular. then the DFA reaches three different states after reading those three strings. For. the index) of and let .
Let us now show that this machine is in fact a DFA and it accepts the language First.. . where [ ] =[
] for all
. then ]'s. [ ]=[ }
]. note that for every string [ ].. .. To prove this.
is in exactly one equivalence class. Hence if there are infinitely many strings to be distinguished with respect to L..
then every
We then show that for every string . let us show that this DFA accepts string in [ ] is also in L.
=[
]. Our proof is by structural induction on string Basis Step: = =[ ]. .
] by the induction hypothesis.
for DFA. . Since . where
. by the definition of = . =[ ]. First some terminology. = [ ]. An equivalence relation on is said to be right invariant if for every . if a string in [
in ] is in
. With these terminology. . if the set of its equivalence classes is finite. ]. this means that the DFA accepts
MyhillNerode Theorem Let us here state MyhillNerode Theorem. Hence
Hence we have shown that for every string ].
. by the definition of = . where ].Next. Also an equivalence relation is said to be of finite index. if
then for every . where ] is the equivalence class that belongs to.
. first note that if
. Hence =[ = . Then by the definition of
.
. MyhillNerode Theorem can now be stated as follows: The following three statements are equivalent: (1) A language is regular. For that.
Inductive Step: Assume Then for every But =[ Hence =[ ].
S > ab }.
ContextFree Languages
ContextFree Grammar
Subjects to be Learned
• • •
ContextFree Grammar ContextFree Languages Push Down Automata
Contents
Earlier in the discussion of grammars we saw contextfree grammars. b } and P1 = { S > aSb . The set of strings generated by a contextfree grammar is called a contextfree language and contextfree languages can describe many practically important systems.(2) L is the union of some of the equivalence classes of a right invariant equivalent relation of finite index. S . They are grammars whose productions have the form X > . P1 > generates L1 : V1 = { S } . Definition (ContextFree Grammar) : A 4tuple G = < V . grammar (CFG) if V and . S . = { a . where X is a nonterminal and is a nonempty string of terminals and nonterminals. For the following contextfree grammar G1 = < V1 . Example 1: L1 = { anbn  n is a positive integer } is a contextfree language. Let us define contextfree grammars and contextfree languages here. S . (3) is of finite index.
.
Proofs are omitted. and P is a finite set of productions of the form X >
and (V )* . where X
the start symbol. Most programming languages can be approximated by contextfree grammar and compilers for them have been developed based on properties of contextfree languages. .
are finite sets sharing no elements between them. A language is a contextfree language (CFL) if all of its strings are generated by a contextfree grammar. P > is a contextfree V is V.
. . if necessary. Example 4: Portions of the syntaxes of programming languages can be described by contextfree grammars. P2 > be contextfree grammars generating L1 and L2 . . b } and P2 = { S > aSa . Next define Vu = V1 V2 { Su } and Pu = P1 P2 { Su > S1 . S > S*S . S . Ss > L2 . For example { < statement > > < ifstatement > . . ) . < ifstatement > > if ( < expression > ) < statement > . S . < expression > > < logicalexpression > . if necessary. 3 = { x . For the following contextfree grammar G3 = < V3 . Su > S2 } . concatenation and Kleene star of contextfree grammars as follows: Let G1 = < V1 . . S > bSb . Then it can be easily seen that Gu = < Vu . P3 > generates L3 : V3 = { S } . * } and P3 = { S > ( S + S ) . . S2 . b }+ } is a contextfree language . y . S1 . operations + and * and left and right parentheses. first relabel symbols of V2 .Example 2: L2 = { wwr w {a. S > x . } . Then let Su be a symbol which is not in V1 V2 . < statement > > < forstatement > . Su . S > bb }. so that V1 and V2 don't share any symbols. L1L2 . w is spelled backward to obtain wr . first relabel symbols of V2 . For L1* . Sc . Then L3 is a contextfree language. Then for L1 L2 . . respectively. P1 > and G2 = < V2 . . < expression > > < algebraicexpression > . < expression > ) < statement > . 3. . where w is a nonempty string and wr denotes the reversal of string w. Then it can be easily seen that Gc = < Vc . so that V1 and V2 don't share any symbols. that is. < expression > . Similarly for L1L2 . Outline of Proof This theorem can be verified by constructing contextfree grammars for union. = { a . Then L1 contextfree languages. < forstatement > > for ( < expression > . let Ss be a symbol which is not in V1 . . For the following contextfree grammar G2 = < V2 . Properties of ContextFree Language Theorem 1: Let L1 and L2 be contextfree languages. Then let Sc be a symbol which is not in V1 V2 . Example 3: Let L3 be the set of algebraic expressions involving identifiers x and y. < statement > > < assignment > . + . . . . S > aa . and L1* are
. . Then let Ps = P1 { Ss > SsS1 . ( . S > y }. Pc > is a contextfree grammar that generates the language L1L2 . Pu > is a contextfree grammar that generates the language L1 L2 . . P2 > generates L2 : V2 = { S } . Next define Vc = V1 V2 { Sc } and Pc = P1 P2 { Sc > S1S2 } . .
but also by the symbol at the top of the stack. Ss . generates the language L1* . q0 is the initial state. Otherwise reject it. Ps > is a contextfree grammar that
Like regular languages which are accepted by finite automata. A pushdown automaton ( or PDA for short ) is a 7tuple M = < Q . contextfree languages are also accepted by automata but not finite automata. and it replaces X with the string at the top of the stack. X ) = ( q . q0 . Second. reject the string. respectively ). For example. a . They need a little more complex automata called pushdown automata. Example 1 : Let us consider the pushdown automaton < Q . push them into the stack. . A . the contents of the stack can also be changed every time an input symbol is read. Any string of this language can be tested for the membership for the language by a finite automaton if there is a memory such as a pushdown stack that can store a's of a given input string. > . ) means the following: The automaton moves from the current state of p to the next state q when it sees an input symbol a at the input and X at the top of the stack. Let us define this new type of automaton formally.
. and are finite sets ( the input and stack alphabet. accept the string.} . It can be seen that the grammar Gs = < Vs . q0 .
. Pushdown Automata
. As soon as the symbol b appears stop storing a's and start popping a's one by one every time a b is read. Z0 is the initial stack symbol and it is a member of .
.
Thus ( p . check the stack. A is the set of accepting states is the transition function and :Q ( ( } > 2 Q
*
. > . This automaton behaves like a finite automaton except the following two points: First. Thus its transition function specifies the new top of the stack contents as well as the next state. When all the symbols of the input string are read. Z0 . its next state is determined not only by the input symbol being read. A . . If another a (or anything other than b) is read after the first b. as a's are read by the finite automaton. where Q is a finite set of states. where Q = { q0 . Z0 . If it is empty. Let us consider a contextfree language anbn .
Z0 ) . we write
) is reached from ( p . aZ0 ). After reading the second a. another a is popped from the top of the stack and the PDA stays in state q1 . = { a . To express that the PDA moves from configuration ( p . ) ) (q. Z0 )
*
( q2 .x. where the input is read from left to right and the top of the stack corresponds to the leftmost symbol of . Z0 ) (
If we are not interested in the intermediate steps. ) in a single move (a single application of the transition function) we write (p. . b . aabb . .q1 . abb . x . When the second b is read. it is ( q0 . y . . aabb . where q is the state the PDA is currently in. Z0 ). ) to configuration ( q . . aZ 0 ) q0 a a ( q0 . Next it moves to the state q2 which is the accepting state. After reading the first a.
Let us now see how the PDA of Example 1 operates when it is given the string aabb . it moves to state q1 and pops a from the top of the stack. b . aZ0 ) ( q0 . y . ) ( q2 . aZ0 ) ( q1 . b } . for example. following table:
= { a . A = { q2 } and let
be as given in the
State Input Top of Stack Move q0 a Z0 ( q0 . Initially its configuration is ( q0 .x. aaZ0 ) ( q1 . aa ) q0 q1 q1 b b a a Z0 ( q1 . A . Z0 ). abb . q2 } . ). aabb . This entire process can be expressed using the configurations as ( q0 . A configuration of a PDA M = < Q . Z0 } . aaZ0 ). ) by a sequence of zero or more moves. aZ0 ). (p.
). q0 . Thus aabb is accepted by this PDA. ( q0 . x . ) . we can also write ( q0 . Z0 ). b . bb . ) ( q1 .y. To describe the operation of a PDA we are going to use a configuration of PDA. Z0 . x is the unread portion of the input string and is the current stack contents. bb . its configuration is ( q0 . Thus the configuration is ( q1 .y. If ( q . > is a triple ( q . Thus the configuration is ( q1 . .
. x . Then when the first b is read. Z0 )
This pushdown automaton accepts the language anbn .
*
(q. Z0 ) q2 .
.
= { a . X / ) indicating that X at the top of the stack is replaced by upon reading a from the input. b . ) . q2 } .k. where Q = { q0 . A . Z 0 )
In this table
represents either a or b. and an accepting state q. acceptance by final state) if (q0. ) ( q2 . = { a .
Example 2 : Let us consider the pushdown automaton < Q .
*
(q. a .
Like FAs. b . then an arc from state p to state q is added to the diagram and it is labeled with ( a . For PDAs. Z0) . ( q1 . . b ( q1 . for some in *. a ( q0 . however. Z0 } . q0 . aZ 0 ) q0 b Z0 ( q0 . > . Z0 .A string x is accepted by a PDA (a. q1 . If ( q . c } .
. x. For example the transition diagram of the PDA of Example 1 is as shown below.a. . ) ) ) )
( q1 . A = { q2 } and let be as given in the following table: State Input Top of Stack Move q0 a Z0 ( q0 . ). PDAs can also be represented by transition diagrams. X ) = ( p . bZ 0 ) q0 q0 q0 q1 q1 q1 a b c a b a b Z0 ( q0 . arcs are labeled differently than FAs.
In the figure and 2 represent a or b. aZ0 ) ( q1 . it ignores c and from that point on if the top of the stack matches the input symbol. . For example for the input abbcbba. aZ0 ) ( q0 . cbba . it pops the stack. Otherwise it rejects the input string. The transition diagram of the PDA of Example 2 is as shown below. it goes through the following configurations and accepts it. When there are no more unread input symbols and Z0 is at the top of the stack. . bbaZ0 ) .
1
Further topics on CFL
•
PDA and ContextFree Language There is a procedure to construct a PDA that accepts the language generated by a given contextfree grammar and conversely. bcbba . bbaZ0 ) ( q1 .This pushdown automaton accepts the language { wcwr  w { a . When c is detected. bba . a . which is the set of palindromes with c in the middle. Z 0 ) ( q2 . baZ0 ) ( q1 . Z 0 ) . ( q0 . abbcbba . b }* } . bbcbba . Z0 ) ( q0 . ba . That means that a language is
. it accepts the input string. ( q0 . baZ0 ) ( q1 .
This PDA pushes all the a's and b's in the input into stack until c is encountered.
However. The parsing for contextfree languages and regular languages have been extensively studied. Though we are accustomed to interpreting this as x + (yz) i. Thus if a computer is given the string x + yz. w. then multiply the result by z. Then there is a positive integer n such that for any string u in L with u n . a computer like nonEnglish speaking people must be told how to interpret sentences such as the first noun phrase (" A dog") is usually the subject of a sentence. native English speakers know that it is the dog that bites and not the other way round. Contectfree grammars are powerful grammars. By parsing sentences we identify the parts of the sentences and determine the strutures of the sentences so that their meanings can be understood correctly. x. "bites" is the verb and "a man" is the object of the verb. a verb phrase usually follow the noun phrase and the first word in the verb phrase is the verb and it is followed by noun phrases reprtesenting object(s) of the verb.e. we are not going to study parsing here. Those procedures are omitted here. it does not know which interpretation to use unless it is explicitly instructed to follow one or the other. ???? references on Parsing ????
. They can describe much of programming languages and basic structures of natural languages. compute yz first. Similar things happen when English sentences are processed by computers (or people as well for that matter). Interested readers are referred to the textbook and other sources. it could also be interpreted as ( x + y )z meaning that first compute x + y. However. vwmxymz L
for every integer m
•
Parsing and Parsers for CFL Consider the algebraic expression x + yz.
•
Pumping Lemma for ContextFree Language Let L be a CFL.e. productions of grammars.contextfree if and only if there is a PDA that accepts it. Parsing is the process of interpreting given input strings according to predetermined rules i. "A dog" is the subject. there are strings v. then add the result to x. For example in the sentence "A man bites a dog". Thus they are widely used for compilers for high level programming languages and natural language processing systems. y and z which satisfy
u = vwxyz wy > 0 wxy n 0 .
however.Test Your Understanding of ContectFree Language
Indicate which of the following statements are correct and which are not. Click True or Fals . These languages can describe many practically important systems and so they are heavily used in practice. the phrase structure languages (also called Type 0 languages). and the machines that can process them: Turing machines. then Submit. Turing machines were conceived of by the English mathematician Alan
. They are.Turing Machines Back to Schedule Back to Table of Contents
Turing Machines
Turing Machines
Subjects to be Learned
• • •
Definition of Turing Machine Configuration Operation of Turing Machine
Contents
Introduction We have studied two types of languages from the Chomsky hierarchy: regular languages and contextfree languages. In this chapter we are going to study the most general of the languages in Chomsky hierarchy. of limited capability and there are many languages that they can not process.
Next .
right or stay at the same square after a read or write. it stops its operation. At any state it reads the symbol under the head. unlike finite automata. Later Alonzo Church conjectured that any computation done by humans or computers can be carried out by some Turing machine.Turing as a model of human "computation". a Turing machine starts at the initial state. either erases it or replaces it with a symbol (possibly the same symbol). One of its states is the halt state and when the Turing machine goes into the halt state. its head is a readwrite head and it can move left. It is also divided into squares and a symbol can be written in each square.
Given a string of symbols on the tape. consists of a finite control and a tape. like finite automata. We are going to study Turing machines here and through that limitations of computers and computation as we know today. However.
. At any time it is in one of the finite number of states. Definition Conceptually a Turing machine. It then moves the head to left or right or does not move it and goes to the next state which may be the same as the current state. The tape has the left end but it extends infinitely to the right. Computers we use today are as powerful as Turing machines except that computers have finite memory while Turing machines have infinite memory. This conjecture is known as Church's thesis and today it is generally accepted as true.
= { a . where D represents R. S}.
State (q) Input (X) Move ( q0 q1 q2 q3 q3 a b a
(q. respectively. which is assumed not to contain the symbol h. Example 1 : The following Turing machine < Q1 . X ) = ( r. A transition diagram can also be drawn for a Turing machine.S)
A transition diagram of this Turing machine is given below. > accepts the language aba* . b . > .
. Here denotes the blank and R. The states are represented by vertices and for a transition ( q. q0 . where Q is a finite set of states. Y.Formally a Turing machine is a 5tuple T = < Q. X) )
( q1 . . . a . an arc from q to r is drawn with label ( X/Y . q0 is the initial state. the symbol X currently being read is changed to Y and the tape head is moved as directed by D. where Q1 = { q0. L or S . The symbol h is used to denote the halt state. R ) ( q3 . q3 } .L. is a finite set of symbols and it is the input alphabet. D ) . . = { a . left and do not move it. b } and is as given by the table below. q2. R ) ( q2 . . a . It is assumed that the tape has at the left end and the head is initially at the left end of the tape. is a finite set of symbols containing as its subset and it is the set of tape symbols. It is a mapping from Q ( { } ) to ( Q { h } ) ( { }) {R. R ) ( q3 . . D ) indicating that the state is changed from q to r. q1. L and S denote move the head right. q0. R ) (h. is the transition function but its value may not be defined for certain points. b } . .
zbw ) if the Turing machine goes from the first configuration to the second in one move. aababb ) shows that the Turing machine is currently in state q.Turing Machine that accepts aba*
To describe the operation of Turing machine we use configuration. ( q0 . xay ) ( q . For example ( q . For example the Turing machine of Example 1 above goes through the following sequence of configurations to accept the string aba: ( q0 . yaz ) for some symbol a { } and some strings y and z in ( * { } ) . A configuration for a Turing machine is an ordered pair of the current state and the tape contents with the symbol currently under the head marked with underscore. aba ) ( q2 . xay ) * ( q . If the Turing machine needs to be explicitly indicated T or T* is used. aba ) (h. In this case we also say that the Turing machine halts on input x. and ( p . A Turing machine T is said to decide a language L if and only if T writes "yes" and halts if a string is in L and T writes "no" and halts if a string is not in L. . > if x ) * ( h. zbw ) if the Turing machine goes from the first configuration to the second in zero or more moves. We write ( p . the taper contents are the string aababb and the head is reading the last a of the string. aba ) ( q3 . The set of strings accepted by a Turing machine is the language accepted by the Turing machine. Note that the Turing machine does not stop if a string is not in the language. A string x is said to be accepted by a Turing machine* T = < Q . aba )
. aba ) ( q1 . q0 . .
the second is a Turing machine that accepts { a } but goes into a loop if a string is not in the language (hence it accepts but doe not decide { a }) and the third decides { a }.
. where = { a }.The first of the following figures shows a Turing machine that accepts but does not decide the language { a }.
.
It is denoted by TR .
. It is assumed that initially the tape has at the left end.
to the right of
Example 3 : The following Turing machine erases the string on the tape and moves the head to the left end.Example 2 : The following Turing machine moves the head to the first the current position. This Turing machine is denoted by TE.
Strings not Accepted by Turing Machines When a string is not accepted by a Turing machine. (2) no transition is specified for the current configuration and (3) the head is at the left end and it is instructed to move left. In cases (2) and (3). the operation of the Turing machine is aborted. one of the following three things happens: (1) The Turing machine goes into an infinite loop. For example the following Turing machine accepts the language a+.
. that is when a Turing machine does not halt on a string. but it goes into an infinite loop for any strings that are not in the language.
Turing machine accepting a+
Computabler Function Let S
*
and let f be a function f : S > S. A Turing machine thus may accept a string and halt. or loop. a string is accepted by a Turing machine if given the string. there are two halt states: "accept halt" and "reject halt". the Turing machine eventually goes into the accept halt state. A language is a phrase structure (type 0) langauage if and only if it is Turingacceptable in either sense and it has no effects on decidablility.
* Note on "Turingacceptable": Some books define "acceptance by Turing machine" slightly differently. x)
*
( h. in the Turing machines those books define.
. reject a string and halt. Then we say T computes f or f is
computable if for every x ( q0 . That is. T does not halt on x. With this definition. As far as the material discussed in this class note. f(x) )
*
. there is no difference between these two definitions of "accept".
*
and for every x
that is not in S.
Furthermore according to the Church's thesis. Let us start with some basic Turing machines.Test Your Understanding of Turing Machines
Indicate which of the following statements are correct and which are not. It moves the head to the first symbol (which may be )
.Combination of Turing Machines Back to Study Schedule Back to Table of Contents
Turing Machines
Combination of Turing Machines
Subjects to be Learned
•
Combination of Turing Machines
Contents
Combination of Turing Machines We have seen the definition of Turing machine and a few simple examples. There are two sets of questions. One can construct many more Turing machines that perform various functions. Click True or Fals . then Submit. In fact Turing machines that simulate computers and Turing machines that perform computations done by any algorithm can be constructed. Here we are going to study how complex Turing machines can be constructed using simple Turing machines and how computers can be simulated by Turing machines. any "computation" done by human beings or machines can be done by some Turing machine. We have already seen TR .
Next .
T1T2 and T1 > T2 denote the Turing machine that behaves initially like T1 and when T1 halts T2 takes over inheriting the head position and the tape contents of T1 . let us construct a little more complex Turing machines. T1 > T2 denote the Turing machine that first executes T1. Also by TR and TL we denote Turing machines that move the head to right and left one position. Example 4: The following machine shifts the tape contents to the left one position. To combine Turing machines we use the following conventions: Let T1 and T2 represent arbitrary Turing machines. then T2 is started as in the case of T1T2 .
. The halt state of T1 becomes the initial state of T2 . takes the head to the right end of the string and halts. Using these basic machines and the convention. Similarly by TL we denote a Turing machine that moves the head to the first symbol (which may be ) to the left of the current position. Otherwise it crashes. Below is assumed to be at the left end of the tape initially.to the right of the current position. Then by T we denote a Turing machine that writes symbol at the current position and does not move the head (stays). Then if T1 halts and if the symbol currently under the head is . respectively.
k consecutive I's are put on the tape. it goes through the following
> aab > a > a > abb > ab > ab > ab Example 5: The leftshift machine of Example 4 can be used to construct an adder for natural numbers. An adder can be constructed for example as TR > TSL TL . Example 6: The following Turing machine copies the tape contents at the left end to their right separated by a blank . that is ( q0 . First.For example with the initial tape contents of sequence of tape contents and ends with ab ab > ab :
ab
. m I's and n I's with a blank between them are placed on the tape.
. So the initial configuration for adding 2 and 3 is ( q0 . In general to represent a natural number k. For example the number 3 is represented by three consecutive I's on the tape and 5 by five I's. After the addition the configuration becomes ( h .
x
x). II III ) . x)
*
b b
(h. IIIII ) . After adding two numbers placed on the tape it moves the head to the left end and halts. natural numbers are represented on a Turing machine using symbol I. To add two numbers m and n.
the branch operation is already in Turing machines because next configurations are determined based on the current state and tape symbol being looked at. In fact many of the earlier computers had a much smaller instruction set but still could do everything today's computers can do albeit much more slowly. there is a Turing machine that performs addition. Click True or Fals . store and load operations. one can say that computers can be simulated by Turing machines. A bare minimum instruction set would contain addition. then Submit. However. All the other operations can be realized by using those basic operations. The following notations are used in the questions:
. Since any complex computer instructions can be realized using those basic instructions. all of those instructions can be realized using combinations of a small number of basic instructions. On the other hand as we have seen above. branching.Today's computers are very complex machines and their instruction sets contain complicated operations. Furthermore if the subtraction operation is necessary. Thus by combining appropriate Turing machines a computer with a minimal instruction set can be constructed. and store and load operations can be taken care of by a Turing machine that copies tape contents. it is not difficult to construct a Turing machine that performs subtraction using the same representation of numbers as for the addition.
Test Your Understanding of Combination of Turing Machines
Indicate which of the following statements are correct and which are not.
how fast they can compute. It is divided into rows of small squares. may vary. ones having one tape but with multiple heads. that is. That is. For any Turing machine of this type there is a Turing machine with a one dimensional tape that is equally powerful.
Turing Machines with Two Dimensional Tapes
This is a kind of Turing machines that have one finite control. ones with two dimensional tapes.Types of Turing Machines Back to Study Schedule Back to Table of Contents
Turing Machines
Types of Turing Machines
Subjects to be Learned
•
Variation of Turing Machine
Contents
There are a number of other types of Turing machines in addition to the one we have seen such as Turing machines with multiple tapes. that is. To simulate a two dimensional tape with a one dimensional tape. nondeterministic Turing machines etc. the former can be simulated by the latter. The tape has the top end and the left end but extends indefinitely to the right and down.T_a : Ta T_R : TR >^b : >b
Next . the efficiency of computation. what one type can compute any other can also compute. However. one readwrite head and one two dimensional tape. It turns out that computationally all these Turing machines are equally powerful. first we map the squares of the two dimensional tape to those of the one dimensional tape diagonally as shown in
.
the following tables: Two Dimensional Tape v v h 1 h 3 h 4 h 10 h 11 h 21 .. . That is the head position of the one dimensional tape corresponding to 13 on the two dimensional tape.
... . .... . Thus this time the third square is the head position of the one dimensional tape corresponding to 9 on the two dimensional tape... then from h move the head of the one dimensional tape further right to the kth square from h. ... That is the square corresponding to the square below i in the two dimensional tape. v 6 8 13 19 23 ... One Dimensional Tape
v 1 v 2 3 h 4 5 6 v 7 8 9 10 h 11 . . . down. . . then move the head of the one dimensional tape to right until it hits h or v counting the number of squares it has visited after i.. left or right. . then on the one dimensional tape the head moves to the right and it hits v first... If h was hit first. v 16 26 . which is the third square from 8.. For example... respectively.. . . .... .. .. which is the second square from i = 5. v 2 5 9 12 20 22 .. . If the head moves down from i.. . .. move 3 positions to the right.. then for the one dimensional tape. . . .. . Let us simulate this head move with a one dimensional tape. Then it meets h first.. Let k be the number of squares visited by the head of the one dimensional tape.
Here the numbers indicate the correspondence of squares in the two tapes: square i of the two dimensional tape is mapped to square i of the one dimensional tape.
The head of a two dimensional tape moves one square up.. the head moves from position 8 to right.... Thus from h. If v was hit first..> ..... . suppose that the head position is at 8 for the two dimensional tape in the above table... .... Let i be the head position of the two dimensional tape.. .. v 15 17 25 .< . then (k+1)th square to the right from v is the new head position.... . .. . v 7 14 18 24 . that is i = 8..... ... If i = 5 and the head moves down on the other hand. . h and v are symbols which are not in the tape alphabet and they are used to mark the left and the top end of the tape...... If the head moves down to position 13. ..... .
.
Turing Machines with Multiple Tapes :
This is a kind of Turing machines that have one finite control and more than one tapes each with its own readwrite head. It is denoted by a 5tuple < Q . it can be said that they are equally powerful.. right or left on the two dimensional tape. . Its transition function is a partial function :Q ( { } )n > ( Q { h } ) ( { } )n { R . It is denoted by a 5tuple < Q . q0.Similarly formulas can be found for the head position on the one dimensional tape corresponding to move up. Details are omitted.L. H2 . . H2 . Hn denote the tape heads. Since the converses are obviously true. Thus some Turing machines with a one dimensional tape can simulate every move of a Turing machine with one two dimensional tape.
Turing Machines with Multiple Heads :
This is a kind of Turing machines that have one finite control and one tape but more than one readwrite heads. . It can be proven that any language accepted by an ntape Turing machine can be accepted by a one tape Turing machine and that any function computed by an ntape Turing machine can be computed by a one tape Turing machine. Since Turing machines with a two dimensional tape obviously can simulate Turing machines with a one dimensional tape. where H1 .
Nondeterministic Turing Machines
.S
It can be easily seen that this type of Turing machines are as powerful as one tape Turing machines. one can say that one tape Turing machines are as powerful as ntape Turing machines. > . . L . . S } n . In each state only one of the heads is allowed to read and write.. >. {h}) ( { } {R. Hence they are at least as powerful as Turing machines with a two dimensional tape. It turns out that this type of Turing machines are only as powerful as one tape Turing machines whose tape has a left end.. q0. Hn } ( { } ) > ( Q }.
Turing Machines with Infinite Tape :
This is a kind of Turing machines that have one finite control and one tape which extends infinitely in both directions. A configuration for this kind of Turing machine must show the current state the machine is in and the state of each tape. The transition function is a partial function : Q { H1 . .
Here an action means the combination of writing a symbol on the tape. Given a string x . They form level 1. All possible configurations that are reachable by applying the transition function of TN once form the children of the initial configuration. Proof : Let TN denote a nondeterministic Turing machine. Given a string x. tape symbol and head movement out of the set of triples without following any specific predetermined rule. For example consider the following nondeterministic Turing machine that accepts a+ . . In general for each vertex of level i all possible configurations that are reachable by applying the transition function of TN are its children. a nondeterministic Turing machine that accepts this language L would first guess the midpoint of x. Theorem Any language accepted by a nondeterministic Turing machine is also accepted by some deterministic Turing machine.. Formally a nondeterministic Turing machine is a Turing machine whose transition function takes values that are subsets of ( Q {h}) ( { } {R.A nondeterministic Turing machine is a Turing machine which.
As in the case of NFA. can take any action selecting from a set of specified actions rather than taking one definite predetermined action. it is understood that a nondeterministic Turing machine at any configuration selects one combination of next state. . can not guess the midpoint of the string x. on the other hand. A deterministic Turing machine. goes into an infinite loop or aborts. b }* } . 2. At any point in the process TN is in some configuration and has a finite set of configurations to choose from for its next configuration. like nondeterministic finite automata. It can be shown that a nondeterministic Turing machine is only as powerful as a deterministic Turing machine. TN starts at the initial configuration and goes through a sequence of configurations until it reaches a halt configuration . at any state it is in and for the tape symbol it is reading. that is the place where the second half of x starts. It must find the midpoint by for example pairing off symbols from either end of x.
.S}. The children of all the vertices of level i form level i+1. Note that the number of children for a vertex in this tree is finite because the number of states is finite and there are a finite number of tape symbols. moving the tape head and going to a next state. The root of the tree is the initial configuration and it is the only vertex of level 0. For example let us consider the language L = { ww : w { a .L. The set of all possible computations that TN can perform for a given string x can be represented by a rooted tree as follows.. Then it would compare the first half of x with the second half by comparing the ith symbol of the first half with the ith symbol of the second half for i = 1. Even in the same situation it may take different actions at different times.
it would proceed as follows to accept it: ( q0 . aa ) ( q1 . At the second and third configurations in the above sequence. aa ) for the third. and ( q1 . The tree for this case would be as follows:
. aa )
and ( q2 . aa ) ( h .Turing machine accepting a+
Given the string aa. aa ) and ( q2 . it has two candidates for the next configuration: ( q1 . aa ) for the second. aa ) ( q1 . aa ) . aa ) ( q2 .
Test Your Understanding of Different Types of Turing Machines
Indicate which of the following statements are correct and which are not. with a deterministic one. call it T2. call it T1. then T2 accepts the string and halts.One way to simulate a nondeterministic Turing machine. However.
Next . T2 applies the transition function of T1 to each configuration at that level and computes its children. though a deterministic Turing machine might take much more time than a nondeterministic Turing machine to accept a string. then Submit. Click True or Fals . is to traverse this tree breadthfirst way from the root until the halt state is reached. Many other variations of Turing machine are possible.Unsolvable Problems Back to Study Schedule Back to Table of Contents
Turing Machines
Unsolvable Problems
. In fact the Church's thesis conjectures that any so called computation done by humans or computers can be performed by a basic deterministic Turing machine. These children are the configurations of the next level and they are stored on the tape (if necessary a second tape may be used). It can be easily seen that T2 accepts a string if and only if T1 accepts it. Thus any language accepted by a nondeterministic Turing machine is also accepted by a deterministic Turing machine. If there is the halting state among these children. At each level of the tree. it has been shown that none of them exceed the capability of basic deterministic Turing machine as far as accepting languages is concerned.
the question is unanswerable that is the problem is unsolvable. there is no way of telling whether it is in an infinite loop or along the way to a solution and it needs more time. Thus if a language is not decidable. and an arbitrary string w over . that is computationally they are equally powerful. T writes "yes" if M halts on w and "no" if M does not halt on w. L. So we are going to look at the unsolvability in terms of language recognition. Suppose that the halting problem is decidable. Suppose that a language is acceptable but not decidable. does M halt when it is given w as an input ? It can be shown that the halting problem is not decidable. Halting Problem One of well known unsolvable problems is the halting problem. It asks the following question: Given an arbitrary Turing machine M over alphabet = { a . the question of whether or not a string is in the language may not be answered in any finite amount of time. if the Turing machine is running. It is also conjectured that any "computation" human beings perform can be done by Turing machines (Church's thesis). Theorem 1 : The halting problem is undecidable. and then T halts. First recall that solving a problem can be viewed as recognizing a language (see Problem Solving as Language Recognition). hence unsolvable. In this chapter we are going to learn that there are problems that can not be solved by Turing machines hence by computers. Then given a string a Turing machine that accept the language starts the computation. Since we can not wait forever for an answer. Minsky): This is going to be proven by "proof by contradiction".Subjects to be Learned
• • •
Halting Problem Languages not Accepted by Turing Machines Other Unsolvable Problems
Contents
We have learned that deterministic Turing machines are capable of doing any computation that computers can do. Proof (by M. At any point in time. and that any of their variations do not exceed the computational power of deterministic Turing machines. Here "unsolvability" is in the following sense.
. given a description of a Turing machine M (over the alphabet ) and a string w. b } . Then there is a Turing machine T that solves the halting problem. That is. Below we are going to see some well known unsolvable problems and see why we can say they are unsolvable.
then Tm goes into an infinite loop (Tm halts if the original T rejects a string and halts). denoted by d(M).
Next using Tm we are going to construct another Turing machine Tc as follows: Tc takes as input a description of a Turing machine M. First we construct a Turing machine Tm by modifying T so that if T accepts a string and halts.We are now going to construct the following new Turing machine Tc. where * is a symbol that separates the two copies of d(M) and then supplies d(M)*d(M) to the Turing machine Tm . copies it to obtain the string d(M)*d(M).
.
it makes a copy. Thus one implication of the halting problem is that there can be no computer programs (Turing machines) that check whether or not any arbitrary computer program stops for a given input.
. Hence that assumption must be wrong. constructs the string d(Tc)*d(Tc) and gives it to the modified T. Thus the question of whether or not a program halts for a given input is nothing but the halting problem. This contradiction has been deduced from our assumption that there is a Turing machine that solves the halting problem. This is a contradiction. Thus Tc goes into an infinite loop if Tc halts on d(Tc) and it halts if Tc does not halt on d(Tc).
The way T was modified the modified T is going to go into an infinite loop if Tc halts on d(Tc) and halts if Tc does not halt on d(Tc).Let us now see what Tc does when a string describing Tc itself is given to it. Thus the modified T is given a description of Turing machine Tc and the string d(Tc). When Tc gets the input d(Tc) . Program correctness and Halting Problem Note that for any computer program a Turing machine can be constructed that performs the task of the program. Hence there is no Turing machine that solves the halting problem.
then Submit.Test Your Understanding of Unsolvable Problems
Indicate which of the following statements are correct and which are not. Click True or Fals .More Unsolvable Preoblems Back to Study Schedule Back to Table of Contents
Turing Machines
More Unsolvable Problems
Subjects to be Learned
• •
Languages not Accepted by Turing Machines Other Unsolvable Problems
Contents
The next unsolvable problem is in a sense more difficult than the halting problem. Language NonSelfAccepting Let us first define two languages NSA1 and NSA2 as follows:
. It is presented as a language and it can be shown that there are no Turing machines that accept the language.
Next .
Then this w is a description of a Turing machine but it must be longer than one symbol. b }*. NSA2 is the set of strings that do not describe any Turing machine. w = d(T) for a Turing machine T and T does not accept
NSA2 = { w  w { a. NSA1 is the set of strings that describe a Turing machine but that are not accepted by the Turing machine they describe. Problem Accepts( ) The problem Accepts(
) asks whetehr or not a given Turing machine T accepts
. This is a contradiction. let w = a. that accepts NonSelfAccepting. Certainly more symbols than a single a are needed to describe even the simplest Turing machine. Thus there can not be Turing machine T0 that accepts the language SelfAccepting . w d(T) for any Turing machine T } . We are going to see that T0 neither accepts w0 nor rejects it. Then there is no Turing machine that can be described by the string a. Neither NSA1 nor NSA2 is empty. It
. For NSA2. This means that there can not be any Turing machine that accepts the language NonSelfAccepting. Hence by the definition of NSA1 . which is absurd. that is w0 is a description of the Turing machine T0 . Hence w0 is not in NonSelfAccepting . Also T0 does not accept w0 . For let T be a Turing machine that accepts { a } and let w = d(T). then w0 NonSelfAccepting because T0 accepts NonSelfAccepting. (1) If T0 accepts w0. w0 is in NSA1 . then w0 is not in NonSelfAccepting because T0 accepts NonSelfAccepting. b }*. This is again a contradiction. Suppose there is a Turing machine.NSA1 = { w  w w}
{ a. Hence either T0 accepts w0 or rejects it. (2) If T0 does not accept w0 . Hence it is in SelfAccepting . Hence T0 can not accept w0 . However. by the definitions of NSA1 and NSA2. Proof: This is going to be proven by contradiction. Hence it is not accepted by T. where d(T) is a description of the Turing machine T. w0 is in neither NSA1 nor NSA2 . But w0 = d( T0 ) because that is how we selected w0 . either w0 is in NonSelfAccepting or it isn't. Hence w is in NSA1 .
Theorem 2 There are no Turing machines that accept the language NonSelfAccepting. Knowing the unsolvability of the halting problem some other problems can be shown to be unsolvable. call it T0. Let us define the language NonSelfAccepting as NonSelfAccepting = NSA1 Then we can prove the following theorem: NSA2 . Hence a is in NSA2 . Let w0 = d( T0 ). Since NonSelfAccepting is a language. Thus neither NSA1 nor NSA2 is empty.
a Turing machine. this means that Accepts( ) is unsolvable. Consider a Turing machine T = TwT'. call it M. Let T' be an instance of Accepts( ). M writes the string d( T ) on the tape and let A take over.
Then M halts on d(T') and w if and only if T' halts on w. which is an instance of the halting problem. Thus if Accepts( ) is solvable. Then there is a Turing machine that solves it. Then consider the Turing machine T = TeraseT' . that solves Accepts( ) can be constructed as
. That is. where Terase is a Turing machine that erases the input on the tape and halts. the halting problem can be solved. This T halts on if and only if T' halts on w. Since the halting problem is unsolvable. Let A be a Turing machine that solves Accepts( ). a Turing machine. call it M. Let a Turing machine T' and a string w be an instance of the halting problem. where machine Tw is a Turing machine that writes w. Using this T. Using this T. Suppose that AcceptsEverything is solvable. Suppose that Accepts( ) is solvable. This T halts on every string over if and only if T1 halts on . Problem AcceptsEverything The problem AcceptsEverything asks whether or not a given Turing machine T halts on every string over a given alphabet . that solves the halting problem can be constructed as follows: Given a description d(T') of a Turing machine T' and a string w as inputs. Using a similar idea the following problem can also be shown to be unsolvable. Let A be a Turing machine that solves AcceptsEverything.can be shown to be unsolvable. We are going to show that Accepts( ) can be solved using the solution to it. M solves the halting problem. We are going to show that the halting problem becomes solvable using this A.
Other Unsolvable Problems Let G1 and G2 be contextfree grammars and let L(G) denote the language generated by grammar G. It can be shown to be unsolvable using AcceptsEverything. It can be shown to be unsolvable using Accepts( ) .
Since Accepts(
) is unsolvable.
By similar arguments the following problems can be shown to be unsolvable. Equivalence This problem asks whether or not two Turing machines accept the same language. AcceptsNothing This problem asks whether or not a Turing machine accepts nothing. it means that AcceptsEverything is unsolvable. Is L( G1 ) L( G2 ) ?
Is L( G1 ) L( G2 ) = ? finite ? infinite ? contextfree ? Is L( G1 ) = L( G2 ) ? Is L( G1 ) = * ? Is the complement of L( G1 ) contextfree ?
. Then the following problems are all unsolvable.shown below.
Next . Click True or Fals .Time Complexity of Problem Back to Study Schedule Back to Table of Contents
Complexity
Time Complexity
Subjects to be Learned
• • • • •
Time Complexity of Problems Decision Tree Class NP Polynomial Time Transformation NPComplete Problems
Contents
In the previous sections we have learned that some problems are unsolvable by Turing machines hence by computers. No one can write computer programs that solve those problems and halt after a finite amount of time. then Submit. A problem is solvable if some Turing
.Test Your Understanding of Unsolvable Problems
Indicate which of the following statements are correct and which are not.
One can ask the same question for formulas of first order predicate logic. They take much more time to execute than polynomial time algorithms.machine can solve it in finite time. However. In this section we are going to study solvable problems and learn a hierarchy of solvable problems based on the computation time required to solve them. are propositions of PDL. where A is an algorithm and S is a statement. For example a binary search takes O(lg n) time. then Y Z [ X = Y + Z ] ] is a proposition of Presburger arithmetic. variables taking positive integers as their values. as well as all the connectives such as and. where k is a natural number. it is as good (or bad) as unsolvable. a quick sort needs O(n2) time. X [ if ~( X = 1 ). etc. that is it requires at least O( aan ) time to solve in the worst case. S). problems that need kexponential time algorithms. The problems that can not be solved with any polynomial time algorithm are called intractable problems . For more detailed review of this. Among the solvable problems there are problems that can be solved by algorithms with the worst case time which is a polynomial in the problem size (polynomial time algorithms). They are all polynomial time algorithms. Q )" . But (P ^ ~P ) is always false. Then there are problems that require double exponential ( e. or etc. where P is a propositional variable. S) says that S is true after executing A. Presburger arithmetic is a logic that allows statements involving positive integers. The satisfiability problem for PDL is known to take at least exponential time to solve in the worst case. it is still solvable. Before proceeding to predicate logic let us consider the following logic called propositional dynamic logic (PDL for short). and there are problems that require algorithms with the worst case time worse than kexponential time for any natural number k. second order logic. It is estimated by counting the largest possible number of key operations to be performed in terms of the input size. the satisfiability problem for the propositional dynamic logic is proven to take exponential time to solve in the worst case. So it is certainly satisfiable. O(f(x)) (bigoh) and other related subjects click here. So it is not satisfiable. For example.g. This is a propositional logic with an extra construct (proposition) after(A. the equality symbol = and quantifiers and . For example the formula ( P V ~P ) is always true. They are both satisfiable. in practice if it takes a million years. Let us see some of those intractable problems. as we are going to see below. For many problems a day or even an hour would be too long. etc. ~Q )" and "if P then after( if P then Q else ~Q. 22n ) time algorithms. For example "after( if P then Q else ~Q. a heap sort needs O(n lg n) time. For example.
. Even if it takes a million years to solve a problem. the addition operation +. For example the satisfiability problem for Presburger arithmetic is doubleexponential (2fold exponentail). where P and Q are propositions. There are also problems that must be solved at best by exponential time algorithms in the worst case. This is the problem of asking whether or not a given formula can take the value true for some values of its variables. The satisfiability problem becomes even harder when logic becomes more complex. The measure for computation time we use is the worst case time. after(A. Similarly ( P V Q ) is also satisfiable. In logic there is a well known problem of "satisfiability".
In Presburger arithmetic (minus addition operation). These problems are called decision problems. Let us here review nondeterministic Turing machines. This problem is called "Graph Coloring" problem or more precisely "Vertex Color" problem. Now let us go back to the satisfiability problem of propositional logic. in addition. there are no Kfold exponential time algorithms to solve it for any number K. sets of integers and the predicate "belongs to" (an element X belongs to a set S) are allowed. Below we are going to characterize this class of problems. For the satisfiability problem of WS1S. is it possible to color its vertices with three or less colors ?
. For the problems of this class there are no known polynomial time algorithms for solving them nor are they known to be unsolvable with polynomial time algorithms. Consider the problem of coloring vertices of a graph with a given number of colors or less so that no two vertices connected directly by an edge have the same color assigned. " Is it possible to assign colors to vertices of a given graph using a given number of colors or less so that no two vertices connected directly by an edge have the same color assigned ? " etc. For example. if. Some of these decision problems are NPcomplete problems. Let us try to solve the following instances of this graph coloring problem: Given the following graph. "Is it possible to schedule committee meetings without conflicts into a given number of time slots ? " . First. the consensus is that they ca not be solved with polynomial time algorithms. "Is a string w in the language a*b ? ". there are problems that are solved by answering with yes or no. however. the logic is called WS1S (Weak Secondorder theory of 1 Successor). At the moment. Such a problem (having no Kfold exponential time algorithms) is called nonelementary. This problem belongs to a peculiar class of problems called NPComplete problems.
For the graphs of (a) and (b), you could find a solution very easily by inspection. You could see a right coloring as soon as you saw the graphs. However, you can most likely not tell how you arrived at your solutions. You probably don't have any algorithms you could use to solve them. You could somehow see the solutions. This is basically the idea of nondeterministic (Turing) machine. There is no fixed procedure which you can use repeatedly to solve instance after instance of this problem. But you can somehow solve them. Let us move on to a slightly more complex example of (c). For this graph to find a right coloring you could start with vertex 1 and assign color a. Then move on to vertex 2 and assign color b(it has to be something other than a ). Then go to vertex 3 and assign a third color, say c. Then at vertex 4 select color b and for vertex 5 use color a. In this process we make a decision as to what color to use for each vertex and when a decision is made for all the vertices we have a solution to the problem. This process applies to any decision problem. That is to solve a decision problem a number of smaller decisions are made one after another and as a result a solution to the problem is obtained. This process can be represented by a tree called decision tree. For example, for the graph coloring problem let us first decide on the order of vertices we color in, say 1, 2, 3, 4 and 5 for the graph of (c) above. Then the root of its decision tree corresponds to the vertex we assign a color to first (vertex 1 in this example). Then for each possible color for the first vertex, a child is created for the first vertex of the tree. So the second level of the
decision tree corresponds to the second vertex to be colored. Then in general, for each possible color for each vertex of level i of the decision tree, a child is created. Those children form level i+1 of the decision tree. The decision tree for the graph of (c) is given below. Since any color can be assigned to vertex 1 without loss of generality, it has just one child in the actual decision tree. Also since in this case the ith and (i+1)th vertices are connected by an edge for i = 1, 2, 3, 4, they can not have the same color. So each vertex after vertex 1 has two colors to choose from. So they each have two children in the decision tree.
Thus during the process of solving the problem a decision is made at each level and when all levels are covered, the problem is solved. A path from the root to a leaf corresponds to a coloring of the vertices of the given graph. A decision tree, however, does not tell us how to make decisions. Also a decision tree does not tell how to order the vertices for coloring, that is which vertex to color first, second etc. A deterministic machine (or algorithm) has a specific fixed set of rules for making a decision at each level of the decision tree. Although it knows what to do at every stage of problem solving, the decisions it makes are not necessarily the right ones. When it makes wrong decisions, it must retract earlier decisions and try different paths, which is called backtracking. For the graph coloring problem a deterministic algorithm might first order the vertices of the graph in decreasing order of their degree and also order colors. Then, following the
order of the vertices, assign to each vertex the highest order color available for the vertex. Since that kind of algorithm is not guaranteed to use the minimum number of colors, it may produce a wrong answer unless there is some provision for backtracking. A nondeterministic (Turing) machine, on the other hand, is a fictitious machine and somehow knows which branch (child) to select at each step. It always makes a right selection. A decision problem is said to belong to class NP if each vertex in its decision tree has a finite number of children and if it can be solved by a nondeterministic (Turing) machine in polynomial time. The graph coloring problem is in class NP, so are the satisfiability problem for propositional logic and most of the scheduling problems just to name a few. Also there are other characterizations of class NP. Interested readers click here. At this moment it is not known whether or not problems in class NP can be solved with a polynomial time algorithm in the worst case. The consensus is that there is no polynomial time algorithm to solve them. It would take at least exponential time. Among the problems in class NP, there are problems which all problems of class NP can be transformed to in polynomial time. Those problems are called NPcomplete problems. If a polynomial time algorithm is found for any one of the NPcomplete problems, all the problems in NP can be solved in polynomial time. Below we are going to study NPcomplete problems. We start our discussion with the concept of polynomial time transformation (reduction). Basically we say a decision problem Q1 is polynomially reducible to a decision problem Q2 if and only if there is a transformation that transforms any arbitrary instance of Q1 into an instance of Q2 in polynomial time such that the answer to Q1 is yes if and only if the answer to Q2 is yes. A little more formally we define this in terms of languages. Note that a decision problem can be viewed as a language of its instances and that solving it can be considered as recognizing the language as we have seen earlier. Let L1 and L2 be languages over alphabets 1 and 2, respectively. We say that L1 is polynomialtime reducible to L2 if and only if there is a function f from 1* to 2* such that for any string x in polynomial time.
1 *
,x
L1 if and only if f(x)
L2 and f can be computed
For example let us consider the following two problems: graph coloring and scheduling of committee meetings. The graph coloring problem is as given above. In the scheduling of committee meetings problem, committees with their members and a positive integer k are given. The problem is whether or not the meetings of the committees can be scheduled in k or less time slots so that everyone can attend one's meetings. Note that some people may be in more than one committee. Let us try to show that this scheduling problem is polynomial time reducible to the graph coloring problem.
Then the committees can be grouped into p groups so that the committees in the same group can meet at the same time. Thus the scheduling problem asks whether or not the meetings of the given committees can be scheduled in 3 time slots without any conflicts. d } and { a. { b.
Suppose that the meetings can be scheduled in p time slots. b }. Then since committees 1 and 2 share a. c }. Proceeding similarly the following graph is obtained corresponding to the committee memberships. add vertices 1. Then the meetings can be scheduled in k or less time slots if and only if the graph can be colored with k or less colors. connect with an edge the vertices corresponding to the committees. 3 and 4 to the graph. an edge is inserted between vertices 1 and 2. 3 and 4 with the memberships { a. respectively. where p k. The corresponding graph for the graph coloring problem can be constructed as follows: Corresponding to the committees 1. c. and 1 and 4. For example suppose that we are given the committees 1.What we need to do is given an instance of the scheduling problem construct an instance of the graph coloring problem. Similarly since committees 1 and 3. and if and only if two committee have some members in common. Corresponding to this grouping assign colors to the vertices of the graph so that the vertices in the same group are given the same color and those in
. d }. Let us consider the following transformation: For each committee add a vertex to the graph. 2. Suppose also that k = 3. that is construct a graph and give the number of colors to be used to color its vertices so that the meetings can be scheduled if and only if graph can be colored. 3 and 4. edges are added between 1 and 3. 2. 2. {a. and 1 and 4 share members.
their sizes and a number of bins of the same size. Some of them are listed below. Conversely if the graph can be colored with k or less colors. 6. Formally a problem is NPhard if every problem in class NP can be polynomial time reducible to it. Subsequently the satisfiability problem was found to be polynomial time reducible to many other problems. This group of problems are called NPcomplete problems. For if any two vertices are connected with an edge. that is the construction of graph for a given set of committees. fast algorithms exist to solve it. We are now ready to discuss NPcompleteness. can be done in time polynomial in the size of the problem. group them into two groups so that the sum of the numbers of one group is equal to that of the other group. then it can be easily seen that the committees can meet in k or less time slots.e. If a problem is NPcomplete. As a consequence if a polynomial time algorithm is found for any one of those problems. This coloring uses p colors which does not exceed k. 7. Partition Problem Given a set of integers. and vertices connected with an edge have different colors. It is also easily seen that the transformation. Traveling Salesman Problem Given cities and traveling times between cities. NPcomplete Problems 1. Thus these two vertices must get different colors. then P is also NPcomplete. Cook that the problems of class NP can be polynomial time reducible to the satisfiability problem of propositional logic. For all the problems in class NP can be reduced to P through the known NPcomplete problem in polynomial time. find out whether or not one is a subgraph of the other. Subgraph Isomorphism Problem Given two graphs.different groups are given different colors. 5.
. then that means that the corresponding committees share some members and that they are scheduled to meet in different time slots. 4. It was first proven by S. Graph Color Problem 3. then the consensus today is that it is most likely that no polynomial time algorithms i. Satisfiability Problem for Propositional Logic 2. Today hundreds of problems are known to be NPcomplete. which in this case can be taken as the number of committees. all the problems can be solved with polynomial time algorithms. a traveling salesman wants to know a shortest route to visit all cities exactly once and come back to where he/she started. Committee Meeting Schedule Problem In fact most scheduling problems are NPcomplete. A problem is NPcomplete if it is in class NP and NPhard. It can be easily seen that if a problem P at hand is NPhard and if a problem known to be NPcomplete can be polynomial time reducible to P. Bin Packing Problem Given a set of objects. find out whether or not the objects can be put into the bins.
a.a. their values and an integer V. their sizes. Note that {(1. a set of objects.(2.b.2}. B and C of the same size. and S = {(1. is it possible to select objects so that the sum of their sizes does not exceed S and the sum of their values is V or larger ? 10. (2. let A = {1.y)}. (2. a collection of subsets of S and an integer k.8. (2.x)} is not a matching. Then T = {(1. B. of S such that every element of A. Set Cover Problem Given a set S.x). and C appears exactly once in T ? For example.x)} is a desired set satisfying all the requirements.b}.y). find out whether or not there are k or less subsets in the collection whose union is S.b. Knapsack Problem Given a knapsack of size S.b.a.x). and a subset S of the Cartesian product A X B X C. B = {a. Is there a subset T.
.y). 3Dimensional Matching Given three sets A.x).b. 9. and C = {x.(1. called a matching.b.y}.