INFORMATION AND CONTROL 9.

, 1 3 7 - 1 6 7

(1959)

On Certain Formal Properties of Grammars*
NOAM CHOMSKY
Massachusetts Institute of Technology, Cambridge, Massachusetts and The Institute for Advanced Study, Princeton, New Jersey
A grammar can be regarded as a device that enumerates the sentences of a language. We study a sequence of restrictions that limit grammars first to Turing machines, then to two types of system from which a phrase structure description of the generated language can be drawn, and finally to finite state IV[arkov sources (finite automata). These restrictions are shown to be increasingly heavy in the sense that the languages that can be generated by grammars meeting a given restriction constitute a proper subset of those that can be generated by grammars meeting the preceding restriction. Various formulations of phrase structure description are considered, and the source of their excess generative power over finite state sources is investigated in greater detail. SECTION 1

A language is a collection of sentences of finite length all constructed from a finite alphabet (or, where our concern is limited to syntax, a finite vocabulary) of symbols. Since any language L in which we are likely to be interested is an infinite set, we can investigate the structure of L only through the study of the finite devices (grammars) which are capable of enumerating its sentences. A grammar of L can be regarded as a function whose range is exactly L. Such devices have been called "sentence-generating grammars. ''z A theory of language will contain, then, a specifica* This work was supported in part by the U. S. Army (Signal Corps), the U. S. Air Force (Office of Scientific Research, Air Research and Development Command), and the U. S. Navy (Office of Naval Research). This work was also supported in part by the Transformations Project on Information Retrieval of the University of Pennsylvania. I am indebted to George A. Miller for several important observations about the systems under consideration here, and to I~. B. Lees for material improvements in presentation. i Following a familiar technical use of the term "generate," cf. Post (1944). This locution has, however, been misleading, since it has erroneously been interpreted as indicating that such sentence-generating grammars consider language 137

138

CHOMSKY

tion of the class F of functions from which g r a m m a r s for particular languages m a y be drawn. T h e weakest condition t h a t can significantly be placed on g r a m m a r s is t h a t F be included in the class of general, unrestricted T u r i n g machines. T h e strongest, m o s t limiting condition t h a t has been suggested is t h a t each g r a m m a r be a finite M a r k o v i a n source (finite automaton).2 T h e latter condition is k n o w n t o be too strong; if F is limited in this w a y it will not contain a g r a m m a r for English ( C h o m s k y , 1956). T h e former condition, on the other hand, has no interest. We learn n o t h i n g a b o u t a natural language f r o m the fact t h a t its sentences can be effectively displayed, i.e., t h a t t h e y constitute a reeursively enumerable set. T h e reason for this is d e a r . Along with a specification of the class F of grammars, a t h e o r y of language m u s t also indicate how, in general, relev a n t structural information can be obtained for a particular sentence generated b y a particular g r a m m a r . T h a t is, the t h e o r y m u s t specify a class ~ of " s t r u c t u r a l descriptions" and a functional • such t h a t given f 6 F and x in the range of f, ~(f,x) 6 Z is a structural description of x (with respect to the g r a m m a r f ) giving certain information which will facilitate and serve as the basis for an a c c o u n t of how x is used and understood b y speakers of the language whose g r a m m a r is f; i.e., which will indicate whether x is ambiguous, to w h a t other sentences it is structurally similar, etc. These empirical conditions t h a t lead us to characterize F in one w a y or a n o t h e r are of critical importance. T h e y will not be further discussed in this paper, 3 b u t it is clear t h a t we will not be able to defrom the point of view of the speaker rather than the hearer. Actually, such grammars take a completely neutral point of view. Compare Chomsky (1957, p. 48). We can consider a grammar of L to be a function mapping the integers onto L, order of enumeration being immaterial (and easily specifiable, in many ways) to this purely syntactic study, though the question of the particular "inputs" required to produce a particular sentence may be of great interest for other investigations which can build on syntactic work of this more restricted kind. 2 Compare Definition 9, See. 5. Except briefly in §2. In Chomsky (1956, 1957), an appropriate ~ and ~ (i.e., an appropriate method for determining structural information in a uniform manner from the grammar) are described informally for several types of grammar, including those that will be studied here. It is, incidentally, important to recognize that a grammar of a language that succeeds in enumerating the sentences will (although it is far from easy to obtain even this result) nevertheless be of quite limited interest unless the underlying principles of construction are such as to provide a useful structural description.

ON CERTAIN FORMAL PROPERTIES OF GRAMMARS

139

velop an adequate formulation of ¢ and % if the elements of F are specified only as such "unstructured" devices as general Turing machines. Interest in structural properties of natural language thus serves as an empirical motivation for investigation of devices with more generative power than finite automata, and more special structure than Turing machines. This paper is concerned with the effects of a sequence of increasing heavy restrictions on the class F which limit it first to Turing machines and finally to finite automata and, in the intermediate stages, to devices which have linguistic significance in that generation of a sentence automatically provides a meaningful structural description. We shall find that these restrictions are increasingly heavy in the sense that each limits more severely the set of languages that can be generated. The intermediate systems are those that assign a phrase structure descritption to the resulting sentence. Given such a classification of special kinds of Turing machines, the main problem of immediate relevance to the theory of language is that of determining where in the hierarchy of devices the grammars of natural languages lie. It would, for example, be extremely interesting to know whether it is in principle possible to construct a phrase structure grammar for English (even though there is good motivation of other kinds for not doing so). Before we can hope to answer this, it will be necessary to discover the structural properties that characterize the languages that can be enumerated by grammars of these various types. If the classification of generating devices is reasonable (from the point of view of the empirical motivation), such purely mathematical investigation may provide deeper insight into the formal properties that distinguish natural languages, among all sets of finite strings in a finite alphabet. Questions of this nature appear to be quite difficult in the case of the special classes of Turing machines that have the required linguistic significance. 4 This paper is devoted to a preliminary study of the properties of such special devices, viewed as grammars. It should be mentioned that there appears to be good evidence that devices of the kinds studied here are not adequate for formulation of a full grammar for a natural language (see Chomsky, 1956, §4; 1957, Chapter 5). Left out of consideration here are what have elsewhere been
4 In C h o m s k y and Miller (1958), a s t r u c t u r a l characterization t h e o r e m is s t a t e d for languages t h a t can be e n u m e r a t e d by finite automata, in t e r m s of t h e cyclical s t r u c t u r e of these a u t o m a t a . The basic characterization theorem for

finite automata is proven in Kleene (1956).

SECTION 2 A phrase structure grammar consists of a finite set of "rewriting rules" of the form ~ --* ¢. for example. These are complex operations that convert sentences with a phrase structure description into other sentences with a phrase structure description. 1956. the condition that ~ differ from m by replacement of a single . negatively. #Cb#. Hence investigation of these devices is important as a preliminary to the far more difficult study of the generative power of transformational grammars (as well as. we generate a sentence by writing down the initial string #S#. #CB#. and hence providing the derivation D = (#S#. Chomsky. Some of the symbols of the grammar stand for words and morphemes (grammatically significant parts of words). and so on. applying another rule to form a new string #e2#. 1957. until we reach a string #~# which consists solely of terminal symbols and cannot be further rewritten. where e and ~b are strings of symbols. 1957). Consider. We can represent D diagrammatically in the form S /\ A C B b I I (1) If appropriate restrictions are placed on the form of the rules m --~ ~ (in particular. Nevertheless. #ab#).140 CHOMSKY called "grammatical transformations" (Harris. Given such a grammar. applying one of the rewriting rules to form a new string #~1# (that is. it appears that devices of the kind studied in the following pages must function as essential components in adequate grammars for natural languages. for the information it should provide about what it is in natural language that makes a transformational grammar necessary). a grammar containing the rules: S ~ AB." Other symbols stand for phrases. These constitute the "terminal vocabulary. b. CB ~ Cb. A --~ C. #AB#. standing for the "longest" phrase). C --> a. It contains a special "initial" symbol S (standing for "sentence") and a boundary symbol # indicating the beginning and end of sentences. 1952a. and constitute the "nonterminal vocabulary" (S is one of these. we might have applied the rule #S# --~ #el# or the rule S --~ ¢~). The sequence of strings constructed in this way will be called a "derivation" of #e~#.

¢.A¢ --+ ~co¢. which indicates that John is a Singular Noun and a Noun Phrase. Axiom 3. ~2. co such that ~ --+ ¢#co. . coJ) whose existence is guaranteed by Axiom 4 give a finite specification of the relation --~. cos) such that for all ~. immediate constituent analysis." Vr contains I and a " b o u n d a r y " element #. 1957) for further ]iseussion. C as Singular Noun. V~ contains an element S (sentence). --~ coi which completely determine all possible derivations. A x l o ~ 4. that comes is a Verb Phrase. ~ --~ ¢ if and only if there are ~1. where Vr is the "terminal vocabulary" and VN the "nonterminal vocabulary. A substring x of the terminal string of a g i v e n derivation will be called a phrase of type A just in case it can be traced back to a point labeled A in the associated tree (thus. A two-place relation -~ is defined on elements of G. ¢. Grammars containing rules formulated in such a way that trees can be associated with derivations will thus have a certain linguistic significance in that they provide a precise reconstruction of large parts of the traditional notion of "parsing" or." This relation satisfies the following conditions: Axiom 1. given a derivation. --* is irreflexive.. VN disjoint). read "can be rewritten as. a as John. and the method of constructing them.) The basic system of description that we shall consider is a system G of the following form: G is a semi-group under concatenation with strings in a finite set V of symbols as its elements. and that John comes is a Sentence. (Cf. If in the example given above we interpret A as Noun Phrase. and I as the identity element. and b as comes. AXIOM 2. the substring enclosed within the boundaries is a phrase of the type "sentence"). it will always be possible to associate with a derivation a labeled tree in the same way. co such that ~. ¢. In other words. A C VN if and only if there are ~. Chomsky (1956. V is called the "vocabulary" of G. we m a y think of the grammar as containing a finite number of rules x. V = Vr u VN(Vr. col). B as Verb Phrase. There is a finite set of pairs (Xi. "'" . The presentation will be greatly facilitated by the adoption of the following notational convention (which was in fact followed above). These trees can be taken as the structural descriptions discussed in Sec. we can regard D as a derivation of John comes providing the structural description (1). will (when stated precisely) be a definition of the functional ~. in its more modern version. and j _= n such that ~ = ~ixj~2 and ¢ = ~coj~2 • Thus the pairs (xJ. for example.ON C E R T A I N FORMAL P R O P E R T I E S OF GRAMMARS 14%1 symbol of ~ b y a non-null string). 1. There are no ~. (x~.

92. "'" . the grammar contains such rules as ab . and that each such rule may be applied independently of the context in which A appears. then there are A. ~b -. Restriction 1 requires that the rules of the grammar [i. RESTRICTION i.A--~. above. (which is the ordinary ancestral of --~) is thus a partial ordering of strings in G. late letters of all alphabets for arbitrary strings. 9J. then there are A.e. e.~b91~92.g. then there are A. The nature of these restrictions is clarified by comparison with Axiom 4.92.~ cd). This paper will be devoted to a study of the effect of imposing the following additional restrictions on grammars of the type described above. RESTRICTION 3. ~b = 91w92.~0 ~I. 6 Thus the terminal language LG consists only of those strings of V r which are derivable from #S# but which cannot head a derivation (of > 2 lines). and 9~-+9i+1(1 =< i < n). the minimal pairs (x~. Restriction 3 5 Note that a terminated derivation need not terminate in a string of Vr (i. and ~ ~ I. . but A -~ w.142 CHOMSKY CONVENTION 1: We shall use capital letters for strings in V~ . where A is a single symbol and w ~ I. in Chomsky (1956. 6 DEFINITION 4. 91.but¢o ~. and that a derivation ending with a string of VT need not be terminated (if. A 9-derivation is terminated if it is not a proper initial subsequence of any 9-derivation. The terminal language La generated b y G is the set of strings x such that there is a terminated #S#-derivation of x. RESTRICTION 2. 9n)(n > 1) is a ¢J-derivation of~ if ~b = ~i. = 9~. 92. DEFINITION 1. where A is a single symbol. it may be "blocked" at a nonterminal string). If 9 -~ #. early letters of all alphabets for single symbols (members of V). B such that 9.aBor~o = a. Such a rule asserts that A -~ in the context 91--~2 (which may be null). Greek letters for arbitrary strings.91~92. G is equivalent to G* if La = L a . in slightly different form.. small Latin letters for strings in Vr . w~) of Axiom 4] all of be the form 91A92 --+ 91~q~2. that is.e. ~ such that 9 = 91A92. 9 ~ ~b if there is a 9-derivation of ~. ~0 # I. Restriction 2 requires that the limiting context indeed be null. w. ~ DEFINITION 3. DEFINITION 5. a. DEFINITION 2. These notions appear..91A92. . (91. 1957). ~ such that 9 -91A92. If 9 --* ~b. 91.. If 9 -~ ~b. that the rules all be of the form A -+ o~.

and a is a single terminal symbol). ~+1 are successive lines of a derivation produced b y a t y p e 1 g r a m m a r . since ~+1 is formed f r o m ~ b y replacing a single symbol A of ~ b y a non-null string ~.L2 .. a type i grammar is one meeting restriction i. This follows f r o m the fact t h a t if ¢~..L2. there is an effective procedure for determining w h e t h e r an a r b i t r a r y string x is in the language e n u m e r a t e d b y G. Chap. 7~ T h a t is. . 2. . E a c h t y p e 1 language is a decidable set of strings. 6. THEOREM 3. Davis (1958. Conceptually. t y p e 0 D t y p e 1 t y p e 2 ___ t y p e 3. the t h e o r y of g r a m m a r can be viewed as a s t u d y of special classes of recursive functions. It is easily shown that the further structure in type 0 grammars over the combinatorial systems there described does not affect this result. .be an effective enumeration of all finite strings in what we can assume (without restriction) to be the common. T h e following is. we have immediately a decision procedure for this language. T y p e 0 g r a m m a r s are essentially Turing machines. B are single nonterminal symbols. Let sl. and a type i language is one with a t y p e i g r a m m a r . finite alphabet of L1. 7~ But not conversely.. well known. T h e t h e o r y of t y p e 0 g r a m m a r s and t y p e 0 languages is t h u s p a r t of a rapidly developing b r a n c h of m a t h e m a t i c s (recursive function t h e o r y ) . F o r b o t h g r a m m a r s a n d languages. 3. SECTION 3 T h e o r e m 1 follows immediately f r o m the definitions. t y p e 3 grammars.s~ . given a t y p e 1 g r a m m a r G. Given the index oi a language in the enumeration L~ . . Clearly a n y string x which has a 7 See. t h e n ~+1 c a n n o t contain fewer symbols t h a n ~ . F o r i = 1. I am indebted to Hilary Putnam for this observation. THEOREM 1.ON C E R T A I N F O R M A L P R O P E R T I E S OF GRAMMARS 143 limits the rules t o the form A ---> aB or A --+ a (where A . finite a u t o m a t a . thus enumerating type 1 languages as L1.. furthermore.. E v e r y recursively enumerable set of strings is a t y p e 0 language (and conversely). v T h a t is.--. for example. L ~ . at least. T y p e 1 and 2 g r a m m a r s can be interpreted as systems of phrase structure description.-. A type 0 grammar (language) is one t h a t is unrestricted. TtIEOREM 2. Then M is a decidable language not in the enumeration. DEFINITION 6. §2). Let M be the "diagonal" language containing just those strings sl such that si@ Li. .. a g r a m m a r of t y p e 0 is a device with the generative power of a T u r i n g machine. For suppose we give an effective enumeration of type 1 grammars.

only a finite number of derivations (those with no repetitions and no lines longer than x) need be investigated to determine whether x C L o . s We can thus associate a labeled tree with each derivation as a structural description of the generated sentence. if. We can then associate with this pair the element A (2) (T 1 O~2 • • • O/m_ 10/m Corresponding to any derivation D we can construct a tree formed from the elements (2) associated with the transitions between successive lines of D. these grammars provide a precise reconstruction of much of what is traditionally called "parsing" or "immediate constituent analysis. ." Type 1 grammars are the phrase structure grammars considered in Chomsky (1957. 1. and X .144 CHOMSKY #S#-derivation.2. Suppose that ~ = ax • • • a ~ . B are particular strings of G. The basic relation -~ of a type 1 grammar is specified completely b y a finite set of pairs of the form (¢1A@~. has a #S#-derivation in which no line repeats. P~ooF. Consequently. Cn+l new and distinct.p1AB~. there were a derivation containing the successive lines . Let G' be the grammar formed by adding X B ~ B X to G. It is possible to add conditions on G that guarantee uniqueness without affecting the set of generated languages. Let Q be the sequence of rules 8 This associated tree might not be unique. that Restriction 1 provides an essentially more limited type of grammar than type 0. for example. since lines between repetitions can be deleted. ~IACB~2. Chap. The restriction on the rules ~ -+ ~ which leads to type 1 grammars thus has a certain linguistic significance since. @~¢~). 4). given a grammar G of type 1 and a string x. • • • . We see. Suppose t h a t X = A1 • • • A n . since this step in the derivation might have used either of the rules A --~ A¢ or B --~ CB. as pointed out in Sec. Suppose that G is a type 1 grammar. therefore. SECTION 4 LEMMA 1. adding elements to the tree from the appropriate node as the derivation progresses. Then there is a type 1 grammar G* equivalent to G'. Choose C1.

/~.ON CERTAIN FORMAL PROPERTIES OF GRAMMARS 145 A1 "'" A n B . C.b.~ C1A2 "'" A.. . A. and the following rules: (I) (a) S ~ CDS~S2F (b) S~ -+ S:S~ (c) [S2B --->B B J (d) $1 "-+ S1S~ ~Sl u =---+ AB~ ( e ) [ S I A ---+ A A / . Consequently the only effect of adding the rules of Q to G is to permit a string ~.~B C1 . B. . By a similar argument it can easily be shown that type 1 languages are those whose grammars meet the condition that if ~ --~ ~b. C~B BC2 . weakening Restriction 1 to this extent will not increase the class of generated languages. It is obvious that if there is a #S#-derivation of x in G* using rules of Q. It is clear that La* contains all the sentences of Lo. contains only sentences of L~.XB~ to be rewritten ~ B X ¢ . D.4. That is. E. F}. Let L be the language containing all and only the sentences of the form #a~bma%'~ccc#(m. LEMMA2.n ~ 1).#}. and that G* meets Restriction 1. VN = {S. PRoof. and La.c. .~ where the left-hand side of each rule is the right-hand side of the immediately preceding rule. $2. . then ~b is at least as long as ~. $1 . then there is a #S#-derivation of x in G* in which the rules are applied only in the sequence Q.I. with no other rules interspersed (note that x is a terminal string). Let G* be formed by adding the rules of Q to G. Cn+l BA1 " " A. Consider the grammar G with V r = la. . Then L is a type 1 language.

(b). F}. and (e) n times. I t can n o w be determined t h a t the only #S#-derivations of G t h a t terminate in strings of VT are produced in the following m a n n e r : (1) the rules of ( I ) are applied as follows: (a) once. .~.+. . (c) m times. Note thut use of rules of the type of (II). . as in (2). .~. and (III) is justified by Lemma 1. .~+~CDFal .. . . f~ range over {A. giving #CDo~. ~ = /~ if a~ = B. .1 times for some n => 1. ol~Fal# (3) the rules of ( I I ) are applied.1 times for some m = 1.146 C~OMSXY {CDA --+ CE~A ( I I ) (a) ICDB --+ C E B B S (b) [CE~ ~ ~CE J (c) E ~ -~ ~Ea (d) E a # --+ D a # (e) ~ --~ D a ( I I I ) CDFa ~ a C D F (IV) (a) ~B. . . B. o~n+~FDal# (e) n + m times. giving #alCDa~ . #al "'" a. (c). (e).F# whereat =Afori~n.~+. n + m giving 1 more times. giving #alCa~ . a.~# 9 Where here and henceforth. (b) m .~+mF#9 (c) n + m times and (d) once. 3 -~ bJ ~CDF#---+ CDc#] (b) ~CDc ~ Ccc LCc -~ cc J where a. ..a~= Bfori>n (2) the rules of ( I I ) are applied as follows: (a) once and (b) once. giving #alCEal . . a~ = fi~ if a~ = A. . . (d) n -.

by Axiom 4. that grammars meeting Restriction 2 are essentially . Notice that the form of the terminal string is completely determined b y step (1) above. giving #anbma% '%cc# Any other sequence of rules (except for a certain freedom in point of application of [IVa]) will fail to produce a derivation terminating in a string of Vr.ON CERTAIN FORMAL PROPERTIES OF GRAMMARS 147 (4) the rule ( I I I ) is applied n + m times. select a sentence s = #a~b'~a'b'%cc# such that m -t. then no m a t t e r what x and y m a y be. . since A appears in the next-to-last line of a derivation of a terminal string. there is a type 1 grammar G* equivalent to G. Rules ( I I ) and (III) are nothing but a copying device that carries any string of the from #CDXF# (where X is any string of A's and B's) to the corresponding string #XXCDF#. (~1. There are type 1 languages which are not type 2 languages.n > r. giving # ~ "'" ~. (a) 2 (n + m) times. We can assume for each A in the vocabulary of G that there are infinitely m a n y x's such that A ~ x (otherwise A can be eliminated from G in favor of a finite number of rules of the form B -+ ~lz~ whenever G contains the rule B --~ ~1A~2 and A ~ z). From among these. "'" . where n and m are selected. • -. . (b) once. as was to be proven. which is converted b y (IV) into terminal form. ~ t ) is the initial part of infinitely m a n y derivations of terminal strings not in L. ~t+~) is a #S#-derivation of s for which ~t = #xAy#. and. Hence G is not a grammar of L.~+. B y Lemma 1. We have seen that the language L consisting of all and only the strings #a%'~a%%cc# is a type 1 language.~CDF# (5) the rules of (IV) are applied.~+. TI~EOREM 4. Suppose t h a t G is a type 2 grammar of L. We see.. but G contains only finitely m a n y symbols.e. PRooF. therefore. an is the longest string z such that A --+ z (note that there must be a z such t h a t A -+ z. . B u t now it is immediately clear that ff ( ~ . A is its only nonterminal symbol). L contains intlnitely m a n y sentences.~1 "'" a. where al . Therefore we can find an A such that for infinitely m a n y sentences of L there is an #S#-derivation the next-to-last line of which is of the form xAy (i. there are only finitely m a n y such z's).

t h a t neither Restriction 1 nor Restriction 2 is exactly what is required for the complete reconstruction of immediate constituent analysis. t h a t A --~ o~ in the context ~ . However.~ .B A . .. Thus the extra power of type 1 grammars is as much a defect as was the still greater power of unrestricted Turing machines (type 0 grammars). I t seems clear. 2. specify t h a t the segment derived ultimately from the B of • • . The purpose of permitting only a single symbol to be rewritten was to permit the construction of a tree (as in Sec. I n type 2 grammars. in the example in Sec. The tree associated with a derivation such as t h a t in the proof of L e m m a 1 will. . . from the above results. in effect. ~ and ~2 must be null in this case. Consequently the extra flexibility permitted in type 1 g r a m m a r s is important.B A • • • is an A. contrary to intention. A type 1 g r a m m a r m a y contain minimal rules of the form ~ I A ~ ~ 1 ~ 2 . the extra power of g r a m m a r s t h a t do not meet Restriction 2 appears. where will J o h n come is produced b y a permutation. is a B. I t is not obvious what further qualification would be appropriate. . it is nevertheless possible in effect to incorporate a permutation such as A B ~ B A ( L e m m a 1). J o h n is a N o u n Phrase). whereas in a t y p e 2 grammar.g.148 CHOMSKY less powerful t h a n those meeting only Restriction 1. with regard to the intended interpretation. and the segment derived from the A of . then. The final line of each terminated derivation is a string in V r . where it incorporates a permutation A B --~ B A . 2) as a structural description which specifies t h a t a certain segment x of the generated sentence is an A (e. A ~ ~A~b. For example.) if it contains an A such t h a t for some ~. and no string in Vr can head a derivation of more t h a n one line. . Contextual restrictions of this type are often found necessary in construction of phrase structure descriptions for natural languages. SECTION 5 We consider now g r a m m a r s meeting Restriction 2. at least) from the fact t h a t even though only a single symbol is rewritten with each addition of a new line to a derivation. A rule of the type 1 form asserts. to be a defect of such grammars. would specify t h a t will in will J o h n come is a N o u n Phrase and J o h n a Modal.e. a type 1 g r a m m a r in which both J o h n will come and will J o h n come are derived from an earlier line N o u n Phrase-Modal-Verb.~b(~ ~ I ~ ¢). The extra power of type 1 grammars comes (in part. A g r a m m a r is self-embedding (s. DEFINITION 7.. the anomalies mentioned in footnote 5 are avoided.

Suppose t h a t G1 contains rules R~ and R2 : R1 : A -+ ~iB~o~ = o01c02~a~(~ ~ I ) R~ : A --~ ~B¢~ where 9~ ~ ~bl or 92 # ~2 • Replace R~ b y the three rules RI~ : A ---+ C D R ~ : C --+ ~ where C and D are new and distinct. Continuing in this way.e. where B ~ C. form G2 equivalent to G~ . replace it b y . if G is non-s. C -+ b. Define L(~) (i.c. Clearly G~ is a type 2 g r a m m a r equivalent to G. if G~ is non-s.c. there is a regular g r a m m a r G* which is equivalent to G and which. A g r a m m a r G is r e g u l a r if it contains only rules of the form A --+ a or A ---+ B C . replace it b y A ~ B C . • ".e. and meeting the second of the regularity conditions. a. then ~o~ = ~b~(i = 1.c. PROOF. form Ga. replace it b y t h e rules R~ : A ---+ a l . B ---+ a. Given a type 2 g r a m m a r G.ON CERTAIN FORMAL PROPERTIES OF GRAMMARS 149 DEFINITION 8. .~_~B where B is new.. non-s. if G is non-s. consider all derivations D = (91. Continuing in this way. Let G1 be the gramm a r containing the minimal rule ~ -+ ~bjust in case for some such derivation D. where a~ 7 I . where B and C are new. . and if whenever A -+ ~1B~2 and A --+ ~blB¢~2are rules of G. 9t) meeting the following four conditions: (a) for some A.. If G is a type 2 grammar.~(a~ ~ I . is non-s. n > 2). If Ga contains A ---+ a B . and is non-s.c. furthermore. Clearly there is a finite number of such derivations. If Ga contains A --+ a b ( a ~ I ~ b)..e. since ~o -+ ~bin G1 only if ~o ~ ¢ in G. If G~ contains a rule A --+ a~ • • • oe. ~ = ~ and ~ = c t . THEOREM 5. 2).. always adding new symbols. length of ~) to be m if ~ = al • • • am. 91 = A (b) D contains no repeating lines (c) L(~ot_~) < 4 (d) L(¢t) _-__ 4 or ~ot is terminal.

Suppose that 2~ is a finite state Markov source with a symbol emitted at each inter-state transition. PROOF. THEOREM 6. Continuing in this way form G4. T h a t is. Suppose that G is a type 3 grammar.. The type 3 languages are the finite state languages. We interpret the symbols of V~ as designations of states and the symbols of Vr as transition symbols. a lowest level phrase. and nowhere else. as the set of input sequences that carry it from So to a first recurrence of S0 . z° Since Restriction 3 limits the rules to the form A --+ aB or A -~ a. where C is new. and with no transition from Sf except to So.e. i." . Theorem 5 asserts in particular that all type 2 languages can be generated b y grammars which yield only trees with no more than two branches from each node. Chomsky and Miller (1958) for a discussion of properties of finite state languages and systems that generate them from a point of view related to that of this paper. from the point of view of generative power. C --+ a. lO Alternatively. An #S#-derivation of G can involve only one application of a rule of the form A --+ a. G4 then is the grammar G* required for the theorem. Thus G can be interpreted as a system of the type described in Definition 9. This can be interpreted as indicating transition from A to a final state with a emitted. Similarly. we immediately conclude the following. an immediate constituent in this case is a member of the class). replace this by A ~ BC. C --> a. Then a rule of the form A --~ aB is interpreted as meaning that a is emitted on transition from A to B. a "phrase" has one immediate constituent just in ease it is interpreted as a word or morpheme class. DEFINITION 9. and the generated finite state language. Cf. If it contains A --+ Ba. with # emitted. each such system can be described as a type 3 grammar.150 CHOMSKY A -+ CB. with # emitted on transition from So and from Sf to So. Then the set of sentences that can be emitted by Z is a finite state language. where C is new. ~ can be considered as a finite automaton. and as a requirement that the only transition from the final state is to So. A finite state language is essentially what is called in Kleene (1956) a "regular event. Define a sentence as a string of symbols emitted as the system moves from So to a first recurrence of So. with a designated initial state So and a designated final state Sy . we do not restrict grammars b y requiring that each phrase have at most two immediate constituents (note that in a regular grammar. The fact that # bounds each sentence of L~ can be understood as indicating the presence of an initial state So with # emitted on transition from So to S.

. and L3 are not type 3 languages. Several different systems of this general type are studied by Schiitzenberger. Chomsky. THEOREM 8. A language generated (as in Definition 9) by a system of this sort (where each counter begins in a fixed position) will be called a counter language. L2 is not a counter language. where x is a string of a's and b's and y is the mirror image of x. From Theorem 5 we see that Restriction 2 amounts to a limitation of the rules to the form A ~ aB. L2 is a type 2 language but not a counter language (hence not a type 3 language) . but not a type 3 (finite state) language. (Cf. This is true for a t y p e 2. the language containing just the strings xy.TMTo summarize. Hence the fundamental feature distinguishing type 2 grammars (systems of phrase structure) from type 3 grammars (finite automata) is the possibility of rules of the form A ~ B C in the former. as can be shown by proofs similar to those of Lemma 2 and Theorem 4. b u t n o t a t y p e 1 p h r a s e s t r u c t u r e system. and we permit the next transition to be determined by the present state and the present readings of the counters. We permit each counter to shift position in a fixed way with each inter-state transition. L2. L~ is a counter language and a type 2 language. the language consisting of all strings x x where x is a string of a's and b's. Then L1. 12 T h e f u r t h e r question w h e t h e r all counter languages are t y p e 2 languages (i. w h e t h e r counter languages c o n s t i t u t e a step between types 2 a n d 3 in t h e hiera r c h y being considered here) has not been investigated. is proven. where the following. is a counter language. 119) a n d C h o m s k y (1957. THEORE~ 7. L3 is a type 1 language but not a type 2 language.. 11 I n C h o m s k y (1956. 1956. though not a finite state (type 3) language. LI and L2 are type 2 languages (cf. or A --~ B C (with the first type dispensable).e. and L3 is a type 1 language but not a type 2 language. L~. This leads to an important difference in generative power. 1957. each of which can assume infinitely many positions. Let L1 be the language containing just the strings a'b~. 34). in particular. it was erroneously s t a t e d t h a t La c a n n o t be g e n e r a t e d b y a phrase s t r u c t u r e system. three examples of non-type 3 languages were presented. 1956).ON CERTAIN FORMAL PROPERTIES OF GRAMMARS 151 Restriction 3 limits the rules to the form A ~ a B or A --~ a. Clearly L1.1~ Suppose that we extend the power of a finite automaton by equipping it with a finite number of counters. A --~ a. p. Chomsky. Thus there are type 2 languages that are not counter languages. p. There exist type 2 languages that are not type 3 languages. L~. (1957).) In Chomsky (1956).

We shall now show that the source of the excess of power of type 2 grammars over type 3 grammars lies in the fact that the former m a y be self-embedding (Definition 7). . (i) If Bn --+ a in G. B~]2.e. then ( a ) [B~ . Restrictions 1. We can conclude from this that finite automata (even with a finite number of infinite counters) that produce sentences from "left to right" in the manner of Definition 9 cannot constitute the class F (cf.. . 1) from which grammars are drawn.. 2 and 3 are increasingly heavy.D]~ (c) [B~ . then [B1 .C]I (b) [B1 . Construction: Let G be a non-s. . and A~#A~}. the devices that generate language cannot be of this character.152 CHOMSKY From Theorems 2. . . i. 1957). where the B / s are in turn nontermihal symbols of G. . since English has the essential properties of L~ (Chomsky. .. 13Since the nonterminal symbols of G and G' are represented in different forms. B. 1956. Because of Theorem 5 we can restrict our attention to regular grammars. 2).e. both for grammars (trivially) and for languages. we conclude: THEOREM 9. ." B. . regular (type 2) grammar.. B. 1 < i < j < m.A~) for [m = 1 or.." B~]~(i = 1. B~]~ ~ [B~ .. . B=]2. Sec.C]2 --+ [B1 . 5. . SECTION 6 The importance of gaining a better understanding of the difference in generative power between phrase structure grammars and finite state sources is clear from the considerations reviewed in Sec.. 3. as follows: 13 Suppose that (BI. . . B. Let K = {(A1. . The fact that L~ is a type 2 language but neither a type 3 nor a counter language is important.~D]~ -+ [B~ . T h a t is. (ii) If B~ ---+CD where C # B~ ~ D ( i <= n). we can use the symbols --~ and ~ for both G and G' without ambiguity. the inclusion in Theorem 1 is proper inclusion." B~]~ -+ a[B1 . Ai--->~Ai+l¢~ = We construct the grammar G' with each nonterminal symbol represented in the form [B1 . Bn) C K... 4 and 7. . .

If we have a configuration of the type (3c). .ON CERTAIN FORMAL PROPERTIES OF GRAMMARS 153 (iii) If B . Ej ~ xj). (i) of the construction corresponds to case (3a). in case (b). then (a) [B~ . we can have substrings of the form (xl • . In the case of (3d) we can have substrings of the form (yXn--i "'" Xl) ~ (where D ~ y.]I . B~]2. (ii) to (3b). . (3e) and (3d) are the only possible kinds of recursion. (iv) If B~ ---> C D where B~ -. . . Sec.~_~y) k (where Ej ~ x~-. (iii) to (3c). . B~h-+ [B~ . B.C]2 --~ [B~ .~D]2 ~ [BI . . . . . and in each case. Since G is regular and non-s. B. To the earliest (highest) occurrence of a particular nonterminal symbol . no node dominated by B~ is labeled B i ( i <= n).D]~ (b) [B~ . B~]~ ~ [BI . . x.C for some i ~ n.. ~ C D where B~ = D for some i < n. . . B1 = S. C ~ y ) in the resulting terminal strings. The character of this construction can be clarified by consideration of the trees generated by a grammar (cf. B.e. (iii) and (iv) accommodate these possibilities by permitting the appropriate cycles in Gt. We shall prove that G' is equivalent to G (when sfightly modified).~C]I (b) [B1 . we have to consider only the following configurations: (a) B1 (b) B1 (c) B1 (d) B1 /1\ B2 /1\ B2 /!\ B~ /1\ B2 / 1: \ B~ / 1: \ /\ C B~ D / 1: \ /\ E1 E~ B~ B~+I Bi+~ /1\ B~ B~+I E1 B~+~ E2 i a (3) /\ C Bn B~ /\ Bi Bn D where at most two of the branches proceeding from a given node are non-null. and (iv) to (3d). . . then (a) [B1 . B. B. . 3). . .

1958). " B~]2. This result. . where B1 . . For example. . . . B~-I are the labels of the nodes dominating this occurrence of B~. 4. The derivation in G' corresponding to the given tree will contain a subsequence (z[B1 . B~]I and [B1 . namely. however. Bn]~. the corresponding G' will generate the derivation (5) with the accompanying tree: 1. . Chomsky and Miller. . . . 5. . 2. . . B~]2).c. 6. cf. From this we will conclude that given any non-s.154 CI-IOMSKY B~ in a particular branch of a tree. the construction associates two nonterminal symbols [B1 . From this follows the main result of the paper (Theorem 11). which requires a long sequence of introductory lemmas. where B~ ~ x and z is the string preceding this occurrence of x in the given derivation in G. . We now proceed to establish that the grammar G' given by this construction is actually equivalent (with slight modification) to the given grammar G. that the extra power of phrase structure grammars over finite automata as language generators lies in the fact that phrase structure grammars m a y be self-embedding. is stated in a following paragraph as Theorem 10. type 2 grammar. we can construct an equivalent type 3 grammar (with m a n y vacuous transitions which are. 3. eliminable. . zx[B~ . corresponding to a tree of the form S A B (4) I a J b generated by a grammar G. [S]1 [SA]t a[SA]2 a[SB]I ab[SB]2 ab[S]2 (iia) (i) (lib) (i) (iic) a [S]1 I /\ [SAh [SA]2 ] (5) [SB]I b [SB]2 [S]2 I where the step of the construction permitting each line is indicated at the right.

n ) .~o~2B~3. T h e n ~ = Q~ is formed from Q~-I b y a rule whose source is (iia) or (iiia). B u t for some ~ Unless the initial line contained more than one nonterminal symbol. .'. Similarly. . . B1 ~ x~B~x2 Xlo~iB~2Bmo~3X2 ~ ~Blo~Bl¢o~ ~ o~TBlo~sBl~DB~o~6. Bm ~ ~ 1 B ~ 2 ~ ~lBm~b~2. LEMM~ 4. for 1 =< j <= k =< m. . the n u m b e r of non-null symbols a m o n g a l . B~ ~ ~1B. B~]I and B~ ~ x B 1 . B~ ~ B~¢. . since now B ~ is self-embedded. Suppose t h a t D = (~1. . m 1 zn --. ~ Note that a~+~will always be I unless the step of the construction justifying ~ --~ ~+~ is (i). it is n o t the case t h a t B~ ~ ~ B m ( i <= n. ' . a~Q~ (where Q~ is the unique non~ 15 terminal symbol t h a t can a p p e a r in a derivation~). --. t h e n [CBI . . . ~ ) is a derivation in G' where Q~ = [B~]2.. t h e n A j ~ A k ~ b . . Suppose t h a t B ~ ~ ~B1 and for some i ~ m. q~ = [ " ' ] 1 and q~+~ = [ ..e. Suppose t h a t ( ~ .~B~h.. (s.. " " . . . " ' " .~2Bmo~3(i <= n ) PROOF. " . ( I I ) if ~ = [B1 . i ~ m ) (c) if Bm ~ ~B~b.+ x[B1 " " B. C~---+A~+~C~+I (for 1 < i < m ) . Suppose t h a t the length of z~ is 1 . . Cm+l) ~ K .ON C E R T A I N F O R M A L P R O P E R T I E S OF G R A M M A R S 155 LEMMA 3. If (B1. Suppose Bm ~ ~B~b and for some i.a m " " a. there is one and only one i s. z~[C~]~) in G'. Contra. . PROOF. . .~]j. it is n o t the case t h a t B~ ~ ~lB. .e. a~. A m ) ~ K . B y l e m m a 3. . i. a n d C ---> aBl~.).'.zn = zn. B~) C K and 1 < m < n. (a) Suppose t h a t i > 1.] 2 . ~ ~ I ~ ~b. T o facilitate proofs. • • • . . t h e n there is a derivation ([C~ . B~ ~ ~ 1 B ~ 2 . ' . t h e n there is a derivation ([B~h. B~]i ---+ x[CB1 . P r o o f is b y simultaneous induction on the length of z~. If ( A 1 . T h e n : ( I ) if~l = [B~]I. we a d o p t the following notational convention: CONVENTmN 2. C ~ B k ( k <= re. If [B1 " ' " B~]~ . a n d Cm+l = B1. t h e n (a) if B ~ ~ ~B1. C. LEMMA 5. . Contra.~. B. " • • .t. Proofs are immediate. (C1. . LEMMA 6. it is n o t the case t h a t B~ ~ B m ~ ( i <= n.+ a ~+l~z~+~.. a n d ~+~ = a~+lQ~+~ is f o r m e d f r o m ~+1 = a~+~Q~+~ b y a rule whose source is (iic) or (ivb). T h e n ~ = a~ . i ~ m ) (b) if Bm ~ BI~. a case which will never arise below. ~ ) is a derivation in G' f o r m e d b y construction. z~[B~]~) in G'. a~ will generally be I in this sequence of theorems.~]j. • • • . Q~ . where K is as in the construction. case (b).

'. [C~ .. g+a[B. (lib). .can only have been introduced b y (lib).. . i = 1. Suppose it is true in all cases where z~ is of length < t. Qi = [B~ . ¢ # I. ' . it follows t h a t [Ca . . " C m ] 2 . Let D be such t h a t z~ is of length t.. j > 1. . This proves the l e m m a for the ease of z~ of length 1. Since i = 1.o~Bi+q¢o~ . .]~ --~ [CI .. B.. B~+q_t ~ B~B~+q. C~ ~ Ba. [Ca "'" C.B~) ~ K.. ([C~ .~Ba]2 ~ [Ca .. • • • . and Bq ~ C~D..'. . If none of Ca. . Q~-a and (Ca. . ..156 CHOMSKY = [B1 "'" . • • • . ] 1 to [.~B1]a --* z2[C~ " " CraB1]2. Qi = [B~.. b y (i) of c o n s t r u c t i o n . Bk --+ EBk+a . Since i = 1. since there is at least one transition from [ . where C~ = B ~ (1 < s =< q). B. Bn "-" E C k . Bi+q]~ (q >= 1). ' .-.E]~----> [ C a ' . the rule Q~'-I --+ a j Q j used to form ~. .' .'.. .. Bk --~ Bk+ID for some D . . . Contra.'.B~C~]I. . for some E .]2) (S) ~ It can only have been introduced by (iia). B . (iva). which m u s t therefore have been introduced b y (i) of the construction. r = 2 . . . by Lemma 5 (a). . .- Bk+l]2. Suppose (iva). . BqC~]. . F u r t h e r m o r e z j_l is not null. (b) Consider now ( I ) . or C~ will appear in Q~. . then the proof is just like (a). But C~ ~ ebb. .. Qj = [ B a ' . (e) Consider now ( I I ) .'. ([Bn]~ . since C1. C~+~) C K b y assumption). B u t Ca.-'-. But C~ ~ ¢B~ . .. Bd= . B~ --~ z2 and [Bn]a --~ z2[Bn]~.c. . Consider ( I ) . (ilia). . Qi--1 Bk]l. z 2 [ C a " " CraBs]2. contra. . CmBa]a. B y assumption of nons. z2[Bn]2) is the required derivation. C~]~ (7) [by (iiib)]. Q~--j = [Ba " " B n E ] 2 . zj_~[C~ . Qi-* = [B~ .. B y assumption a b o u t the C?s and m applications of L e m m a 4. Suppose (ilia). . appears in Q j .]2 in (6). ~6 .~ ---+ E C ~ . Qi = [B1 "'" Bk+ila. and (i) of the construction. (6) is a derivation.Qi+~ = [B1 • • • Bk]2.~E]~) C~. C~ do not occur in Q1. B u t B. ' . . Since Ci --~ Ai+aCi+l(Ci ~ C j for 1 _-< i < j _< m + 1. C~ ~ ~colBi+q_lW2 --> ¢o~lBiBi+qCO2 :=~ ~bwaCte. . . --+ z2. C r a B s . . "'" . say Ck. z 2 [ C ~ . C~Ba]~ . B. . Same. .. . since (Ca. . . Q¢+I = [B1 . " ' . Qi-* = [B~ ._. .. . . F u r t h e r m o r e we know t h a t ([B~ ... . . . Suppose (iia). introduced by (lib). Bq]. b y L e m m a 4. ' .~C~]a.t~.. " ' " . . C. . . Suppose t h a t ~j is the earliest line in which one of Ca. C ~ .'. . C. z_~[Ca]2) is the required derivation. ---* [Ca]2.'. above.'. C~ appears in a n y of the Q¢'s in D. Cm-~]2 . . . C r a B s . which contradicts the assumption t h a t G is r e g u l a r . ([C~ .

m <= n). . d+'[c&) . • • • .e. . ~ j . B. .. (9) (10) z~-t-1[C112) C o m b i n i n g (6). . t h e r e is a d e r i v a t i o n (IV1 ' ' ' Ck]l. . . a d e r i v a t i o n . • "]2 • . B u t ([B. Bm ~ yB~. I t is clear t h a t we can c o n s t r u c t ¢1.. hence is of l e n g t h < t .B. ' . ~p = z~Qp'. . w i t h t h e i n i t i a l s e g m e n t zj d e l e t e d f r o m each of ¢~.~+v]~. B~C~]I = Qj . L e t q = m i n ( j . A s s u m e n > 1. • • • . . L e t ~ j c o n t a i n t h e first Q of t h e f o r m [B. . Zq_~Q'q_~) is a (13) (14) derivation. . F r o m L e m m a 5 it follows t h a t t h e rule used t o f o r m ~k+~ m u s t be justified b y (iic) or ( i v b ) of t h e c o n s t r u c t i o n .. " • " . . is. . If n . . Ck ~ xB1. 5 t h a t Bm ~ yB1. B~]2. . i. z~+~ is s h o r t e r t h a n z~. derivation z~+1[B ~12) " (J1) b y i n d u c t i v e h y p o t h e s i s ( I I ) . z~ = zk. t h e proof is t r i v i a l . " ' " . . • • • .'.. B~+~]i w h e n Q. t h e r e w a s a t l e a s t one a p p l i c a t i o n of (i) in f o r m i n g (~1.~]2(m <= n).ON CERTAIN FORMAL PROPERTIES OF GRAMMARS 157 m u s t be a d e r i v a t i o n . . ) .]~) is a d e r i v a t i o n . .B~-112 • S i m i l a r l y . • • • . b y a s s u m p t i o n . . As a b o v e . . zj_l is n o t null. Cq-~ s. . . (8) is j u s t t h e t a i l e n d (¢5. T h e n all of Q2. B A 1 . for p < q. C o n s e q u e n t l y ( [ B d ~ . Qj-1 are of t h e f o r m [. b y i n d u c t i v e h y p o t h e s i s ( I ) .t. . . we see b y checking t h r o u g h t h e possibilities in t h e c o n s t r u c t i o n t h a t n o t all of Q1. . Qq-1 are of t h e f o r m [BI " ' " B~+~]i . like ( 8 ) . L e t ek c o n t a i n t h e first Q of t h e f o r m [B~ . . Cr) of D.. b y i n d u c t i v e h y p o t h e s i s ( I I ) . Since z j_l is n o t null. we show t h a t ([Bi .1 or t h e r e is no such d e r i v a t i o n of l e n g t h l. z~+~[BmD (12) w h e r e z~+1 is s h o r t e r t h a n z~. [B. . .Bm]. . ' . . we h a v e t h e r e q u i r e d d e r i v a t i o n .'. .'. Cr. k ) . . Since m <= n. since [B1 . Also. (10). ' . .. it follows f r o m L e m m a s 3. . ~ xB~.. I n e i t h e r case. w h e r e Q~' = [B~ . ' . t h e r e is a d e r i v a t i o n ([ck]l.. Since B . . .. (7).(j > 1. . QI~+~ = [BI " . C o n s i d e r n o w ( I I ) . = [B1 . t h e r e is a ([B~]I. .

Combining this with (14) we have the required derivation. B~x.. where m =< n < n + v . . Suppose t h a t D = ( ~ .+~]2 --~ ak[B.]2.. B. -.~ B ~ + ~ B ~ .'. LEI~MA 7. . . we take Q~ as the last of the Q's in which one of C~. B. B. . . . . . Bn+~. C~ . . C~+~) C K. . (16). C~ ~ C~+~A. • • • . .]~--~ a~[B1 . .. b y inductive _s+~ hypothesis ( I ) there is a derivation s < v ( l B . We have seen t h a t in this case z~ = zk. B. ~ ) is a derivation in G' where QI = [B111 . I n the inductive step. .]I = [B. B u t (12) is a derivation where z. [ B . . ' . is of length < t .+. . We have the required derivation. . (C~. ' .~]z = Qj . this rule can only have been introduced b y (iic) or (ivb). CmB~]~) ( I I ) if ¢.]I. T h e proof is analogous to t h a t of L e m m a 6. Bm]~ .'. . B u t Qq-1 = [BI .]I (15) Case 2.. . (17). We have thus shown t h a t the l e m m a is true in case z~ is of length 1. . . . m = n. . + . . B. = [B~ . . t h e n there is a derivation ([B. B .+~_~ w ~ B ~ . ' . . ~ Bn. t h e n there is a derivation ([C~]~. . . z~+*[Bn]2) (17) Combining (14). t h a t q = k.+~. B. . .]~ Combining (14).+~ (1 < i < m). . (16) b y (iib). . . . --~ E~B. and O~_.'. (15).+~_~B. Suppose. ' . = [B~B. we have the required derivation. this rule can only have been introduced b y (iiib) of the c o n s t r u c t i o n . (12). [B.158 CItOMSKY Suppose q = j . . case ( I ) . .. of length < t . i = 2 and B~+~-I . ' . . B . for 1. . . Case 1. Qq-1 = [B1 . and C~+I = B~. Therefore it holds for e v e r y derivation D. finally. We have seen t h a t B . . a n d t h a t it is true for z~ of length t on the a s s u m p t i o n t h a t it holds for z..+~]~ --~ [B. .+. B. . z~[C~ . B~+~]¢ --~ ak[Bi . . .]2 and B~ ~ in G'.'. B~+~_IB~]~ . v = 1. Suppose m = n . T h e n ( I ) if ~ = [B~h. Suppose m < n . ~ y B ~ . b y L e m m a 5. . i = 2.+~]2 ~ [ B . . z~[Bn]~) in G'. I n either ease. . . . ' . . where m < n < n + v .~ .

.. (D3 is thus a C-derivation of xy.. where the proof' of the second is analogous to the proof of the first. LEMMA 8.k) [instead of min(j. Suppose that G' is formed from G by the construction and D is an a-derivation of x in G. ~ b ~ ) be derivations in G. where (~1. . . m <. .k)]. L e t D 1 = ( ~ . then Da = (C~1 . B~E]~ by (ira).max(j. ~ m ) and D2 = ( ~ l . there is a derivation ([C]~.~) = D~*D2.z[C]~) in G' (b) if C --~ B A . Then (a) if C --~ AB. Taking q -. . In case ( I I ) of the inductive step we let Qj be the last Q of the form [ B 1 . D will be said to be represented in G ~ if and only if ~ = a or a = A and there is a derivation ([All. ' " . B. If Di and D2 are represented in G ~ and C ~ A B . The proof goes through as above. . with similar modifications throughout. we form [Ci " " G]2--* [ G " " C. x[A]2) in G'. Let Di be an A-derivation of x and D2 a B-derivation of y in G. most of the results obtained come in symmetrical pairs. ..~BI . LEMMA 9. . the proof is analogous throughout. . Then DI*D2 is the derivation ( ~ 1 ~ . . z[CB]2) in G'. as above. .n ) . DEFrNITIoNll.. . . . We will require below only the following special case of (I) of Lemmas 6.ON CERTAIN FORMAL PROPERTIES OF GRAMMARS 159 appears. ~2¢1. . . because of the symmetries in case (iii). with (iva) taking the place of (iiib). Only one of the pair of proofs will actually be presented. ~. however. 7 (which. . In general.~ ) is represented in G'. ~m¢~). . • "" .¢~. DEFINITION 10.~]2(j < r .) .n ) . z[B]2) is a derivation in G' and that C ~ B. ~. (3d) for recursion].. . ~ . could not be proved without the general case).-. . . . . B~]l(m <. (iv) of the construction [reflecting the parallel possibilities (30). . there is a derivation ( [ C B h . andQk the last Q of the form [B1 " .-. and instead of (iiib) in (7). . W h a t we are now trying to prove is that every S-derivation of G is represented in G'. Suppose that D = ([B]~. .

Since for i > 1. Case 3. C = B .'. (22). B y Lemma 8. (19). ])ROOF. .* [CB]~. (23) Combining (18). LEY~MA 11. . [CB]I (22) Combining (20). . . D3 is a x~-derivation and D~ is an wl-derivation. we have (20). (24) Combining (20). LE~MA 10. ~r) is a Xl~l-derivation. x[A]=) ([B]I. B y (ira) of the construction. [CA]~ --~ [C]~ = [B]~. Case 1.~. B y Lemma 8. there are derivations (18) (19) ([C]1. Suppose A ~ C ~ B.'. . then there is a derivation D2 = D~*D~ = (g. we have the required derivation. ' ' . however. . . . . q~ is formed from ~_~ by replacement of a single symbol of ~_~ . C = A . Compare footnote 8. y[C]2) in G'. . Case 2. Then by Lemma 8. case (b). (24). we have the required derivation. ~ s.~7 we can clearly find X~. and (21). these are the only possible cases. we have again the derivation (21).. [A]~ = [C]2 . where Xl I ~ ~ . . " . If D1 = ( ~ . . B y (iiib). Since C ---+ C C is ruled out by assumption of regularity. . . y[B]~) in G'. B y (iib) of the construction. (23). ~ = x ~ where either (a) xi = x~-i and w~-1 --~ ~ or (b) xi-~ --~ x~ and ~i = o~_~ (X~-~-~ = ~-~). case (a). "'" . . . x[CA ]~) (20) (21) ([CB]I. ~ ) formed b y dropping repetitions.. . . . If G' is formed from G by the construction. B y hypothesis. then every a-derivation D in G is represented in G'. C ~ B by assumption of regularity of G. may not be uniquely determined.t. . ¢~r) such that t r = ~r. ~ Which. . C ~ A . [CA]~ --. . . there are derivations ([A]I. Then D~ is the subsequence of (X~. (21) we have the required derivation.160 CHOMSK¥ PROOF. X.) formed b y dropping repetitions and D4 is the subsequence of ( ~ .

. . x[A]2) is a derivation in G'. C.~. I t remains t o show t h a t if (JAIl.. and t h a t (a) D = ( ~ l . . ~k~) s. Q. . Q~ --. Q~_~ = [A0 . . Let D = (~i. " A ~ _ I B o . . m ' > 1. Qqis t h e s h o r t e s t Q of t h i s f o r m b e t w e e n ~ i a n d f ~ : . . (b) there is no u.. ~2 = B C ._~Co . ~ ) . Q~ = [A1 . Ak+t]~(t > 0). where ~+~ is f o r m e d b y ( i r a ) of the construction. m = 1. ~s Suppose t h a t G' is formed b y the construction f r o m G. where ~ is formed b y (iiib) of the construction.. . Q~ = [A~ • • • A~]:. . . p < s (since (iiib) gives the only possibility for decreasing the ~s W e c o n t i n u e t o e m p l o y C o n v e n t i o n 2. A.. Co = A ~ . . . B0 = A k . ~ q .. A~ ~ ~ A ~ B ~ ¢ ... ~0 T h a t is. . a = A . D4 a C-derivation.~_I --> A ~ B .t. ~ T h a ~ is. A0 = I . • . A~]~. Q~0 = [A1 .t. . . A0 = I .c. . Q~ = Q~ = [B1 . t h e n s > f 0 T h e n it follows t h a t (A) if n -.ON CERTAIN FORMAL PROPERTIES OF GRAMMARS 161 PROOF. in case D contains 2 lines.t. .. Qml = Q ~ = [A1 . .B~]t. where ~1 = a. Therefore. ] t . B y L e l n m a 10. Ak]l. A~]n. ~ . . Q. ) is a derivation in G'. u ¢ v. there is a Dz = D~*D~ = (¢~2. If there is no m0 < ~ s. there is an m0 < m~ such t h a t Q~0 = [A~ . . is a B-derivation. Q~+I = [A0 .Aj]~. B y L e m m a 9. . . D is represented in G'. and s < /c (since ( i r a ) gives the only possibility for increasing the length of Q b y more t h a n 1 ) . Qqis n o t s h o r t e r t h a n Q~I = Q~e - . . . 19 T h a t is. Ak+~]~. Since r > 2. regular a n d non-s. x) in G. -. . D. where Q~ = [A1].]~. there is an m~ > m2 such t h a t Qm = [Aj .a-~ = [A1 . A . ~ l . Obvious. ' . . a b o v e . ( ~ . t h e n there m u s t be a u < r~ s. LEMMA 12. ". . t h e n there is a derivation (A.t. • . . ~ ) is a BC-derivation. ' .2.[A1 . . . ~ m 2 . . = [A~ . . . B u t Q~ = [A~ . . Suppose true for all derivations of fewer t h a n r lines (r > 2). . Q~I = Q~: is t h e s h o r t e s t Q of D t h a t r e p e a t s .t. . b o t h D3 and D4 are represented in G'. . • "" . . and s < k 19 (c) for ml < u < m2.. v s. (A) Suppose n = 2. Assume ~ to be the earliest line to cont a i n [A1 . . there m u s t be a v > m2 s. B. . Ak]t (B) if n = 1. . .~]I. Ak]2 (C) j = ~ PROOF. A~]2 cannot recur in a n y line following f ~ [this would contradict assumption (b)]. . and ~kr = ~r • B y inductive hypothesis. .. Clearly there is an ~ =< m~ s. . if Q~ = [A1 . . iust as above. . " ' " .t. B. . A~]2. .

then ml < u < m3.'. . Proof is analogous. . A~]~ [which contradicts assumption (b)] or Q.]~. either Q~ = [A~ . Ao = I . Furthermore. .Ap~2 ~ ~ 1 C ~ .. Q~+I = [A0 . since G is assumed to be non-s.A. .AkBo . Co = A i . Bin]l. . A . A i ~ xaA ~Dt~. Contradiction follows as above. . . ... . and A~.'. Contra. ~m1+1 is formed b y ( i r a ) of the construction (B) if n = 1. Let D = (~i.'. ' . there is an m0 < r~ _= m~ s. s > 0 if t # n. . A j ~ ~ C m A . ~ 3 ~ A ~ B . . ~ ) be a derivation in G' formed .'.I . Bo =. .162 CttOMSK¥ length of Q b y more t h a n 1 ) ._~ = [A0 . (C) ( I ) . ~ 4 . where ~+1 is formed b y (ira). ~m2 is formed b y (iiib) of the construction (C) Q~ is of the form [A1 .I ) . Ao . . contrary to assumption. L e m m a 12. p < j [as in the second paragraph of the proof of (A)]. it follows t h a t ~q+~ can only have been formed b y (iva) of the construction. " Ai_~Do . ' . A i ~ o~fC~o~A i x ~ D t ~ .t.t. since p < j. where A0 = I . . Proof is analogous. Ak]~ (B) Suppose n = 1.. where f~ is formed b y (iiib) of the construction. .c. C~_~ --* C m A ~ . Qq+l = [A0 . t ~ 1. it follows t h a t Cq can only have been formed b y (iiib). D t _ ~ . " Dt]:t. m => 1. Assuming the latter.A~ . This completes the proof. . . there is an m0 < m~ s.-1~2 ( L e m m a 3 ) . Qm0 = [A~ . Aj_1Bo . m >= 1. for ml < u < m2. A~ ~ elC~. . F r o m L e m m a 12 it follows readily b y the same kind of reasoning as above t h a t COROLLARY. = [A1 . as above. Clearly there m u s t be a v > m~ s. .t. s < j . But A~ ~ ~1C. . ¢ . .'. Suppose the l a t t e r . A~. A ~~ ~C~A ~. = [A~ . for u such t h a t : (a) where n = 2 and m0is a s i n (A).. DEFINITION 12. F r o m assumption (c) and assumption of regularity of G.> A i D t . ] ~ . L e m m a 12. S u p p o s e j < k. Bo = A ~ . we conclude t h a t A j ~ ~IAjw2Bm¢~. ~ A ~ 4 ~ ~ I C ~ . ' . Suppose n = 2. . Q~0 = [At • .. Aj-~Co . ' • • .. A~. Suppose i (in Qq) is 2..'.. C~]~ . F r o m assumption (c) and assumption of regularity of G.~.t. is self-embedded. ' . then m0 < u < m2 . Do = A~. . Cm. . . ( I I ) Suppose n = 1.]2 [which contradicts assumption (b)] or Q. either Q~ = [A1 . .. (b) where n = 1 and m~ is as in (B). .]~ .-1 ---> Cm. there is a u < m0 s. Q~. B y (A). Furthermore.. Under the assumptions of L e m m a 12. (A) if n = 2. . B~]t (s >-_ O.~. Suppose t h a t i (in Qq) is 1. .

Suppose t h a t there are no recurrences of a n y Q~ in D. j > s. • • -. there is no j > m2 s. . ' . .. . or Q~ is repeated as Q~. Qr = [A~ .. m~ s. Clearly there is an m > i ~. Proof is b y induction on the n u m b e r of recurrences of symbols Qi in D (i. i < p < j. I. ) corresponding t o D. c o n t r a r y t o the assumption t h a t G is non-s.t. . Also. LEMMA 13. Let D = (~1. q s.~. Aq]~. k ( i < j ) such t h a t (a) ~i is the earliest line containing [AI . ' . . .. A~]2. m~ < m~ . . 22 Compare Convention 2. ~ . Suppose t h a t Q1 = [A~ .. Similarly. Ak]2 (c) there is no p. and there is no p. . • A~]I. T h e n D ~ corresponds t o D if D ' is a derivation of z~22 in G and for each i. F o r suppose there were such a p a i r . . Aj]2.1 s. . Ak]l (b) ~j is the latest line containing [AI .. . .t. .t. 1.e. A~]~. Qi = [A1 . Qt+l = [A1 . .. m~ < i < m~. Aq]~. Q~ = [A~ . .ON CERTAIN FORMAL PROPERTIES OF GRAMMARS 163 b y the construction f r o m G. .e. • • ". ~r) be a derivation in G t f o r m e d by the construction f r o m a regular. .e. the n u m b e r of cycles in the derivation). . I t follows t h a t there can have been no applications of (iva) in the construction of D. PnOOF. . . q such t h a t 1 < p < r.e.t. . Suppose t h a t the shortest recurring Q in D is [A~ • • • Akin. non-s.[A1 .. there is no i. A~]~ = Q~: . . T h e n there is a derivation D r = (~1 ~ . . there is a subsequence (ziA~b. Ak_~ ~ A ~ A k . .. Suppose now t h a t the l e m m a is true for every derivation containing < n occurrences of repeating Q's. As+~]2. no pairs Qi -. . Aj]2 ( c o n t r a r y t o a s s u m p t i o n of no repetitions) or Qt = [A1 . Qp = [ A 1 . B u t now the proof for this case follows immediately b y induction on the length of D.. there can be no applications of (iiib) in the construction of D. Select m l .t. v > 1). A~]I where j < h. and Qp = [A1 . i. q < h. Qj = [A1 . s.G. . Qi+~ = [A1 . Suppose t h a t D contains n such occurrences. where ~+1 is f o r m e d b y ( i i i b ) . there is a t > m s. Q ~ = [A~ . either Qt = [A1 . ' . q < s. . . A~]~ . j.A~+~]~ (n > 0 ) . 2. zj~b) in D'. . . Aj-. .]I (u.

a n d Qt+l = [A0 .'.-.~Co . B~+.). Since v was selected in s t e p 4 t o be m a x i m a l . Consequently (~+i. m >= 1. Qv+~ = [Ao . . S u p p o s e t h a t i = 1. B~+t]2. we can select v as t h e l a r g e s t i n t e g e r ( m 2 s.e. . Q.. t h e r e is a v(m~ < v < m2) s._~ --+ B m .1 ~ t ~ v) s. . b y reasoning similar t o t h a t i n v o l v e d in ~ Recall that z ~ . .-.~ ~ ~ B m . ' . . . i. Qm0 = [A1 . z m)58 c o r r e s p o n d i n g t o ( ~ o . .A~_~Bo . S u p p o s e n = 0. C~]i ( ~ >= O. 5.. Q~ = [A0 . c o n t r a r y t o a s s u m p t i o n of r e g u l a r i t y . . . where u.(z~+~Bm. ... ~t~v)" 7. ..~o+h~. .t. z~+~). A~]I. . B y t h e C o r o l l a r y t o L e m m a 12. i = 2.'. .~0+~. L e t t b e t h e l a r g e s t i n t e g e r (ml ~. ~ 1 ) . B. zm0z~ .l B o "'" Bm-u]l. ~ x 4 . A k _ l B o .A~_IBo . A ~ . . Select m0 as t h e earliest such ( t h e r e is in f a c t o n l y o n e ) . c o n t r a r y t o a s s u m p t i o n in s t e p 2.I B o . a n d t h e i n d u c t i v e h y p o t h e s i s .~l[Ao " ' " A k . B y C o r o l l a r y ( A ) . t > 1 a n d ~+1 is f o r m e d b y (iiib) [note C o r o l l a r y (C)]. u = 1 a n d Bm_~ . = [A0 • • • Ak_~Bo • • • B ~ ] ~ . . 4.i B o " ' " B m ] l .]1(n _-> 0).164 CHOMSKY 3. . B m ~ ~ 3 ~ A ~ B . since B~_~ ~ A k B m . F r o m t h e l a t t e r a s s u m p t i o n we can d e d u c e self-embedding. . "'. .. w h e r e A o = I . ~) meets the assumption of the inductive hypothesis e~ and there is a derivation D~ -. • •. .e. .~ . Qt = [A~ • • • A~-]2.'. there is no t such as that postulated in step 5. t h e r e is a d e r i v a t i o n D~ = (zmoAk .I ~ 2 B ~ .I . .t. . . ' . ~ : B ~ + .A k .t. z.t. b y r e g u l a r i t y . n ~ O..~ B m .~ B. . Bm]2 or Qv = [A0 ..~ B m C . . there is a derivation (A~. Bm-~ --~ A k B ~ . ~ + ~ _ ~ 4 . w h e r e ~t+l is f o r m e d b y ( i r a ) of t h e c o n s t r u c t i o n ._~. as a b o v e . B m _ u ] i . ~ That is. B m + . ' . • • • . i t follows t h a t ~. there is a derivation (B. . . we k n o w t h a t t h e r e is a n m0 < ml s. . B m ~ ~ B . .~ . . . . " " " . B~_~ ---A k . O b v i o u s l y . u > O.B~+. Bo = A k . ( C ) . T h e n B. c o n t r a .+~ c a n n o t be f o r m e d b y ( i v a ) . (s.Zm~) ~ corresponding to (~m1+1 . + ~ --~ ~IAkBm~2Bm+. Co ~ I). e i t h e r Q~ = [A0 .'. so t h a t . 6. Ak_~Bo . ' . "" ". B u t ~t+1 m u s t be f o r m e d b y (iia) or (ilia) of t h e c o n s t r u c t i o n .'. B y L e m m a 12 ( A ) . Qt = [A0 . we k n o w t h a t ~ml+~ = z. . . . . . ~a From nonexistence of such a t it follows at once that for u such that mt u < v.

v -t. . Q~+2 = [A0 • . and AIo ~ AkB~.'.. • •. since B. where for i < m~. L e t D2' be the derivation f o r m e d f r o m D~ (cf. . B. . Ak]~ (as a b o v e ) . e ~ ) . Bm-2C]l. ~b~ -= z ~ z m _ ~ + ~ 2 _ ~ 1 + ~ . ' . Q~+I -. ~b~ = ~ . . and so t h a t no occurrences of Q ~ occur . . . ~1. Q~ = [A0 .~_2---+ B1C. 10. . Suppose m = 1. . . there is a derivation D~ corresponding to D~. L e t (~1. and ~ the latest occurrence of Q ~ = Qm~ = [A~ . Consider now the derivation D6 f o r m e d b y deleting f r o m t h e original D the lines ~ + ~ ..m~)). . .~oAl~.. . .]2. • • ". a n d for i > m~. ~ ) = DI*D2' (cf. B y a s s u m p t i o n t h a t G is non-s. ~b~)*D4'. • • -. ~ ) . DI as in step 3). B~_2 --> Bm_lC. ' .-I ---+ A k B ~ . it cannot h a v e been f o r m e d b y (lib) or (iiib) of the construction.~oAk. . . b y a s s u m p t i o n of step 2.t.[A0 ..~-2C]2 a n d s. ..t.A~]~. . ~p) be as in step 8. .~-112.Ak-IBo . • •. . ..+~. Suppose m _>.A~]2. 11. Q~+I = lAx . . L e t ( x l . Clearly D3 = (z. (¢. 12.2. ~ ) is a derivation corresponding to ( ~ 0 . "" ". Definition 11. ~ and the medial segment z ~ +~ from each later line. . . . As above. ~ ) . we can find a derivation A = (Z. xq) = (z~oB1. . . I n steps 2 and 3.A~. . B. T h a t is. B. . 8. . Ak-~Bo . L e t D4' be the derivation f o r m e d f r o m D4 b y deleting initial z~+2 f r o m each line. step 6) b y deleting initial z~1+~ f r o m each line. . "" ". w h a t e v e r m is.(m~ . L e t (~b. . Suppose m = 2 . m~ were chosen so t h a t ~ 0 contains the earliest occurrence of Q~0 = [A~ .. Similarly. ~t) (t = r . D6 = (~1. . ' . Xl. . z ~ ) corresponding to ( ~ 0 . ~ . we can find a vl which is the largest integer < m 2 s.ON CERTAIN FORMAL PROPERTIES OF GRAMMARS 165 step 4. . so t h a t Q~+I = [AI . . ' . ~ 1 ) m e e t s the inductive h y p o t h e s i s .. Xq) is a derivation corresponding to ( ~ 0 . . . . . B y regularity assumption. . 9. there is a derivation D4 = (z~+2C. a n d t h a t v is m a x i m a l (in step 4) we can show t h a t e~+~ m u s t be f o r m e d b y (iib) of the construction (all other cases lead to c o n t r a d i c t i o n ) . . Clearly D5 = (zmoAk. Ak-lBo . ~ = ) . m~. .1 = m2. m0.e. z~) corresponding to ( ~ + ~ .. " " . By inductive hypothesis. . " " . . vl + 1 = m2. "" ".

' . . z ~ ) corresponding to ( ~ 0 . • . A corollary follows immediately. . If G' is formed from G by the construction. . z[S]2) in G'. then there is a derivation (S.. From this result and Lemma 11. it follows that D7 contains a subsequenee D7 = (z~oAk~b. and that it holds of a derivation with n occurrences of recurring Q's on the assumption that it holds for all derivations with < n such occurrences. . .. "" ". . -. . We have shown that the lemma holds for derivations with no recursions. .. . . and Q1 is of the form [.]1.. by inductive hypothesis and the definition of correspondence. .. . there is no Q shorter than Qm0 between ~m0 and ~ 1 . But step 11 guarantees us a derivation A = (z. 1958). ' .Q~ (n _ . "]2 for 1 < i =< n. Therefore L~ could have been generated by a finite Markov source of the usual type. .. and inserting z~°+~ after z ~ in all lines of D7 following the subsequenee/)7.166 CHOMSKY between ~ 1 and ~ . it is true of all derivations. THEO~E~ 10. z ~ ) . . and the language L~ generated by G could have been generated by a finite state Markov source (of.2) such that Q1 -~ aQ2 and Q~ --~ Q3 --~ " .. An analogous proof can be given for the case in which the shortest recurring Q is of the form [.. Theorem 6) with many vacuous transitions. That is.. then there is a derivation D = (A.. .]~. . if [S]~ in G' plays the role of S in G. z m ~ ) . by Corollary (C). Clearly/)8 corresponds to D. x) in G.. Furthermore.'.]~. But in the grammar thus formed all rules are of the form A --~ aB (where a is I unless the rule was formed by step (i) of the construction) or A --~ a. Qi is of the form [-. .-.. every type 3 grammar is .[S]~. Cmo contains the earliest occurrence of Q. formed by suffixing ~ to each line of 4.~o and ¢/ml the latest occurrence of Q~I. But for every such source. COROLLaR:C. . ~ ) . in Ds.. there is an equivalent source with no identity transitions (el. --+ _> Q~. It is thus a type 3 grammar. . Chomsky and Miller. then G and G' are equivalent if we emend the construction by adding the rule Q1 --~ a wherever there are Q~. z) in G if and only if there is a derivation ([S]1. x[A]~) is a derivation in G'. . we draw the following conclusion. which is the required result in case the shortest recurring Q is of the form [. If G' is formed from G by the construction and D' = ([A]~. Obviously.. where Q~ -.~oAk . . We now construct Ds by replacing 1)7 in D7 by A = (zmoAk~. II.

E. (1958). Recursively enumerable sets of positive integers and their decision problems. Three models for the description of language. "Syntactic Structures.. Discourse analysis: A sample text. KLEENE. In particular. (1957). it would be interesting to determine a necessary and sufficient structural property that marks languages as being of type 2 but not type 3. M.e. Z. Language 28. Theory IT-2. HA~nIS. If L is a type 2 language. Soc. Princeton. Discourse analysis. A. S. New York. eds." McGraw-Hill. This is a task of considerable difficulty for which investigations of the type presented here are a necessary prerequisite.ON CERTAIN FORMAL PROPERTIES OF GRAMMARS 167 non-s. Princeton Univ. and Control 1. (1958). pp. Math. N. DAVIS. Press. No. Consequently: THEOREM 11. it seems particularly important to t r y to arrive at some characterization of the languages of these 2s various types 27 and of the languages that belong to one type but not the next lower type in the classification. REFERENCES CnOMSKY. Z. 1958. 3. Language 33. (1952b). S. 91-112.. 113-124. Unpublished paper. (1957). (1944). Inform. POST. investigations of this kind will be of limited significance for natural languages until the results are extended to transformational grammars. 283-340. C~O~SKY. then it is not a type 3 (finite state) language if and only if all of its grammars are self-embedding. (the lines of its A-derivations are all of the form xB). it does not appear easy to arrive at such a structural characterization theorem for those type 2 languages that are beyond the bounds of type 3 description. 284-316. SCHOTZENBERGSR. Z. New Jersey. Among the m a n y open questions in this area. E. 3-40. CnOMSKY. "Computability and Unsolvability. Cooccurrence and transformation in linguistic structure. N. HARRIS. S. 2s And several other types. 1-30. (1957). McCarthy. Finite state languages. 474-494. HAnnms. for example. C. RECEIVED: October 28. P. Am. (1956). Representation of events in nerve nets. The Hague. (1956). and MILLER. N. ~ As. IRE on Inform. Bull. Trans." Mouton and Co. (1952a). S. Language 28. Even given Theorem 11. 50. Shannon and J. L. . G.M.). the results cited in footnote 4 characterize finite state languages. In "Automata Studies" (C. In particular.

Sign up to vote on this title
UsefulNot useful