TIIKEE MODELS FOR TIE DESCRIPTION OF LANGUAGE*

Nom Chomsky
Department of Modern Languages and Research Laboratory of Electronics
Massachusetts Institute of Technology
Cambridge, Massachusetts

Abstract

We investigate several conceptions of observations, to show how they are interrelated,

linguistic structure to determine whether or and to predict an indefinite number of new
not they can provide simple and sreveallngs phenomena. A mathematical theory has the
grammars that generate all of the sentences additional property that predictions follow
of English and only these. We find that no rigorously from the body of theory. Similarly,
finite-state Markov process that produces a grammar is based on a finite number of observed
symbols with transition from state to state sentences (the linguist’s corpus) and it
can serve as an English grammar. Fnrthenuore, sprojectss this set to an infinite set of
the particular subclass of such processes that grammatical sentences- by establishing general
produce n-order statistical approximations to “laws” (grammatical rnles) framed in terms of
English do not come closer, with increasing n, such bpothetical constructs as the particular
to matching the output of an English grammar. phonemes, words. phrases, and so on, of the
We formalisa the notions of lphrase structures language under analysis. A properly formulated
and show that this gives us a method for grammar should determine unambiguously the set
describing language which is essentially more of grammatical sentences.
powerful, though still representable as a rather
elementary type of finite-state process. Never- General linguistic theory can be viewed as
theless, it is successful only when limited to a a metatheory which is concerned with the problem
small subset of simple sentences. We study the of how to choose such a grammar in the case of
formal properties of a set of grammatical trans- each particular language on the basis of a finite
formations that carry sentences with phra.se corpus of sentences. In particular, it will
structure into new sentences with derived phrase consider and attempt to explicate the relation
structure, showing that transformational grammars between the set of grammatical sentences and the
are processes of the same elementary type as set of observed sentences. In other wards,
phrase-structure grammars; that the grammar Of linguistic theory attempts to explain the ability
English is materially simplifisd if phrase of a speaker to produce and understand- new
structure description is limited to a kernel of sentences, and to reject as ungrammatical other
simple sentences from which all other sentences new sequences, on the basis of his limited
are constructed by repeated transformations; and linguistic experience.
that this view of linguistic structure gives a
certain insight into the use and understanding Suppose that for many languages there are
sf language. certain clear cases of grammatical sentences and
certain clear cases of ungrammatical sequences,
1. Introduction e-e., (1) and (2). respectively, in English.

There are two central problems in the (1) John ate a sandwich
descriptive study of language. One primary (2) Sandwich a ate John.
concern of the linguist is to discover simple
and srevealing* grammars for natural languages. In this case, we can test the adequacy of a
At the same time, by studying the properties of proposed linguistic theory by determining, for
such successful grammars and clarifying the basic each language, whether or not the clear cases
conceptions that underlie them, he hopes to are handled properly by the grammars constrncted
arrive at a general theory of linguistic in accordauce with this theory. For example, if
structure. We shall examine certain features of a large corpus of English does not happen to
these related inquiries. contain either (1) or (2), we ask whether the
grammar that is determined for this corpus will
The grammar of a language can be viewed as project the corpus to include (1) and exclude (21
a theory of the structure of this language. Any Even though such clear cases may provide only a
scientific theory is based on a certain finite weak test of adequacy for the grammar of a given
set of observations and, by establishing general language taken in isolation, they provide a very
laws stated in terms of certain wpothetical strong test for any general linguistic theory and
constructs, it attempts to account for these for the set of grammars to which it leads, since
we insist that in the case of each language the
-.- clear cases be handled properly in a fixed and
4Thi8 work was supported in part by the Army predetermined manner. We can take certain steps
(Signal Corps), the Air Force (Office of Scientific towards the construction of an operational
Research, Air Research and Development Command), characterization of ngrammatical sentences that
and the Navy (Office of Naval Research), and in will provide us with the clear cases required to
part by a grant from Eastman Kodak Company. set the task of linguistics significantly.
Observe, for example. that (1) will be read by an of linguistic structure in terms of the possi-
English speaker with the normal intonation of a bility and complexity of description (questions
sentence of the corpus, while (2) will be read (31, (4)). Then, in 6 6, we shall briefly
with a falling intonation on each word, as will consider the same theories in terms of (5) , and
any sequence of unrelated words. Other dis- shall see that we are Independently led to the
tinguishing criteria of the same sort can be same conclusions as to relative adequacy for the
described. purposes of linguistics.

Before we can hope to provide a satisfactow 2. Finite State Markov Processes.

account of the general relation between observed
sentences and grammatical sentences, we must 2.1 The most elementary grammars which, with
learn a great deal more about the formal proper- a finite amount of apparatus, will generate an
ties of each of these sets. This paper is ccn- Infinite number of sentences, are those based on
cerned with the formal structure of the set of a familiar conception of language as a
grammatical sentences. We shall limit ourselves particularly simple type of information source,
to English, and shall assume intuitive knowledge namely, a finite-state Markov proce6s.l Spe-
of English sentences and nonsentences. We then cifically, we define a finite-state grammar G
ask what sort of linguistic theory is required as as a system with finite number of states
a basis for an English grammar that will describe s o’-I s a set A= aijk 05 i,jlq; 15 klNij for
the set of English sentences in an interestirg 9’ 9 1
each i,j of transition symbols, and a set
and satisfactory manner. 1 of certain pairs of states of Gthat
c={ (s@j))
The first step in the linguistic analysis of are said to be connected. As the system moves
a language is to provide a finite system of from state Si to S it produces a symbol a
j’ i jkcA’
representation for its sentences. We shall assume Suppose that
that this step has been carried out, and we
shall deal with languages only in phonemic (6) Scr,‘..‘Sa
or alphabetic transcription. By a language
then, we shall mean a set (finite or infinib) is a sequenci of stttes of G with al=a,,,41, a&
of sentences, each of finite length, all or each i<m. A0 the aystem smvas
constructed from a finite alphabet of sysbols. @cAi %
i+l)cc f
If A is an alphabet, we shall say that anything from S to s it produces the symbol
formed by concatenating ths symbols of A is a al “i+l
string in A. By a grammar of the langnage L we
mean a device of some sort that produces all of (7) a
the strings that are sentences of L and only ai=i+lk
these. n
for some k(N . Using the arch to signify
No matter how we ultimately decide to OiUi+l
construct linguistic theory, we shall surely concatenation ,2 we sey that the sequence (6)
require that the grammar of any language must be generates all sentences
finite. It follows that only a countable set of
grammars is made available by any linguistic (8) a * na
theory; hence that uncountably many languages, %.kl~ao2$k2 l ** am-lamkm-l
in our general sense, are literally not dsscrlbable
in terms of the conception of linguistic structure for all appropriate choices of k (i.e., for
provided by aw particular theory. Given a k <N ). The language Lo chntaining all and
proposed theory of linguistic structure, then, it I- aiai+1
is always appropriate to ask the following question: only such sentences is called the language
generated by G.
(3) Are there interesting languages that are
simply outside the range of description of the Thus, to produce a sentence of LG we set the
proposed type? system G in the initial state So and we run
through a sequence of connected states. ending
In particular, we shall ask whether English is again with S , and producing one of the associated
such a language. If it is. then the proposed transition s$mbols of A with each transition from
conception of linguistic structure must be judged one ate te to the next. We say that a language L
inad.equate. If the answer to (3) is negative. we is a --finite-state language If L is the set of
go on to ask such questions as the following: sentences genera.ted by some finite-ste.te grammar G.

(4) Can we COnStNCt reasonably simple 2.2. Suppose that we take the set A of transition
grammars for all interesting languages? symbols to be the set of English phonemes. We
(5) Are such grammars n reveali@ In the can attempt to construct a finite state grammar G
sense that the syntactic structure that they which will generate every string of English
exhibit can support semantic analysis, can provide phonemes which is a grammatical sentence of
insight into the use and understanding of langusge, English, and only such strings. It is immediately
etc. 7 evident that ths task of constructing a finite-
state rammar for English can be considerab
We shall first examine various conception6 simpli fi ied if we take A as the set of Englis 1%

ll4
morpheme3 or words. and construct G so that it exemple, the languages k L2, 3 described in
will generate exactly the grammatical stringsof (12) are not describable y any inite-state
these units. We can then complete the grammar
by giving a finite set of rules that give the gTj(i) L contains anb. aoar\ bn b ,
phonemic spelling of each word or morpheme in A
a ananbnbnb,..., and in
each context in which it occurs. We shall general, all sentences consisting
consider briefly the status of such rules in of n occurrences of a followed by
4 4.1 and 4 5.3. exactly n occurrences of b, and only
these ;
Before inquiring directly into the problem (fi) L2 contains afia, bnb, anbnbna.
of constructing a finite-state grammar for
Englieh morpheme or word sequences, let US bnananb, ananbnbnana,. ..,
lnvestlgate the absolute limite’of the set of and in general, all nmlrror-imagen
finite-state languages. Suppose that A is the sentencee consisting of a string X
alphabet of a language L, that al,. . *an are followed by X In reverse, and on4
these ;
eymbole of thir alphabet, and that S=aln.. .nan (W contains ana, b-b, anbnanb,
is a sentence of L. We sey that’s has an (i, j)- ZA
b anbna, ananbnananb, ...,
dependency with respect to L if and only if the and in general, all sentencee con-
following conditions are met: eietlng of a string X followed by
the identical string X. and only
(9)(i) lSi<j<n these.
(ii) -there are symbols bi,b cAwith the In 12, for example, for agy m we can find a
property that S1 is no d asentence of sentence with a dependency set.
L, and S2 is a sentence of L, where Sl Dm=((1,2m),(2.2m-1) ,.., (m,m+l)) .4
is formed from S by replacing the ith
symbol of S (namely, ai) by bi, and S2 2.3. Turning now to English, we find that there
is formed from Sl by replacing the are infinite sets of sentences that have depenw
jth symbol of Sl (namely, aj) by bj. sets with more than any fixed number of terms.
For exemple, let Sl,S2, . . . be declarative eentsrces.
fn other words, S has an (I, j)-depenency with Then the following are all English sentencee:
reepsct to L if replacement of the I symbol a. (13)(i) If Sl, then S2.
1
of S by bi ( bi#ai) requires a corresponding (ii) Either S3, or S4.
(iii) The men who said that S5, is
replacement of the jth symbol aj of S by b j arriving today.
(b #a ) for the resulting string to belong to L. These sentencee have dependencies between sifv-
J il neither”-“or”, ‘mans-siss. But we csn
“then”,
We sey that D- (q,@l) , . . .(a,,B,)}ie a choose Sl, S3, S5 which appear between the inter-
i dependent words, as (ISi), (13ii), or (13111) them-
dependency set for S in L if and only if the selves. Proceeding to construct eentencee in this
following c;;;;;iitions are met: way we arrive at subparts of English with just the
mirror image properties of the languages Ll and L2
(10)(i) For’l<i<m, S hae an (ci,Si)- of (12). Consequently, English falls condition x
dependency with respect to L (11). English is not a finite-state language, aha
we are forced to reject the theory of language
(II) for each l,j, ui< pj under discussion as failing condition (3).
(iii) for each i,j such that i#j. uibj We might avoid this consequence ti an
arbitrary decree that there is a finite upper
aha B,#Bj. limit to sentence length in English. This would
Thus, in a dependency set for S in L every two serve no useful purpose, however. The point is
dependencies are distinct in both terms and each that there are processes of sentence formation
sdetermininge element in S precedes all ede- that this elementary model for language is
terminede elements, where we picture a as intrinsically incapable of handling. If no
determining the choice of a ai finite limit is set for the operation of these
%’ processes, we can prove the literal lnapplica-
Evidently, if S has an m-termed dependency set bility of this model. If the processes have a
in L, at least 2” states are necessary In the limit, then the construction of a finite-state
finite-state grammar that generates the grammar will not be literally impossible (since
langnage L. a list is a trivial finite-state grammar), but
this grammar will be so complex as to be of little
Thie observation enables ue to state a use or interest. Below, we shall study a model
necessary condition for finite-state languages. for grammars that can handle mirror-image lan-
guagee. The extra power of such a model in the
(11) Suppoee that L is a finite-state infinite case is reflected in the fact that it is
bsguaSe. Then there is an m such that no much more ueeful and revealing if an upper limit
sentence S of L has a dependency set of more is set. In general, the aeeumption that a
than m terms in L. are infinite Is made for the purpose of simpli-
With thie condition in mind, we can easily fying the description.5 If a g-r has no
conetNct many nonfinite-state languages. For recursive steps (closed loops, in the model
diacuaaed above) it will be prohibitively complex- that there is apparently no significant approach
it will, in fact, turn out to be little better to the problems of grammar in this direction.
then a list of strings or of morpheme class
sequences in the case of natural languages. If it Botice that although for every n, a process
does have recursive devices, it will produce of n-order approximation can be represented as a
infinitely many sentencee. finite-state Markov process, the converse is not
true. For example, consider the three-state
2.4 Although we have found that no finite-state proceee with (So.S1), (S1,S1) .(S1,So>.
Markov process that produces sentencee from left
to right can serve as an English grammar, we (So,S2) ,(S2,S2) ,(S2,So) as its only connected
might inquire into the possibility of constructing states, and with a, b, a, c, b, c as the reapact-
a sequence of such devices that, in some nontrivial ive transition symbols. This process can be
way. come closer and closer to-matching the outnut represented by the following state diagram:
of-a satisfactory English grammar. Suppose, foI
example, that for fixed n we construct a finite-
state grammar in the following manner: one a tate
of the grammar is associated with each sequence of
English words of length n and the probability that
the word X will be produced when the system is in
the state Si is equal to the conditionalproba-
bility of X, given the sequence of n words which C a
defines S . The output of such grammar is
cuatomariiy called an n+lat order approximation to
English. Evidently, as n increases, the output of This process can2 roauce the sentencee ana,
such grammars will come to look more and mora like anbrra, a”bc’b a, a”b-bOb-a ,..., c-c,
English, since longer and longer sequences bavea cfibnc, chbnb*c cc\b”b’\bnc,..., but not
high probability of being taken directly from the ahbnbnc, chbnb’a, etc. The generated
sample of English in which the probabilities were language has sentencee with dependencies of any
determined. This fact has occasionally led to finite length.
the suggestion that a theory of linguistic
structure might be fashioned on such a model. In 8 2.4 we argued that there is no
significant correlation between order of approxi-
Whatever the other interest of statistical mation and gremmaticalneaa. If we order the
approximation in this sense may be, it is clear strings of a given length in terms of order of
that it can shed no light on the problems of approximation to English, we shall find both
grammar. There is no general relation betweenthe grammatical and ungrammatical strings scattered
frequency of a string (or its component parts) and throughout the list, from top to bottom. Hence
its grammaticalneas. We can see this moat clearly the notion of statistical approximation appears
by considering such strings as to be irrelevant to grammar. In 4 2.3 we pointed
out that a much broader class of processes,
(14) colorleaa green ideaa sleep furiously nemely, all finite-state Markov processes that
produce transition symbols. does not include an
which is a grammatical sentence, even though It is English grammar. That la, if we construct a
fair to assume that no pair of ita words may ever finite-state gremmar that produces only English
have occurred together in the past. Botice that a sentencee, we know that it will fail to produce
speaker of English will read (14) with the an infinite number of these sentences; in par-
ordinary intonation pattern of an English sentence, ticular, it will fail to produce an infinite
while he will read the equally unfamiliar string number of true sentences. false sentencee,
reasonable questions that could be intelligibly
(15) furiously sleep ideas green colorleaa asked, and the like. Below, we shall investigate
e still broader claes of processes that might
with a falling intonation on each word. as In provide us with an English gremmar.
the case of any ungrammatical string. Thus (14)
differs from (15) exactly as (1) differs from (2); 3. Phrase Structure.
our tentative operational criterion for grem-
maticalneas supports our intuitive feeling that 3.1. Customarily, syntactic description is
(14) is a grammatical sentence and that (15) is given in terms of what is called nimmediate
not. We might state the problem of grammar, in constituent analyeie.n In description of this
pert, as that of explaining and reconstructing sort the words of a sentence are grouped into
the ability of an English speaker to recogniae phrases, these are grouped into smaller conati-
(l), (14), etc., as grammatical, while rejecting tuent phrases and so on, ‘until the ultimate
(2) , 05.). etc. But no order of approximation constituents (generally morphemes3) are reached.
model can distinguish (14) from (15) (or an These phrases are then classified as noun
indefinite number of similar pairs). As n phrases (EP), verb phrases (VP), etc. For
increases, an nth order annroximation to Eneliah example, the sentence (17) might be analyaed as
will exclude (as more and-more imnrobabla) an in the accompanying diagram.
ever-increasing number of arammatical sentencee _
while it still-contain2 vast numbers of completely
ungrammatical strings. We are forced to conclude
(17) that are derivable from some L x ,F] grammar,
and we eey that L is a terminal lm if it is
the set of terminal strings from some system
c 2 4’1.
In every Interesting case there will be a
terminal vocabulary VT (VT C VP) that
exactly characteriaer the terminal strings, in
Evidently, description of aentencea in such tenaa the sense that every terminal string la a string
permita considerable simplification over the in VT and no symbol of VT la rewritten in any of
word-by-word model, since the composition of a the rules of F. In such a case we can interpret
complex claaa of expreealona such as XP Fan be the terminal strings as constituting the law
stated just once in the grammar, and this class under analysis (with Y aa its vocabulary), and
can be used as a building block at various the derivations of thege strings as providing
points in the construction of sentencee. We now their phrase structure.
aak what fow of grammar corresponds to this
conception of lingulatic structure. 3.3. Aa a simple example of a system of the form
(18). consider- the foliowing smell part of Pngliah
3.2. A phrase-structure grammar is defined by a grammar:
finite vocabulary (alphabet) Y , a finite aet 2
of initial strings in Y end E finite aet Fof (20)
C : WSentencen#
rules of the form: X +?i, where X and Y axe I: Sentence - l$VP
strings in Y . Path such rule is Interpretedas VP - Verb*NP
the instruct P on: rewrite X as Y. For reaaona NP - the-man, the” book
that will appear directly, we require that In Verb - took
each such [ 2 ,F] grammar Among the derivations from (20) we have, in
particular :
(18) I: : xl,.., cn
(21) Dl: $~%nfit;~~;#
F: ..- - - -..
x1 - y1
.
. #?henmannYerbnXPn#
#“thenmannYerbn thenbookn#
‘rn - ‘rn #“the” man” tookm thenbook” #

D2 : #“Sentence”#
Yi is formed from Xi by the replacement of a WXPnYPn#
single symbol of Xi by some string. Neither #C\thenmannYPP”#
#%renmanr\VerbnXPn#
the replaced symbol nor the replacing string #*the* mann tooknl?Pn#
may be the identity element U of footnote 4. #nthenmanntookn thefibookn#

Given the [ c ,F] grammar (18). we say that: These derivations are evidently equivalent; they
differ only in the order in which the rules are
(19)(i) a atring BRollowa from a string a applied. We can represent this equivalence
if a&xi W end $4*YinW, for graphically by constructing diagrams that
some i I rni7 correapondd, in an obvious wey, to derivations.
a derivation of the string S is a Both Dl and D2 reduce to the diagram:
(ii)
sequence D=(Sl,. . ,St) of atr 1 ngn,
where Sle c and foreach i< t, Si+l (22) #*Sentencefi #

followa from Si: /\

a atring S is derivable from (18) WP VP
(W
If there is a derivation
terms of (18);
of S in
tkL Yed\Ap
(iv) a derivation of St is terminated if
tcaok d>ok
there la no atring that followsfrom The diagram (22) gives the phrase structure of
St; the terminal sentence a the man took the book,”
(4 a string St la a terminal string if just as in (17). In general, given a derivation
It la the last line of a terminated D of a string S, we say that a substring a of S
derivation. is an X if in the diagram corresponding to D, a
is traceable back to a single node, end this node
A derivation is thua roughly analogous toa ta labelled X. Thus given Dl or D , correapond-
proof, with c taken as the axiom system and F ing to (22), we say that sthenmanl 3 is an NP,
as the rule* of Inference. We say that L La “tooknthenbookfi is a VP, nthe”bookll ie an
derivable languape if L la the set of strings HP, athenmann tooknthenbooka is a Sentence.
flmanntook,ll. however, is not a phrase of this
 string at all, eince it Is not traceable back such rules. In 3 3.2 we required that In such
to any node. a rule as (25)) X must be a single symbol. This
ensures that a phrase-structure diagram will be
When we attempt t0 COnStNCt the Simplest constructible from any derivation. The grammar
poesible [ c ,F] grammar for English we find that can also be simplified very greatly If we order
certain sentences automaticallv receive non- the rules end require that they be applied in
equivalent derivations. Along-with (20), tti sequence (beginning again with the first rule after
grammar of English will certainly have to ccmtain applying the final rule of the sequence), and if
such rules as we distinguish between obligatox rules which
must be applied when we reach them inthe sequence
(23) Verb - are n f lying
and o tional rules which msy or may not be
Verb - are
HP - they applh se revisions do not modify the
generative power of the grammar, although they
HP -planes
lead to co-nslderable simplification.
IPP-flying”planes
in order to account for such sentences as “they It seems ‘reasonable to require for
are flying - a plane” (IIT-Verb-BP) , a ( flying) significance some guarentee that the grammar will
planes - are - noisy” (m-Verb-Ad jet tive) , etc. actually generate a large number of sentences in
Hut this set of rules provides us with two ZYJDD a limited amount of time; more specifically, that
equivalent derivations of the sentence “they are it be impossible to run through the sequence of
flying planeenS reducing to the diagrams: rules vacuous4 (applying no role) unless the
last line of the derivation under construction is
(24,) #?eyty “# a terminal string. We can meet this requirement
r .
by posing certain conditions on the occurrence of
obligatory rules in the sequence of rules. - - We
define a proper gremmar as a system [ I; ,Q],
the VerF!\ f 2-l where 2 is a set of initial strings and Q a
sequence of rules X - T as in (lay, with the
I
are fly i<r>anes ‘byl&z>ing p1r.m. additional conditiod that for each I there must
be at least one j such that X =X and X -f Is
an obligatory rule. Thus, ea=!h,jeft-ha& te& of
Hence this sentence will have two phrase
the rules of (18) must appear in at least one
structures assigned to It; it can be analyaed as obligatory rule.
“they - are - flying plane@ or “they - are flying This is the weakest simple
condition that guarantees that a nonterminated
- planes.” And in fact, this sentence ia
derivation must advance at least one step every
ambiguous in just this way ; we can understand it time we run through the rules. It provides that
as meaning that “those specks on the horiaon - if Xi can be rewritten as one of yi ,. . ,Ti
are - flying planes” or othose pilots -are flying
When the simplest grammar automatio- 1 k
- planes.” then at least one of these rewritings must take
ally provides nonequivalent dsrivations for some However,
place. roper grammars are essentially
sentence, we say that we have a case of different from [ E .F] gramars.
constructional homovmitg and we can suggest I& D(G) be
the set of derivations producible from a phrase
this formal property as d erplenation for the
semantic ambiguity of the sentence In question.
In 41 we posed the requirement that grammare
offer insight into the use and understanding of a proper grammar Then
language (cf.(5)). One wey to test the adequacy
of a grmmar is by determining whether or not are incomparable; i.e.,
the cases of constructional homomlty are
actually cases of semantic emblgulty, as In (24)
We return to this important problem in # 6.
That Is, there are systems of phrase structure
In (20)-(24) the element # Indicated that can be described by [ 2 ,F] gremmars but not
sentence (later, word) boundary. It ten be by proper grammars, and others that can be
taken as an element of the terminal vocabulary described by proper grammars but not by [ z ,F]
VT discussed in the final paragraph of l 3.2. grammars.
3.5. We have defined three types of language:
3.4. These segments of English grammar are much finite-state languages (in $2.1). derivable and
oversimpllfied In several respects. For one terminal laugoages (in $3.2). These are related
thing, each rule of (20) and (23) has only a in the following way:
single symbol on the left, although we placed no
such limitation on [ 1 ,F] grammars in 0 3.2. (27)(i) every finite-state language Is a
A rule of the form terminal language, but not Eoniersely;
(ii> every derivable lanme is a terminal
(25) Z-X-W-ZhThW language,.but not converiel;;
(iii) there are derivable, nonfinite-state
indicates that X can be rewritten as I only in languages and finite-state, nonderivable
the context Z--W. It can easily be shown that languages.
the grammar will be much simplified if we permit tippose that LG i8 a finite-state language
with the finite-state grammar G as in 8 2.1. The major import of Theorem (27) is that
We construct a [ c ,F] grammar in the following description in terms of phrase structure is
manner: ItI ={So) ; F contains a. rule of the form essentially more powerful (not just simpler) than
(281) for each 1,j.k such that (Si, s j ICC, j#O, description in terms of the finite-state grammars
that produce sentences from left to right. In
and k < H, ,; F contains a rule of the form (2811) # 2.3 we found that Xnglish is literally beyond
for each ;:‘k such that (si,s,)cC and k 5 Bio. the bounds of these grammars because of mirror-
Image properties that it shares with Ll and L2
(28) (1) “S of (12). We have just seen, however, that Ll
’ i -&ijk j
(if) ’ i -aiok is a terminal language and the same is true of
Hence, the considerations that led us to
Lz-
reject the finite-state model do not similarly
Clearly, the terminal language from this [x ,F]
grammar will be.- exactly LG. establishing the lead us to reject the more powerful phrase-
.
first part of (271). - StNCtW'S model,

In f 2.2 we found that Ll, L2 and 5 of Wote that the latter is more abstract than
the finite-state model in the sense that symbols
(12) were not finite-state langnsges. Ll and Lzl that are not included in the vocabulary of a
however, are terminal languages. For LIB e-e., language enter into the description of this
language. In the terms of 4 3.2, V P properly
we have the [ I: ,F] grammar
includes VT. Thus in the cs.se of (29) , we
(29) x : Z c\b describe Ll in terms of an element Z which is not
F: Z-a ; and in .the case of (20)-(24). we introduce
Z -anZhb in L1 symbols as Sentence, NP, VP, etc., which are
such
not words of English. into the description of
This establishes (271). English structure.
Suppose that L4 is a derivable language with 3.6. We can interpret a [ c ,F] grammar of the
form (18) as 8. rather elementary finite-state
process in the following way. Consider a system
that has a finite number of states So,..,S .
a -cb i, where the bins are not in VP and are 9
i When in state So, it can produce any cP the
all distinct. Then this new grammar gives a
terminal language which is simply a notational strings of X , thereby moving into a new state.
variant of L . Thus every derivable language Its state at aq point is determined by the sub-
is also term f nal. set of elements of Xl,..,Xm contained as sub-
As an example of a terminal, nonderivable strings in the last produced string, and it moves
language cohsider the languege L5 containing just to a new state by applying one of the rules to
the strings this string, thus producing &new string. The
system returns to state S with the production
(30) anb, chaObnd, chcnanbhdhd, of a terminal string. Th& system thus produces
0-c “c*a-b^d^d”d,... derivations, in the sense of 5 3.2. The process
is determined at any point by its present state
An infinite derivable language must contain an and by the last string that has been produced,
infinite set of strings that can be arranged in a and there is a.finite upper bound on the amount
sequence Sl,S2,... in such a way that for saeo~ of inspection of this string that is necessary
before the process can continue, producing a new
rule X-T, Si follows from Si-1 by application string that differs in one of a finite number of
of this rule, for each i >l. And Y in this rule ways from its last output.
must be formed from X by replacement of a alnple
symbol of X by a string (cf. (18)). This is
evidently impossible in the case of L . This It is not difficult to construct language8
language is, however, the terminal la d guage given that are beyond the range of description of
by the following grammar: [ x ,F] grammars. In fact, the language L3 of
(12111) is evidently not a terminal language. I
do not know whether English is actually a terminal
language or whether there are other actual
Z -cnZ”d languages that are literally beyond the bounds of
phrase structure description. Hence I see no wsy
An example of a finite-sta.te, nonderivable to disqualify this theory of linguistic structure
langoage is the language L6 containing all and on the basis of consideration (3). When we turn
only the strings consisting of 2n or 3x1 to the question of the complexity of description
occurrences of a, for nP1.2,. . . . Language Ll (cf. (4)), however, we find that there are ample
of (12) is a derivable, nonfinite-state language. grounds for the conclusion that this theory of
with the Initial string anb and the rule: linguistic structure is fundamentally inadequate.
a-b -ra”a-b* b. We shall now investigate a few of the problems
that arise when we attempt to extend (20) to a ,Applying this rule to each of the three Afnv
full-scale grammar of English. sequences in the last line of (33), we derive
4. Inadeauacies of Phrase-Structure Grammar (35) #rrthenmannhavenpastn #%enenn #^
takening” #nthenbook”#.
4.1. In (20) we considered on4 one wey of
developing theelement Verb, namely, as “tookn. In the first paragraph of 8 2.2 we mentioned
But even with the verb stem fixed there are that a grammar will contain a set of rules (called
a great many other forms that could appear in morphophonemic rules) which convert strings of
the context sthe man -- the book,s e.g., stakeq” morphemes into strings of phonemes. In the
“has taken,” shas been taking,” sfs teking,s morphophonemics of English, we shall have such
“has been taken,” swill be takiog,n and se 011. rules as the following (we use conventional,
A direct description of this set of elements rather than phonemic orthography) :
would be fairly complex, because of the heavy
dependencies among them (e.g., shas taken” but (36) havenpast - had
not “has taking.5 Ye being taken5 but not “1s benen - been
being taking,s etc.). We can, in fact, give a take-ing - taking
very simple analysis of “Verb” as a sequence of will” past - would
independent elements, but only by selecting as can~past -) could
elements certain discontinuous strings. For M*present - M
example, in the phrase “has been takiDg5 we can walk-past - walked
separate out the discontinuous elements shas..er$ takenpast - took
Nba..ing,5 and stakes, and we can then say that etc.
these eleeents combine freely. Following this
course systematically, we replace the last rule Applying the morphophonemic rules to (35) we
in (20) by derive the sentence:

(32) (i) Verb -AuxillarynV (37) the man had been taking the book.
(ii> V-take, eat,...
(iii) Auxiliary --?!8Mz Qd (be-w Similarly, with one major exception to be
discussed below (and several minor ones that we
( -1 M-will*, can, shall, mey, must shall overlook here), the rules (32), (34) will
(v) C - past, present give all the other forms of the verb in
declarative sentences, and only theee forms.
The notations in (32111) are to be inter-
preted .a5 follows: in developing sAuxiliar+ This very simple ana4eis, however, goes
in a derivation we must choose the unparenthe- beyond the bounds of [ x .F] grammars in several
sised element C, and we may choose sero or more respects. The rule (34). although it is quite
of the parenthesised elements, in the order simple, cannot be incorporated within a [ 2 ,F]
given. Thus, in continuing the derivation grammar, which has no place for discontinuous
D of (21) below line five, we might proceed elements. Furthermore, to apply the rule (34)
a t follows: to the last line of (33) we must know that stakes
is a V, hence, a Y. In other words, in order to
(33)#nthenmannVerbn thenbook ^# apply this rule It is necessary to inspect more
[from Dl of (21)] than just the string to which the rule applies;
#^ the;(~~,Auxiliary” V” the* book * # it is necessary to know some of the constituent
structure of this string, or equivalently
(cf. 3 3.3), t 0 i nspect certain earlier lines in
#~thenmannAuxiliaryntakenthe”bookn# Its derivation. Since (34) requires knowledge of
1:(3211)1 the ‘history of derivation’ of a string, it violates
#^then mann C” have” enc\ hen ing n take n the elementary property of [ 2 ,F] grammars
discussed in f 3.6.
thenbookn #
C(32fii). choosing the elements C,
4.2. The fact that this simple analysis of the
havenen, and bening] verb phrase as a sequence of independently chosen
#nthenman*past”5ha~e’5en”beningntake’5 units goes beyond the bounds of-[ c .F] grammars,
thenbook” # suggests that such grammars are too limited to
C(32dl give a true picture of linguistic structure.
Further study of the verb phrase lends additional
Suppose that we define the class Af as containing support to this conclusion. There is one major
the-afflxes *ens, s I@, and the C’s; and the limitation on the independence of the elements
class v as includine all V's.. MIS. shaves,and she .m introduced in (32). If we choose an Intransitive
We can then convert-the last.llne-of (33) into a verb (e.g., ncome,n 5occur.s etc.) as V in (32).
properly ordered sequence of morphemes by the we cannot select be-en as an auxiliary. We can-
following rule : not have such phrases as “John has been come,s
“John is occurred ,” and the like. Furthermore,
(34) Afnv -vnAfn # the element beAen cannot be chosen independently
of the context of the phrase sVerb.n If we have

320
 the element “Verb” in the context *the man -- the 5.Transformational Grammar.
food,” we are constrained not to select benen in 5..1. Each grammatical transformation T will
applying (32). although we are free to choose any essentially be a rule that converts every sentence
other element of (32). That is, we can have “‘the with a given constituent structure into a new
man is eating the food,s “the man would have been sentence with derived constituent structure. The
eating the food,’ etc., but not n the man is eaten transform and its derived structure must be related
the food,* “the man would have been eaten the food,* in e fixed and constant wey to the structure of
etc. On the other hand, if the context of the the transformed string, for each T. We can
phrase “Verb” Is, e.g., sthe food - by the man,” characterize T by stating, in StNCtUrd terms,
we are required to select benen. We can have the
_-- domain
-.--. of strings to which it applies and the
s the food is eaten by the man,” but not sthe food cmee that it effeke on any such string.
is eating by the man,” etc. In short, we find that Let us suppose in the following discussion
the element be-en enters into a detailed network that ve have a [ H ,P] grammar with a vocabulary
of restrictions which distinguish it from all the VP and a terminal vocabulary VT C VP, as in 4 3.2.
other elements introduced in the analysis of “Verb”
in (32). This complex and unique behavior of In 5 3.3 we showed that a [ 2 ,F] grammar
benen auggests that it would be desirable to permits the derivation of terminal strings, and we
exclude it from (32) and to introduce passives into pointed out that In general a given terminal string
the grammar in some other way. will have several equivalent derivations. Two
derivations were said to be equivalent if they 9
There is, in fact, a very simple way to reduce to the same diagram of the form (22), etc.
incorporate sentences with benen (i.e., passives) Suppose that Dl, . . , Dn constitute a maximal set
into the grammar. notice that for every active of equivalent derivations of a terminal string
sentence such as sthe man ate the foods we have a S. Then we define a phrase marker of S as the set
corresponding passive “the food was eaten by the of strings that occur as lines in the derivations
man” and conversely. Suppose then that we drop the A string will have than one phrase more
D1.. l ,Dn.
element be-en from (32111). and then add to the
grammar the following rule: marker if and only if it has nonequivalent
derivations (cf. (24)).

(38) If S is a sentence of the form Npl- Suppose that K is a phrase marker of S. We

Auxiliary-V-KP2, then the corresponding string sey that
of the form lW~Auxiliarynbe*en-V-by”~l is
also a sentence. (39) (S,K) is analyzable into $,..,X )lf
and onls if there are atrinse a . ..*.a such ?hat
For example, if “the man - past - eat the ii) S”sl~...~sn - I- - n
foods (KPl-Auxiliary-V-NP,) is a sentence, then
(ii) for each iln, K contains the string
sthe food - past be en - eat - by the mans (WP2- al- . .fi 6i-l- xi- si+l- . .* sin
Auxiliarynbenen-V-bynHP1) is also a sentence.
Rules (34) and (36) would convert the first of
these into “the man ate the food11 and the (40) In this case, si is an Xi in S with
second into “the food was eaten by the man.s respect to K.10
The edvantages of this analysis or passives The relation defined ID (40) Is exactly the
are unmistakable. Since the element be-en has relation “is aa defined in 5 3.3; i.e., si is an
been dropped from (32) it is no longer necessary to X in the sense of (40) if and-only if a Is a
qualify (32) with the complex of restrictions suib string of S which is traceable beck ti a single
discussed above. ’ The fact that be “en can occur node of the diagram of the form (22). etc., end
only with transitive verbs, that it Is excluded in this node is labelled Xi.
the context *the man -- the foods and that it is
required In the context sthe food -- by the man,” The notion of analysability defined above
is now, in each case, an automatic consequence of allows us to specify precisely the domain of
the analysis we have just given. application of any transformation. We associate
A rule of the form (38), however, is well with each transformation a restricting class B
beyond the limits of phrase-structure grammars. defined as follows:
Like (34)) it rearranges the elements of the string
to which it applies, end it requires considerable (41) B is a restricting class if and only if
information about the constituent structure of this for some r.m, B Is the set of Gnces:
string. When we carry the detailed study of English
syntax further, we find that there are many other
cases in which the grammar can be simplified if the
[ 2 ,P] system is supplemented by ralee of the same
general form as (38). Let us cell each such rule a
grammatical transformation. As our third model for
the description of linguistic structure, we now where Xi is a string in the vocabulary VP, for each
consider briefly the formal properties of a trans-
formational grammar that can be adjoined to the 1.J. We then say that a string S with the phrase
[ C ,F] grammar of phrase structure.8 marker K belongs to the domain of the transformaUon

I21
T if the restricting class II associated with T tpol J2 WY3;y3 J4) = y3
contains a sequence ($, . . ,X$ into which (S,K) is tp(Y1. a. ,Y4;Y4> = by”Y1
The domain of a tra sformation is
% % “%*of ordered pairs (S,Kf of a string S t,(Yl,.. ,Yn;Yn ,.., Yr) = Yn for all x&r++.
and a phrase marker K of S. A transformation rosy
be applicable to S with one phrase marker, but not The derived transformation tIf thus has the follow-
with a second phrase marker, in the case of a string ing effect:
S with ambiguous constituent structure. (WJ(U t$Y,,.. ,Y,) = Y1 - Y2%inen - I3 - l$Yl
In particular, the paSBiVe transformation (ii> tG( tb”man, past, eat, then food) =
described in (38) has associated with it a the”food - past” be” en - eat - bynthen
restricting class Il, Containing just one sequence: man.
The rule8 (34),(36) carry the right-hand side of
(42) BP= { (&. Auxiliary, (4&i) into “the food was eaten by the men,@ just
as they carry (43) into the corresponding active
This transformation can thus be applied to eny sthe man ate the food.”
string that is analyzable into an SF followed by The pair (R ,t ) as in (42),(47) completely
an Auxiliary followed by a V followed by an HP. characterizes thg p&.eive transformation as
For example, it can be applied to the string (43) described in (38). B tells us to which strings
analyzed into substrings sl,..,s4 in accordance this transforxation applies (given the phrase
with the dashes. markers of these strings) and how to subdivide
these strings in order to apply the transformation,
(43) the man - past - eat - the food. and t tells us what structural change to effect
on thg subdivided string.
5.2. In this way, we can describe in structural
texms the set of strings (with phrase markers) to A grammatical transformation is specified
which eny transformation applies. We must now completely by a restricting class R and an
specify the structural change that a transformation elementary transformation t, each of which is
effects on any string in its domain. An element- finitely characterizable, as in the case of the
transformation t is defined by the following paesive. It is not difficult to define rigorously
property : the manner of this specification, along the lines
she tched above. To complete the development of
(44) for each pair of integers n,r (nsr), transformational grammar it is necessary to show
there is a unique sequence of integers (ao,al,. . ,cg) how a transformation automatically assigns a
derived phrase marker to each transform and to
and a unique sequence of strings in VP (Z,,. . ,Zk+l) generalize to transformations on sets of strings.
ruchthat (i)ao=O; k>O;llajl r for 15 j<k;Yoi[J1l (These and related topiqs are treated in reference
[3].) A transformation will then carry a string S
(ii) for each Tl,. . ,Yr, with a phrase marker K (or a Bet of such pairs)
into a string St with a derived phrase marker Kc.
t(Y1,..,Yn;Yu,..,Yr)=Ya* znY”***Y* cYn
al 2 a2 0 5.3. From these considers.tions we are led to a
Thus t can be understood as converting the picture of gl’emmars as pOBBeSSing a t,riwrtitS
occurrence of Yn in the context structure. Corresponding to the phrase structure
WbBiS We have a sequence of rules of the form
(45) Y1
A
. . - Ynwl* --^Yn+l^ . .hYr X-Y, e.g.. (20). (23). (32). Following this we
have a se Pence of transformational rules such a*
(34) and ?38). Finally, we have a sequence of
into a certain string
n Zlh . .n Y n Zk+l Y morphophonemic rules such as (36). sgain of the
aO % form X--Y. To generate a sentence from such a
which is unique, given the sequence of terms grammar we construct an extended derivation
0 Y ) into which Ylh ..nY, is subdivided. beginning with an initial string of the phrase
t bbliier the string Y -. . -Yr into a new string structure grammar, e.g., #^Sentence^#, as in
wl-. . qr which is rela t ed in a fixed way to (20). We then rnn through the rules of phrase
Y1-. .-Yr. More precisely, we associate with t the structure, producing a terminal string. We then
derived transformation t*: apply certain transformations, giving a string of
morphemes in the correct order, perhaps quite a
(46) t* is the derived transformation of t if different string from the original terminal string
and only if for all Yl,.., Y,, t*(Yl,.., Yr)=Wln ..^s, Application of the morphophonemic rules Converts
where W,==t(Yl, . . ,Yn;Tn,. . ,Y,) for each n L r. this into a string of phonemes. We might run
through the phrase structure grammar Several times
We now associate with each transformation T and then apply a generalized transformation to the
an elementary transformation t. For example, with resulting set of terminal strings.
the passive transformation (38) we associate the In ! 3.4 we noted that it is advantageous to
elementary transformation t defined es follows: order the rules of phrase structure into a
P eequence, and to distinguish obligatory from
optional rules. The aeme is true of the trans-
(47) tpul;Y1’ ‘Y4) = ‘4 * l

formational part of the grammar. In Q4 we dis-

tp(Yl,Y2;Y2,Y3,Y4) = YZh be-en cussed the transformation (34). which converts a

I.22
sequence affix-verb into the sequence verb-effix, in terms of such essentially formal criteria. as
and the passive transformation (38). Notice that simplicity. In d 1 we SIIgg85ted that there are
(34) must be applied in every extended derivation, Other relevant consideration5 of adequacy for
or the result will not be a grammatical sentence. such theories. We can ask (cf.(S)) whether or
Rule (3b), then, is sn obligatory transformation. not the syntactic structure revealed by these
The passive transformation, however, may or may theories provides insight into the use and under-
not be applied; either way we have a sentence. The standing of language. We can barely touch on
passive is thus an optional transformation. This this problem here, but even this brief discussion
distinction between optional and obligatory trans- will suggest that this criterion provides the
formation5 leads us to distinguish between two same order of reletive adequacy for the three
classes of sentences of the language. We have, on models we have considered.
the one hs.nd, a kernel of basic sentences that are
derived from th8xnal strings Of the phrase- If the grammar of a langoege is to provide
structure grammar by application of Only insight into the way the language is Understood,
obligatory transformations. We then have a set of it mUst be true, in particular, that if a sentence
derived 58nt8nC85 that are generated by applying Is ambiguous (Understood in more than one way),
optional transformations to the strings underlying then this sentence is provided with alternative
kern81 Sentences. analyses by the grammar. In other words, if a
When we actually carry out a detailed study certain sentence S is ambiguous, we can test the
of English StrUcture, we find that the grammar can adequacy of a given linguistic theory by asking
be greatly simplified if we limit the kernel to a whether or not the simplest grammar constructible
very small eet of simple, active, declarative in terms of this theory for the languege in
sentence5 (in fact, probably a finite set) such as question automatically provide6 distinct ways of
"the man ate the food,5 etc. We then derive generating the sentence S. It is instructive to
questions, paseivee, sentences with conjunction, compare the Markov prOC8Sf3, phrase-structure, and
sentence5 with compound noun phrases (e.g., transformational models in the light of this test.
"proving that thsorem was difficult.5 with the NP
aproving that theoremn),li! etc., by transformation. In 53.3 we pointed out that the simplest
Since the result of a transformation is a sentence [ x ,F] grsmmsr for English happens to prOVid8
with derived constituent etructure, transfofiaaticns nonequivalent derivations for the sentence nthey
can be compounded, ana we can form question5 from are flying planes,H which is, in fact, ambiguous.
~SSiV8S (e.g.) "was the food eaten by the man"), Thie reasoning does not appear to carry over for
etc. The actual 58nt8nc85 of real life are finite-state grammars, however. That IS, there
usually not kernel sentences, but rather is no obvious motivation for assigning two
complicated transforms of these. We find, however, different paths to this ambigaous sentence in any
that the tr5nsformations are, by and large, meanin& finite-state grammar that might be proposed for
preserving, so that we can view the kernel a part of English. Such examples of construction-
sentences Underlying a given sentence as being, in al homonymity (there are many others) constitute
some sense, the elementary 5content elements" in independent evidence for the superiority of th8
terms of which the actual transform i~%nderstood.~ phrase-etructure model over finite-state grammars.
We discuss this problem briefly in 4 6, more
extensively in references Cl], [2]. Ptrrther investigation of English brings to
IZI 83.6 W8 pointed Out that a grammar of light examples that are not easily explained in
phrase strUCtur8 is a rather elementary type of terms of phrase structure. Consider the phrase
finite-State process that is determined at each
point by its present state and a bounded amount, of (49) the shooting of the hunters.
its last output. W8 discovered in $! 4 that this
limitation is too severe, and that the grammar can
b8 simplified by adding tran5formational l’,l18S We can understand this phrase with nhuntersII as
that take into account a certain amount of the subject, analogously to (50), or as the
constituent structure (i.e., a certain history of object; analogously to (51).
derivation). Hovever, each transformation is- still
finitely charactarieable (cf. $5 5.1-2). and th8 (50) the growling of lions
finite restrlctin class (41) aSSOCiat8d with a
transformation in f icates how mUch information
about a string is needed in order to apply this (51) the raising of flowers.
transformation. The grammar can therefore still
be regarded
- as an elementary finite-state process Phrases (50) and (51)) however, are not similarly
Of the type Corresponding t0 phrase strUCtUr8. ambiguous. Yet in terms of phrase structure, each
There is still a bound, for each grammar, on how of the58 phrases is represented as: the - V*ing-
nruch of the past output must be inspected in order of9P.
for the process of derivation to Continue, even
though more than just the last output (the last Careful analysis of English show5 that we can
line of the derivation) must be known. simplify the grammar if we strike the phrases
(49)-(51) out of the kern81 and reintroduce them
6. Explanatory Power of Linguistic Theories transformetionally by a transformation Tl that
carries such sentences as "lions growl" into (50).
We have thus far COnSid8red the relative and a transformation T2 that carries such sentences
adequacy of theories of linguistic structure only

I.23
as "they raise flowerss into (51). T1 and T2 will replace "is" by nare 'I in (13iii), and choose
be similar to the nominalizing transformation S5 of any required length).
described in fn.12, when they are correctly
But both shunters shootH and zthey 7.2 or W may be the identity element U (cf..fn.4)
constructed. in this case. Note that since we limited (18)
shoot the hunterss are kernel sentences; and
so as to exclude U from figuring significantlv
application of Tl to the former and T2 to the on either the right- @r the left-hand side
latter yields the result (49). Hence (49) has of a rule of B, and since we reouired that
two distinct transformational origins. It is a only a single symbol of the left-hand side
case of constructional homonymity on the trans- msy be-replaced in any rule, it follows that
formational level. The ambiguity of the grammat- Yi must be at least as long as Xi. Thus we
ical relation in (49) is a consequence of the fact
have a simple decision procedure for deriv-
that the relation of "shoot" to lfhuntersz
differs in the two underlying kernel sentences. ability and terminality in the sense of
We do not have this smbiguity in the case of (50). ( 19iii) , (19p) .
(51)s since neither "they growl lions" nor
8. See [3] for a detailed development of an
"flowers raise" is a grammatical kernel sentence.
algebra of transformations for linguistic
There are many other examples Of the same description and an account of transforma-
tional grammar. For further application of
m-ma1 kind (cf. [11,[21), and to my mind, they this type of description to linguistic
provide quit6 convincing evidence not only for
material. see Cl], [2], and from a somewhat
the greater adequacy of the transformational
different point of view, ['cl.
conception Of linguistic structure, but also for
the view expressed in $5.4 that transformational
analysis enables us to reduce partially the 9. It is not difficult to give a rigorous
definition of the equivalence relation in
pro3lem of explaining howwe understand a sentence
to that of explaining how we understand a kernel question, though this is fairly tedious.
sentence.
10. The notion "is an should actually be
In summary, then, we picture a language as relatioized further to a given occurrence
having a small, possibly finite kern& of basic of si in S. We can define an occurrence of
sentences with phrase structure in the sense of ai in 5 as an ordered pair (zi,X), where X
331 along with a set of transformations which
is an initial substring of.S, and si is a
Can be applied to kernel sentences or to earlier
transforms to produce new and more complicated final substring of X. Cf. 651, p.297.
sentences from elementary components. We have 11. Where U is the identity, as in fn. 4.
seen certain indications that this approach may
enable us to reduce the immense complexity of 12. Notice that this sentence requires a
actual language to manageable proportions and; in generalized transformation that operates on
addition, that it may provide considerable insight a pair of strings with their phrase markers.
into the actual use and understanding of language. Thus we have a transformation that converts
Footnotes Sl,S2 of the forms NP-VPl, it-VP2, respect-

1. Cf. c73. Finite-state grammars can be ivekf , into the string: ingnVPl - VP2. It
represented graphically by state diagrams, as converts Sl= "they - prove that theorem",
in C7]. P.13f. s2= "it - was difficultI' into "ing prove
2. See [6], Appendix 2, for an axiomatization Of that theorem - was difficult," which by (34)
concatenation algebras. becomes sproving that theorem was difficu1t.z
Cf. il], [3] for details.
3. By 'morphemes' we refer to the smallest Bibliogranny
grammatically functioning elements of the 611 Chomsky, N., The Logical Structure of
language, e.g., "boys, "run", "ing" in Linguistic Theory (mimeographed).
"mining" , flsll in "bookss, etc. c21 Chomsky, N., Syntactic Structures, to be
published by Mouton & Co.. 'S-Gravebe. - ~
4. In the case of Ll, bJ of (9ii) can be taken Gtherlands;
as an identity element U which has the 633 Chomskv. M.. Transformational Analyzis,Ph. D.
Dissertation, University of Pennsylvania,
property that for all X, UAX=X"U=X. Then
June, 1955.
Dm will also be a dependency set for a Harris ,Z.S., Discourse Analysis, Language
c41
sentence of length 2m in Ll. 2891 (1952).
c51 Quine.
- . W.V.. Mathematical Logic, revised
5. Note that a grammar must reflect and explain edition,HarvardUniversity Press., Cambridge,
the ability of a speaker to produce and underc- 1951.
stand new sentences which msy be mush longer C61 Rosenbloom, P.. Elements of Mathematical
than any he has previously heard. Logic, Dover, New York, 1950
6. Tnus we can always fina sequences of n+l c73 Shannon & Weaver, The Mathematical Theory of
words whose first n words and last n words Communication, University of Illinois Press,
msy occur, but not in the same sentence (e.g. Urbana, 1949.