This action might not be possible to undo. Are you sure you want to continue?
https://www.scribd.com/doc/55534562/AnIntroductiontoFormalLanguagesandAutomata
10/06/2011
text
original
An Introduction to Formal Languages andAutomata
Third Edition
PeterLinz
University Californiaat Davis of
filru;;;:
IONES AND BARTLETT P,UBLISHERS
.BOSTON Stdlnry, Massnclrrsrtr TORONT'O LONDON SINGAPORE
6l;l .rf;ti.
etig*
t' o
dtry'l,tlti,tFI, hgfryfl6a
';;'ut: \ n qf I" A\ ,r'f7 lA ,obi
Workl Headquerters Iones and Bartlett Puhlishers
40 Tall PineDrive MA Sudbury, 01776 9784435000 info@jbpub.corn www.jbpub.com
Jonesand Bartlett Publishers Canada 2406 Nikanna Road Mississauga, ON L5C 2W6 CANADA
Jonesand BarJlettPublishers International Barb House,Barb Mews London W6 7PA UK
Inc. Copyright O 2001 by Jonesand Bartlett Publishers, or All rights reserved.No part of the materialprotectedby this copyrightnotice may be reproduced recording,or any infotmation including photocopying, or mechanical, utilized in any fonn, elcctronic storage retrievalsy$tem,without written permissionf'romthe copyright owner. or Library of Congress CataloginginPuhtication Data Linz, Peter. / and automata PeterLinz'3'd cd An introductionto formal languages p. cm. and Includesbi hliographicalref'erences index. G' A
26+,3 . L5 4 LooI
rsBN0763714224
l. Formal languages. 2. Machine theory. l. Title. QA267.3.Ls6 2000 5  1.3dc2l Chief ExecutiveOfficer: Clayton Jones Chief OperatingOfficer: Don W. Jones,Jr. and Publisher: Tom Manning ExecutiveVicc President V.P., ManagingEditor: Judith H. Hauck V.P.. Collese Editorial Director: Brian L. McKean V.P;, Dcsigir'and"Prodgction: \ Anne $pencer arket+rgiFauI Shefiardson V. P., Salcs anit*ffr . V. P., Man uf aeturingjandilnhrr'trrry dpntrol : ThereseBriiucr SeniorAgquisitionsEditor; Michacl $tranz f)evelopment and Product Managcr: f,lny Rose Markcting Director: Jennifer.Iacobson ect Production CoordinationI Tri{ litrm Pt'oj M anagcment Cover Design; Night & Day Design Composition:NortheastCompositors Printing and Binding: Courier Westford Cover printing: John Pow Cotnpany,Inc.
;..#F*F*.,.
00062546
Covel Imasc O Jim Wehtie This book was typesetin Texturcs2. I on a MacintoshG4. The fbnt families usedwere Computer The first printing was printed on 50 lb. Decision94 Opaque. Modern, Optima, and F'utura.
States Arnerica _. '_ of Printed the United in 04030201 lo987654321
I
L
lch4
I ,r./1,il.t!\
his book is designed for an introductory course orr forrnir,l larrguages, autornatir, txlmputability, and rclated matters. These topics form a major part of whnt is known as tht: theory of cornputation. A course on this strbitx:t rnatter is now stir,nda,rdin the comprrter science curriculurn ancl is oftrlrr ta,ught fairly early irr the prograrn. Hence, the Jrrospective audience for this book consists prirnrr,rily of sophomores and juniors rnirjrlring in computer scicntxlor computer errgirrwring. Prerequisites for the material in this book are a knowledge of sorne higherlevel prograrnrning la,nguage (cornmonly C, C++, or .Iava) and fatrrilinritv with ihe furrdarn<lnta,ls data structures and algoriihms. A colrr$e of in discretc mathematics that irx:hrcles theory, furrctions, relations, logic, set and elernerrtsof mathematical reasorringis essential. Such a corlrse is part of the standard introductory computer sciencecurriculum. The study of the theory of cornputa.tionhas several purposc$, most imprortantly (1) to fa,miliarizestuderrts with the fbundations and principles of computer sciettce,(2) to teach tnaterial that is useful in subsequerrt colrrres! rrnd (3) to strengtlrcrr utudents' ability tu t:ilrry out formal and rigorous rrrirthematical argurnerrts. The presentatiorr I ha,ve chosen for this text fa
III
lv
F RHrAciE
vors the first two purpose$r although I would rr.rguethat it a,lsoserves the give strrdcrrtsinsight into the material, thircl. To prt:sent ideas clenrly arrd 1,o rnotivation and ilhrstration of idcir.sthrough ex* tlte text stressesintuitive a,m1llcs.When there is ir choice, I prefcr arguments thtr,t a,reeasily grer,sptxl and elegant brrt rlifficult in concellt. I state clefinito thosr.'tlnt are concisr,l tiorrs ancl theorems llrecisely and givt: the tnotiva,tion ftlr proofs, brrt tlf'tt:rr k:ave out the rorrtirre and tediorrs rlctails. I believe tlrrr.tthis is desirnblc for peclagogir:nl rcasdhs, Many proofs are unexc:itirrgapplications of irrduction that are sptx:ific to particuLrr llrobletns. with diff'ererrt:es or contra,clit:tiotr, arguments in full detail is not orrly ullllecessary,lrtrt interPresenting $rrdr feres with the flow of the storv. Therefore, quite a few of the proofs are Inay consitlclr tltern lacksketchy irrrcl someone wlxl irrsists on complerttlrress in cletrr.il. I do not seq:this as a clrawback. Mathematica,l skills are uot ing else's argutttents, but comc frorn thinkthe byproduct of reading sorrreorle ing atrout the essenrxlof a problem, disrxlvtlrirrg ideassrritatllc to make the poirrt, thel carrying tltetn out in prtruistldetail. The kr,tter skill certainly arrd I lhink th.r,t the proof sketches irr this text providc has to be lea,rnerd, very appropriir,tcstartiug points fbr such a practitx:. sornetitnes vi(lw a course in the theory of StudentS irr courputer sclit1rrce computation aa urlnecessarily abstract and of little practical con$(xpelrce. 'Ib convinr:c thetn otherwi$e, t)nc treeclsto appeir.l tcl their specific irrterests and strengths, suclt a,stena,t:ity and inventivttntlss itt clealing with hardtoof solverllroblettts. Beca,user tltis, tny a,pprtlitt:h empha,sizeslea.rnirrgthrough probletn solving. I By a problemsolvitrg approa,ch, rrteattthat students learn the material prirnarily througlt problemtype illustrative examplcs that show the motiwell as their conncction to the theorcrns attd vation bohirrd the concepts, a^s tirne, the examples rrriryinvolve a nontrivial aspect, clefinitiotrs. At the sa,me a for whir:h students must dist:ovc:r solution. In such an approach, htlrnework contribute to ir,rrrajor part of the leartting procefJs.The exercises exrrrc:ises :rt the end of each sectiorr are designed to illutrftrate and ilhrstrate the malevels. Some tr:rial and call orr sttrdents' problemsolving ability a,t vtr,riotrs of the exerci$csare fairly sirnple, pickirrg up where the discussiotrin the text Ieaves ofl and asking students to carry ou for antlther step or two. Other extrrcisesare very difficult, challenging evtrrrthe best ntinds. A good rnix of such exercisest:ilrr be a very eff'ectivt:teaching tool. Ttr help instructors, I have provitled separately an instructor's guide thrr.t outlines the sohrtitlrrs of the exercise$irrrd suggeststheir pcdagogical value. Students need not trrr asked to solvc all problems bqt should be assigned those which support tlte goals of the course and the viewpoint of the instnrt:tor. Computer sr:ience currir:ulir tliffer from institrrtiorr to iilstitutiorr; while a few emphasize the theoretir:nl side, others are alrnost entirely orientt:d toward practiclnl application. I believe that this tt:xt can serve eitlNlr of these extremes, llrclvided that the exercises a,re stllected carefully witli the students' btr,c:kgroundatld intertlsts in mind. At ttle same time, the irrstructor needs to irrform tlle
PRnrecn
students aborrt the level of abstraction that is expected of tlxrm. This is particularly tnre of the prooforietrttxl exercises. When I say "llrove that" or "show that," I hrr,ve mind that the student should think about how a, in proof rnight be cclnstrur:ted ancl then produr:e a, clear argurnent. How fbrrrtal srrch a, proof should bc needs to be deterrnined by the instructor, ancl stutltlnts should be given guitlrllines on this early irr the txlrrse. Tltc content of the text, is allllropriate for a onesernestcr txrurse. Most of the nraterial can be covered, although some choice of errrpha.sis will have to be rnirde. In my classes,I gencrirlly gloss over proofs, skilr4rv as they are itr tlte tcxt. I usually give just enough coverageto make the rcsult plausible, asking strrdents to read the rest orr their own. Overall, though, little can be skippexl entirely witltout potential difficulties later on. A few uections, which are rnrlrked with an asterisk, c:rr,n omitted without loss to later be material. Most of tht: m:r,teria,l,however. is esscrrtial ancl must be covered. The first edition of this book wrr,u published in 1990,thc:stxxrnda,ppeared in 1906. The need for yet another cdition is gratifying and irrtlic;atesthat tny a1l1lrorr,ch, languagesrathcr than computations, is still viable. The via charrgcs ftrr the second edition wercl t)volutionary rather than rcvolrrtionary and addressedthe inevitable itrirct:rrra,c:ies obscurities of thtl Iirst edition. and It seertrs, however, that the second r:dition had reached a point of strrbility that requires f'ew changes,so thc tlrlk of the third editiorr is idcntical to the previous one. The major new featurtl of the third edition is the irrc:hrsion of a set of solved exercises. Initially, I felt that giving solutions to exerciseswas undesirable hecause it lirrritcd the number of problerrts thir.t r:a,nbe a,ssigned hourework. Howfor ever, over tlre years I have received so rrrany requests for assistance from students evt:rywhere that I concluded that it is time to relent. In this edition I havc irrcluded solutions to a srnall rrumber of exercises. I have also added solrro rrew exercises to keep frorn rtxhrcing the unsolved problems too much. Irr strlec:ting exercises for solutiorr, I have favored those that have signiflcant instructioner,l ver,lues. For this reasorr, I givc not onlv the answers, brrt show the reasonirrg that is the ba,sisfor the firml result. Merny exercises have thtl ser,me theme; often I choose a rupresentative case to solve, hoping that a studerrt who can follow the reasorrirrgwill be able to transfer it to a set of similar instances. I bclicrve that soluiions to a carcfirlly selected set ttf exercises can help studerrts irrr:rea"re their problernsolvirrg skills and still lcave instructors a good set of unuolved exercises. In the text, {lxercises for whir:h rr,solution or a hint is givcrrrr,rqr identified with {ffi. Also in response to suggcstitlns, I have identified sonre of ther harder exercist:s. This is not always easv, sirrt:e the exercises span a spectrrrm of diffic;ulty and because a problen that seems easy to one student rnay givr: considerable trouble to another. But thcre are some exercises that havcl posed a challcnge fbr a majority of my studcnts. These are rnarked witlr a single star (*). There are also a few exercisosthat are different from most in that they have rro r:learcut answer. They rnay crrll f'or upeculation,
vt
PRnrncp
suggest additional reading, or require some computer programming. While they,are not suitable for routine homework assignment, they can serve &s entry points for furtlter study. Such exercisesare marked with a double star (** ). Over the last ten years I have received helpful suggestions from numerous reviewers, instructors, and students. While there are too many individuals to mention by name, I am grateful to all of them. Their feedback has been in'aluable in my attempts to improve the text. Peter Linz
Chapter
1
fntroduction
to the Theory
of Computation 3
1.1 Matlrenratical Prelirrrirrrlricu ar,nd Notation Sets 3 Functions and Relations 5 Craphs and l}'ees 7 Proof Techniques I 1.2 Three Basic Concepts 15 Lirrrgrrir,ges 15 Grarnrnilrs 19 Automala 25 +1.3 Some Applications 29
Chapter 2 Finite Autornata 35 2 , 1 I)eterrrrinistit: Finite Accepters 36
I)ctc:rrnirristic Accepters and'IIrrnsitiorr Grir,phs Languir,gcsand Dfa,s 38 R.t:gulil,rL:lngrrages 42 2 . 2 Nondeterrriinistit:FiniteAccepters 47 Definilion of a Nonrleterministic Accepler 48 Whv Notxlctt:rrninism'1 52
36
vll
2 Parsing and Ambiguity 136 Parsing and Mcnbership 136 Anlbiguity in Grarnrnars and Latrguages 141 5.r Expressions for Regular Languages 81 Regular Expressions for Describing Simple Patterns 85 3.2 Connection Between Regular Expressions and Regular Languages 78 Regular Expressions Denote Regular Languages 78 Regula.1 CorrtextFree Grammars 126 Exarrrples of ContextFlee Languages 127 Leftntost and Rightmost Dt'rivations 129 Derivation Tl'ees 130 R.anrl LeftLinear Grammars 89 RightLinear Grammars Generate Regular Languages 91 RightLinear Grammars for Regular Languages 93 Equivalence Bctween Regular Languages and Regular Gra.2 Elementary Qrrestionsabout Regular Languages 111 4.3 Identifving Nonregular Languages 114 llsirrg the Pigeonhole Principle 114 A Pumping Lemma 115 5 ContextFree Languages L25 Chapter 5.3 Regular Gra.mma.3 ContextFtcc Gramrnars and Programmirtg Ltr.3 Equivalence of Deterministic and Notrdeterministic Finite Accepters 55 +2.1 Regular Expressions 7I Forma.l Delinition of a Regular Expression 72 Languages Associated with Regular Expressions 73 3.rrgrrages 146 .vlll CoNrnNrs 2.1 Closure Propertitrs of Regular Languages 100 Closure under Simple Set Operations 100 Closure under Otlter Operations 103 4.elation Between Sentential Fttrms and Derivation 'llees 13? 5.rs 95 Chapter 99 4 Properties of Regular Languages 4.trrnars 89 Right.4 Reduction of the Number of States in Finite Automata 62 Chapter 3 Regular Languages and Regular Grammars fl 3.
2 Pushdown Automata and ContextFree Larrguagcs 184 Pushdown Autorrrata fbr ContextFlee Languages 184 CorrtcxtFloe Grammars for Pushdown Autorrrata 189 7.1 Two Pumping Lemmas 206 A Purnpirrg Lcrnrna fbr ContextFlee Languages 206 A Purnping Letrrnil firr Linear La.1 Methods for Tfansforrrring Grammars 150 A Useful Substitution Rule 150 Removing UselessProductions 15? Removing.ges 200 Chapter 8 Properties of ContextFlee Languages 205 8.ges 195 *7.3 A Me:mbership Algorithm for ContextF]'eeGrarnrnrr.ngua.chines as Tlansducers 232 9.rsfbr Deterministic CorrtextF}ct:Langua.r.1 The Standard T\rring Machine 222 Definition of a Thring Machine 222 T\rring Machines as Language Accepters 229 Tlrring Ma.3 T\rring's Thesis 244 .2 Closure Propcrtien and Decision Algorithrns for ContextFree Languages 213 Closure of ContextFree LangJuages ?13 Some Decidable Properties of ContcxtFre.l Form 168 +6.2 Combining Tlrring Machines for Cornplicated Tasks 238 9.7 Nondeterrnirfstic Pushdown Automata 176 Definition of a Pushdown Arrtomaton tTti A LangrrageAccepted by a Pushdowrr Automaton I79 7.ges 210 8.72 Chapter 7 Pushdown Automata 175 7.4 Gramma.2 Two Important Normal Forrns 165 Chomsky Normal Form 165 Greibach Normtr.rs 1.'e Languages 218 Chapter 9 Turing Machines 221 9.\Productions 156 Removing UnitProductiorrs 158 6.3 DerterrrinisticPushdown Autornataand Deterrrfnistir: ContextFr{lc Lirnglrri.CoNrErrts ix Chapter 6 Simplification of ContextFlee Grammars 149 6.
chirrt:s 261 10.3 ContextSensitivc (]rarnrna.3 Norrtletertninistic T\rring Ma.4 [Jndccidable Problems for ContextFree Lir.(.4 I'he Chomskv Hierarchv 29Ir Chapter 12 Limits of Algorithrnic Cornputatiorr 299 12.chines 263 10.3 Tlte I'osL Correspondence 12.etlucitrgOne Undecidable Problem to Another 304 12.rrnotBc: Solved By l\rring Machines 300 The T\ring Machine llalting Problem 301 H.4 A lJrriversal I\rring Machine 266 10. A Hierarchy of Formal Languages and Autornata 278 11.2 Uritlt:c:itlrrble I'robletns for Recursivelv llnrtmertr. Sta.rsarrd Lirnguages 289 ConterxtSrlnsitivc Languages and Litrear Bounded Aulomata 29t) Relation Betweeu Recursive and Ctlrrtt:xtSetuitive Languages 2gz 11.ltlrr Languages 308 Ptoblem Sl2 12. 250 clf Ttrrirrg Machines with a.yOption 251 Thring Machines with SemiInfinitc Tape 253 The OffLine Tttrirrg Mat:hine 255 'I\rring 10.nguages 318 .t:hint:Therne 25t) Eqrrivalcrrt:tl Classesof Autonrata.1 Mirxlr Virriatiotrs on the T\rring Ma.5 Liricar Bouttded Autotnata 270 Chapter Ll.2 Uurestricted Grarnmars 283 11.l 11.1 Recursive and Reclrrsively Euurnerable Languages 276 Enumera.chiners 258 Mttltidimensional T[rring Mtr.2 Ma.tx:ursively A Language That Is Not R.chineswith Morc Cotttplex Storage 258 Mullitape Ttrring Ma.ble 278 Languages That Art: Not R.1 Some Probletns That (ltr.t:cursively Enumerable 279 Erlrrrterable But Not A Language That Is Rer:rrrsivr:ly Recursive 28.'onrnrlrs Chapter 10 Other Models of Turing Machines 249 10.
Cot{reNrs xi Chapter L3 Other Models of Computation 323 13.3 R.ewriting Systems 337 Markov Algorithms 339 LSystems 340 Chapter 14 14.2 14.2 Post Systern$ 334 13.1 14.3 I4.4 An Introduction to Computational Complexity Efficiencyof Computation 344 Ttrring Machinesand Complexity 346 LanguageFamiliesand Complexity Classes 350 The Complexity Classes and NP 353 P 357 343 Answers to Selected Exercises References 405 Index 4OT .1 Recursive Functions 325 PrimitiveRecursiveF\nctions 326 Ackermann'sF\rnction 330 13.
.
inly true of computer science studcrrts who rrru interested rna.re.ctical discipline. This attitude is appropriirte. f'rom machine design to progratntrtittg. But givcrr ihis practical oritlrrtir. Tlte use of cornputtlrs irr thel rea.l world involves a wealth of specific detail that must lre lerirrrrcxl a uuccessfirla.pplication.tiorr. Even . there are soure colrtlrlotr urrclcrlyirrg prirrt:ipltrs.reessential to many of the special and complex corrstructs we crrcourrtrlr while wclrking with computers.l topics. Tcl strrdy these basic principles.pra. onr: rnight well a.ndthat a. Tlteoretical qucstions arcrinteresting to them only if they help in finding good solutions.mrlrked pref'erencefbr useful and tangible problerns ovt:r theoreticrrl spt:c:ulirtion. we construct abstract rnodels of corrrllrtcrs and comprrtation. sinr:e without npplications there would be little interest in cornputers. But in spite of this diversity.I N T R O D U C T I OO TN T H E H E O RO F T Y COMPUTATION otrprrterr science is ir. These ruodels embody the important features tlnt are cornnron to both harrlwarc and softwtr.inly in working on difficult applicatious from the real world. The field of computer scienceincludes a wide rarrgr:of sper:irr.sk "why study theory?" 'Ihe first arrswer is that tlrrxrry provides concepts and principles that help us understand tlrtl gerrcral rrirturt: of the discipline. This ftrr makes computer science a very diverse arxl lrroarl rlis<:ipline. I'hose who work irr it oftcn hirve ir. Thiu is certa.rr.
ndrrirnpilt:rs are the most obvious erxarnplcs.ny crha. from opera.ting systerrrs to pa. the insiglrts wt: gain frotn studying them provide the foundations on which sptx:ific.ges. T!'eesand graph structures will be rmul f'requently. construr:t thir.'first chapter.rxxlpts porary utorilgrl. An automaton is a.tirrg atrd furr.t possessesall Lhe indispensable f'eatrrrt:stlf a irrput. Trr rrrodel the hardware of a comprrtt:r. the conchrsionrr wu draw will be based on rigorous arguments.sin a vcry broad way to set thtl stagc for later work.rrnlxrtcrscience.lthoughlittle is needed beyond the definition of a.nsformirrg the input into tlte output.rrguirgesfrom formal languages.directed graph. This will involve sclmelrnilthernatical machinery.but there are rnanv othcrs. we will look at models that represcrrtfcatures at the core of all c:ornputers and their applica.bout programming lir. tr.lly.tlrttrnaticsthat will be required. automata).lir. rlevelopment is ba*sed. we look at these ba.renot) suitable fbr solution try srrclt trtechatrical Ineans. although these requirementn alrelnot t:xterrsive. yet powerfirl. The fields of digital design. In this hook. Perhaps the rnost stringent requirement is thu rrtrility to follow proofs aud . A ftrrmal lirrrgrrage cotrsists of a set of symbols irrrd some rules of forma. prograrntning laugua. anrl rclatiorrs.rxlllcrhaps not so obvious answer. Fina. The third irlrswer is oue of which we hclpc to txlrtvittce the reader. A f'ormell lirnguage is the set of all strings permitted by the rules of fi)rrnirtiorr.lsleepless nights. In the cour$e of orrr stutly.beled. This is lenging. wtr will forrrralize the concept of a rnechanical computation by givirrg a precise clefinition of the term algorithrn and study tlrt: kittds of problems that are (and tr. The cotrcepts we study hert: nrrr like a thread through mrrr:h of txrrrrputer sciettce.retoo simplc to be applicable immediately to realworld situations. and can make decisions in tra. Although sorne of the formal langrrirgcs we study here are simplt:r thirrr prograurmitrg langua.sicidea.nt applit:atiorrs.tiorru.ges.ttern rtxxrgrritiorr. we revit:w thc rrrain ideas from ma. Tlte reader will need a rea^sonably good gra. a. In tlx. The srrtricc:trnatter is intellectually stimrrltr.tputarlott whertr such moclelsa. may have somtl tcrndigital computer.s. prrzzlelikeproblems that can lead to ir()rrrc probkrrnsolvittgin its pure essence.bstractiorr of the general characteristics of prograrnming languages. we will show the clo$er(xlrrrrc(:tiotr between these abstractions and irrvc:stigate the conclusions we carr tlcrive from them.tion by whit:h thcse sytnbols can be cotnbined into crrtities called sentences. In Section 1. The construc:titlrrclf rnodels is one of the esdisciplitte. It :r.tr. and the usefiilnessof a discipline is often sentials of any sc:iurrtific clf clependenton the exi$ttrrrt:c simple. While intuition will frcquently be our guide irr exploring idea.they have rnarry of the same esserrtial features. We cau learn a great deal ir. A formal language is arnir.1.spof the terminology and of the elementary results of set thtxrry. It provides ma.pproach is of course not unique to rx. A second. we introcluce the notion of iln automaton (plural.Chopter I IurnorrucjrloN To rHE Tsr:enY ol' Col. ftnetions. thtxlric:satrd laws. is that the ideas we will discuss have srlmt: irnrnediate and itnporta. produces output. This tr.
Mothemoticol Preliminoriesond Nototion WffiWmHilW Sets A set is a collectiott rtf t:lclrno'rrts. To indicate that r is arr clcrnrrnt of the set 5..ndrj is even.pplica.3. wc givc some simple a. we write r € . gralrllnar$. in S. . . . 9 z } .but for lhe ntotttettt. intersection (n).rssion these two scc:tionswill be intuitive in rather tltirrr rigororrs. trrrd a.9is sc:t of irll ri. We read this as tt.2}.ter tltatr zero.6.} denotes the set of all positive everr irrtcgrlrs. Tltus. 5 1 1 5 2 : { z : r € S r a r r rr E .sicopera. consists of a. The sta.diction..9.Sz : 5r {z : z € Sr arxl r fr 52}. When the neecl arises.ll elernenls not. {a." implying of course that z is irrr irrteger.2.1 MarrrnlrATrcAl PRr:r. thtl goal is to get a.z} slands for is all the lowercaseletters of thc Engliuh a.t rl is grea.J 1.9. In Section 1. Sectiott 1. These cortcepts oc{:rrr irr rnarry specific fbrms throughout the book. 1. and difference (). in which we write S= {i:i >0.The discr. b.sto illustrate that thesc c:tlnr:rrpts have widespread uses itt cornputcr ur:ience. 2 is shown as 5 : {0. we use rrrore explicit notation.sthis necessarybackground. exa. Anothttr bir. we will make all of this rmrr:hrnoro precise.lphabet. The cotrplerntlnt tlf a set . induction. a.l idea.. srx:h thrr. The usual set operations arc union (U).mple.l) frrr the ltr.92}. In Section 1. Later. To rnakc this . ancl proof by c:clrrtrir. defined as 51U52 : {z : r e Sr or r €. l . Ellipses are usetl wltcncvc:rtlNl rneir.while {2. This includes farniliarity with the hasic proof techniques of dcrluction.1 is induded to review some of the rrririrr results that will be used arrrl to entahlish a notational colrurrorr grourrrl f'rrr subsequentdiscussion.ning clear.tementthat r is not in S is written r f 5.tion is complementation.We will assurne that thc rcirrlrlr ha. we take afirst look at thc r:entral concepts of languages.clear picture of tire corrceptswith which we are derrling. without any structure olher tharr rnr:rnhership. A set is specified by cnr:losingsome description of ils elernentsin curly bracxrs.lnrwnRrES AND Norauou atr utrderstarrdingof what constitutes proper rnathcrnirtical reasoning.4. denotecl by F.. ziseven} (f.utomata.the fbr set of irrtt:gers0.st example.tions of tlrr:sc gerrera.1.
9r is said to be a subset of 5 if every element of 5r is also atr element of S. r : } { b . finite. Exottplq l'. it is obvious that Sus: Sfr:5. (t 2) (1. b } . 5 rU S z . { o } . set 5 is callecl the powerset of S ir. TIte set of all subsets of a. Wc write this as 5rc5' If St C S. b . are needed orr $eivc:ral A set . Observe that 2s is rr. Sr n 5'2 = fl. A given set norrnally has marry sutrsets. this is denoted bV l5l.rlsl I . that is. 'Ihe otherwise it is infinite. " } } . { .t the universal elements is.n#S}. then its powerset z s : { f r . hut 5 rxrrrtilirrs ir. A set is said to be linite if it contains a finite nlrmbcr of elemenls. The following useful identities.4 Chopter I llqrnooucrroN To rrrn THnoRv cln Cor. n . ffi: F l n S z : .9. set of sets. If U is specified.b.3) occasions. { c } . thcrr set U of a'll possitrlt: F:{*:rr(I. then This is arr instirrrce a general of result. { a .we need to know whir.t b } ..f I is If $ is thc set {a. c } . Here lSl : 3 and lZtl :8. If 51 a. Snfr=9. { o .ncl5'z have IIo coilllrron elemeut. tltett the sets are said to berdisjoint. fr =U. wc write tltis as $rCS.nd is denoted by 2's. called the ernpty set or the null set is denoted by #. size of a finite sct is tht: rrurrtberof eletnents in it. Srl3z.if 5 is lrsl .c}. known a. 5::5.sthe DeMorgants laws.tpu'rn'rtou rnerarrirrgful. The sct witlt no elements.rrelernetrt not in 51 we say that Sr is a proper subset of . Flom the definitiorr rtf a set.
generally S r x 5 ' r x ' . .f is said Lo be a partial function. . the donaiu and rauge of the firrrt:tiottsinvolved In ma.( 4 . For the Ca.q.rtcsiarrproduct of two sets.then the flrst set is t:ir. 1 ' r()4 . . 5 1.virlrof tltese functions as their arguments btlclottte arr such c:asers urrrlerstanding of the growth rtr. 6 ) . 3 ) .:. Srrr:h$ets arc said to be the Cartesian product of other sets. positive constant c such that for all rz f(n)tcs(n). Thc pair (4.2) is in 51 x 5'2. ( 2 . We write this ir.ny applir:rrtiorrs. subst:t of the positive itrtegers.lltxl sct is its range. n . 2 .. otherwist: . .sordcr at most g. If / dcrxrtt's a futrcLion. .) and . 3 . Notc that tlte order in which the elements of a.4) is not. .\ AND 1. € S r } . 2 ) .in is a.rcoften interested only very large.Ee Sz}. T 6 c ' 6 .( 4 .mples. The nolation is extendeclin a. Let / (n.tes is oftetr sullicient and a corrrrrrorr order of magnitude nota. we say thrlt / is a total function on 5r. 5 ) .s f ( n): o ( s( n) ) . are in the set of positive integers. which itself is a set of orclerednairs.ngeof / is a subset of 52. I Functions Relotions ond A function is a rukt that assignsto elements of one set a. Itr in the heha.IEs Norauolt In rnany of our exa. and the serxrnd / : .n.tion carr be used.( 4 . ( 2 . 3 ) .the elements of il stlt irre ordered sequencesof elements frorn otJrer sets. $ 2 of to itrdicate thal the doma. 2 ) .ltvttw. . ' x 5 r : { ( r 1 . 6 ) } .51atrd that the ra. )r: . Furthermorel we il.in of / is a strtrsc:t .but (2. If thcre exists a.q(n) be functions whose doma.unirptl cletrtetrtof the domain another set. 1 ' r ./) :r€S'1.1 MnrHntutnrtcAL PRt:t. we sav that . llnir are written matters. . If tht: tlornirirr of / is all of 5r. We write of /.nobvirlrs firshiorrto tlte Cartesian product of rnr)rt)than two sets. we writer S:Sr x52:{(*.f ha. (2 S r x 5 ' z : { ( 2 . T 2 .
'ConrurArrorv 'r'o ChopterI IurR. 2 Thcn / (t) : o (s (rl)). .lly. s (n) : o (ft. expresseda.nalysisof algorillurs. there exist constants c1 and c2 such tlnt if cr lg(?l)l l/ (")l ! czle(rl)l. if used properly. I'rrr whit:h we tuJe /(") :o(g(")). the . ea. If ttris is not satisfied. wh{:rc il.For such a set to delirrc a firnt:tion.lity a. a*swe will see in later r:hirpturson the a. wtr ignore multiplicative constants and lower order tertns that becotnencgligibkl as n increases. In order of rnagnitude notatiorr.ncl order of magnitude expressiorrs r:annet be treated like ordirrirry cxl)r{}ir$ions.'l'roN 'r'Hu If .(ru)). u z ). .Still.s I(n)*o(g("))' In this order of magtritude trotittitlrr. ( r z ..'l'Hnonyor. < / and g have the same ordcr of magnitude.nge. the order of magnitude argurnents tlrrrr tlr: effective. tlrrl syrnllol : should not be interpreted irs txlra. l/ (")l l c ls(?z)l then / has order at least g.ch can 11 occur at rno$t on(:ea. Finar.onuc.n element in the clornain of t}re furrc:tion.)). I Some functiorls can be rtlprt:srlrrttxl by a set of pairs { ( " r . and gti is the corresportdirrgvilhrelin its ra. ( n ) : 1 o r z+ 1 o o . Ma. ExompleI.nipulations such as O (rz)+ i) (n) = 20 (n) a. is a.3 Let f (n) : 2nz + iJtt. } . .sthe first element of a pair. / (n) : o (h (r. y r ) . I (rt) : "'t.renot sensibleand catr lead to irtr:clrr(lct rxrnclusions. h .
l. Ea.rATrcAL PnnLrNrrNanrES AND No'r. the set V : {tt1. a generalization of thc is concept of equality (identity)..e2.n orrtgtlirrg edge for ?ri and an incoming edge forr. I f xsnrslF f . I Grophsond Trees A graph is a construct consistirrgof two fitilte sets. the syrnrnetry rule if '': r .sexir. frlr irrstance e. Such a construct is actually ir.sirrce we associate a direction (fiorn ui to u6) with each edge. a label being a ntrme or other itrformation with parts of the graph.tethat a.'r..tion on thc sct of rrotrttegativeiulegers defined hy 'l': l' if and only if rmocl 3 .tion tht:re miry trcl scvtlral such elernenls in the range. A relatiori rlcrrotexl lry : i.nge.i: \U.ctly orrclitssociated eletnent itt the ra.. :y r' dt and the transitivity nrlc If fr: A a n c ly : a. we write :I:='!J. To indica.pair (r:..ll z.. in a relir....t'l'tor'r set is called a relation. and l. Orre kirrd of relatiott is that of equivalence...l M.in ha. Clearly this is atr equivalencerelation.ph). Graphs miry bc labeled.Ltk) is an edge from ui to tr4. directed graph (digrrr.err} of edges.uml. a*ssor:iated . Both vt'rtices atrd edgesmay be lahclctl. then n # E.q.chedgttis a pair of vertices fiom V. and 0 = il6.'tt2.'Dn} of vertices and the set E: {e1.i. consiclered atr equivalence if it satisfies three mlcs: the reflexivity rule r: = r: for ir.4 ' ' Consider the rela. irs it satisfies reflexivitv. Wc srry that the edge e. is a. 12 = 0. Relatious are Inore general thtlrr firrrt:tions: in a function each element of the doma.37mod3. ll r r a u hvn o .37) arr crpivirlcrrce relation. syrntttetry.ransitivity. 'Ihen 2 : 5.
we have all paths of length orrt: txlgt:s (u. The level associated with each vertex is the nunber of edgesin the path from the root to the vertex.llerla.lly. The length of a walk is the total nurrrber of rxlgcs travr:rscrl . . After we do this. If there is an edge from ua to ui.in going from the initial vertex to the final orre. ( u i ... .1 there is a loop on vertex u3. .onuc'l'roN'l'o tsr: Tttr:rtHv ou Cotr. a. A wrrlk in which no eclge is repeated is said to be a pathl rr path is simple if no vertex is repeated. and that htust)ne rlistinct vertex..slong as the. ) . .then it is sa.. These are called the leaves of the tree.th we are rxlnstnrcting.. . u r ) . .rt) is ir cyc:le. The graph with vertices {u1. we will refer to atr algoritlun for lindirrg all sirnpkr paths between two given vertices (or all siurplc c:yrlcs bn^sed rr.1 f \ \'otl t Graphs are conveniently visualized by diagrarns in which the vertirx:s are represented as circles and the edges as lines with arrows corrrrt'cting tho verti<x.t)t.u3} and edges {(u1. A walk fron ui to itself with rro rcpcir.vertex). The height of the tree is the Iargest level number of any vertex..u'r'n'r'tow Figurc 1. .nd ui the child of u1. we will eventually lisi all sirnple paths beginning at rr. we will have all sinrple paths of lerrgth two origirrrrtirrgat a. ( ' . we carr llsc tlrrl following obvious method. In Figure 1. These terms are illustrated in Figure 1. on If we do not concern ourselves with efficiency. we a. (u3. u * .1 so rea. On several occasiotts. called the root.sirnple one.w a l k o ) fiom rri to urr.rr). (r*. If the edges of a.116).'s. u3) . ' u i ) .u2) is a simple perthfiom ?rrto ??. A s e q u e n c e f e d g e s( a t . Fina.y do not lead to arry vcrtclx alrtlirdy rrsed in the pa. Starting frotn tlte giverr vcrtcxr say ?ri. Since there ate orrly ir finite number of vertices. we can talk about the label ofa walk. 1 .ched. u 3 )  i s d e p i c t e di n F i g u r e 1 . Iist all orrtgoing (ui. For a. r r " i. u2.1.cycle with base u4. Tlees are a particular type of graph. such that there is exactly one path frorrr the root to every other vertex. then ua is said to be the parent ()f rrj.tt. This definition implies that the root ha^srro irrcoming edges and that there &re some vertices without outgoing edges..u1) . list all outgoing edges .llverrticesuk1.id !o be simple. ( u 3 . A tree is a directed graph that has no cycles. ( t r . The sequence edges(ut. This label is thc scqucrrr:c r:dgo ler.trxltxlges is ca.s s a i d t o h e t r . In Figure 1. il.. (rr. At this point.n eclgefrom a vertex to itself is calk:d a loop.Chopter I Itrrn. graph are labeled. of llrt rxit ir.. If no vertices other thatt tlte base are rrlllc:itttxliri ir r:yr:le. . (rr. . .2. startittg at u4.bels of encorrnteredwhen the path is traversed.rr).. We r:ontinue this until all possibilities are accounted for.ur).ur) . Flom these we select those ending at the desired vertex. (z1.
we want to prove to be true.P2.nulumnn. we want to a*ssocirr. Induction is a technique by which the truth of a number of statements can be inf'erred from the trr.2 Root Height = 3 . Brrt now thir. Pk are true.1 MarHu. Two special of proofteclufques are used so frequently that it is approprintc to rcvielwthem briefly. . every statement is true.1... These are proof by induction and proof by contradiction. . Pz. ProofTechniques An important requirement for reading this text is the ability to follow proofs..t we know that the first h * 1 statements are trlrc.. The problern is such that for any z ) A.1. ilnd rnilny proofs artl simllly a ..rATrcALPn.rth of a few specific instances. We can then use induction to show that everv statement in this sequenceis tnre. Then Condition ? tells us that P611 alsrl rmrst btr tnre..L"'"13t I I At times. tlrt: trutlm of P1.te ordering with the nodes at each level. $rrpposealso that the following holds: 1. we know that Pt. 2. Irr a pro<lf by irrclucticln. we More details on graphs and trees can be found irr rnost books on discrcte mathematics. . we employ the accepted rules of deductive reasorring. .sequence such steps...P. imply the truth of P. we rrrguo as follows: Ftom Condition 1 we know that the first k statements are true.ticalarguments. Therefclrc. Thc cltairr of rcit*sorrirrg t:itrr btl cxtended to any strrtcrnerrt. In mathema. arrd so on. . an in srrch ciL$e$ talk aborrt ordered trees. we r:arr allply Contlitiorr 2 agairr to tlaim that P61z must be true. Suppose we have a sequenceof statements Pr . Pz. Furthermore...rEs AND Norauoru Figure 1. \Me need not explicitly continue this argument because the patterrr is clcrrr. For some fu ) [.
utornittorr Exatnple1.ninstarrt:t: clf ir firrite acceuter.ph.n input file cannot be rewritten.l ur introduction in the first cha.rigorous results. provide formal defiuitions. we must be more prct:istt. I'his type cia.FIN ITE AUTOMATA tho".a finite a'utomat(lrr (:arronly deal with situations in which tlrt: infortnalion to tre stored at irr arry tirrre is strictly bounded. Wt: llcgitr with linite accepters.tirlrr. particularlv tIrc tlist:ussiotr autornata' was hrief antl irrftlrrnirl. orrly ir general understanditrg of whir. ir.pacityte "rcmcmbcr" things during the comprrtir. a firritc itutornittott is severely limitcd irr its A ca. 1ti is tt.tt:s firriLe. Ttl llrogress.t. linite atnoutrt of inf'ormir. But sitrcethe number of srxrtrstir.l case of thc of autclmittotr is characterized hy having no tt:rnporary storage.r tomaton is and how it c:itrrbe representedby a gra. we harvt. is sptx:ifi<: state.r't: sirnple' spea gcrreral scherneiutroduced in the ltrst drapter.tiorr carr be relained in the control rrnit tly placittg Lhe uttit itrto a. The a.ptcrto the basic cotrceptsof comprrtaof tion.t irrl auAt this point.rrrlstirrt to clevelop . whit:h ir. Since a. 35 .
if rI (qo. tlrtl irrlxrt meclha. Rrr tlxtr. Drrring eelchmove advancesone position ttt tlrc right. The initial . t J O € Q is the initial state. internal state to another are govcrned bv the transition futrctiorr frorn one d. ond Grophs Accepters Tronsilion Deterministic sniiit$lil]tnir tiftffif[j! f. with its input mechanisrrrort the lefhmost symbol of the input strirrg.qs.ton we study in detail are firrite accepters that are deternfnistic in their opera. then if tlrtl rlfa iu in sta.Tln input rnechauismcan Inove otrly frorn 'I'he tratrsitiorrs Icft to right and reads exactly ontl symbol on each step.tesclf some clfa M.mple. the dfa will go itrto stirtc' q1. itrlrut syrnhol is a.ssurnecl be irr thc initial state q0. Otherwise the string is rejer:tcxl. the string is ircx:clltedif the automaton is in one of its Iirtr. At to thc initinl time.bclcda represetrtsthe [ratuitiorr d (gs. E i s a flnitc sct of symbols called the input alphabet. where Q i s a firtite st:t of internal states.ffifrtll A deterministic finite accepter or dfa is defined by the quintulller M : (Q. Ibr exirtrllle.l stir.Q iu el tota. Thc Inhels on tlre vertices are Lherrirlrr()s tlx: sttr. An edge (qn.Chopter 2 AUToMATA ffi FiniteAccepters Deierministic The first type of automa. clt:iu and intr"ritive we picture to work with.n. of the automaton. A deterministic finitt: ir.lizeand represetrt fittitr: irtrt{lrna. We start with a precise frrrrnir"ldefinition of deterministic accepters.) = q't.a) : gr .if qp and qt are interlral strr. it is essential to havc a.F). then the graph associatcd with M will have one vertex lalrclcdq0 irnd another labeled q1. in whir:h the vertices represerrt stattls arrtl the edges of represent transitiorrs.qt) la. Q x X . use transition graphs. while the labels orr the eclgcs art: the current values of the input sytnbol. In dist:ussirrg arrtomata. it is a.E.l function called the transition function. T C Q is a set of final states. When the etttl of tlxr string is so each rnove colrsurrrrlri reached.nhm one input symbol.d.rx:epteroperates in the following tttanrrt:r. Trr visutr.tir..tes.te4u ancl the current. d .tion.tes.
the string 01 is a. :i. if M : (Q. 4 0 { q r } ) . q0.tctlo..plr:RS 87 Figure 2.redrilwn with ir.we $eethat the autouraton will accept thtr strirrgs 101.1I) tlc:firritionof a dfa to its tra.. qi) Iabeled a. and its va. c:orrsecutive 0's.nsitiorrgraph Gna has exactly lQl vcrtitr:s. d . we see that the arrtrlmatott remains intcl state 4r' We in state qs.# T . the l is relirrlirntl the autotnaton goerJ strirtg and. represents tlrc dfh } . the 'Ihe with q6 is called graph has a. at the same time.tc will be identilied by a.q t .f. 6 r"L+Q It is convenient to irrtrorlut:e the extendecltra.i:i. The ser:orrdargutrtettLof d* is a string.l ma.Q. Final sta.2. finitt: ttt:t:cpter.double circle. For exermplc. hrrt tlf b . is More forma.lo.crt:cpted.nedge (qr.. IUI= ( { g u .luegivtls tlre stale the autotna. vertex assocria. Next.t sta.cccptthe string 00' Therefore. are now at the end of thrl The dfa does not a. r 1 z . X. 1 } .nsition firrx:tiotr d* : Q x X* . the syrnbol 0 is rcird fi. not 100or 1100.') a cletertnitristic: then its associatedtrtr. d.lly. whcrc d is given by This dfa accepts the strirtg [Jl. Stnrting irr state gs. 11001.bt:lt:r1 arry vertex. irr a fltral state q1. t:itch otte Iabeledwith a diff'erentq..tecl the initial vertex. 0111' and :.. d. Looking at the edgesof thrl graph.tesa.ttcr to converl from the (8. ) : q' n .trincoming rrnlir.rst. It is a trivia. rathrlr tltatr a single symbol.li:i ii =r. { 0 .a): qi.1 I arrow ttot origiuating a. X.tonwill llc irr after reading that string. while tltose labeleclwith qy € F arc tht: final vertices. it will be in uttr.For every transitiott rrtle d(rJ.if d (qo.nsitiorrgraplt represeutatiou and vice veruir. By similar since after reading two reasoning.1 DnrnnluNrsrrc Frurrn Accl.i €Q.
q) d (go'a) (2.0) : d (d. Formally.renow ready to define formally what we mean by an associated language. : d(o:. But d" (q0. (2l) 1' r)\ &t+!qF Ir u € Ej. The associatiorr is ob. ab): qr. d. s EA To seewhy this is appropriate. we a. longuoges ond Dfq's Having made a precise definition of an accepter. we use (2.3) : Substitutingthis into (2. lsfi.F) is the set of all strings on X accepted by M. e F} .!: oz ab) as expected. (9. b ): q z . go. L (M): ur) {ru e E* : d* (q6. First.l) . we can define d* recrrrsively by d.A) : q.2) to get U_!no' :_d (d*(qs.3). .we get !'!'' d*(qo. In formal notation.ffl$ifii .38 Chopler 2 FIt'rIrp AuToMATA and 6 ( q t . then d* (qn.(qo. r The language accepted hy a dfa M : (8. b) ab] a).E. vious: the language is the set of aJl the strings accepted by the automaton. let us apply thesedefinitions to the simple caseabove.
narrtornatott deterministit:.2 wt: allowecl the use of two labels on a sitrgle edge. until the The automaton irr Figure 2.renttt rtfsled bV the rtlpresentation antl thal arguments ba.2).phs. a urrique tnove is dtlfirrcd. thg autorlatotr a.sedon graphs are emvalid as those that use the fbrrrtrl properties 6f d' Thel following preliminary result gives us this assura. To do so. Wtr see clea.which ptlssible.cr:ttlttsall slrings clclnsistirrgof au arbitrtr. string is accepted.nt:tl.r assllrirnce that we a.1) arrcl rMe (2.''b:rt. I These exarnples sltow htlw t:ottvenient transitiott graphs artl for workiug with finite irutotnata.te frotn which it can never 'I'ire sla.chstep.gea. Note that we rt:cpire that 15.ccolltit. In our discrrssiorr. then the q2. Figtrre 2.2 reurtrins in its initia'l strr. wc tttust of cotrrse irave sonre are more intrritive.r. w f F I . At ea.ry number of c. followed by a single b. so ther.nt:tt in a nrlrtlirral state.lling such a.1 Dnrurl. the results are hard to fbllow. so t'hat we irr'(l justifiecl in ca.cc:cpt or not a. ttre langua. Tf nqt.> 0l .t L(M): ( ) { r ue X * : t 5 * q o .2 In drawirtg Figure 2.llNIS'tICiFtNtre Accr:p'tuns 39 and trlrrstltluenlly d*.tiorr. If this is also the la"rt syrnbol of the input.cr:cptedby the a'rrttlrlatotr is f.rlyfiorn tlte graph tha. Whilc it is possible to hase all arguments strictly on thc properties ofthc transition functiorr attcl its extensiorrtlrrough (2.2 . Sut:h rnultiply labelerdcdges are shorthand for two or mor(r distittct trirrrsitiols: the trelnsition is taken whenr:vcr the input syrrrbol nratches any of the edge labels. bc trltal functions.irr Figure 2. * lu.'tt.ystritrg in X* ir. All other irrput strings are rejtN:tt:d. Consider the dfir. the clfa goes into trtir.teq11 first b is etrcountered. as far i. A dfa will pror:tlsstlver. Nonaccepta.t e$(rirpe.2. In set nota.te q2 is ir trap state. use gra.rrd rnealls that thtr tlfa stops it either ir.
Tra. Thus there is a walk in Gnr labeled ua: u between {a and qi. It r:a.a) : {i. yet typiur.. we errrllloy reasoning sirnilirr to that for prograrnming in higherlevel languages.2. 6* (rqi. 'l'herr frrr every q?.nguagesdefinerrl informally.irrrtl k:t Gna be its a.mination of srrr:h simple carrcs Ilxample 2.ri. Whilt: graphs are convcrrir.ur): qi.(qi.4). Fbr exa.3 is equivalent to Figure 2.ei € Q. ilire secorrdis that the result will be rrsed over and over. Agir.ll strings u with lrl < n.F) be a deterrninisticfinite accepter. arrtl . Here the row Iabel is thc r:urrent state.1air walk with label zu frorn q.l example of an inductive proof in connection with automata. we can represent the functiorr d as a table.mplesand proofs more transpa"rent than they worrld be if we used thc properties of d'*. It is apparent f'rom this exarnplc that a dfa can easily be implemented a^s ir {xrmputer prograrn.w) : qt (.n proved rigorously using an induction on the ir$ be Icrrgth of ur. We went through the details for two rt:a^sons.40 Chopter 2 Frr.w) : qi if antl only if there is in G. we can clainr by induction that.u :. whilcl the column label representsthe currerrt input symbol.1. Since the result is ohviously true for n: l.. there rnust be a walk in Gy labeled u from qi t.in. so wr) rr$e them in rnost of our discussions. Proof: This clairn is fa.. fclr example.urof lr:rrgth n * 1 and write it ns . The errrtrv in the table tlcfines the next state. other representations art) also useful. The best implernentation or representation depends on tlte specific applicatiorr. thus completing the 1rr{lof.. Corrsirlr:r then any . then M rmrst have a tranuition d(qr.€ X*. 6. But the . The first is that it is a simple. I lJt_lr.ting and proving it as a theorerrr lets us argue quitc confidently usittg graphs. The table in Figure 2. The argurnerrt r:irn be turned around in a.qo. to qi.mple.() qk. But if d* (r71.u): q6. straiglrtforward way to show that the existence of such a path irrrplit:s (2.et M: (Q.nsition graphs are very corrvcrrient for the kinds of argurnents we warrt to make here. so that by construction Gy has arr cdge (Sn.nt fbr visualizirrg irutomata.sa sequence of "if" statements. f'or every 'ur€ I+. In construc:ting automata ftrr la. Supposenow that 6* (qi.DQ. the result of tlx: theorem is so itfrtititfr' obvious thrrt a formal proof seems unrlecessary. Assurne thnt the claim is truc frrr a.ru: Aurovarn I.X. as a simple tablelookup or a.rr.irly obvious frorrr irn exa..l. $o sta. This rnakes orrr exa.qi) with label a. t\ implies that there is a walk in Gy from qi to qj laheled u. Since lul: n.ssociatedtrarrsition graph.
the automaton enters the nonfirlal trap state' The simple solution is shown in Figure 2. where it will stay since the rest of the input does not matter. if the first symbol i$ not an a or the $econd one is not f. and the second is a b. and one nonfinal trap state.4 . the automaton goes to the final trap state. I Figure 2. b.4. We can therefore solve the problem with an automaton that has four states.I DETERMINISTIC FINTTU ACCEPTERS 4l Figure 2.$ b' tl b !o ?o ?t q. The only issue here is the first two symbols in the stringl after they have beerr read. Itwo statesrfor recognizing ab ending in a final trap state.2.". b} starting with the prefix ab. an initial"state. On the other hand. no further decisions need to be made. Iz {t 9z !t ?z programming of a dfa is tediou$ and sometimes conceptually complicated bv the fact that such an automaton has few powerful fbatures. If the first symbol is an a.J Find a deterministic finite accepter that recognizes the set of all strings on X : {a. .
heling them accordilgly. but we still rnust add provisions for the srrbstring 001 occurring in the middle of the input. tht:n it must be rejected.ted such a set of Ianguages a family. If we consider all possihle with thertr.tra q Figure 2.rral sttr. This state mtrst be a trap stelte.cceptsall the strings on {0. a fact we remember by keeping the dfa in the state 00. Fcrr cxrnvenience. becausc this situation arises only if there are three consecutive 0s. except those corrtaining the substring 001. hecause Iater synrbcllsdo not matter.state names are arbitrary arrtl r:an be chosen for ttttternorrit: rea^rons. All other stateri are acceptirrg state$. f i ( fixotrtpl* [. In decidirrg whether the substritrg 001 ha.this nonfi.0 0 . We see frorrr this exarnplc how useful mnemorricrlabels on the states are for kceping track of things. Like variable narrr()sin ir prograrnming languager.soccurred.4 Find a dfa thtit tr.5. This gives us the ha"sicstructure of thc solrrtion. I Regulor [onguoges Every finite arrtomaton accepts sorne larrguage. it is vr:ry ea.Chopier 2 }'rllrn Aurolr. d ( 0 0 . We are only interestcd in the la.te is labeled 001.to see that the solution is irrdt+ed correct.l state.te in which two 0s were the immediately prrrcding symbols can be labcled simplv 00. such as 100100 and 1010100. The family of languages that is accepted lty detcrministic finite acceptersis quite limited. we need to krrow not only the currcnt input symbol. If tht: string starts with 001. but wt: also need to remember whethrlr or not it has been preceded by one or two 0s.5 liol*r# riilIr. 0 ). We can keep track of this hy pr. Tbar:ea fbw strings.syto see what the trrrnsitions must be. tlx: sta. whether or not need to know sclrnel the two previous syrnbols were 00. we part of string to the left.ssocia. In this case. A complete solution is shown in Figure 2. . If we labcl the states with the relevarrt symbols.st two. We will call finite autornatal. This implies that therc must be a path labeled 001 lrom the initial state to a rronfina.whett a synrLrol is read. 1}. we get a set of larrguagcs ir. For exantple.rtting the autornaton into specific slates arrrl la. For example.rrr I 1 ' ( o t lc ) o . Thc strrrcture and properties . for example. We must define Q and d so thnt whatever we need to rnirkn the correct decision is rernerrbered by the autornaton.
t the langrra.ckto its finir.tt: 'u e {o. we is can claim that. all we have to clo is find a.r to Exrrrnpkr 2. } .mple 2.l.. What this dfn. Aga'in we is show tha. nanely.2 : {rz.gt: 1.1 Dulnnl. e {a.s 2. Exolmple f .nningcontinrresin this wiry. but a little more complicated.lstiltc. 7. We carr writc a.in. and another b is ftlrrrd.* 1". Sca.11t2 b}. trace a few exa.3. Aga. b}* } is regular. must do is check whethcr a striug begins and ends with au a. tlrc lir.t the dfh. wltat is between is immateritr.tza i . This clifficulty is ever(xrrnc try sirrtply prrtting thrl rlfir irrto a final state whenever the second a is enrxlrrrtcrtxl.L is called regular if and only if there exists sornedetertnitristic flnite accepter M such that L: L(M). To show that this or any other la.2 regular. Aftcr orre or two to tests.ngualgcis rcgular.LL)1. Thc: cornplete solution is shown in Figure 2. diagra..rrgrrirgc regular.tonba.qftfr f. and ends with an a.l irr value).trNrsTrc F'tt'tt'l'Fl 43 in of the la.!. Slxrw that 1.o. Therrcftlrc. caclt a taking the autorna.5 Show that the lirngrrtr.ges this family will bet:ome clearer irs our study proceeds.fi ffi $$lliRtl$ri tffi uil.6.Lr.ngua. " F A language .nytil. it will be obvious tha.nexplicit exprussirlnf<tr L?.5.m tially tlrc sartreforttt (but noi necessa. Thc txrrrstructiorrof a dfa for this language is similn. Since we have txlrrstnrc:tc:drr dfa for the lauguage.tnples see why this works.dfa for it. by definition. a.ccepts strirrg if atttl otrly if it begins rr. dfa fbr it.wcl rrt:cxlrr dfa that recognizestwo conseclttivc strings of essen'Ihe identica. filr ttxl trorn(lrrt wtl will sirnply attach a rrarrreto this farnilv.AccpprpR.r. : {utL. R. L. I is Let tr be the language in flxa.gc rr:gular by corrstructing a.rily . lf this is rxrt tirc end of the string. Tlrl sohrtiorr is cotnplicated by the fact that there is no explir:it way of tr:stirrg tlrtl errd of the string. it will take the dfa out of the final state.
44
Chopter 2 FtuIrn Aurolvlare
Figure 2.6
\=.
tl/
I t Jt bJL r i f+_!4 " * ' A ' * t ' r A (fitl,r aa YJ aa 9F{ r?,F j * _.v  t
in Figure 2.6 can be used as a starting point, but the vertex gs has to be modified. This state cafl no longer be final since, at this point, we must start to look for a secondsubstring of the form awa. To recognize secthe ond substring, we replicate the states of the first part (with new names), with q3 as the beginningof the secondpart. Sincethe completestring can be broken into its constituent parts whereveraa occurs,we let the first occurrenceof two consecutive be the trigger that gets the automaton into a's its secondpart. We can do this by making d(gs,a) : qa. The complete solution is in Figure 2.7. This dfa accepts,L2,which is thereforeregular.a
l
v
tnlr
, ,q ^ w
,T.u.rp"o u r t f t  i l
={ tu f)J^
4111 61aujLft
ri,,
Figure 2.7
t?rfl'b,'
t L,ilat"
t
t r ,,1,86 4."
r\t
?.1 DETERMTNTSTTC Frrrrrn Accnprnns
46
The last examplesuggests conjecturethat if a language is regular, the "L so are .L2.,L3. We will seelater that this is indeedcorrect, ....
1. Which of the strings 0001,01001,0000110 are accepted by the dfa in Figure
2.r?
f)t* E = {a,b}, constructdfa's that acceptthe setsconsisting of
(a) all strings with exactly one a, (b) all strings with at least one a, (c) all strings with no more than three a's, ffi
(d) all strings with at least one a and exactly two b's. (e) all the strings with exactly two a's and more than two b's.
3 . Show that if we change Figure 2.6, making qs a nonfinal state and making
qo1qt1 qz final states, the resulting dfa accepts Z.
4 . Generalize the observation in the previous exercise, Specifically show that if nI
"(fr)
A* G"ive
(a) I=
M : (Q,8, d, go, F) and
= (Q,8, d, qo,Q  F) are two dfa's then T(6
:
dfa's for the languages
{abswba:tue{a,b}} ffi
(b) fI : {wpbwz:
tor € {a, bI* ,wz € {a,b}}
Give a set notation description of the language accepted by the automaton depicted in the following diagram, Can you think of a simple verbal characterization 6f the language?
b
I
(T
g
( z) fi"a
dfa's for the following languages 61 5 = {a, b}.
(a) I=
{ta: ltalmodS=0} ffi
(b) Z, = {tr : ltrl rnodSI 0} (c) I = {w : no(trr)mod3 1} > (d) 3 {w : n* (tu)moct > n6 (w)mod 3} W
46
Chopter 2
FtmIrn Aurouare
(e) r,:
nu(tu))mod:} >0} {ur I (n*(ur) (f) , : {u : ln* (w)  nr, ('u)l rnotl:l < 2}
Nole
tl^t
* 7 rna:/ :t" !
q$
e run in a string is a silbstrirrg of length at least two, as long as possible anrJ conrristing entirely of the satre symbol, For instance, the string a,bbhaa,lt contains a rul of b's of length three and a nrn of n's of lertgth two. Find dla's for the following languages on {4, h}. (u) t : {tr : ru cxrntairlrr IIo rtllls of length less than four} (b) .L : {'ur : every rtrrr rtf a's }ras lengt,ir either two or three} (c) I : {'ur : t}rere are at tnost two runs of a's of length three} (.1) f : {tl : there are exactlv two ruus of a's of length 3}
t.}tg
Consider the set of striflgs on {0, 1} delined hy thc rcquircments below. For each construct an acccptirrg dfa. (a) Every 00 is fbllowed irnrnediatel.y by a 1, For cxample, the strings 101, 0010, 0010011001 are in thc languagc, but 0001 and 0010t) are rrot. ffi (lr) all strirrgs containitrg 00 but not 000, (c) The lef'tmost' symbol diffcrs frorn thc rightrnost one. (d) Every substring of four symbols has at most two 0's. Ftrr exarnple, but 10010 is not since one 001110 arrd 011001 are in the latrg;uage, of its substrings, 0010, contains three zeros. {il (e) AII strings of length five or rnore in which the fourth syrrrbol lrom the right erxl is tliffererrt frorrr the leftrnost sytrbol' (f) All strings in which the leftrnost two syrnbols a,rrd the righttnost two syrnbols are iderrtit:al.
*10.
Corrstruct a clla that accepts strings on {0, 1} if and only if thc value of the string, intcrprctcd as a binary representation ofan integer, is zero morlulo five' lbr example,0101 and 1L11, representing the integers 5 and 15, respectivel.y, are to be acceptcd' 11. Show that the language 7 : {uwu : 'u,w E {o, b}* , lrl : 2} is rcgular. L2. Show that tr : {.a," : ",> 4} is regular. 13. Show that the language L: It^: rt.) 0'n I Sf]o* that the langua,geL : {a" : n,: i, I ula,r. ) is rcgular. ffi (1,1,2, "'} is regih,i,,k fixerl, j :
[iQ ."
15. Show that the set of all rcal numbcrs in C is a regular lauguage. L6. Show that if L is regular, so is Z  {I}. Qf Use (2.1) and (2,2) to show that
d " ( s ,t u u ): d " ( d " ( q , w ), u )
frrr all tr,u € E'
?.2 NorurErEFMrNlsTrc Flrulrl
Accnpr'trrs
47
1 8 . Let L be thc language ar:cepted lry thc autorrraton in Figure ?.2. Find a dfa that at:cepts L2.
1 9 , I,et .L be the langrrage acceptcd by t,he automaton in "Figure 2.2. Firxl a dfa
for tlre larrguage Lz  L,
2 0 . Let I, be the language in Example 2,5, Show that .L* is regular, 2L. Let G,r.rhe the transition graph for some dfa M, Prove the following,
(a) If f (M) is infirrite, then G,y must have at least one cycle lbr which there is a path fronl thc initial vertex to some vertcx in the cyclc and a path frorn some vertex in the cyr:le to some final vertex. (b) If , (M) is finite, then no such cycle exists. ffi
2'
Let rrs define an opcration trun,t:a,te, which removes the rightmost symbol lrorrr arry utring. l,br example, trurtt:a,te(aaliba) is aaab. The operation can be extenderl to languages bv :ate(L): {truncate (tu) : trr € I}.
Show how, given a dfa for any regular langrrage 1,, onc can construct a dfa for truntate (tr). Flonr this, prove that if L is a regular language not containing ,\, then truncaLe (.L) is also regular. L e t r : a o e , r , , ' a m t a: b o b r . . . b n r z : c o c l ' , , c " , b c b i n a r y n u r r l b e r s s a defined in Exarnple 1.17. Show that the set of strings of triplets
(il)(li) (r)
where the di, lt.i, c; are such that ;r * U : z is a regular languagc, 24. Wltile the language accepted by a given dfa is unique, there are normally man.y clfa's tirat accept a language. Find a dfa with exactly six states that accepts the sarne larigrrage as the dfa in Figure 2.4. m
N o n d e t e r m i n i s tFc n i t eA c c e p t e r s ii
Finite act:rlpters rnorecomplicatedif we allow them to act rrondetermina,re istically.Nondeterrninisnr a powerful,but at firs{Er.Sh is unusualidea..$,3 gormally thirrk of computersas contplettilydeterministic,and the elernt:rrt of chijiEedeeihs out of piritie. Nevertheless, rioridctermintsmis a useful notion, as we shall seeas wc prot:eed.
48
Chopter 2 .l'l.rrtr: Aurounre
Accepter of Definition o Nondeterministic
Norrdeterminism mea,nsa choice of moves for atr automaton. Ratlter than prescribing a,uniqrrt: rnttve itt each situtr.titlrr, we allow a set of possible tnoves. Formally, we a,chievcthis by delining the trarrsition function so thirt its ratrge is a set of possible states.
onfilWin
A nondeterministic finite accepter or nfa is definedbv thc quitrtuple F) M : (e,X, rl,r/0, , but fitrite accepters, as whereQ, E, {0, F are deiinecl for deterministic d : Q x ( E u { A } )  2 Q .
Note that there are three major difli:rerrces between this definitiott ancl the definitiorr of a dfa. In a, nondctcrrninistic accepter, the rarrge of d is in the powerset 2Q, so that its vtr,hreis ttot a single element of Q, but a subset of it. This subset defines the set of all possible states that can be reached try the transition. If, firr irrstance, the current stattr is q1, the symbol a is read, atrd ,5(qr, *) : {qo, qz} , therr either 8o or 8z could be the uext state of thtl rrfa' AIso, we allow .\ as the second irrgutnetrt of d. This mcirns that the nfa can mtr.kt: a transition without corrsutningau input symbol. Although we still assurnethat the input rrrechanismcan only travel to the right, it is possible that it is statiorrary on some Inove$. Finrr,lly, in an nfa, the set ,) (ql, *) ttray be empty, mearrirrg that there is no trarrsition defined for this specific situation' Like dfa's, nondeterrnirristic accepters can btl representedby transititlrr graphs. Ihe vertices are rlt:tertnined by Q, while arr edge (q1,qr) with labcl a is in the graph if arrd otily if d(qi,a) contairrs {i. Note that sintxr n rllay be the empty string, there can be some edges labeled .\' A string is accepted lry au ltfa if thcrc is sone sequenceof possitrle rrroves that will put tlte machiue in a,firral state at the end of thc string. A string is rejcctecl (that is, not accepted) only if there is no possible sequenceof moves by which a firral state c:arrbe reached. Nondetcrrnitrism can therefortl be viewed as involving "irrtuitive" insight by which the best move ctrn be chosetr at every sta,te (assurning that the nfa warrts to accept everv strirrg)'
2.2 NoNnnTERMINIS'r'tc t'rrurlr:AccnrrnRs
49
Figure 2.E
i.WWMorrsidertItetrarrsititl'gpr'inFigure2.8.Itdescribesarrtrrrrltttttrministic accepter sirrcc tlx)rc irre two transitionslabeleda out of gg. I
WtrorrcIeterrrrirristit:a,rrtornatonisshowninFigure2'9.Itisrrorrd(lterministic not otily bcc:au$c several edgeswith the same label origirratr: from one vertex, but also bct:iurser has a Atransition. Some trarrsitiorr, suc:has,i(q2,0) are it unspecified irr tIrc graph. This is to be interpreted as a transition to the empty set, that is, d(g2,0) : g. The automaton acceptsstritrgs .\, 1010, and 101010,but not 110 and 10100. Notc tha,t for 1.0there are two alternative walks, one leading to qe, tlte other to q2. Even though q2 is not a final sta,te, the string is acccpted hs:ause one walk leads to a final stattl. Again, the trarsiticlrr firrrr:tion can he extended so its second argurnrlnt is a string. We require of tlrt: t:xtended transition function d* that if d. (q1, : Qr, tu) then Qi is the set of rrll possible sta,testhe automaton may be in, having uttr.rtedin state {1 and }ravittg rearl rir. A recursive definition of d*, analogous to (2.1) a,nd (2.2), is possible, but rrot particularly enlightening. A rnore easily appreciated defirrition t:an be made through transition graphs. I
Figure 2.9
'r^Y
0
Y,_
1'=l*^7
l.
\
0,r
Iz
50
Chopter 2
FINITE Aurolvrrrrn
Definilion
?.5
Fbr an nfa, tlre extendctl tratrsitiou function is dt:firred so that d. (qr,r) contains 4i if rr,nrlorrly if there is a walk in thc: transition graph fiom q, to e qy lir,bclt:d'u. This irolcls for all r7n,t1, Q and ru e E*.
Figure 2. l0 reprtlserrts att nfa. It has severtr,lAtrarrsitiotrs aud some undeflned tra,nsitirlrrssuch as d (q2,a). 'fhere is a walk labelod wc Sr,4rpost: wattt to find d* (qr, a) nrxl d* (,1r,.\). n, involvirrg two .\transitions from q1 to itself. tsy using some of the Atrdgtls arrd 92. twicur, wr: see that there are also walks irrvolvitrg .\transitions to q11 Thus , d * ( , 1 t a ) : { g o 'q r i q z } Sincethere is rr .\edgebetween {2 alrd q0rwe have irnrnediately that d. (q2,A) contains gs. Also, since any state can be reat:hetlfrom itself by making no move, a,ndrxlrrscquentlyusing no input symtrcll,tl* (q2,.\) also contains q2. There,ftlrtl 6* (qr, A) : {So,q'i} . Using as mir,ny .\tratrsitions as needed, you (rirrl also check that d * ( q r ,a a ) : { q o ,Q t , Q z I
I
The definition of d* through labeled wirlks is somewhat informal, so it is rrscful to look at it a little more t:loscly. Definition 2.5 is proper, sincc' between any vertices ui and r.r; there is either a walk labeled tu or there is not, inclicating that d* is cottrpletely defined. What is perltaps a little harder to scc is that this definition urn irlways be used to flnd d. (gi,ru). Itr Section 1,1, we descrihcd art algorithm for finding all sirnple paths bctween two vertices. We crr,nrrotuse this algorithm directly flirrce, as Exilrnple 2.9 shows, a labeled walk is not always a simple path. We catt ttrodify the sirnple path algorithrn, rernovirrg the restriction that no vettex or edge
Figurc ?.10
2.2 NoNDHTERMrNrsrrc FrrrrrnAccEprERs
51
can be repeated. The new algorithrn will now generate successivelyall walks of length one, length two, length tlrree, arrd so on. There is still a difliculty. Given a ru, how lorrg can a walk labeled trr be? This is not immediately obvious. In Exa"rrrple 2.9, the walk labeled a between {1 antl q2 has length four. Ihe problem i$ caused by the .\transitions, which lengthen the walk but do not corrtritnrte to the label. The situation is saved by this obscrvation: If between two vertices r.',.and ui thercl is rr.ny wa,lk labeled ,ar, t,herr thr:rc mrrst be sorne walk labeled u.' of lerrgth rro more than A + (1+ A) lrl, whcrc A is the number of .\edgcs in tlte grir.ph. The a,rgurnent for this is: While Aedges may be repeated, there is always a walk in which every repeatexl Aedge is separated by arr edge labeled with rr nonempty symbol. Otherwise, thc walk contains a cycle labeled .\, whir:h can be replaced by a sirnple path without changing the Iabel of the walk. We leave a fbrmal proof of tlfs clairn ir,$an exercise. With this observation, we have a,urethod for computing d* (q,;,ru). We elvalua,teall walks of lerrgth at trxrst A + (1 + A) ltul originating at tr,;. We select fiom them those that are labeled zr. The terminating vertices of the selectedwalks are the elements of the set 6* (qi,,ut). As we have rt:rnarked, it is possible to define d* irr ir, rercursivefashion as was done for tlte tleterministic case. The result is urrfcrrturrtr,telynot very transparent, arrd arguments with the extended transitiorr fun<:tion defined this way are hard to follow. We prefer to use the rnore irrtuitivc and more manageablealternative irr Definition ?.5. As for dfa's, the larrguagc acr:cpted by an nfa is definecl forrnally by the cxtended transition function.
lrR,Ffin',tllllellrii ,ti
The language L accepted by a,rrnfa, M : (Q,X,d,qo,F) is defined as the set of all strings accepted in the abovtl scrrsel.Fcrrma,lly, L ( M ) : { r i r € X * : d * ( q o , w )n I ' I n } . Irr words, the language consists of all strirrgs ur fbr which there is a walk labeled 'u I'rom the initia,l vertex of the transitiorr graph to some final vertex.
,
i
t
;
Exumple 2.10
What is the latrguage accepttxl by the a,utomaton in Figure 2.9? It is car,sy to see from the graph that tlrtl orrly way the nfa can stop in a final state is if the input is either a repetition of the string 10 or the empty strirrg. Theref'orethe automaton accepts tlr: larrgrrer,ge = {(10)"'; n > 0}. I
62
Chopter 2 FINITEAurouil,rt
What happens wherr this automaton is presented with the string tu 110? After reading the prefix 11, the automaton linds itself in state q2, with the transition d (q2,0) undefined. We call such a,situatiotr a dead configuration, and we r:an visualize it as the automaton sirnply stopping without further action. But we must always keep in mind that such visualizations are imprecise and carry with them some darrger of misinterpretation. What wc carr say precisely is that ) d * ( q 0 , 1 1 0= o . Thus, no final state can be rttached by processingu,  110, arxl hence the string is not accepted.
I
Why Nondeterminism?
we In reasorritrg about nondeterministic mat:hirres, should be quite cautious in using irrtuitive notions. Intuition c:an tlasily lead us astray, and we mrrst be able to give precise arguments to substarrtiatc our conclusions. Nondeterrrrinisrn is a difficult concept. Digital conrputers are completely deterministic; tlteir state at anv time is uniqucly predictable from the input and the initial state, Thus it is natural to ask why we study nondeterttftristic rnachifies at all. We are trying to rrrodel real systems, so why includc such nonrnechartical features as choice? We ca,n an$wer this question in various wiiys. Many deterministic algorithrns rcrluire that one make a choice at $omc) stage. A typical example is a gameplarying progrant. Flequently, the best move is not known, but ca,nbe f'rrrrndusirrg arr exhaustive search with backtracking. When several tr,ltt:rnatives are possible, we choose one arrd follow it until it becomes clear whcther or not it was best. If not, w€l retreat to the last decision point and explore the othtrr <:hoices.A notrdeterministic algorithur that can tnake the best choice would bc able to solve the problern withoul backtra,cking,brit a,dtltcrrtritristicolre can simulate nondeltcrmirristtt calt serve with some extra wrlrk. F<rrthis reason. noncleterministir:rnat:hirrrls rrr morlt:ls of searchandbacktracka,lgorithms. Nondctcrrnirristn is sometimes helpfirl in solvirrg probletns easily. Look at the rrfir itr F'igure 2.8. It is clear that thertt is a cltoice to be made. The first alternaLive leads to the acceptarr:t:of tlte string ail, while the seconrl accepts all strings with an even mrmhnr of a's. The Ianguage accepted bv the nfa is {a3} u {ot" : n > 1}. While it is possible to find a dfa fcrt this Ianguage, the nondeterminisrn is quite natural. The language is the urriotr of two quite difftrrcrrt sets, and the uondeterminism lets us decide at the olrtset whir:h case we want. The deterministic sohrtion is not as obviouslv
2.2 NounptrrRMtNrsTrc FrnrrrnAccnlrnns
5S
rclated to the definition. As we go orr, we will seeother and more convincing exarnplt:sof the rmefulness nondeterrnirrisrn. of In the sarrrcvein, nondeterminism is an effective rrrcr:hani$mfor describing some cornplicated ltr,ngua,ges concisely. Notice that tlrc definition of a gralrlrrlar invtllves a nondeterministic element. Irr ,9 a,9bl.\
we can at any point chooseeither tlrc first or the second production. This It:tu rrs specify many different strittgs usirrg only two rules. Firrally, therreis a technica,lreason for irrtroducirrg rrondctcrminism. As we will see, tltlrtirirr results a,re more easily established for rrfats thtr,n for dfats. Our rrext maior resrilt indica,testhat there is rro essential diffcrcnce betweetr tlrt:sc two types of automata. Consequently, allowing rron(lcterminism ofterr sirrrplifies f'rrrmrr,l arguments without affecting the gerreralitv of the conc:lusiorr.
l.
Prove in detail the claim made in the previous section that il in a trarrsitiorr graph there is a walk labelerl rl, there must be some walk labeled tu of length rro rrrore tharr A + (1 + A) l,rrrl. Fitrd a dfa that at:r:eptsthe langua,gedefined by thc nfa'in Figure 2.8. I n F i g u r e 2 . 9 , f i n d d * ( q 6 ,1 0 1 1 )a n d d * ( g 1 , 0 1 ) . ffi
, 3.
4 . In Figure 2.10, Iincl d (qo,a) and d* (r;r,l)
5 . Fbr the nfa,in Figurc ?.9, find d (qo,1010) and d* (t71,00).
6
O
Design at nfa with rto rnore than five states for thc sct {abab" ; rr.} 0} U {a,ha'o:rr,>0}. C.rrr"t.,rct an nfa with three statcs that accepts the language {tr,b,abc}. W
8 . Do yorr think Exercise 7 can be solvccl with fewer than three states'l ffi 9 . (a) Firrrl an nfa with three states that acccpts the language L : {a" : rz > 1} u {I,*aA' : rrr } fi, fr;> n ) ' t
(b) Do you think the larrgrragein pa,rt (a) can bc a,cccptcd lry an nfa with fcwcr than three states'/ \lpt' @ l,'ind an nfa with lbur states lbr L : {a" : rr > 0} U {h"u.: n } I}. Wtli"tr of thc strings 00, 01001, 10010, 000, 0000 are arceptetl by the following rrfa?
54
Chopter ? Flt{trr Aurolrere
0
/^\f \?o/
\__.\
12. What is the complement of the language accepted by the nfa in Figure 2,10? 13. Let .L be the language accepted by the nfa in Figure 2.8. Find an nfa that accepts I u {a5}. 14. Give a simple description of the Ianguage in Exercise 1.2. (rs.)fina an nfa that accepts {a}* and is such that if in its transition graph a single edge is removed (without any other changes), the resulting automaton accepts i") W 16. Can Exercise 15 be solved using a dfa? If so, give the solution; if not, give convincing arguments for your conclusion. 17. Consider the following modification of Definition 2,6. An nfa with multiple initial states is defined by the quintuple M :(Q,E,d,Qo,F),
where 8o C Q is a set of possible initial states. The language accepted by such an automaton is defined as L (M) : {tr.' : d* (q6,trr) contains gy, for any q0 € Qo,St € F} . Show that for every nfa with multiple initial states there exists an nfa with a single initial state that accepts the same language. Nft 18. Suppose that in Exercise 17 we made the restriction Qo fl F : fi. Would this affect the conclusion? ( rg./Use Definition 2.5 to show that for any nfa
d (q,uu) :
U
pE6* (tt,u)
d* (p,r) ,
for all q € Q and all trr,u € D*, 2o. An nfa in which (a) there are no trtrartsitions, and (b) for all g e I and all a e E, d (q, a) contains at most one element, is sometimes called an incomplete dfa. This is reasorrable sirrce the conditions make it such that there is never arry choice of moves.'
2.3 EQUIvaIENCE DptpRturt'ttsrrcANDNoNnnrnnN.rrNrsrrc oF FInrIre AccnRrnns
DD
For E : {a, b}, convert the incomplete dfa below into a standard dfa.
W
E q u i v o l e n co f D e t e r m i n i s t ocn d e i N o n d e t e r m i n i s t iF i n i t eA c c e p t e r s c
We now come to a fundamental question. In what sense are dfats and nfa's differerrt? Obviously, there is a difference irr their definition, but this does not imply that there is any essential distinction between them. To explore this question, we introduce the notion of equivalence between automata.
lIM
Two finite accepters.Ml and M2 are said to be equivalentif L(Mr): L(M2,, that is, if they both acceptthe samelanguage.
As mentioned, there are generally many accepters for a given language, so eny dfa or nfa has many equivalent accepters.
.11 is equivalent to the nfa in Figure 2.9 since they both accept the language {(10)" : n, } 0}.
Figttre 2.11
The classesof dfa's and nfa's are equally powerful: For every language accepted by sorrrerfa there is a dfa that accepts the same language. therefore d ({sn}. we may not know exactly what state it will be irr. Most of the work in this suggested construction Iies in the analysis of the nfa to get the correspondence betrveen possible states and inputs.nite number of states.inly lms to be demonstrated. Iet us illustrate it with a simple example.qz}. But the converse is not so obvious.. A state labeled g represents an impossible move ftrr the nfa and. like most argurnents in this book. it is clear that any language that is accepted by a dfa is also accepted by some nfa. the tfa has no specified transitiort wltetr the input is b. After arr rrfa has read a string ?u. so it is at least corrceivable that there is ir larrguage accepted by some nfa for which we cannot find a dfa.. so the initial state of the dfa will be labeled {qs}. means nonacceptance of the string. a ) : {qt. Let us look at this cluestiott for finite accepters. The rationale for the construction is the following.12 to arr equivalent dfa. the nfa c:an be itt state q1 or. once the principle is clear it becomes the starting point for a rigorous argument.. Before getting to the formal description of this. the corresponding dfa will have a fi. Conseqrrently.. after reading ru. How can we make these two situations correspond? The answer is a nice trick: Iabel the states of the dfa with a set of states in such a way that. Since for a set of lQl states there are exa.\transition. This result is ttot obvious and certa.Ui.. The argument. therefore.QnI be in orre state of a set of possibltr dfa after reading the same string rrrust be in some definite state. But it turrrs out that this is not so.56 Chopter 2 Frrurrp Auronara When we colrlpare different classesof automata.. will be constructive. in state q2. the question invariably ariscs whether one class is more powerful than the other. saY {qz. Since a dfa is in essencea restricted kind of nfa.t:tly2lQl subsets. by making a . This me&ns that we can actually give a way of corrverting any nfa into an equivalent dfa. tlre equivalentdfa will be in a sitrglestate labeled {Uo. In state q0. Therefore the corresponding dfa must have a state labeled {gr. qz} and a transition d ( { s n }. By rnore powerful we mean that an automaton of one kind can a<:hicvesomething that cannot be done by any rrutomaton of the other kintl. The nfa starts in state gs.Qjr. The corrstruction is not hard to understand.qa}.. Convert the nfa in Figure 2. We htrve added nondeterminism. After reading an a.An equivalent states. . but we can say that it must . this state in the dfa mrrst be a nonfinal trap state.b) : @.
shown in Figure 2.ENCE DnrnRrr.1S . equivalent to the nfa with which we started.12 accepts any string for which d* (q0. If. qz}. F\rthermore. I Figure 2.13. b ) : {qo} . so we need to find the transitions out of this state. every state has all transitions defined. Therefore d ( { q t . then there is no specified transition. Remember that this state of the dfa corresponds to two possible states of the nfa. For the corresponding dfa to accept every such w7 araystate whose label includes 91 must be made a final state. so we must refer back to the nfa. a ) : Similarly. for the same input.ru) contains 91. At this point. The nfa in Figure 2. { g r .12 We have now introduced into the dfa the state {h.uNrsrrc oF ANDNoFrnErrgRMINrsTIc FrNrrn AccBnrens 57 Figure 2. q z }.q r } . q z }. is a dfa. it can go to q1.2. If the nfa is in state Q1and reads an a.3 Eeurvar. from 91 the nfa can make a .\transition to q2. the nfa is in state q2. The result. d ( { q t .
E. Fp) such that L: L (Mp). and label it 3. Therr form the uniotr of aII these d*... Proof: Given M1y. Bui by construction.q*.rrfrom qo to q. To understand the constructitln? relllefiIber that Gp has to have certain properties. but the until they are all there.L be the language accepted by a nondeterministic finite accepter M1" : (8r.gi.iand an edge (or a sequence of edges) labeled a liom qt to qt. C o m p u t e d * ( g 1 . By the irrductive assurrrption. we use the procedurc nf atotlfa below to construct the trarsition graph Gp for X.Fry). Identify this vertex as the initial vertex.. 2. dry.. Repeat the following steps until IIo lrlore edges are missing' Take any vertex qo. Q*.'}. Each pass through the lXl Ioop in Step 2 adds arredge to Gn' But Gp has at mo$ 218'^/l edgcs.. the presence in G1y of a walk labeled u from qo to qi implies that in Gn there is a walk labeled tr from tsol to a state 8l : {. Then there exists a deterministic firrite accepter Mn : (8o. each labeled with a different element of E. During procedUre continues construction. d * ( q j .qi. '. {t} of Gn that has no outgoing edge fbr sorne a€8. we argue by induction on the length of the input strirrg' Assume that for every u of length less than or equal to n.ny qy € Flv is identified as a final vertex.'. some of the edgesmay be missing. there will be an edge from Qa to sone state whose label contains ql. {qo} ...Consider rrow any u : ua and look at a walk in G1r labeled r. E.{'} with a.. in Gp there will be a walk labeleclu from {qo} to Qa. If MN accepts )."'. . d. .Chopter 2 FIruIrn Aurovete Let .. so that the loop eventually stops.qj. .tIp.. Thus the inductive iutsumption . Every vertex must have exactly the lxl outgoing edges. a ) . d * ( q * . create a graph Gn with vertex {qs}. a ). o .{'} exist.4t} to tqt. Every state of GIr whose label containu a.. There must then be a walk labeled u frorn qo to Q. . yielding the set IU..QnI' if it does not already Create a vertex for Gn labeled {qt. Add to Gp alr edge from {qr. 4.qo. procedure: nfato dfa L.l. To show that the constru<:tiorr also gives the correct aIISwer. the vertex {So} i" Gsr is also made a final vertex' It is clear that this procedure always terminates.."'.q*.
The construction in the above proof is tedious but irnportarrt. Q z I. i : 0. so tl(xruthe label of di(gs. Since there a.t. are admittedly somewhat terse. J : 1. showing only tlte ntajor steps. tr. There a. rnnsidering dlu (qo.rrNrsTr(l !rNrrp AflfletrnRs oF 59 lnlds fbr all strings of length rz * 1 As it is otrviously trrrt: for z : 1.2. we reverse the argrrtrent to show that if the label of di(qs. The result then is that whenever dir(So.q.mple to make sure we understand all the steps.r r s i n ga .HN(rE DnrnnMlr'usrrc AND Nolotrpn.l : I.(q0.16.2.s r } . With o : 0.re now a number of missing edges. 2. In the $irrnewity. e r .ils.we continue until we obtain the complete sohrtion in Figure 2. emphasizing the basic ideas irt ir proof arrd omitting minor deta.etransition d n ( { q o .15.qr} in Gt and add an edge labeled 0 between {q6} and {. so we continue. To complete the proof. using the (:orrutructionof Theorem ?..ez} and th. we have the partially constructed autornaton shown in Figure 2. sr] makes it necessaryto introduce yet another state {q1. dfu 1)u (ti/(q'. t) : {qt} gives us Lhe trew state {q1} arrtl an txlgc labclud 1 between it and {(ro}. (sr.14intoatlequivalentdetennitristictrraclrine. we introduce the state {qo.restill some missing edges.qt}. I Figure 2. : I .lo.sr}.ncltherexa.q.lthough correct. (so. .g'.n .trr) contains a firml state 91. : Sittcc t)1. This givus us thc new state. ri : 0.ru).It.0) {qo.qzI .tu) contains g/. We will fbllow this practice in the rest of the book. which you may want to fill in yourself. k = 2.14 . r Ihe argumerrts irr this proof.tt). Lnt uu rlrl rr.1)u dfu 1): tsr.'{qo. it is true for all n.rr.qz}. At t}ris point. we compute di' (Sn'0) U div (qr' 0) : {. 0 ) : { { 0 . T l r c r r . so rrrust dh (qo.ll EeuIv.
/ { .Chopter 2 Frutre. Figure 2.15 (lrl r. Autottl'rn Figrue ?.16 @ v'l\ lrrJ) @ 't\ V. ___. ./') \./ .
ur) contairrs 97. . \0 4. / . Carefully cornplete the arguments in the proof of Theorcm 2.\transitions and with a single final state that accepts I.u) nF: J counterexample.2. > 1}.\trarrsitirrns and with a single firral state that accepts thc sct {a} u {b"' : n. the complement of I (M) is F) ft? fr it trr. tXZ/\ p.d..re that ftrr every nfa M * f eQual to the set {tr e E.2 into an cquivalent rlfa.x.ur)rr(Q . II not' give a equal to thc set {u e X* : d* (qo.o"* ttrat for every nfa with an arbitrary numbcr of final rtfates there is an eqrrivalent nfa with only one final state. r7o. Use the construction of 1lheorem 2.10 to a dfa. erpivaletrt to thc given one.tr. d.{o.16..2. then dfr (So. '? : (Q.. Convert the nla in Exercise 11. Show that thcre exists arr rrfa without . ffi Show that if /.prove it.2.d" (qo. Can you see a sinrpler an$wer rrrore directly? 2. Show irr tletail that if the label of di. Define a dta with multiple irritial states in an analogorrsway to thc corresponding nfa in Flxercise 17. Can we make a siurilar claim for clfh's'l ffi {a tl t i" + @) j #) Firrd an nfa without . ffi /ftor'rve. if not' give a r:outtterexantple. Give a sirnplc verbal descriptiot of thc la. 1.F) # s)'l If .nguagear:r:eptedby thc dfa in Figure 2. Sectiorr 2.')also corrtairrs q1' it true that ftrr arqynfa M : (Q. Section 2. prove it.F) the cornplcment of I(M) is 4 (DIs * @}? If Bo.t the ftrllowing nfa into an equivalerrt tlfa.is regular' so is IE' 13.2 to convert thc nfa in Figrrre 2. L0. .l. but with fcwer statreB. Use this to find another dfa.. X.(q6.3 EeUIVALIINCEoF DErnRuIuIsuc AND Nounu'rpRltlNISTIC FINITE AccEP'I'ERS 61 One important conclusiolr we can draw from Theorem 2'2 is that every Ianguage accepted by an nfa is regular.so. f)oes there always exist au equivalerrt clfa with a single initial state'l Provc that all finite languagcs are regrtlar. ffi 9J Let Z be a regular langrrage that does not contain.
In terms of the questions we have corrsideredso far. For a givcrr language.. so is euen (I)..and it carr btr removed (along with tr..a2o"3a 4.. ffi by renxrvirtg the two Provc that if Z is regular.L we create a rre.'Ti ...17(a).wlanguage chopL (l) 'efrmosr svmbors :.chedfrom the initial state q0. t t e n\ w ) : tt'2a4.ll transitions relalilg to it) without affec:tingthe lrrrrguageaccepted by the irrrtomaton. Corresponding to this. ffiL.Chopter 2 FIuIrn Aurov. :TjTl'.op2 is also regulat.. 15. that is.L is regular then cft.0) rrrirror those reachable frotn a first move d (So.. we can definc a language eaen(L): l e u e r t( u .but the colrvortteis not true. I .re to be applicd in a practical settitrg. ffi R e d u c t i o n f t h e N u m b e ro f $ t o t e si n o Finite utomofo* A Any dfa defines a uniqrte la. there are rnany dfa's that accept it. The states rcachable subsequent to the Iirst move d(S0.nguage. the first tlrttomatotr has sollre redlrndant parts. but if the results a. (tr) Show that if .17(b) are equivalent' a$ ir fcw test strings will ctuickly reveal. Thcre may be a considerable diflerence in the numlrer of states of such equivalertt inrtomata. Frtrm a Ianguage . The state q5 plays absolrrtelv no role in the autornrrton since it can nevet tlt: retr. Such a statt: is inaccessible.L be any langrrage. We notice sottre obviottsly unnecessary fcatrrres of Figure 2.arm {i+\  f. The secorrdeurtomaton cotnbines these two options. then e . .red positions. f)efine euen (trr) as the string obtained by extracting from tu the letters irt evenmrmbe. tht:re may be reasolls fcrr preferring one over antlther..1). if w : at."t . all solutiorrs arc equally satisfactory.17(a) and 2. But even aftcr the removal of q5. The two dfa's depictcrl in Figure 2. ): ' r r re I ) .
or vice vcrsa. r) € F implies d. If. We now describe an algorithm that accomplishes this. (9.17(a). For storage efficiency. there is little reason for preferrirrg the automaton in Figure 2. there exists some string u e E* such that d* (p. then the states p and g are said to be distinguishable slrrng ?{).1? @ From a strictly theoretica. on the othcr harrd.r) € F and 6* (q. by a .u) f F. it is desirable to reduce the number of stir. rii0it iltlLfi i if Two states p and q of a dfa are called indistinguishable .u. lbr all tu € E*. However.tes as far as possible.l point of view. ro) € f'.') f f' irnplies 6* (rt.5*(p.4 RpnucuoN oF rnn NuunnR on Smrus IN FINITEAurouare 63 Figure 2. in terms of simplicitv. Representation of an automatorr for the purpose of computation requires space proportional to the number of states. and d* (p. the second alternative is clearly preferable.2.17(b) over that in Figure ?.ut)f F.
mark (p. One method for reducing the states of a dfa is ba^sedon finding and combining indistinguishable states. all states distinguishableby strings of length rz or lesshave beenmarked. q ) a n d a l l a e X . and all three states are indistinguishable.q). We claim that this procedure constitutes an algorithm for marking all disiinguishable pairs. mark the pair (p... then fro are p and r. F o r a l l p a i r s ( p . 3.a) qn and 6 (qi. In step 2. Note first that states q. two states are either indistinguishable or distinguishable. Consider all pairs of states (p. a ) : eo. with q6 and g1distinguishableby a string of length n .\. terminates and determines all pairs of distinguishable states. If p e F and q fr F or vice versa. o ) : p o a n d 6 ( q .and qj are distinguishable with a string of length ru.q0. If the pair (po. We now a.ssume .Chopter 2 Furtre Auroneta Clearly.rked pairs are marked.F).q) as distinguishable. 2. E.eo) is nmrked as distinguishable. The only claim that requires elaboration is that the procedure fiuds all distinguishable pairs. procedure: mark 1. c o m p u t e5 ( : p . We first describe a mothod for finding pairs of distinguishable states. Remove all inaccessible states. q) as distinguishable.6) for somea € X. we mark all pairs indistinguishableby . since there are only a finite number of pairs that can be marked.5) (2. The procedure nlurlr. This can be done by enumerating all simple paths of the graph of the dfa starting at the initial state' Any state not part of sonte path is inaccessible.lWe usethis first to showthat at the completionof the nth passthrough the loop in step 3. 6. if and only if there are transitions : 6 (qr. applied to any dfa M : (8. the procedure terminates. Repeat the following step until no previously unma. Indistinguishability has the propertie$ of an equivalence relations: if p and q are indistinguishable and if q and r are also indistirrguishable. Proof: Obviously.o) : qu (2. It is also easy to see that the states of any pair so marked are distinguishable. so we have that the claim is trrte a basiswith rz : 0 for an induction.
1 ...qj. Qj. h for M.1.1n .add to d a rule TU. Use procedure mark to lind all pairs of distinguishable states.4 RbDUCT'roN rsp Nulrnnn.j . By induction then.. such that any (t E I occurs in exactly one of these subsets.'...5) and (2.1 have been marked.i'"k. all pairs distinguishable by strings of length rz or less have been marked.q^....'. find the sets of all indistinguishable states.'. we can claim that. To show that this procedure marks all distinguishable states. as describedabove' 2 .8'}. For each transition rule of M of ihe form 6 (q. 1. assume that the loop terminates afber rz pas$e$.rQn}. .6) above. at the end of this pass. . {qr.tQn}. that elements in each subset are indistinguishable. and so on.. create a state labeled i..q0..F)..d.. we use the results to partition the state set Q of the dfa into disjoint subsets {qn. find the setsto which q' Ttd qo belong. But if there are no states distinguishable onlv by strings of length n. Qj.. all states distinguishable by strings of length up to rl . {qr. By this inductive assumption..a):Im.whett the loop terminates. Becauseof (2..a) : qe. we construct a reduceddfa as follows. and that any two elements from different subsets are distinguishable.q*r. it can be shown that such a partitionirrg can always be found.6).qr} and € Qp {qt. Flom these subsets we construct the minimal automaton by the next procedure.5) and (2.. Then from this. procedurer Given reduce fr: (8. all distinguishable ptrirs have been marked..2. . enl of such irrdistinguishable states. for any TL. For each set {qa. Using the results sketched in Exercise 11 at the end of this section. all states distinguishable by strings of length up to n will be marked.. If q' € {U. 3..'. Flom (2.n.. at the beginning of the nth pass through the loop. As a consequence. r After the marking algorithm has been executed. op Srarns ru Frlrrrn Aurotrlnrn oF 65 for all i : 0. but not distinguishable by any shorter string. . there cannot be any states distingrrishable orrly by strings of length n * 1. This means that during the nth pass no new states were marked..Q*. etc'.E..Qnl1...at the completion of the nth pass. it then follows that there cannot be any states distirrguishable by a string of Iength rz.ei. say {ft.Qr}.
qs} ancl {ga}.(qr. the procedrre mark will iclentify distinguishable pairs (qo.g+) is a distinguishable pair.q+). the states q1.1 ) : g r . o.qn).gt). and (g3.66 Chopter 2 Fmrrn Aurorr.l (gr.Therefore. The initial state f11is that state of M whose label includes the 0.{2.19. the procedure computes .rnra 4 . the marking algorithm eventually marks the pairs (qo.qz). Consider the automaton depicted in Figure 2.q+).qn) and (qs.qr) is also marked. kh. (qo.19 . qz.qa)as distinguishable.18 Figure 2. F ir th* set of all the states whose label contains i such that ga € F. 1) : q+ and d ( q u .q3) and (qr. Since (qs.gt). (qo.n3 &r€ all indistinguishable. the pair (qo. (qr.ga)' In some pass tlrrough the step 3 loop.q+). qz).g+). and all ofthe states have been partitioned into the sets {So}. I Figure 2. In step 2. {qr. (qz. (qo.m). (h. Continuing this way.18. Applying steps 2 and 3 of the procedure reduce therr yields the dfa in Figure 2. leaving the indistinguishable pairs (q1.
wn srtch that .Tr. .Auroraarn 2..): qi if arrd only if the label of F(ga.wk) .such that and d i ( q o .. Assume that there is art equivalent dfa M1. Sincepl andp. 2 .j....r or rejects both.. ) . . contradicting the assumption that M and M1 are equivalent. application of the procedure reduce yields another dfa fr suchthat L(M):"(fr) F\rthermore.'trz... This is relatively easy and we can use inductive argument$ similar to those used in establishing the equivalence of dfa's and nfa's.r*) : d* (pt'z) is a nonfinal state (or vice versa). trrr) .tupn): di (di (qo.. there must be some strin&z such that d.l. n) * dl (di (qo.pt. . rurr is accepted by M and rurr is not. We will leave this as an exercise. In other words. Since there are no inaccessible states in M. Thus. d " ( p o w t ): P t . but with fewer states. This contradiction proves that M' cannot exist.. (po. withps the initial state. m . Proof: There are two parts. I . Suppose M has states {ps. But note that di (so. equivalelt to M.pz.. M1 either acceg$ both tlrz and ur. The second part.ut). to show that M is minimal.r) : . M is minimal in the sense that there is no other dfa with a smaller number of states which also accepts L(M). . sa. are distinguishable. . . with transition function d1 and initial state gn. (g.Fm. The first is to show that the dfa created by reduce is equivalent to the original dfa.ur) is of the form. But since Mr has fewerstates than fr.4 RpnucuoN oF rHn NuNaenR Smrns IN FINTTE on 67 Given any dfa M.li (qo. there must be at least two of these strings. upr) : 6* (pn.. there must be distinct strings 'trr./?116 ur1. All we have to do is to show that d. i : 1 ....z) is a final state. and d* (qn. 1 .w n ) : d i ( s 0 . is harder.
Prove or disprove the following conjecture. Write a cornputer pr()grarn xhat produce. Show the explicit steps of thc suggested proof of the first part of Theorem 2. f 1 states. arxl g.> 0 . namely. Minirnizethe rturrrber statesin the dfa in Figure 2.E.. ffi and q..L. ** g.l in . F) is a' minimal dfa for a regrrlar la.d. If M : (Q. / .L is a norrernPty langrrage .. ffi f1/ Strow that if .go.tn'r'e of 1.l (d ltrrq below. is re.Q .16.. 8.ce deterministic..Chopter 2 FIullp Aulor. 10. then any dfa accepting L rnust have at least n.such that any r. and if q. X.l dfa. . Showthat the automatongenerated prot:etlure fr) Virri*i"e the states in thc dfa depicteclitr the ftrllowing diagram. that M is equivalent to the original dfa.* that inrlistinguishability is an equivaletrcerelation but that rlistinguishabilitv is not. for Z. mrtst be distinguishablc. In eachcaseprove that the rcsult minimal dfa's for the larrguages rs mlnlmal. Prove the fbllowirrg: If the states g* and qa are indistinguishable. (*) fI:{o"b"''n}Z.f') is a minima. by 3. 6.gc then M : (Q.L has length at lea^strz.t. ffi f.4. q6. r l l 3 } ffi {a^:nt'2 andn+41. therr r71.du.mlll (b) r : {a"h: n> 0} U {b'a : n } 1} (c) I: (d)r: : { o " o n .ngua.ga rninimal dfh for any given clfa. arc distinguishable.Z) Sl. .
. saY. Then take another state. Put all states rrot marked distinguishable from ge irrto an equivalerrce set with qcl. and prove that this algorithm does indeed partition the original state set into equivalence sets. not in the precedirrg equivalence set. Then formalize this suggestion to make it an algorithm. Start with some state.2. to be done after the corrrpletion of the procedure rnar. {0.' rxn Nuvel.b.4 RnnucrloN oI. and do the sarne thing' Repeat until there are no more states aw. Consider the following process.R oF STATESIN F'INITE AUToMATA 69 11.ilable.
.
for exarnple.pter for it. and +. if wr: vrant to show the logic by which we decide if a given string is in a certairt language. But in many instances. . every regular language can be described by some dfa or some rrfa. Such a description can be very useful. we Iook at other ways of representinpJregular languages. Expressions ffiM Regulor One way of describing regJular languages is via the notation of regular expressions. for which. Slightly rrrore complicated is the language {a. These representatiorrs have important practical applications. b. Therefore.. a Ianguage is regular if there exists a finite acce. a rnatter that is touched on in some of the examples and exercises.R * g u l q rL q n g u q g e s ond Regulqr Grtt m mq rs ccording to our definition. c}. which will be denotetl by the regular expressiona. In this chapter. This notation involves a combination of strings of symbols from some alphabet E. The simplest case is the language {a}. we need more (roncise ways of desr:ribitrg regular languages. parentheses. 71 . and the operators *.
it$.aa. ri. (a + b+) is not a regular expression.. a.\. we hirve the regular expressiona+b+o We use ' for cont:eltenation aud + for starclosttre in a.. hcbc. and (r1).g.(c+o) is a regular expression. that is. If 11 a. Then l . a.abc. {.72 nruDReculAR GRAMMARS Chopter 3 Reculan LANGUAcE$ usirrgthe * to denote uttiott. #.. similar way. we eventually generate the whole string.ril Let E he a. Repeating this. of Expression Formol Definition o Regulor from prirnitive constituentsby repeatedly We construct regulrrrexpressions applying certain recursiverules. bc. since it is constructed by application of the above rules. c)* sta.These are called prirnitive regular expression$.1b. and n € X are all regular expresrtions. are rt*rz. I . On the other hand.rfltltil .. if we take 11 = c and rz : fr. the language hco. we find that c * fi and (c * o) a"re also regular expressions. (a+b'c)'.rrdsfrrr the starclosure of {a} U {bc}. }.lr.nd12 are regular expressions. 3 ..ii.\.firril. since there is no way it carr be constructed from the primitive rcgrrlar expressions.given alphabet. The expression (o.q. A string is a regular expression if and only if it can be derived from the primitive regular expressions by a firritc mrmber of applications of the rules in (2).rtrz. aaa. so 2 . ubr:. This is similar to the way we con$truct f amiliar arithnteticerxpressions. For example.
. 3. I((r1)): tr(rr). } . Exhibit the language (a* . ( a + b ) ): L ( a * )L ( a + b ) : (r (a)). 7.a a br .b . a ) . L(rfi: (I(rr)). b } . If 11 and rz are regulaxexpressions.nculan ExpRpssrorls 7I Longuoges Associoted Regulor with Expressions Regular expressions be used to describesomesimple languages. 5 . for everya € E. a is a regularexpre$$ion denoting{a}.f. l. . 6. .we apply theserules repeatedly. This languageis definedas follows: The languagetr (r) denoted by any regular expre$sion is defined by the r following rules.} : { a .3.L (rr) u L (rr). L L ( o * . a . (a + b)) in set notation. $*qmpfq 5. e a r . . will let . 2. L ( r r . L (r1 r r2) . . r r ) : L ( r 1 )L ( r 2 ) .1 R.{ o . I .L (r) denote the languageassociated we with r. The last four rules of this definition are usedto reduceI (r) to simpler componentsrecursively. e .rexpression.If r is can a regula.\ is a regularexpre$sion denoting{A}. a a .a b . . a a r o . (o)) (r : { t r .(a)u r. the first three are the termination conditions for this recursion. To seewhat languagea given expression denotes. fi is a regular expression denoting the empty set. e . then 4.
the expression r=(&+b)(a+bb) is regular. The first part. I . Consider. but this gives * crrmbersome results. The second part.Chopter 3 Ruculnn Laucuecns aNo RBctiLA. To overcomethis. It denotes the Ianguage L (r) : {a. for example.rz. They define a Ianguage precisely if 11 and r? are given.n. Goirrg from an informal description or set notation to a regular expression tends to be a little harder. GR. We establish a set of precedence rules for evaluation in which starclosure precedes concatenation and concatenation precedes union.aa.2.3 For E: {a. so we can write r1r2 for rL . the regular expression a ' b + c. c } ..} . L(ab*c) we could require that all expressiolrs be fully parenthesized. I n t h i s c a $ ew e f i n d t r ( a ' b + c ) : { a b . . (a * b)*. I The expression y : (aa)" (bb).b}. L (r) is the set of all strings on {a. Exomple3. r L : & ' b a n d r z : c .bbb. Wo can consider this as being made up of . the symbol for concatenation may be omitted. terminated by either an a or a bb.b denotes the set of aII strings with an even number of a's followed by arr odd number of b's.I\. We now get a different result.IaRS to*bK\"^"{ There is otte problem with rules (a) to (7) in Definition 3. With a little practi<re.abb. * > 0}. B r r t t h e r e is nothing in Definition 3.. (a * bb) represents either an a or & double b.bb. We can seethis by consideringthe various parts of r.b}.ac}. we use a convention familiar from mathematics artd programrning languages.ba. Instead. Consequently.Euity in breaking a complicated expression into parts.c\. that is L(r) : {a2"b2*+t : n } o. : {ab.2 to stop u$ from taking rt : a and 12 : b * c. we can see quickly what language a particular regular expression denotes. stands for any strirrg of a's and b's. Also. but there may be some amhri.
is sirrcrrthe utringu ending in 0 or c:onsistingof all l's a.rexlrressionsfor any given langua.nswer still incornplete. that is. ' 1.r expression suchthat r L(r): one pair of consequtive zeros}.5. One helpful observation is that whenever a 0 occurs.n derive a variety of rules fbr simplifvirrg rtrgrrlirr .cur.ngua. 101 . .ge.rrswcrs are corl'ect. I r the la.. 1}.regula. Putting these observations together. givea.he close relationship between the languages.Although tlrc two erxpressions krok rliflilront. However.1 Rl.st One catt arrive at alt arrswer by relirsclrrirrg sorntlthing likc this: Evcry utring in I (r) must conlaitr 00 somewhere. we arrive at the solution r . the answer is harder to construct. Howtlvtlr.rnlimited nunrber:of regula.nn FlxpREsSroN$ . the shorter expression ?':(1+01)(0+I) rniglrt llu rua.ccollntedf'or.r cxllrcssiorrs ar(l rrot verv sirnililr irrrrl do ttot suggest clearly l. the a.^ uctr subsrrirffi )7 : ( r * 0 1 1 *( 0+ A )+ 1 ( 0+ A ) .t lea. 0 0( 0+ l ) . D r H*Wnrple 3.r expressions.sa.rexpression(IilI I+)". {rrr e X* :ur ha.ly by atr arbitrary nutrrber of 1's. An arbitrary strirrg orr {0. tlrc rc:gtrlir.5. but what rxlrnc:sllc:frrrr:irnrl what gorrs after is completely arbitrary. After takirrg care of these special caseswe arrivc at thc arrswcr s a H_Li. If we see . I The last example introduces the notion of equivalence of regula..re una. Generally. 1} carr bc dr:rrotrxl by (0 + 1).ge. Even though this looks similar to Exanple 3. One ca.'guage 1 : {tu € {0.t thr: irrrswcr irrvcllvr:stlrrl repetition of strings of the form l . lhere are an r.: ( 0+ l ) . 1}.t:Ircrl.ts.We say the two regular expressionsare equivalent if they denote lhe same la. as they denote the sarne language. we rniglrt corrrc up with arrother arrswer.rrguagcis tlrc txlrnlllcmt:nt of thc languagc irr Exarnple 3. : rl }rirsrro llair of rxrnser:rrtivo zeros} .' . both ir. If we reasorr sliglrtly differently. the language denoted by the regula. This suggcsts thrr.L as the repetition of the strings 1 arrd 01. Notc that this la.S For X = {0. it rnust be lg[g*gg"gi.
. < (r:) The corrrplelnent Ir. ffi (#..N'lN{. ffi 12.5'l ffi Show that r : (1 +01 ). but since we this.itritrg IIL) Illore than three a's.denotc thc languagein F)xarnple 3. Firxl a regular expressiorrfor f : Iu E {0. ( c l all strings that contain at least ollc occurreltce of each symbol in r.n'rn> (9i Cit* a rcgularcxprcssiotrorL: 3]. jl. L".r.ncuLeR GR.'"b"' : Tr. (a) Ir : {1t"ll". fO. ((t) Does the expressiorr + 1) (0 + 1)). L1. have littkr rreed for such rnanipulations we will not pursr"te 1 . b .4' {uut : t. .tu e {a. of tlre larrguage L((au)r b(aa). ancla# clcnote? do Wfrot languages thc cxprcssions Give / \ q a I sirlple / \ + \ verbal ). O (b) . $ f fanb^:rtlL. Give regular e.+ LL \aLL ) Dil.76 Chopter 3 F.uacns aNn R.. (.rrrt. Lauc. I} : ur has exactly one pair of consecutive zeros} .rirrirrg (l') all strings corrtir.q.m.4) Firxl a regular expressiotr fbr the set {a'b"' : ('rr*'rn) is even}.ll strings in I ((a + t?)b (a * ab)) of lerrgth less tirart ftrur. Find two other equivalent expressions. * (") all strings in whir:h all nrrrs of n's have lerrgths that are rrrultiples of three.00 (0 + 1). Give a regtrlar expression fo.) tnir'ra regula.xpressionsfor the following langrrages. a.. where . w e {a.D Ctu" rcgular expressiotrsfor the following languages on { o . \(n.s ( d ) all str:ings that contain no run of a's of length greater than two. of @ 7. strirrgs .rrl { i1}.t:n.rexpressiol for I: {o.lculan. ( " ) all exactly one a. (0 11. Find a regular exptessiorrfrrr the cornpletnent of the language in Example 3. of (rt) The trrrnlrlernetrt L2.b}. L s .I.L is the larrguage in Exert:ise 1. 4.rr} 4. t'ind a. : {u.b}+}.c } .\}ts expresuirlrrs(st:t: Exclrt:isc 18 irr the following exercisesection).} i}.'ntg 3}.).) also denotes the langrragein Exarnple i1'6. Find a regular expressioufor L: 13.lol : ?}.h"t. ) 8.
Arr exarnple o1 this notation is 'urdl. Givc a general method by wirich auy regular expressiortr can be changed into frsuch that (I("))" : L(F). (e) all strirrgs with at most two occurrences of the substring 00.EGULAR EXPRESSIONS Tf .. rcspectively.e. Repeat parts (a). Determine whether or not the following clairns are true for all regular expressions rr arxl rz. right.IJ. Prove rigorously that the cxpressions in Exarnple 3. w}rere t}rese symbols starxl for unitlcngth straiglrt lines irr the dircctions up. S 22. (a) (ri). b. what are sullicicnt conditions orr the exprcssion so that the picture is a r:losed cotrtour iu the sensethat the beginning and erxling point are the sarne? Are thcsc conditions also rrecessarv'l S . S (d) (rrrr). give a set of necessary arrl sufficient couditions that r rrrust satisfy if L (r) is to be infinite. 16J Find regular expressions for the following larrguageson {4. with 5 : {4. (c) all strings txrrttaining an even number of 0's. and (c) of Exercise 16.).\ or @. Draw pictures of the ligurcs denoted try the expressions (r'd). (c) (rr *rr).:curreltces of the substring 00 (Note that witlr the rrsrtal interpretation of a substring.: riri. 23. For the case of a regular expression r that doelt rrot itrvolve . L9. d. (b) all strings not entlirrg irr 01. 000 corrtains two such oct:trrrences). tlrrwn. 20.1 R. 18. Write (a) all strings cnding in 01. l}. arrd left. c}.'1i) regular expressionsfor the ftrllowing languagcs on {0. (b) ri (rr * r:)* : (" + r'r)*. 21. r.6 do indeed rlenote the specified larrguag.: ri. (b). (u) /:: {ru : lurlrrrodS= 0} {ffi (b) f : {tn:no(to)mod3:0} (c) L = {w : no(ur)mod5 > 0} 17. 1}. The svmbol : stands for e<luivalence of regular expressiotts itr the scuse that both expressitlttsdenotc the same language. Formal languages carr be used to dcscribe a variety of twodimcnsional figures.6}. (urrLdru. S (rl) all strings having at least two ot. wirich stands for the squarc with sides of urrit length. alrd (ruldr')'. = (rirfi). Chaincode languages are dcfinccl on the alphabet X = {u. * (f) all strings not corrtainitrg the substring 101. In Exercise ?2.
The constructiorr for this relies orr the recursive definition for I (r).2. Bet:arrseof the equivalence of nfats ir. interpreted as a binary irrteger.tedparts (4).nguage there is a regular exprcssion. (5). We will slmw this in two parts. ffi 26. in Figurtt3. We first construct simple automata for parts (1).tically. and rf. Assrrme now that we havc irutomata.ngua. As irrrlit:a. a language is also regulirr if it is rr.rexpression. respectivcly. (2). r1t s.nR GRAMMAR$ 6+)i"a an nfa that accepts the larrguage L(o. so we lose rrothirrg in rr.m sr:henra.1(a). Find a regular expression for all bit strings. with leading bit 1.a. TIte two concepts artl e$sentia.nddfa's.+run Rr:crJr.ler.r expressiorrso. then show how they can Lrecorrrbirx:dto implement the rnore complir:a. with values not bctween 10 and 30. and (7).lstate.ge regular if it is accc:ptrrrl some is hy dfa.5.ngua. .78 Chopter 3 Rnculen Laucuncns . and fl. M(rr) and M (rz) tha. The constructions are showrr in Figures 3. (b). thc one on the right the firral state.lly the satne.. In this schema.{rcepts (r). when inCE)l'ina terpreted as a binary integer. and (c). then .2 on page 73. With Jl.3 we claimed that frrr every nf'a there is an cqrrivalent one with a sirrgkl fina. a regular expression that rlenotes all bit strings whose value.L (r) is a rcgular la. € E. accept thc ltr. and fbr every regular expression there: is rr regular language. is greater than or equal to 40.t accept larrguagcsrlenoted by regular expressions 11 and 12.ssuming that tlxrrr: is only one final statc.(a+b)). but may represerrttlx. In Exercise 7. Let r' bc ir regrtla. Srxrtion 2.ges the sirnple for regula.ted . C o n n e c t i o n e t w e e n e g u l o r x p r e s s i o nq n d B R E s Regulor onguoges L As the terrninologv srrggests. Tlten thrlre exists some nondeterrrrirristir: finite accepter tlnt ir. rcsptx:tively..f(r1) and M (rr) rt:prcst. the graph as vertex at the lcft representsthe initial state.the connection betwct:rr r{rgrrlar languages and regular expressions is ir close one. We neecl not explicitly r:onstruct these autornata.3 to 3.'nted this in . I'hese are shown in Figure 3. wo then construct autorrrata frlr the regular expressiorrs  12.cceptedby some nfa. for every regular ler. Our definition says that ir. ConsequentlS I (r) is a regular language. Regulor Expressions Denole Regulor [onguoges We first show that if r is a relgular expression.ngua.\. and (3) of Definition 3.1 way.r.ge. L Proof: We begirr with automata thal. We rrow show that if we have arry rcgular expressiorrrr 'we (:irn construct an nfa that arx:eptsI (r).
3 Automaton for L(rt * rz)./.L (r).1 (a) rrfa accepts g. By stringing together several such steps.3 to 3.5 that this cotrstruction works.+ A).\_.2 CoNNEcttot'r BnrwpnN RncuLaR ExpREssIONs Arrrn RuculaR Latqcuecns 79 Figure 3. To argue more rigorously. we can give a formal method for constructing the states and traruitions of the combined machine from the states and transitions of the Jrarts. (c) nfa accepts {a}. It should be clear from the interpretation of the graphs in Figures 3.2 Schematic representation of an nfa accepting Z (r). (b) nfa accepts {tr}. . the initial and final states of tlte constituent machines lose their status and are replaced try rrew initial and final states. where (ba. I . . (e) 6) (c) Figure 3. as it is reasonably obvious that the results are always correct. . : (o+ bb) Figure 3. then prove by induction on the number of operators that the construction yields an automaton that accepts the language dettoted by any particula. t ) in the drawings.3.r regular expression' We will not belabor this point. we call build automata for arbitrary complex regular expres$ions. M(r) '/_\.
+ I)).]\ *\*/*\.7 Automaton accepts . constructeddirectly from first principles. (b) Mz accePts f.raRs Figure 3.. are given in Figure 3.n Lerrcuacns ar'ro Rnculan Gnq. (ba. Putting thesetogether using the construction in Theorem 3. Automata for (a * bb) and (ba* * A). ! b M z r\t fln . we get the solution in Figure 3.0 (a) Mr accepts L (a + bb). d M .l.4 Automaton for L (rrr2)' Figure 3.rr. .80 Chopter 3 Rncul.c.tr).7 I Figure S.5 Automaton for 't (ri).t ((a * bb).rrr. {.rV (a) 1J 6) r /\ Figute 3.6. (ba.
8 represerrts a generalized transitiorr graph. is interpreted as an edge labeled with the expr()ssioll a f b * . A getreralizedtransition graph is a transition graph whosc edges aret lub*l 'Ihe label of a.. thero should exist a corresponding regular exprussiort. one of the tnore intrdtivtt approaches requires a sidel trip into what are called generalized transition graphs.bels of all thc walks from q0 to anY fintr.rarily.nd hence a transitiotr graph.nculeR LANGUAGE$ 81 for Longuoges Regulor Expressions Regulor of It is intrritivt:ly reasonable that the rxtrrvurst: Theoretn 3. as should be clear ffom an inspet:tiorr the go) Iabeled n is a cycle that can genertrtc any trumber graph. itr any order.8 . all we neecl to rlcl is to lind a regula'r expresuitlrr captr. The language acrxlpted of by it is L(o* + o* (o+b) c*). Since this idtlir is in used h<":re a limited way and plirys rro role in our further dist:trssiorr. Flom this observation. I The graph of any nondeterministic finite accepter (:an be considered a generalized transition graph if the edge labels are interpreted properly' An edge labeled with a singlc symbol a is interpreted a^san edge labeled vritlr the expression a.nguagc:ltas an a"ssociatednfa a. The strings denoted by such regular exprtlssiotrs are a subset of the language accepted by the generalized transition graph. of a's.. Tltere are several ways to do this. This creates a bookkeeping probklrn that rnusl be hanclled carrcfully.. it follows that for everv regula'r latrguage.t for clvtlry regular language..r lir.we will deal with it informally.rrywrrlk from the initial sta. it represents L(o. This dot:s not Iook too clifficr. there exists a Figure 3. We could have labelad this edge a* without changing the language accepted by the graph.L should hokl.l state.b.te to ir lirral state is sition graph..hlcof getrerating the la. Sitrce any regultr.). languagc being the union of all srr<:h Figure 3. and hent:tl itself a regular I expression. that is. while atr edge labeled with mrrltiple syrnbols a.AR ExpRpssIols arul R.2 CollruncrroN Bnrwnnrr Rncur. The edge (qo. tfie concatenation of several regular expressions. with the full generated subsets. and thrr.3.rltbrrt it is corttplicated by the existerx:c of cycles thaL catr oftetr be travt:rsc:tlarbil..
. Siriffiiabel Tvery wat[In a genetralized transition graph is a regular expressi6n.1.{q} before removing q.. wtr ornit the corresponding edge in (b). The corntructiorr that achieves this is illustrated in Figure 3. We can assullle without any loss of generality that M has only one flnal state and that q0 f F'.9 e ) a. we will not pursue them here. However.Si) in I .T " \ J (a) generalized transition graph that accepts it.f.en LANGUAGES Figure 3.r language.L: L(r).is. it can be shown that the construction yields an equivalent generalized transition graph.qi). Conversely. where the state q is to be removed a"rrd the edge labels &. To remove a vertex labeled q. we are ready to show how any nfa can be associated with a regular expression.pctll. The construction in Figure 3. Proof: Let M be arr nfa that accepts .stand for general expressions' tai a. every language of iccepted by a generalizecltransitiolr graph. but refer the reader itrstead to Exercise 16. this appears to be an immediate consequenceof Theorenr 3. Then there exists a regular expreti$ion r such that .b.After all the new edgeshave been added. The complete process requires that this be done for all pairs (qi.aNo R. Although we will not formally prove this.9. there are some subtleties in the argument. ..reqular. Section 4.L. we use the scheme in Figure 3. Accepting this.9 shows which edges have to be introduced so that the language of the generalized transition graph does not change when we remove q and all its incoming and outgoiilg edges.1..9 for all pairs (qt.aJ 1 th@ ntosfi'e"eiili" tlie sense' rhe casiaefiiEteilETfie to all three Yg$!c9! qiL+j.il4 as a generalized transition graph and apply the above construction to it. We interpret the graph of. We can d0 this if we do not change the language denoted by the set of labels that can be generated as we go from q0 to q. want to create an eqiiiVEittint'f6neralized transitiorr graph with one less state by removing g. Let tr be a regula. .(l: Irr ca^reswhere an edge is missitrg in (a).cR GRAMMARS Chopter 3 Rncur.3 for details.. L j t .
3.Ncuncps 83 Figure 3.'='/ \+J/ n rr rl \ / (rn (t(rq +rr r.h A l b+abta a+b { r' }11 r'.TB* H.11 a9'__J9"\? o .2 is but it is completelyroutine tedious and tends to give very lengthy answers.rr '*) q with all its incident edge$ can be removed. n b n b t a.phsa^reall equivalent to the initial one.11(a). l"t ab*b A .The corresponding graph after removalof state q1 is shown in Figure 3.xcr )) . We continue this process.1l. we arrive at the a.af b.r' *\. r HxsmplC fl"S generalized transition the Consider nfa in Figure3. I Figure 3. and alwavsworks. T4. ){'L!.2 CoruuncrroNBnrwporu Rnculnn ExpRpsslotrs AND REGULAnL. until we reach the situation shown in Figure 3.f\{ / lt..bb.1 \'l \+J/ (b) 0 F. The construction involved in Theorem 3.". A regular expression that denotes the language accepted by this graph is r : rlr2 (r+ + rsrirz) (31) of Since the $equence generalizedtransition gra.b" rz : o. Making the : identificatio[ ln1 b + a.c.1{) r4 }. lve can prove by an induction on the number of states in in transition graph that the regular expression (3.1) denotes the generalized L .11(b).10. regular expression (a r : (b a ab* ab*b * b)* a)* for the original automaton. removing one vertex after the other.
t n for Firrd a regular expression the language L.12 Figure S.rFumberof a's anran .Chopter 3 Rnculan Laucuaces amo R. : ru.b}.swe rr$evertex labeling effectively..{* e {rr. v/e rcmove the state labeled oE.13 .re. \ rc: I b*a(bb)*ba. . !+' Figure 3. d\Itl) /"\+ \ r+: 0. we applv (3..ba.nculaR GRAMMARS trnq'tmplut.ven number of rr. Next.13.:. An attempt ttt construct a regular expression directly from this description lcacls to all kinds of difficulties. First. 'rz:b+ab(bb)a.(ur) is even and n6(ttr) is odd}. giving the generalized trarrsition graph in Figure 3. On the other hand. we rerllove tlrc vertex labeled O0' This gives Figure 3.Lto detrote an odd this we eaJilil * ott]With .. finding a nfa for it is easy as long ir.'s and b'E with O.1) with rr: aa+ ab(bb). We label the vertices with EE to dcnotc zrrl e.14' Finally.n*t*ne"Iffia*nd get the solFion in Figffe.3'fZ' We can now apply the conversion to a regular expression in a nre<:hanictrl way.
tlrrl eclitor ed in the UNIX operating systenrreqg$ll?fl jlq jgl[t4. or integers and real numher$.Alltexteclitors allow files to ht: st:arrrrcdfor the occurrerlce of a given strin65.anLeNcuecns eur 85 Figure 3.J. Often. rrir"tt:lrirrgoccurSintextecliting..tchirrg. 2 colnpcjrrohr Bp'rwpEN RnculaR Expn..such as identiflers..d* .s staruls fcrr tlte sign. I Regulor Expressions Describing for Simple Pofierns Irr Exarnple 1. mclst rNlitors c:xtt:rrd this to penrtit searching for patterns.ba EE b + ab(bh)*a e9 The final expression is lorrg irrxl corrrplicated.yto see.tccl ancl exterrsive area of computer scienut to which we calt only briefly allude. thc kcy to successfulpattern nratching Lsfincling an effective way to describe the pattertis.ESSroNS R. ir. tivcly straiglrtfbrward. wherc.mple. This is a cornplicir. but rrcvertheless instructive.rgratrrrtirrg larrgrrages.: i .rxl r/ stands frlr the dicits 0 to L Ptrsr:al irrtcgers are a sirnple case of what is sometimes clallccl a "pat$crtt.veta. firr examplc. The example below is a simplifitrd.ncur. Tlxl rclation bctweerr finite autorna[a and regular expressions mean$ thilt wc can also use regular expressions irfJrr wiry of describirrg these features. 'Ihis is eas.tf a1q5brrg ref'ers ttr gories.lkNl attout stl far have been found useful in pattern mtr.14 + ab(bb)rba b + a(blt). For extr. but the way to get it is rela. thc sct of all acceptable Pascal integers is clefined by the regular expression f')r'q' stl. g{tT.I* iry" Ll"t'x "t) .A}. Section 2.we explored the t:orrncx:ti6rr bt'twenn finite accepters and sorne of the simpler constituents rif prr. dernonstration of how tho irluas we ha. with possible values from {t.15 and in Exercise ls." a terrn that refers to a set of objects having $ome common propcrtics.
We have to deal with an unlitnited rrurrrber of rrrbitrtr.tching a. If the pa."+aba. is eft'ectively tltr pl. If memory space is a problem. Tlte constructiorr using Theorems 2. so must be flexible.1. f'ollowed hy a c.GI.cur.ttern description is part of the input. I bln. b)) . This clfa. AII the progranllllel has to do is to proviclt) a rlrive. f b).enLANGUA.ti. Tlxl lffit:iunr:y of the progra.lgorithrn. in the fomr of a trtrnsition tatrltt.*a. the state to yielcl autorrrirta with reductiorr rrrethotl rlt:st:ribedin Section 2. @ [1) Ciu* an nfa that accepts the larrguageL((a+ b). Tlxr pa.4 is helpful. furthertnore. We see from this example tltat the UNIX editor can recogtrize regular expressions (although it uses a sornewhat different convention for specifying regular expressions than the one used here)' A challenging task in suclt ar} applicatioil is to writc an effici<rrrtprogram for rec:ognizingstring patterns.rtion is more complica. but lterc tlie situ. ideils from the recognition pt:ocess arttoma'ta' theorv a. Fiflcl aII rrfa t]rat accepts the complement of the language in Exercise 1.1. I'heoretn 2'2 rrray therr be ust:tl ttl recluce this to a dfa.m must be considered also. / T l llse the conrrtrrrction in L (ub'aa.tterns.rch followed by arr arbitritry ntrmbtlr tlf tz'tl. but created at mrr tirnc:.to find an nfa that arcepts the language 2. the patterrr recxrgrrition prqgram can take tltis description and cotlert it into arr ccrivalerrt nfh.1 tettds of finite autorrrataI'rornrtlgrrlar tlxpression$ rntrny states.reoften used. (a) r(aa.86 nNu Chopter3 Rr.r that gives the general framework for using the table' Irr this way we carl autotnatically }tarrtlltl il lilrge number of patterns that are defined at run titne. as eln instruction to sea. To solvc this probltrm.rilv rrlmplicatefl pa.s llecuI.1 and 3. lsing the constructiorr in Theorem 3. Searching rr file for occurrellces of a givcn string is a very simple progranming exercise.ttern is specified by a regular expression.h).) ffi (b) t (ab (a * ab).an GRAuneRs the file for the flrst occurrence of the string ab.). (a * aa)) (c) I ((obab)" * (aaa. the pattertrs are not fixed beforeharrd.b(o + bb)")' ffi Find clfa's that accept the Ibllowing larrguages.tecl. 'l'heorem 3.) @) r (((aa.tternma.
ffi . section 8. f$.9(a) are missing. Find an nfa for Exercise 15(f).1. (a) L : L (. (a) Find an "^equivalerrt gerreralized transition states. use this to crerive a regular etpression for that language. . !'ind dfa's that accept the following languages.'* qLlwhat langr. a*) (b) L : L (ab*a*) n r ((ab).3.ba).g when various edges in 3. 6 .2 cor'rNncrtor'rBnrwnnrq R.nculnn Expnnssrorrs ehrr R.tu). Give explicit rules for the construction suggested in Figure 3.ncuLaRLANGUAGES 87 5.rageis accepted by the following generalized transition graph? a+b a+b' aab+t Q0r)Find regular expressions for the languages accepted by the following automata.. ffi graph with only two (b) What is the languageacceptedby this graph? W ". t ./Consider the following generalized transition graph.ab* U I ((ab).
rcal ileed t T .10... ('u)) mod it : 1} ) (.: { t n : ( n o ( t n ) n ' 1(. 1 5 .6(rrL) are trot}r everr} (b) . we Inay rtot of t[e pattern.88 Chopter 3 Rlculan AND RECULAR Gnauuil.nt. : {tt : (n. Rework Exarrrple i3. Once the notion arr exact rrratch . Write a regrrlar expression for the tret of all Pa^scalreal rrumbers' lrets whose elementsrare integer mrmbers' 1 6 . : {ur : 2n* (u) + :3n6(u. srrdr asPrograrrfr that cheCk spelliilg.') is even} 1 3 .Rs LANGLTAGES (") 0 11. !'ind a regular expression that generates the set of all strings of triplets defining correct binary addition as in Exercise 23. In sorrre applicatiOns.T a )r n o d 3 l 0 } (d) 1. Section 2'l' r4.this tirrre elinrinating the state OO first' (@ f itta a regular expressiorrfor the following languages on {a'b}' (u) I : {rrt : no (ru) and rr.$u) .) . Find a regular expression for Pa^. Prove that the constructions suggested by Figrrre 3.9 generate equivalent generalized transition graphs. only a1 approxirnate otre.
sinprrt a regular expression for .L). by dropping a single syurtrol tif the string.serl (.ser. Let 1. we look at grarrrmarsthat gurxlrate regular Iangua. lbnnally rlefirre this operation rlrop for lang.t (tr).nd l.3.Grarnmars a.Gir.ions a.3 Rrcur. we err()irrterested in knowing what kind of grammar we can associatewith tlrrl farnily. * (a.q. corrsi<lerall words that can be forrnetl frorn 1.) CJiverran nfa for 1.T. using a. ffi **(b) LJiscusshow you might use this to write a patternrecognitiorr program for i. a spuriorrn^syrrrbol anywhere in a lvrrrtl. regular larrguage on }J and define 'i. ln effer:t. if all .L.y of describing regrrla.alrlr. Ctustnr<:t arr nla lbr droTt(L). E 7'+ A gra. bc a.ges. Aner'logousto the previous cxcrcise. * 18.1 to find nfa's for L(aa) arxl L(O*). As arr i llustratiorr of this. Usc the conrrtnt:tion in Theorem 3.uagcs.nrr.rtsert(L) * {uau: a E X. i'rtsg'L(L) contains all the words crcated from tr by inserting. wlrere A. B E V. utt € L} .rlariguagesis hy means clf certain simple grer.mrniirs.ns 89 of an approximate matt:h has been madc precise.nguages? RegulorGrqmmqrs tffiffiffi A third wa.rB.S. Whcrrrlver we define a.rnrnilris said Lo be leftlinear prloduct. Rightqnd LeftLineor Grqmmqrs A grirrrtrnar G : are of the fbnn (y.ll prorluctions A .P) is said to be rightlinear if a. ltrrrguage family through irrr automaton or in sorncl other way. a.reof'tr:il ail alternative way of specifying languages. shr:w how orre can construct an nfa frrr rirr. First. A + it:.n. given an nIa for . autornata theory can be applied to corrstruct approximate patterrr rnatchers. consider patterrrs derivcd from the original orres by inscrtion of orre syrnbol. Is the result corrsistent with the defirrition of thesc la.L..reof tlu: lirlrl A I1t:.
In a sinilar way. Iinear fbrrn.Pr).{o.t nlost one varia[le appt)ilfs tl1 the tllsistent'lv right sicle of any productiorr. is tine tha. Flom this single instance it is ea"sy conjecture that .L (G1) is thc lirrrguageclenotedby the regular exprtrssionr =.lti$[$ .R or AA regular gramfirar tt. The granunar G2 : ({S. BAh.r. Although everyproduction is either in rightlinearor leftis not rcgrrlrr.rGr:({s}.S.q. we tlirn $eethat L (G2) is the regular language I' (aab (ob).Stab. The stltpttntle: S+a. with producticlrrs S . {o. b} .rs. fz). S.rgramma. or lcftlirreirr.P) with producti''s '5.5r. .}. Both Gr and G2 are regula.$N$iN$Ns[liEihi..t} .withP1givenas SrtbSla is rightlinear.. Sr r Srnbl5zo '52 + ft'' is lcfblinear.4.a. .l.Bl. and . thet Erlablgjlxsl or bir eitherthe rightrnost leftmostsy"tbg]gi$Effi tion. a.). f'\rrtheilnore.90 Chopter 3 Reculln GRAMMARS LeNcuacrs ann R.r.trl.t is either righlliilear Note that in a regular Hrirrnma. A . {o.u. (ab).rc.i. BI .)' I .mmaritself is neitlrt:r rightlinear nor leftlinear. S'..h9+ahab9+rtbaba to is a clerivatiorr with Gr."egramma. the gra. Sz} .
cleir.r Iirrrgua.T.. a. 4I)11)2. Thus. Note that ther scntcrrtial forrns of a rightlinea. tlrt rrot all linear gl'aloqrar$ arreregqtal_ I (Jrrr rrcxt goal will be to show tha.the stir.S.bte.rnmar is tr.nclit rir:<:1rs as l.. The corresponding nftr <:rr..r.UpV71 4 1 1 1 1 ) 2 '"' u k l J l = ' I t J . Then tr (CJ) is a regular ranguage. whilc the part of the string a.}. ' r d E .IrueRs 9l tlrt:refore is not regular.step in tr rlr:r.tV. 1 .rly. If . a.ncul. Srrl4xlsenow Lhat we have a. we coustruct tr. regular grammars a.tcdliy ri rightlinear gra.or V.t)t. To do so. b ' .r grirrnrrriir lul'e the special fonn in whic:h thcrc is exactly one variable a. * __ (3.l ftrrm.il.lreacly pr'cessecl is identical to the terminal prclix of the sentential form. Proof: We assumethat V = {V0..?) rhe automaton to he t:orrstnrcted will reprocluce the rklrivation by . then btx:inrstrof the lorm of the proclu:tiorrs in G. . that s: Vo..rois a stritrg in tr (G)... a.riglrtlirrear grammar.r grirnrrrrirr.rea. we show tlmi a language generir.consuming" ea..4tletjoLwithout restriction orr tht: llositiorr of *+"#.rriveri tr.twr: have of Jrrodrrctiorrs the form Vs i r)tvi1V ...tecl with rcgulirr larrguagesand that fbr every regulirr language there is a.ys rtlgulil. This sirnple icleais the basis for the fbllewirrs theorem.tc of the autornaton will .In this scheme.ges.fcsular qratunalh_alwavslilear.q. +':**::*:* ttris varia.he rightmost symbol. RightLineor Grommors Generote Regutqr longuoges First. svmbol d is r:nrxlurrtered.lJ Fl. t]re derivation must have thc fbrrrr V o1 u t V ..y1.n imitatc this step lry going from statc D to state E w]:err a.t try usirrg a production D + rIE.chof thtlstl u's irt turn.nnfir that ruirnics the clerivations of a. c D+ a . 'Ihe initial stir. 1 tt'ttJ. LeL G : (V.t regulrrr grarrrrnars are associa.n GnAr. one iEliiElEorrr (xr(:rrr on tne rlght $Idetrtf arry ]1lp. P) ht: * riglrtlirrear grarnmar.trlof the automaton corre$poncls te tcl variable in the sententia..ivation a b .nothcr wrry of talking about regu)a.lwir.regula. The gramnlirr is *rr .ncl t}ra.t.uzVj.x*r'ple of a linear A litrear granrmar is a grammilr irr which at most.
. r ) : V y . d* .2) is satisfied' In the nfa there is. the automaton will have transitions to connect V. Thc mrmplete automaton is a. The general scheme from such is shown in Figure 3. labeled u1.'u. a path from V6 to V. Supposenow that w e L (G) so that (3.. ( Y o& r a z ' ' ' a * ) : V i ' For each prodrrction V'ataz"'ftrntt the corresponding transition of the automaton will be 6 * ( U . Therefore.+a 1 a 2 " ' a * V i . e pa.a2.i and Vi that is. ir*uuume M wa. and tl is ar:cepted by M. lr$ing paths labeled u1.. to Vi labeled u2. a^V. to y/.+ Represents arar.sconstructed. Reprcscnts fl=6 \7 \J V. d will be defined so that ./ V+ arar. ) F \ .. innd $o on. by construction..lrun RpcuLRR GRAMMARS btl labeled Vs.Vr.r) .'. where V7 is a final statc.15.th from V. and for each variable Va there will be a nonfinal state labeled Vl..92 Chopter 3 Rncuren Lencuacns . Because of the way in Corrversely.ssetnbled individual parts. d . that trr is accepted by M. For ea<'hproduction f i .. of rr rmrst have the form 'ut = ltt\1'2"'LlnUt (\_/ A n ... so that clearly V1 € d* (H6. & t & 2" ' g .. to accept ur the automaton has to pass through a which seqqence states Vo. The intermediate states that are needed to do this are of no concern and can be given arbitrary labels.
16 and the derivation V " + u t V 4 u 1 u 2 V i3 u p 2 .16.r grammar.L is a regular language on the a.3 RnculnR GnauueRs 93 Figure3. Fbr the se<.rguage L ((aab).tcsof the dfa now become the variahles of the grarnmar. o. . If . The language gerrerated bv the gramma.with vertices Vo.bVolb. t ) 7 y . I Rightlineqr Grqmmqr$for Regulorlonguoges To show that every regular langua.get:irrr be generattxl by some riglrtlinea. . Finally. 5 . . we need to introdurxl att additionnl vertex so tha. Hence ur is in I (G).Vt. t .l.nd accepted hy the automaton is the regular la. and VJ. then there exists rr rightlinear g r a m r n a rG : ( V . irrrd the theorem is proved.ond a rule. The sta. We start the transition gra. we start fronr the dfa fbr thc language ilnd reverse thc construction showrr in Theorem 3. and the symbols causing the tra.rrsitions hetxlrne the terminals in tlte productionu. ' .ph. P ) s r r c ht h a t L : L ( G ) .u n V n4 u 1 u 2 .r ir.t there is a path Iabeled ob between lt and Fo.3. The first pro* drrctiott rule creates an edge la.. we neecl to rrdd an edge labeled b between Vr and Vy.lphabetE.3. giving the autornaton shown irr l'igure 3. I to'that grammar V(t V  accepts the languagr: generated by the aVt. ab). t ) 7 is possible.reled betwecrr tr/sand trzr.
. wc add to P thc productiott {r + .l i n e a r Q grarunar G : (V.g4 GnaltltaRs Chopter 3 Recul.L' We assumethat Proof: Let M: : { q o .dfa that ac:r:epts. P) with v = {qo. the gralnllrar will have orttt productiort ftrr each of tlxrse dts. ai) : e. .'. In addition..Q. .\. For &/ to acccpt this string it must make moves via d(qo'at): qo. a. Consider u € L of the fortrt w : (]. . t q n }a n d E : { a t . .aj"'anilt. 6 (q.r} and .+n LANculcEs ANDREGULAn (Q.5) with the granrlrlar (J. q r 1 . r . (3'4) (3.ataj ' ' ' apal) : Ql t complcting the proof. tht:n its derivation must havc the form (3'5). and u E L(G).d.nsition il (qt. we put irr P the produr:tion qi + a.at):qy e F' By txrnstructiorr."'.v can gelrerate every strirtg in . d ( q " .jqh. (3. C o n u t r u c t t h e r i g l r t .ep aiai4r I aiai 1 u . 4 QoI ut. For each tra.a n ) : q t . S.. 1 a m } .5 : gu. Therefore we cirn make tltc derivatiotr "'(LnQt. a z 1 . Conversely. .qo. if 96 is in F.1f u e L (G). 6(qr.oi) : qn of M.F) bc a. ' ' ( I k & r .t. But this implies tliat ii* (qo..'ia k a r q l 1 a i a i .Qt. E. .3) We first sltow that G dcfined in this wir.1.i.
b) = {qt} E(qyo) = {s} lo d(t {tEtt 9zbg. The string aabcr. tlfa is uot essential to the proof of Theorem 3. I Equivolence Between Regulor Longuoges ond Regulor Grommors The previous two thrlclrerns esta.17. Construct a rightlinear gr&mmirr fot L (aab*a.. lt aQJ ?. .). bc clerivedwith the constructed grarrrrnar can bv Qo 1 (rh ) 0.3 It. togcther with thc corresponding gramrrrar prodrrctions.3. thereby showirrg the complete txlrivalence of rcgular gr&mmirrs and regular languages.) = {92} E(q.r connection between regular langrragesand leftlinttar gramma. With minor modification. ={sJ E(qt.aQ2I aahq2 =+ aabaqs + aaba.4. One can make a simila. The transition firnction for att ttfa'. The result was obtained by sirnply following the construction irr Thtxrrern 3. A lanSirragetr is regrrlar if and only if there exists a leftlinear ggammar G s u c ht h a t L : L ( G ) .17 o) s(so. the sarneconstnr(:tion can be used if M is an nf'a. Proof. Given any leftIirrear grammar with productiorrs of the fbrm A +Bu.bliuli the connectit:rr between regular languagcs and rightlincirr grammars.4. is given in Figure 3.rEF 9t L For therpurpose of constructing a grammar.rs. We orrly outline the main idea. A+rj.[]cuLAnGRnnrnraRs 95 I'igure 3. H$nuwtplo I $ $'. it is usefhl to note that the restrictiorr that M be a.
/ /^\\R A few examples will ma.cilg every suc:h prodrrction of G with A uRB. (d) But thenso are. we useExercise Section verseof any regular larrgrageis also regular. 1 3. they are all cqually powerful' They all give a complete and unambiguous definition of a regular langnage' TIle connection between all tlrcse concepts is established by the four theorems il this c:hapter. We now have several ways of describirrg regular languages: dfa's. Next. (")) 2./. wc a.ke it clear quicklv that .uive a.L(G)' is regular.L((d))R.5 together. Figure 3. regular expressions. r Prrtting Theorems 3. nfa's.whichtells us that the rc12.4 and 3.4 Theorem Regular Brammars .18 Regular er<pressions Theorem3 .cus ANtl RH(IuLAR GnRunlns we r:onstruct from it a rightlinear grammar G try repla.'nd. respectively.18.t the equivalence of regular larrguages and regular grarnmars.2 Theorem dfa or nfa Theorem3 . .t&. and regqlar gra. 1{ _+ t.Chopter 3 Recrrlan Lnuclul.mmars' While in sorne irrstarxle one or the other of these may be most suitable. A Ianguage L is regular if and orrly if there exists a regrrlar grammar G suc:h that L: L(G). Sinced is rightlirrcrrt. as sltowrr in Figure 3.L (G) : (. 3 3.3.
GRerr.) Corrstruct right. Find a leftlirrear grammar f<rr thc language in Exereise 5.rrr)) mod:l I 0} (d) .antl leftlincar grarrlrrrars for the larrguagc L : {atth"' :rr. S 11. : {w : (no (w) . (zrr) arc both even} ffi ' (b) .rrunRs 97 1.1 1 6t r ) l i s o d d } . In Theorem3. 9.n. fi + u.Albb.b}.{a"lt"' tn I mis even}. t Lj.} 2. Constnrct a leftlinear grarnrnar for the larrguage in Exercise 1. A + baB. 7. whereA.ab).provethat l. Find a regular Blarnmar that generates the language L(aa* (ab + a)).).{w : (n* (ur) .r. ' (O f itta a regular grarrrrnar that generates the language on E : {r.. 1 @ linrl a regular grailrrrrar for the larrguagc L . L = {w € {rr.ll. eVardaeT B . 2.no (to)) mod 3 : 1} (c) I . (e) : (f (G))8.3 Rncure. ( (l$y'Show that for every regular language not containing.t n" (w) * 3rr. S. (e) L : {ut : no (u) and nr..: \\ {tn: lno (ur) .\ there exists a rightv linear grammar whose productions are restricted to the forms AaB A+tL. '. consisting b} of all strings wittr no more than three a's.rn > B}. F'ind a regular gramrnar that generates the language (ur) is even} .5. Construct a dfa that accepts the language generated by the grammar S abA. W 1 G Suggesta construction by whicJra leftlinear grammar can be obtairreclfrom an nfa directly. ffi 5. Construct a riglrtlinear grailrrrrar lbr the langrrage L((aah.ru6(. b}.6 (t G Find regular grammals for the following languages on {a.
t'. P r ) b e r i g h t . ( r c .q'r'tM. ) f e t G 1 = ( V r . $l# {5 . Find a regrrlar gramlnar that generates the set of all Pa"scal real numbers.P). atttl a^ssume ({S}UyrUyz. Show that L (G) is regular. E . Consitler the linear lin*u. where. 5 r .s {r+} \'l Sho* that any regular grarrrlrrar G for which L (G) + g must havc at least one procluctiorr of the form 4+r. S r .9. 15.98 Chopter 3 Rnculen Leucuac+psaFIIrRnculan Gn.9is not in VtJVz and P: g r a r n m a rG : * SrlSz} u Pr u Pz.cn.l i n e a ra n d G r = ( V r . P ? ) b e a l e f t V that Vr and l/z are disjoint. grammar. whereA e V and r ET*. X .
r'/ Rrrhaps irrry set we can specify c:a. Frrr r:xirnple. We now raise the qrrcstirlrr of how general regula. Thc fir'st que.s.rnilv ha.closure question.heatrswerto this r:rirrjrx:ture cleliniLely is no. finile llct by automaton.The opera.Properties f o R e g u l q r[ q n g u q g e s e have defincd rergularlanguages.t ha. cari we lell whether a lirrguage 99 .ting hetween thc various language fa. we rnust inqrrirc rnrlrtl cleeplyinto the nature of rcgulirr larrguagesancl see what propt:rtics the whole fa.milieswe will crrt:ounter.studied $om(]wirys irr which they c:arrtre represented. Could it be thnt cvt:rv forrral language is regula. A srx:orrtlset of questions a. albeit velry txlrnplex. Closure propertios.ns wc:ll as operations in whir:h etr. tlltlxrtrgh tnostly of theoretica.r la.a.Jrpr:ns when we perforrn operations on regular latrguages. Is the resulting langua.tiriris corrsiderare sirnple set opuratiorrs. and have $eenir f'cw rlxirrnples of their usefillness.tion.l.sir.ch string of a language is chrrngrxl. ireltrlus in discrirnila.lintorcst.ngrralgcs arc.bout ltrngrra.gc: frrrnilies deals with our ability to tltx:ideon certain properties. As we wiII scc shortl.y.n acx:eptecl some. St:c:tiorr 2. wcr such as concatena.stion we raise is wha.sfrrr inslance in ltlxercise22. But to understarxl why this is so.ge still rtrgular? We refer to this a.1.
Closure under$impleSetOperotions We begin by looking at ther closrtre of regular lattguages urrder thc (:ommon set operations. Closure properties of variorrs language farnilies utrder differcrrt operations are of considerable theoretical interrest. cornplcmentation.Lr and L2 tne regular. clostrrc properties help us answer qrrestions.r is to study the geueral properties of rcgrrlirr Ianguagtrs. irrtersection. brrt are not as easily Finallv we consider the important question: How can we tell whether given language is regular or not? If the language is irr fa<:t regrrlar.7 other. Ont: waY to show a language is not rcgultr.lsignificancethese propcrtieu have. But if it is not. it may rrot bc clear what practica. concatenatioll. we need another line of attack. their union also regular? In spec:ifit: want to addrcss thc problem in general. then tltere exist regula.Chopter 4 Pnorenrrns on RrculAR LANGUAGES fbr is finitc or not? As we will see.Lz are regular liulgtrages. the auswer may be obviotrs. sotne of Lhenr havr: vt:ry little.gefamilies. this leads us to tlte study of the closure properties of larrguagesin general. Irr this chapter. o P C l o s u r e r o p e r t i e s f R e g u l o rL o n g u o g e s Oonsider the following questiorr: Given two regular languages L1 atrd L2. then so are L1U L2.great deal about what regular languages can and canrrot do. a fact we express by saying that the family of regular langtragcs is closed under union. is instances.such qttestions are easily tr. characteristics that are sharcd bv a. a'ttd starclosure.f3) lir. Admittedly. that is. rl *?'2' ?"1 fl'Ild . If we know of some such property. we a can always show it by giving some dfa. attd if we can show that the candidate languagc rloes not have it. By giving us insight iulo the general nirture of language families. Ianguages.nd ?'2' 12 such that.fi arrd. then we catr tell that thc larlguage is not regular. such as ttnion and irrtersection. We catr ask similar qucstion$ about other types of operationu on languages. L1L2.nuwered regular answered for other larrgua. Proof: If .Ll : L(rt) and tr2 : L(rz)'By defilititlrl. more prac:tic. or regular grammar for it. We say that Lhe farrfly of regular languagesis closed ullder ulri()rl. At first sight.Il regular languages. similarities and differences in these propcrtics will allow us to contrast the various language families.ill and Example 4. Is it true for all rt:gular but here we trr and Lz7 It turns out that the arlswur is yes. when we look at the same quttstions fbr other language families.We will see ittstancesof this (Theorem 4.ter in this chapter. Later. L1 tL2.r expressions rr a. If L1 and.Li. but many results are useful. regular expressittn. we look at a variety of propt:rtieu of regular languages' These properticu tell rrs a.
w h t r r cM t : Lt from Ml a.atrd whosc trarrsiliou functiorr d is such thnt M is itr state (q.X.)' 96. Dcrnonstrating cllosureunder intel$t:ction takes a little tnore work.pr) ) : (tr*.): P' such thirt Qr e Ft a. Not orrly does it t:stablish the dcsircd result.rrdM2 is iu stir.f.F2)are clfir.2if a..r'd..F ) accepls Zr. L1L2.Jo.1 CLosuRE orr PRoPIIR:I'IEs Rncur.tcPi' Ihis is:rt:hieved by taking f ((qo.pt). inmedia.:lrj).dz.t ur E tr1 f'l .l tr. Noter tltat in the defirritiotr of a' dftr.rrguages. sitrple matter to show Lha. d* we a. Hert:. f ..Ir. Cotrsequentlyeitlrt:r d* (qo. dfa that accepts tr1. o r t ) * ( q 6 . X.pti). Let (Q.rrr) is deliled fbr all tr € E*.3). by M. To show c:losureunder rcrnlllerneutation. tomatrrrr e x P consists / \ of lrairs (q. 4rr: sfiorter but nttrrt:ottstructive(tlr al least nrtt so obviously r:onstructive) argurrtettts. Erpation (1. ( g o . fnis is rather strariglrtforwa. closure unrler trrrioll. ' u )e Q . Q .we start witlr DeMorgan's la. M : (8.rrdpi € Fz..'rfor lhe intelrscctiouof two rc:gConsJtructivepr:oofstlt:cur throughout this book.i.Po.ssrrrnecl to be n total function.king the complcrnetrt of hoth sicles.nd dz (Pl n. M : ( Q .r la.lgtlrithms. k:t. and Lit and starclttsrrreis respectively. I The proof of t:losureunder irrtt:rsectionis a gtlod exampltl tlf a coustruc:tive proof.'s' We constnrr:t w h o s es t a t e s e t 8 : L 4 : ( A . Thcrr as F is clefitrecl the set of nll (. so that d* (qu. : {r a) a.1. p n ) .trr fl 12 is regrrlirr. X . For tlosure uuder irrtrlrsection.u) is a final state.{o.we havc already slggcstcrd Lhe result irr Exercise 4 in Scrtion 2.a whencvcr d1(q1..nd orrly if it is arx:cptecl it isr. thcy are ula.) whenever M1 is in sta'te qi a.te. irnpurtirtrt because tlx:y give 1rs iniriglrt into the rrlsrtlts and ofttlrt serve a"s there thtr strrrting point fix practica. d . in which caseru € tr. but it also shows explicitly how to construct ir.nrlM2 a combintltlau(P.Then L1) L2: Ltl Lz .nRLANGUAcEs 101 ri are regular exprtlssiottsdenoting thc latrguagestr1 U L2.'f .Fr) and Mz : : L ( M 1 ) a n c lL z : L ( M 2 ) . ta..finile acceptt..F a n d w e L .tiotr. Thrrs.i) be a 'Ihen ther tlfrr.tena. .w. Consequently.1. as itr tnany c:ases.x'. conca. F ) .( 0 .1.
and reverse the directiorr on all the edges. I A variety of other closure properties can be derived directly by elemen* tary arguments. Now. The fact that . Using closure under complementation once more.Lz is regular implies that Ia is also regular.Bc(rrAR Larqcuecps for any languages Lr and tr2. Proof: The proof of this theorem wa^ssuggested as an exercise in Section 2. we see that ffit: is regular. because of the closure of ragular languages rrnder intersection. then by closure under complementation. Then.L is a regular language.identity is immediately obvious from the definition of a set difference. Here are the details. Suppose that . then L1 . t .Chopier 4 Pnopnnrrns ol R. we wallt to show that if . We then construct arr nfa with a single final state for it. we next get that Z1 UL is regular. if tr1 and L2 are regular.L2 is necessarily regular also. The needed set. Lttt Lz The following example is a variation orr the same idea. Section 2. Using closure under union. we know that tr1 rf 12 is regular. Therefore.r languages is closed under reversal. The family of regula. so are Z1 and L2.3. In the transition graph for this nfa we make tha initial vertex a final vertex. this is always possible.Lft.L1 and . namely LrLz:Lrf\Lz. WowthatthefamilyofreguIarlanguagesisclosedunderdifference'In other words. By Exercise 7.3. the final vertex the initial vertex. the modified nfa accepts . It is a fairly straightforward matter to show that the modified nfa accepts ruE if and only if the original nfa accepts u. and the argument is complete.L2 are regular. proving closure under reversaJ.
b. .(ut) : h.nguarges. image is defined a.ny srrr:h results. Supposc X and I a.(h): bbc. la.lt'l't:"ll#1 1i'.tcuu.(n.. substitution is tr. h (a. Others are explored in the exrlrc:ises at tht: crrd of this sectitxr.abbbcabl.4. a hornomrlrphism is a.: Pn. if 1r. thcrr h. Thr:re are ma..h.ba): abbhca. aba} is the Ianguagc h (L) : laha..6.) .c} arrddefine by h'(u):1.geof L : {aa.rl dclirre other operatiorrs atrd investigatc closure propertics for thenr.R LAN(+rrAGEs 103 or underOtherOperotions Closure In acldition to the standarcl operations on la. Tlren h.llcld a homomorphism. h. lf tr is a.nguageon X. we select only two typical ones. . I If we have a regular exprt:ssion r for n latrguage I.1 Let X: h {o.realphabels.1 CI. tlten a.ced with a string.b.function h:}jf* In words.nobvious fashion. Tlte homomorpltic ima. Tlrc dourain of tlrtr function h is c:xtendedto strings in a.. orre ca.(a1) h (az) . in which a single lettcr is repla.Qro. ' lil$finirt.osuRr. Then a.opnnrrns R.b}and f : {a.u Sxsnple 4. tlrcrt its homomorphic h(L): {h(r) twEL}. regular exprrrssion lctr h(L) carr Lreobtnirrcd by simply applyirrg the homomrirphism to each E syrrrbol of r.):uLaz'.
c. The fanrily ttf regula.trt: a. such that u : h(w).Leaving the details as an exert:ise. I The generrrl result on the closure of regular languirges under any horrxlmorphistn folklws f'rom this exarnplc in an obvious lrlanner. .b} '. Proof: Let .gcft.r latrguagesis therefore closed rrrrrlt:ratbitrar.nguage. we claitn that h (tr) is regular. It is equally ea^syto see that thc rcsrilting expressiorrdenotes h (r).. irna.akcir.104 Chopter 4 Pnornnuns op R.Evcry such striug.(a) for each sytnbol o € E of r.(")) tlterc is it ttt itt . Defirxr hv h(a):46""' h (b) : 6. trelongsto L1f L2. tlten its homomorphic Let /r.y }tottrorrurrphisms. . regular languag<l denoted by sonre regular expressioll r. af'tcr removal of this sullix.1.bet.L. thc txrrresponding h (rr') is in L (h (r)) arrd converselythat for every u in L (h.) is also rtlgular.ke t: E: {a.bcctlbt:c)* rlenotes the regulirr lernguageh (L).) (aa).holrorrtorllhism.rrrt h {b.d}. Then the right quotient of tr1 with trz is cleflled rrs € .be a. .nd L2 be latrgurrgcs the sarnealphir.llthe strings in.(I. denotedby If f. I liM on Let L1a.t have a sufrx bclonging to tr2.w€ t.uctrt.'.L1 tha. It r:an be shown directly by an appeal to the definition of a regula'r expressiorr that the result is rr regular expression.ARLAN6SAGES W. L 1f o r s o m e y € t r 2 } Lt/Lz {r:ry (41) To form thr: right quotietrl of L1with /12. We fincl h(r) by substituting h. All we need to do is to show that for ()very ru e L (r'). If L is a regular la.1. is the regular latrgttage r = (e * b. then 11: (tlbtc + (bdc)) (d.
Figure 4.qu.X.rrrl. We start with a.L1. Lp. and Lt/Lz ale all reE.ular. This suggestothat the right quotient of any two regular la.l:cjulan LANCUAGFT$ 105  s'4 Hr+r{rtrplo If L1 : {a"b"' : n } 1.> 0} .rn > 0} U {ba.L2 altd constnrcts from them a. Since iln inrtotttaton for L1f L2 urust accept ir.t I L2 : {att6"' . {l!I : m.Bt:ftrre we describc the r:onstnrction in firll. rz } I.nguagesis also regular.rnple.1 . Note tha.opnRTrns ol' R.4.t tr:rrrrinate with at least one l) as a sulfix.1. tlfa for Lt/Lz.F) irr l'igure 4. We will prove this itr lhe next theorem by a.pplies to this exer. dfir for l1i sa/ thr: automaton Mt : (Q.6. we will trv to rrtodify M1 s{r that it accepts z if there is any y satisfying (4. The strings itt .L2 consist of orre or rrlore b's. constnr:tion that takes the dfa's for L1 ir. we arrive at the answer try rcrnoving one or rtrore b's from those strings in . } 1} .} and L2 : thcln l. ?n. lct us see how it a.1 Clostlns Fn.nyprefix of strings in.1). lllhrlrt:fore.t hcre L1.L1 tha.
2. f<rrca<:hq € 8.li. To solvc it.8.2) : q will be in L1f L2.F). qb to see whether there is a walk labeled bh* to any of the q1. For each et.'R. Q+. We modify the automaton a.X.q0. The resulting automaton for L1f L2 is shown in Figure 4.X. walk to a final sta. eo.) is a dfa. Check it to see that the construction works. This carr be done bv looking at dfa's M. We constnrct u.nguage. Figure 4.94 do not. The arrtomaton Mi is M with the irritial state q0 replaced by gi.where M : (8. € Q. 92.te labeled u sur:h tha.cps Chopter 4 PRopeRuEsor. check each state qq.nn. anY r such that d(q6. or 94.{i.F) there exists a y €. We see that otrly 91 and 92 qualify.F. Qt.5.qo..2 The dififrculty comes in finding whettrur there is some g such that ty E L1 nnd 37€ L2.Bcur. = (Q. we To apply this to our present ca^se. determine if ir. We now determirre whether . Q1. Qz.d. we determille.t u € Lz. The idea is gerreralized in the next theorem I If L1 and L2 axeregular lauguages. We say that the family of regular languages is closed under right quotient with a regular la. If this is $o. Q3.nother rfia fr : (A. L2 such that y) d* (qo.Lnrlcue.te. Proof: Let Lt : L (M).ccordingly to make t7 a finnl sta. whether there is a. tfu:n L1f L2 is also regular. follows. : q1 € F.
L ( M : t )i L 2 : s .3 it is quite evident that L(Mo))L2:s. we have d * ( q u . g € L2 such that rg e trr. Flom the graph in Figure 4. In tlrat case. Therefore is in L1. and fraccepts u because (go. lf there is any path between its initirrl verttrx arrrl any final vertex.c. construction. let r be arry elemetrt of L1f L2. .ry) e F.z) is in by F. Frrr this.this implies that there exists a 37€ tr2 suclt tha. L (Mt). L (Mz). acld qi to F. and a solution is given in Figure 4. we determitre F and thereby construct M. E Q. This is easy.R 107 there exists a E in L(Mt) that is a. and r is in LtlLz. Lz : {o}.L. t Lt: Lz: L(a*batt*).Itcuacns 4. LttLz.3.(fr): and fiom this that LtlLz is regular. r:)q e F . we can use the construction fbr the interscctiorr of tvro regular languages given in Theorem 4.1 Closunr: Pnopr:t{rlr:sor RucuLA. Rut again by rxrnstructiotr.'a)e F. € F. L ( a b * )' We first find a dfa tha. Lz: {o} # a. To lrrove that "L (fr) : Lr/Lz. for Converscly. Then '' ' \ / there must be a.t trt:cepts tr1. The examplc is sitttple etrough so that we can skip the formalities of the r:onstrrrction. (gu.r) : q alnd d* (g.L2 a L (M) is not empty. Repea. finding the transition graph for L2tL (M1).1.1 s.irtoin L2. This implies that d* (qo. so that there rnust be some q € Q such that d. any r acceptedby M. q d* Therefore. then .g) e F'.We therefore r3t corrclude that .td* (q.ting this f'or every et.
the automaton acceptin1 LtlLz is determined. It accepts the language denoted by the regular exL1f pression of a*b* a.which can be simplified to a*ba*.Chopter 4 PRopnnrrEs oF REGuLAR LANGUAGES Figure 4.*batt*. Thh:us L2: L(a"ba"). I .4.4 \s' Therefore.3 Figule 4. The result iq shown in Figure 4.
. : 1 1 .T':. The symmetric difference n d:{ 1. z: h ( L r ) u h ( L { ) *fv* .r ls. In Example 4. The nor of two languages is nor (L7. 6.n} Le of two sets Sr and Sz is defined a.". if L1.L2)= h(Lr) nh(Lz) (c) h(LrL2): h(L1)h(L) . show that the family of regular languagesis closed under finite union and intersection.then Lu= and . L1 and w f L2} .L (baa.1 we showed closure under difference for regular languages. 2..) n L (a*b"a).5r and Se}..e . .. .1 Cr. 4.1 to find nfa's that accept (a) r ((a + b) a. F.L2. Use the construction in Theorem 4.. . 7.Sr or fr € 52..2. ffi (b) L (ab.*r (b) h (21 rr . Provide a constructive argument for this result. Fill in the details of the constructive proof of closure under intersection in Theorem4.osuRE PRopERTTEs Recur. show that the family of regular languages is closed under symmetdc difference.). ffi 8' Denne rhe compremenrffjTl. of the following are true for all regular languages and all homomor (g/Whi"tt phisms? (a) h(hu r .. a.Sz = {n t n E . In the proof of Theorem 4.) n. b /l r Show that the family of regular languages is closed under the cor operation.. show that the family of regula.eR LANcuAcEs or l0g 1.$ Sr $. ) n U Lt Lt: are a"lsoregular.4.r languages is closed under the nor operation.rexpression. .. that is.reregular.. Then show that h (r) denotesh (I).. show that h (r) is a regula.r is not in both .1.."::.3. L2) : {w : w {..a. s.1. lr.L. J". . but the proof was nonconstructive. but.
ft(ru) for some trr € . ' ')' : a s a a ' ' ' with the appropriate extension of this defiflitiorr to languages. E L. t 7 z ' 'f'tn . Show that if . closure of the family of regular larrguages under this operatiotr. Flom this. and.. Theto.^.Lr and ./.'7.u E Zr} is also regular. If tr is a regular language.:. h. thefl all languages would be regular. Defirte an operation thi. 1 8 . we SuDDose krrow that .:.Lr U .L2.. gc ( . that is tu. Prove the 20. Wffi L 7 .r languages is closed under this operation.'r11F.\ fto)t"t . for some 'u e L} ' exch..2 is definecl as LzlLl : {g : r € L7.t(L) : {y : ny € L for some r € t.r'l ffi 13.rd on strings arrtl languages as t h i r d ( a n z o " t a 4 a ! .} .n//. "If 'Lt is regular n'rrtl trr utrz is also regular' then 1 6 . of a language is the set of all prcfixes of its strirrgs. Show that the family of regula. If I/ is a regrrlar language' prove that L1 : {ua : u.'R \ : FirxlL1fL2. Show that regrrlaritv is preserved utder the.lul : Z} is also regular' 14.Lr L(abo')..(rq.rand that L' is finite. Show that.nt. t t ( . F) show that the farnily of regular languages is closed under the left quoticut with a regular language.110 Lat'Icuacns oF 4 Chopter PRopuwrttrs REouLaR . we can define the operation on a language as shift(L): {u:'u : sli.:TT:Trutr.ange(f) : {u : a : exchange (Tr.L is regula. . W t\ The left quotient of a language Lr with respect to 1.} .i. so is toal (l).:::.eo. operation.*haa*) : LrLzlL:r is not true ftrr all languages and Lz' I't \O " fojjEhow that L t . a t a z ' " t t n . that is. The head.{n : ny e L for solne gr€ t. ffi 1 9 .. prove that the langrtage {uu : u E L.shilt \\ c900"' aI]cI e n c h a n ..' 4il lI L..r t r 7 . Can wc " {fi\ \\ U "on"tuae frotn this that Lz is regula.. Lt: L(o.1 I t ' ' ) : Q r y t . Forasrring(rrtrz.L1].Lz is regula.r..tl(L) .) Sbow that the family of regrrlar langualges is clcisetl under euchan'ge.il of a laflgrrage is defined as thc set of all suftixes of its strings. if the statement ^Lr must be regular" were true for all .L}.
tte ..L is defined as rni. we have used several ways of describing regular languages: informal verhal . ..l.de(t): . r o } . Show how()rre derive can regular L(Gt)r L(Gz) ffi L @r) r.+\39Let  Show thal. We first consider whir. * 24.ruruq e L} .2 Elnvnrvrnrr. i u1u2.. The question of the existence arrclnature of rrrerrtbership algorithms will be of great concern in later cliscussiorrsl it is a.given a language.ul € X*) € show that operation./_\ 2.rg1r'rr:rrts.L and a string to. Is the family of regular languages closed under this oper.v QrrusrroNsABour Rncur.L? This is the membership q*estiorr and a method fbr anuwrlrirrg it is called a membership algoritlmr.lr}6?}n1 wtwz.1)m Lz. the farnily of regular languages is crosed under the sh'ffie * 28' Define an operati.4. Show that the family of regrrlar larrguages is closed under the rruinusSoperation.t cxactly we rnearl when wet say .it is irnportant that this be unambiguous. s u c h t h a t u r : . The shulfle of tu'o languages . for all ru.r an(l Zz is rlefinetl as .wm € Lt.r issue that is often difficult. {?." In ma..L bv on Ieftsi.eR LANLITJA#E$ 111 * ??.ation? 25. it is an easy lrratter. Gr and Gz be two regular grammars. The m.L as the set of all strings of tr with the fifth symbol frorn the left rcmoved (strings of length less than fivc are left unr:harrged). very little can be done with languages for which we carrrrot {ind elficierrt niembers}rip algorithms.. u E E + .l (b) ((rJ L (Gr) E l e m e n t o rQ u e s t i o n o b o u t y s Reguloronguoges [ wc rrow conre to a very fundamenttrl issue: Given a language . Definc the operation leftsi.. (Ga) ffi ffi glrammars tbr the larrguages \a. trtrr regular languages. thorrgh.in of a language . shuffie(L1... . can we determine whethel ol not .ar is an elemcnt of . L2) : {w1u1utzuz.ny a. the family of regular larrguages is closed under the rnriruoperation.rlmi'nuss on a language ..n(L): {tu e l: t } r e r ei s n o u E L .
wc thcrcfcrrc sa. sirtce thiS works only fbr finite languages. or clfa's. tltetr test accepted by this automtr. they specity the same language. lf there is ir.st three are sufficiently well defined for use in theorerns. to Nor iu it ea$y see the answer by looking at tlte regular expression$' gramma.rs. representa'tion. 'ur ttl stNrif it is Proof: We represent the lalguage by some dfa.qititln graph of a dta. there exists an algorithm for detertninittg wltether or nttt ru is in I. irr sta.t a regnlar language is given in a standard representation if arrd orrly if it iiJ dQst:rihedby a flnite automaton.rentif we represent t. cvtrn ftrr regula.ndir.ppa.gc irrfirritc.lgorithm fbr determining whether a reglrlar lirrrgrtir. find aII the vertices that are the base of sorlrecyclg.set nqtation. finite altomata.rcl Proof: Tlrg itrrswt:r is a..tt2 Chopter 4 Pnoprnuns or Rpcul. If irlY of these a. or infirritc.he latrguage as ir trirrr. given there exists an a. Often several defilitions of a programming language exist. there t:xists an algoritlrm to determiue whether or not Lt: Lz. It is 1ot possible to arguc on il serrtencebysentencecomparison.qc+l:s rlcsr:riJltiqns. For rtlgular langrtagJes least. I The question of the eqrrality of two languages is also an importailt practica.L1 irntl L2werdefine the la'ngua.r larrguages thel ilrgument is not obvious. a regular expressiotr. Otherwise. standard representation of any regular larrguagc tr on x and any u € X*. attd whether orrc langrttr. and we need to know whether.and regrrlar grarrrrrrars. To dt:termine whethel or ttol a.Ianguage is infinitc. Iitrite. then the la. in spite of their dificrent irppearances. it is finite.sinrpkr pnth fiom the initial vertex to any firLal is velrtex. these questions are easily arrswcrtld.ge a subset of at another. regular expressions. This is gelerally a difficult problertt. 1 6 )  ( t ' t n t ' z ). or a rt:gtrlir'r grilrnfiIirr. Orrly tlxr la.ton. .ngrtage ttot enrpty.ge. Given standard representatiols of two regular lalgtrirgils L1 iilIrl tr2. An elegant solution uses tfue alreatly estahlished closure propr)rties.y tha.reon a path from au initial is to a final vertex.ge t4: ( L 1 . I Other irnportant questiorrsarc whether a language is finite or itrfinite' is whether two languages are the same. Proof: Usirig .nR Lerucu.l issue. Given a.is etnpty. the lattgtttr.
Ac. ffi 2.Lz. t3 is rcgular. termines whether or not L : I. ffi 3..1. I These resrrlts are fundamental. : shuf f Ie (tr. arlcl we ca.fi bc any string irr E*. Iittle.3. for arry given to and any regr. wc will see that the answers ber:gr1(: irrcrcasirrgly rnore difficult.for any regular language tr. ? € X*.J.r..lwir. g. tletenninerr whcther or rrot . 11. Show that there is an algorithrn to detemrine if 1. itr stanclard representtr. A larrguageis said lobe apa.lr)Ns AIlotTTR. 5.L*. For all the exercises in this strtion.tion. such that w : ufru. Let tr be a regular langrrage orr X arrd. 4. tlrat is.t regrrlar lir.the questions raisrld by Theorems 4.L : . Exhibit an algorithrn that. with u.L2. Show that there exists irn algoritlrnr to determine whether or not ur E Lr .t. show that there exists an algorithm for cleteruining if A e . Fi'd an algorithm for dctermining if a given regular language is a Jralildrorne language. Show that for any regular Zr and.1 we see thirt Lt : fr if ancl trnl.rrguages are given 1.irtd'. L.t accepts 1.ter orr.L contains any string rrr such that tuF € 1. Exhibit an algorithm for tleterrnining whcther or not a regular language .yif Lt : Lz. in spite of br:irg obvious ancl unsurprising..Lr arlcl /.Ls._____ 4.ys case when we deal with larger the farnilies of langrrages.l.we will encounter cluestionslikc these on several occasions la.BculeR LnFrL+r.nttnltrnnv Qun$. there is an algoritlrru {.1.L1.tLz. Show that there exists ari algorithm for determirrirrg if tr1 C 1. Anticipating a. giverr any regular language .o dctermine wlrcther or not ^L1: LtlL. Once wtl httvc: M we can theu use the a"lgoritlrrn in I'heorern 4. Section 1.L) fbr any regtilar L.2. The operation tnil(L) is defincd as tail (L) : {u : uu E L.rlar larrguages. € .6 to cletsrrnirre if tr. but this is not ir. for any regular languages Z1 arrd Zl. Find an algorithrn to dctermine if "L coltains any tu such that fi is a substring of it.nfincl a dfa M tha.2 Er. given any three regular languages.2 r:an bc zrnswcredeasily. de g. ffi 6. For tlgular languages.: tr/i. 10.. Exhibit an algorithtr that. assume tha.' X" } .L.u..j is ettrpty.1. But from Ex(n:is(: 8.'r.nrlrlvcrrtually irnpossible to fincJ.rne languagc if .b to 4. 7. tr.r{s 118 Bv closure.
ny stage is strictly lirnitecl. brrt only a finite nunber of states in M. as we show using a Proof bY contradiction' Suppose. Thc ftrct that regular languages are associated with autt>mata tha. in processirrg arry strirrg. the pigeonhole rrurrrber urinciple tells us thtlt there must be sonre state. Therr some dla M : (8' {a. . show that an algorithm exists for determining if tr contairrs arry strings of even length. N r ldentifying onreguloLonguoges R. .L is regular. Intuition tells ns that a language is rtlgulir"r only if. If we put z objects into rn boxes (pigeonholes). o t )f o r i : of zj's.L tail (L) . S i n c et h e r e a r e a n u n l i m i t e d i t .fi'lll Efufffnfm 4rS Is the larrgrragetr . 3 . 2 .{anbn : rz > 0} regular? T}rc answer is no. but has to be shown prtx:iuely to be used in arty rntlarringful way. . the infbrmation that has to be rernernbertxl zrt a. can tell us whelher or not L (G) : E. PrinciPle the Using Pigeonhole The term "pigeonhole principk:" is rrsed by mathematicians to refer to the fbllowing simple observation. Sotne ttarrow rcstritltions mlst be obeyed if regularity is to holcl.cgular languages ca. . N o w l o o k a t d * ( Q o . This is such an obvious far:t that it is surprising how many deep result$ can be obtained frotn it. There are several ways in which this precision can be achieved. e*') : q. Let .114 Chopter 4 PRoenRrrus on Rncttl. b} . . for any regular 12. {0. b}.i.n'n tn'ucu. say q. impOses Some limits on the structure of a regular language. as rlost of our examples have demonstrated. when given a regular gramrnar G. such that d* (qu.t have finite nrcm()ryl however. ffiS 1 S . ancl if TL> rrll then at least one box must have rnore tltarr one item in it. F) exists for I . This is true. .L be any regrrlar larrguage on E : {a. t'ind an algorithm for cletennining whether a regular language tr contains an infinite numher of evenlength strings' Describe arr algorithrn which.L.p be inflnite. d.o") : q d* (q.q.cns : show that there is an algorithm for determining whcther or uot ..
any walk of length z or longer must repeat some vertex. uses the pigeonhole principle in another form. Then there exists some positive integer ?rl such that any w e L with l. . . A Pumping Lemmo The following result. (d. 1 . L f o r a l l i : 0 . such that ^'. wi * .4. and leads us to conclude that tr cannot be regular. I In this argument. it is convenient to codify it as a general theorem. known as the pumping lemma for regular larrguages. . b "t : Af. . the one we give here is perhaps the most famous one. Flom this we can conclude that d* (go. . To accept all anbn. But since there are only a finite number of internal states with which to do this. there are some rr and m fbr which the distinction cannot be made. (g.b') : q7 €. that is. In order to use this type of argurnent in a variety of situations.b") : 6 * ( q . (qo. with l r u l1 m . and l s l> 1 .ul ) m. the pigeonhole principle is just a way of stating precisely what we mean when we say that a finite automaton has a limited merlory. an').a*bn): d. But sirrce M accepts a"bn we must have d. The proof is based on the observation that in a transition graph with n vertices. ' ' z* &g e) (4 2) i s a l s oi n .3 InururrnyrNcNorrRpcuLAR Lnlrcuacos lfE with n t' m. Let L be an infinite regular language. . F. contain a cycle. This contradicts the original assumption that M accepts a^bn only if n : rn. can be decomposed as w egk. an automaton would have to differentiate between all prefixes a* and a. There are several ways to do ttris. 2 .
u) : q. The demonstration rrsed to strow that cert.as we is ftrr hir... Finitt: Ianglra.r.6.vc'stated here.r. d* (q'. Q r t ' . .qr. there exists a dfa that rexrogrfzes Now take a.ngrtages is ir. as well as rf. We have givcrr the pumping lemma only f'or irrfinite languages. everyuuFiciently To paraplrrase of numberof repetitions p. although always tegttla. i . Iike the pigeonhole argtrtnettt in Example 4. . and lyl } t.ry ^. tfuis can always he dqne. 'l'he but it is Vac. r d* (g".{^Jl.llv " creales an infinite set.nllc rrsc:tl proving that a languir. Let such a Proof: If .tt ltence the term puntping lemma for this result. .L is regular. .'l+ I entries. rz) : Qf. and so on. . with lrgl { rz * 1 : m.'.havtl states labeled qo. The pumpirrg lcrnrna. completittg the proof of the theorem.gc regular' it r . . nyzz) : qr.r. Since tr is assumed to be infinittr. Since this sequencehas exactly l.L such that dfn. Ai z of ru such that r d* (qu. so thir.ffiEl] Cornider the set of statcs tlte automaton gocs through as it proces$cs ?rJ' say Q oQ t 'Q i' .UAGES long strjqg in 4 cql_b9*b[aken this. Tlrl tlrtlorerrtdoes hold for finitt: lirrrguages. . . . which ca.: iy"h thc rniddle part yields another string in . (eo.. z) : q. y isg_!L:e.rttnot regular.gr:s. ' t Q f t Q indicatirrg there tnust be substrings 'x.r:annot bc putnped since prrmpirrg automrrtir:a. . Therc is rrothirtg in lhe pumping lerrrrnit. arrd such a repetition rrnrst start tro later than the nth move' Thus rnust Iook like the sequcrrce Q o tQ l . it.r.tqrr. F}om this it immediately follows that .roliE/ nl irr the putnpitrg lemtna is to bt: taken larger tha'tr tlte lotrgest string. d* (80' rY3z) : q7.string trr in. l".t rro strilg can be purnptld.116 Chopter 4 PttopsRrlns or RnculAR LANC. . ) : e.Q f.. ( l r ' ..5*(qn.rT iT anarbitrary . is a.Q2t.L. at lea"stone state must be repeated.lwaysby cotrtracliction. Wtr say that the rtriddlc strittg is "pumped.aitrla.
Orr tlxr otherr hand.inst an opponent. 2.bleto choose tl so that the opponent is very of ) \ T IL\tvdlF** .we must keep in mind what the theorem says.> t Thcrr thel string obtained by using i : 0 irr Erpatiorr (a. wc mrly be rr. it hardest for us to wirr tlrt: girrntl. If we can do so. l'he opponent c:Iroosrls rkx:ornllositionrgra.' Usirrgt]rc prrmping lemma to show LhaL. We have to assurnctirir"tthr: opllonent ma.r. Givt:n ?7?.r. string ..rrd clearly not in . Wc try to pick i in such a way that Lhe purrrpeclstrirrg ti.gc.L. the substring y musl consist c'rrtircly of a's. rk:firrcrl irr Erlrir. l3rl> tlrr: 1. Suppose lyl : ft. I In applying the purnlrirrg l{:rnrna.blishing contradiction of ir.'I'herefore. it+ 1. I'his conLradicts thc pumping lemma and thereby is indit:irtcs thnt the a. There are fbur moves in the garne. thcn the language cannot be regular.rmping lernma.. We are guirrarrttxxl tlrc existence of an nz as well as the decomposition rEz. we We are free to r:]rorisr: irrry ?1r.lue of nt.2) is 'Ll)O : t1711lrhm' rr. subject to lzyl f nz.iirr game we play aga..mountto a. rsr 'ryz.L.jer:t 'ro€ . 'Ihe opporrerrt pir:ks rru.lse. Asstrrnttthrr.llyqrrite difficult) that any pumped stringJrnust be in lhe original larrguir. [ltulnflg [.tion (4. pick a.nalways . We do trot krrow the va. A strategy that rrllows us to wirr whiltcver the opponent's r:hoicesis ta.L : {unltn : n > 0} is not regula.kes choicethat will make the. tr and every 2.. Irr this. The correct argument can be visualiztxl ir. proof thal the language is rxrt rcgulir. we win the garrrt:.of length cquerlor grcatcr tharn rn. the pr.ssumptionthat .L is regular rmrst bc fir.L ancl lrul > rn.8 that irllows us to r:orrrhrrleflrrm this that the language is rcgrrlar.3 Irnrurrr'ylNc NoNREcLTLAR LaNcuacns IIT Even if we coulcl show (ancl this is rmrrnir.nta.urin .. but whatever it i$. wc ctr. there is nothing in the stalernent of Theoretn 4.hodeE t&] Tlxtreftrre.. if the purnpirrg lernrnil e is violatecl everrfor orro 'iu or i. 4. but wc:rkl rrot krrow what they are.t'.t L is regular. Wrile we canuot forc:eLhe opponerrt to pick il pa. Our goal is to witr Lhe gatrte by tlsttr. to 3. srrb. wirile the opponcrrt triels to foil us.4.l.r. the punrping leurua holds fbr cvery ?1. We cannot clain that we have reached a contradiction just bcr:auscthe pumping lenrrna.2).is violated firr some speciflc values of rn.1.rticulilr tltx:omprrsitirlrr 'rl. t  /1 vdr'e . Step 2 is t:rtrt:ia. so thaL Lhe puutping lt:rnrnil must hold. is not in.
trrrtl the requirement that he o f lle Jr) L0 t / t . Nnl( th at' ili sltorla/ 7.'n u). Wha. tts I/g &#tra/ t / a 5 ' or h. g. wc carr always choose a34 aEshetynirr Figure 4. y.cns Chopter 4 Pnornnlus or.ah. the opponetrt need only pick U:ua' Now zr.L. /ht . 'til : ttrZ'o.:{wwfl:rueX*} is trot regrtltr. ir.n Figure 4. atrd we lose. Because of this choice. is irr L for all i.in the case where wu pit:k 'u) : [tr?"'. In that case. But any trrgument that assurne$thtr.118 Lewcua..ha.tcvclr rrl the opponent picks on Step 1.'Rr. then t 0 is a strirrg of odd length ilrrd therefore trot iu .t{ t {o k o{ ' l ffisrestric *cirtirblflbf t e d i n S t e p l J t o c } r o o s i r r g a y t h a t c o n s i s t s a's. we could not have reached a violation of the pumpirrg lerrrrna orr the last step. we use i : 0. F Note that if we irad chosen ru too short. .(e : /t (qna6t .bh. forcing a choice of r.5 u. I . hor{ lro'/..n..rl even numher of bts.l j r'.. flo.f u. then the opponent could have t:hosen a g with a.5.rt:ftrrcL is trot regular.n.o I I which is in L. 6 1 * t.'$ on thc lcft tharr orr the right and so cannot be tlf the fbrrrt u.r. .a F+tE J Z .1l1lly the purnping letntna we cannot ir.rrcl that allows us to produce a.e restricttxl irr Step 3. s )< ..virilrrtion of puttrping letnma on our next rrrtlve.Jsurnc rnirk(r ir wrong rnove. We would also of fir"ilif we were to choosea string cxrnsistirrg all ats.t thc oJrportettt is so accommodating is autornatically incorrer:t.. Tho. If.tus. It. that the opponent will Ttr ir. the opponent wc:rt:to pick a:Q..q...tuft. The strirrg obtained in this fashion has fewer o. / a r n .+rtL.:(+lrl. for. To dcfca... say. In Step 4.
.l with ir.nything br. The languirge 1.b.sour string ut: (ab\"'t7 a"'  which is in L.L. r z > 0 } is not regular. beca.r7:.ny possiblu r:hoir:el rvltich is _Illl_t by the opponent. Supposewe are given rn. b}. the purnping lerrrrna. r!h<rn' We now pump up.lJ Itrurq'r'rr."g!lS.rrchoost)rl :0 rr.use lrgtl cannot be greater than rrz. in which case we can use aill as trr).b)a). : {(ab)" ak : rt> h. Ciiven Lhe opponentts choice for rrz.t L: { e r r l. = lut E X* : rzo(..string not in .rgument.rt pick a g with all a's.irr.gg+cl*UrlrJ:! bqilltp_pg1! g{_tbg p_tlirrg nade up qf. using zi: 2.ru)< n6 (u)] is not regular. wc can dea.L ((a. If our opponent picks so (gL:6. Irr thc sarrrc wily. If the opponent wc piclks. Therefbre. Now we get the string (ab)tt'a"'.we pick a. iqLl. fo > 0} is not regular..4.t ca. ancl L is not regular is The larrguagtl 7. let us seewha. Because of tire constraint l*El < ?r4.we pick as 'u the string a"'l (unless the opponent picks nt 13.gir. Now.nnotdo a. the opllonent r:a.ywcca. I Slmw tha. that is y:ak. t:hoosei = 0 and get ir. The choice of r does rrot affect tht. The various .g. bo!l. Civen rrz.'ylNcNoNREcuLARLexcuacns 119 Let X : {a. wc pick ut * tt"'b""t r. The resultirrg strirrg ?il2: 5*trlE6m+L is not in .'+r. Since we have cornplete freedorn irr choosirrg rrr. thereby proving our clairn.nbe done with y. violated.
h > 0 } is not reeular.r.1)! Therefore.s which has length m.i>U}. Ttrktl : h(a) : u. but it is even easier to use closure uuder homomorphisrn.gt: l:{ar. I In some cases. This ntay be much simpler than a direct application of the putrpirrg lemma. Chopter 4 Pnorenuns or RnGtJt Latqcuecns decotnpositions of tu obviously differ orrly in the lengths of the uubstrirtgs.l .120 an.L carrrrot be regular either.closure propcrties can be used to relate a given problem to one we have a. It is rrot clifficult to apply the pumping lemma.k. . Suppose the opponent picks y srrth thtrt lul: h < m' We then Iook at z. clircctly.h. This string is irr l. I Show that the ltr. but we krrow this language is not regular. since for rn ) 2 and k { m we havc h!:.lready cla"rsified. thereforc .ngrrtr. the lartguage is not regular.. only if there exists a j su<:hthat mth:jl But this is impossible.h(c): c lhF" h ( L ) : { a n t h t " n + k: n * k > 0 } :{aici. Show that the language L : { a " b k c ' ' t k I h > 0 .(b) a.bt.rnltu>(m.:nlI} is not regula.
Its staterrrerrt is complicated.C)onsequently. Let us ta. The riglrt side is tirerefore an inttlgur. there is a.: m.n be decomposed as 'tD: frUE1 .tingthe conditions of the purnping lernrnir. tsut L1 : fntlhtt: rz > 0].l and I : (rn * 1)!.u € Z of lerrgth greater than nr.re to be congratula.* I)k a's. Clhoosinga string with ri. E l*2 will not do. Knowledge of the ntlqrs is esseutial.good game.y rr.rt ihat alotre is rrot errough to plir.ke it impossible to pump thc string orrt of the language (that is. r. Then. The pumping lernrna is likc a girrnc with r:omplica. tirrx:s to generate a string with rn! + (i.ys nt. since our opporrerrrt citrr illwalys choose a decomposition that will ma.tedrules.kl n. ntl . But everr if wr: tnaster the teclurique. irrrtl it is oir. it rniry still be ha. : l+ 1 or n.rT fr arxl A { rn. whicli we have already classifit:das rurrrrcgrrlilr..tecl.stray in applying it. (t Prove the fbllowing version of the pumping lemnra.syto go a. vou a.1. If thc ollporrcnt now choosesa g (by necessitycorrsistirrg of all a's) of length h < rt. We can geL a corrtradir:tion the pumping lemma if we of r:ilrr pick i suclt that nz!f(i1)A:(rrr.*1)! possible since I'his is rrlwir.cns 121 I'Iere we need a trit of irrgemrity to apply the purnping lernrrra dirrx. thcn there is an nz such that. If yoLr can apply the purnpirrg lcmma. every . rnuch more elegarrt way of solvirrg this prriblem. [f L is rcgular..il Innur'lpyrNGNoNRECLILARLar'rcuA. alsrl rrrxrd er. correctly to sotne of the rnorc rliffir:ult r:asesin this book. We rnrrst be rnore inverrtive. Z ancl tire lilnguage I't:T ( r . Lo * b * ) woulcl also be regular'. cannot be regular. irnd we have srrcceedecl irr violtr. we purrrp r. . tr I The purnlling lcmmil is difficult fbr several reasons. by I']rxrrclrn 4.tly.t . Yon good strategy to win. Howcvrlr. br.4.rd to seeexactly how to use it. prrurp it so that it has an equal trutrrlrerof a's arrd b's). ca. firrllpose L were regular.
L fbr all i.Lz are nonregular lan@erov* guages.t)l \J''.^ l ' (a) I : {an : n } 2."L. r}_1..(tu) t' nt (.L .12. therr there exists an m.I: (c) I : { o " 6 l o a: A t I n ..ru)} ffi (f) t .. l_ D n"t*. . r . Fa. t ' .gt*..r. is the product of two pritre numbers} (f) . '.'grl rn.: (.b}} (g) L: t { t u r r l ..L and cvery one of its decornpositions llr : I.*inc ou if the following larrguages X : {a} are regular ffi 1 oqK . o l " . ": . = {an : n is either prirne or the product of two or more prirne numbcrs} 6. :2k (e) I : {an : n. t .. \_/ Lrro.."Itlttk I+} n+l} : (b) .L is regular. 2 Prove the frrlkrwirrg generalization of the prttrrping lemma.t f .t) . 2. : {ut : n.r.tz with u. lgl ) 1.such t. Apply the pumping lcmrna directly to show the result in Example 4. j L'\fl nr la t (r] n>i1r\.8 as well us Exercisc 1. rno* that thefollowinglarrguageis trot regular. . il'he middle string u can bc written asu:'rAz1 < with le.lul ) tn.L:{o.*+(b) tv tL : {a"' : tt' is rxrt a prime number} {ttn : n: A 2 f o r s o m eh > 0 } for some h > 0} 4 .L../t} . (c) l.. ffi (u) f : {a.rr.ulrgi z'uz E L tor all i : 0.g x n L:{*n'h = n t5*'i \ n> lL \ (\( * \ .o"* that the following larrguages are not regular..as special cases. l s l> 1 ' such that nyoz it irr . 1.{""r.hat.1).(i) ordisprove the ftrllowing statement. Rl: e { a .."bt:n{11 (c) Z. " .} " .{unn :'ur€ {4.{a" : n. regular? 3 .t. Lar'{cuacns with lyzl < rn. n' is a prirne nutrber} ..l I ) : I or l f k} {anblak : n (. therr Lr U Lr is also nonregulat.. If .""If trr arrtl . "nl_ _ .r22 Chopler 4 Pnopnnrtns oH ltr:ctrlan. b } " } (fl *. r t _ . such that the following holds for j every sufficiently lurg ur € .. ffi .z€ X*.tltl.. which irrclutles Thcorcm 4. m {w:n*(ut):"r6(tr')} is trot rcgular. Is. S l r o w t h a t t h e l a r r g u a g et ' : (i)e. ' i . r r . .
6}+ .rrz {o.  r.wE la.An. l"t Lr and .L .geregular? [. so tha. kr r l l l k : > 5} m ffi (b) fI : {a"blak:rr. : {w1c'u2 i Lurl.ncuI.utr e LJ} {tu : Apply the pigeonhole arguuent rliret:tly to the larrguage irr Exarnple 4.nguages."ht : n. of .Lu is the union over the infinite set P. Thc smallcst sct containing every . : jrLt {tnuwRui u. :  J L \/. ( a ) Z .ulcrs 123 Consider the latrguagesbelow.{a"bl : nf I is an integer} (d) /. the infinite uniorr is regula.lenotedby Upep1.L2bc regular ltr.3 IuuvrrnyrxcNoun. r ' p ) .your conjecture. Show by cxample that tirc fatrily of regular larrguages is rrot r:losed urrtler irrfirrite urriorr. lrl } lul} dS G4 tr thc following languagcrcgular'l ' "'l l . b}+ } e W ( 1_t:)A* (a) I :  ".r the following langua.}5.1>3. r . I'he r+et walks is genera. L DeP whcrc P is thc sct of all walks through the graph and ru is the expression associated with a walk p. it will be c. : {a*b{ : rz "1.ut€ L1. it tloes rrot irnrnetliately follow that L is regular.n infinite Lrut courrtable set. make a corriecture whether or not it is rcgular.l l : 2 } \fOJ l.h languagc .tNc. i ' r ' i * (b) /.t in light of of Exerr:ise 15. P E P A '. beca.''r i'rtrj'Lt1'u) {a.ure of P..llyinfinite.h Sl} (c) .r. Show that in this case. Then prove . < t. : { a n b t u . arrd associate with ear. Consider the argurnerrt irr Set:tiorr iJ.8.use the special nat.2 that the langrrage associated with any getreralized trartsition graph is regular.)fet f be a. < 2nI ( I ) .p. .. u. For each.bl L J / $ rl+(L5. the languags 7 : Is necessarily rcgular'l ffi .L. ffi * 16.1r) e {a. I : { a " b t : n > 1 0 0 .4. The larrguage assor:iaterlwith sut:h a graph is '/5.is a prirne nurnber} I (e) {a. * t ' r .b}. the followirrg larrguagesregular? {u'*rr.Lo. 15 1 0 0 } ( e )r : { a " b r : l n . ju)r + :iuz} € ( il{ J 2. : { r .
L is not a regular all u e {u.Chopter 4 Pnopnnrlrs or RncutAR LANGUAGES tr /+ the family of regular languagesclosedunder infinite intersection? ffi \L /\ dEJ Supposethat we know that Lt I Lz and trr are regular' Can we conclude .ld}* language. In the chain code language in Exercise 22..L be the set of that describe rectangles. Section 3.1. show that .Lz is regular? 19.r. Iet .r. .# *fipfr trom thls that .
The rclcvartceof these limitations to programming larrguages becomes evirlt:rrt if we reinterpret somt: of tlte exatnples. If in I'or L: {q*\rn : rz > 0} we sutrstitute a left pa. then parenthesesstrings such as (0) and ((0)) are in L' tlrt a (0 is not. illustrating the dqfirritions with some simplc: cxarnples.tion is fir. The la.ngutr. in prrrticular we ask how wt: t:irn tell if a given strirrg is clerivablefiorrr a givtlti graurtnar.'rrtbtlrship problem.gos. ancl grammars. In rlrrlclr to cover this and otlrt:r rnore complicated fuaturcs we tttLtst enla.renthesis a atrd a right parellthesis for b. Whilc rcgular langua.nt nx. wc txrrrsider the importa. flxpla.y 125 .st chapter.rge This leads us to considt:r contextfree langrrir'gcs the farnily of langJrragt:s.r fbr exir.irririg ir setrletrce through its grilrnrnirtical deriva. one does not rrt:c:tlto look very fa. by We begin this r:ha.ContextFree Longuoges n thc la.nguirgcs.nguage therefore clescribes sirnple kincl of nestud stmcindicating that somtr llrollerties of ture fbund irr programmittg la.ges are ttfft:r:tive in describing t:elrtilirr sitttple llatterns. we rliscoverud that rrot all latrgrta'gesartl rcgular.1lter clefining contextf'rtxr gralrunars a'nclltr.rriiliar l.o tnost of rrs f'roru ir stucl. programmirrg lattguages reqrtirer sorncthitrg beyond regrrlar lirrrguages. Next.rnplesof nonregular languages.
ve already shown in Exarnple 1.geis said to be corrtextfrtxr and only if there is a contextI. Every regular grarrurrar is rxrntextfree. The topic of contextfree languagesis perhirys the most important aspect of firrmal la.nother. rxrntextfree one..ges.11 thrrt this language can be generirtctl tly tr. as we krrtlw f'rom simple examples such as {u"h"l. so wc s(xl tltat tlxr fer.nclother translating prograrrrs.as we do frrr irrstirrrce tra.T. A lir.ngua.nslating from one language in to a. In computer science. compilers.milyof regular Iatrguirgesis a proper subset of the fanily of cotrtextfrrt larrgrrir ges grirmmar$ derive their trattrt:frorn thcl fa. thclrc are nonregular languages. What frrrrnal lar.t tr : L(G). does rrot dt:pcnd on the . Wt: ha.crrntextfree grarnmar. a. so a regular langrragr: is trlso a.rs. if freegrarnmarG srrchtha. we get corrtc:xtfi'rxr M A gramrnar G: (V.mffim The procluctions itr it rtlgrrlir.126 Chopter 5 CournxrFRno Lancuacns of natural languagesand is callexl parsing.while tlre right sicle has a spcc:ialforrn. Actual progritrnrning la. but perrnittirrg arrythirrg grarnma.ges well as in the constructiclrr clf clfficient as conpilers. where.4 e V and 'r t (V u ?). To crea.t the subCouLext1'rtx) stituliotr of thc: variable on the left ol a produr:tiori t:ilrr be rnade any time sttch a.nguagetheory as it applies to llrugrilmming la. By retaitritrg the rcstrit:tion on the left side. ContextFree Grommors ffi. lt.P) is said to be contextfree if rrll prodrrctions itr P have Lher ltrrrn A+il.rrgutr.this is rclt:vilnt in interpreters.rgrarnmar are restricled in two ways: the left side musl be a sirrglr:variilblr'.tegl'alrtlrrars thirt irlu rntire powerfirl. we rnust relax sorne of tlrrlsc:rostrictions. variable appears irr il st:rrtential form.ngua. But..nguageshave many fealures that c:arrbo clescribed elegarrtly try means of contextfree languages. We touch rrpon this briefly in Section 5.3. on the right. Parsing is a way of describing sentence structure.S.nguage theory tells us irtrout rxrntextfiee languages has irnportant applic:rrtiorrsin the design of prograrnrnirrg ltr. It is irrrportirrrt wh(lnevrtr we need to understand the meaning of a sentence.ct tha.
but as shown in F.xample 4.ria. Regular ir. but linear.5. sirtgle va. blt lirrc:ar.uaBb. b} .r is S + a.l (J The gramma.r : ({S} .a + aubSbaa ) 'I'his makes it clear that L(G): 'rrr {u. A4. A . R + bbAa. letr.a. is t:orrtextfree. . with productions S +a5a'.:et rrllowittg only a.Su + aaSe.}. to it We is contextfree.l ori of the consecluenr. A typical clerivation irr this gramma. Longuoges Exomples ContextFree of H $*sqtrsf $.8.5 + b$b. it is uot regrrlar S . n.tlt:r showthnt L(G): : {ab(bbaa)" hha. The languagc is contextf'rrlc.b}.(ba)'' n. S.1 Conrnxrli*Rps GnalrunRs 127 'I'his f'eaturtr is frrrtrr (lhe contcxt). symtrols in the rest of the senterrtiir.rrdliuear grafiIIIIarS are clearly croutextfitlt:. > 0} T Both of the above erxanples involvcl gral]lmars thirt ate not only c:ontextfree.bbu'tt'.abR.94. . r).bk: Lhe left sidc of the procluctiorr.ruB. {o. a contextfree gramnlirr is uot neces$rrrily .vt: to the rea. e {a.
We first generate a string with an equal number of ats and b's. (ut) and no (u) } 26 (u) . and ababab. aababb..b} : no (w) : nr.L . This is another gramma. The particular ftrrm of the grarrmrar given here was choserrfrlr the purpose of illustration.} is contextfrrxl.nbuild on that solutiorr.remany other eqrrivalent contextfrrle grammars. and b with lc:ft a.l) We can sec the connection with programming larrguir. 5r .mmar is contuxtfree.cns WWWW rhe language 7.q. A +aAla. The resulting gra. we need to protlurxl a.mple 1.'). B +bBlb. I :IIWiliWWilWWMW Consider the grammar with productions .r that is contextfiee. In fact.respectively. . e {4.9 a.a5rblA. (b. then add extril fl. lb show this. there are some sirrrple linear ones for this language.\.'s on the left. Some strings in . hence tr is a contextfree languager. However. and we get thu answer 5 . but not lirrrrar. In Exert:ise ZE at thc end of this section yolr are asked to firrtl one of them. contextfree grarnmar fbr the language.:{a"b*:nlm.pp Lerrc+u.t (G) are abaabh. The castl rtf n : ?7? was solved in Exa. where u is any prefix of 'u.nd right parerrthtrses. 71+ aAla.. It is not difficult to urnjecture and prove that * L : {.5'bl55l.5r i a$rbltr.SrB. the grammar is rrot lirrerirr. Thc language .11 and we c:a.gesclearly if we rt:place c.ASrl. Take the case ??) rn. We can use sirnilar rea^soningfor the casc n { m.128 Chopter 5 ConrnxrFn. there a. This is done with ^9ASr.
+n 1 e'at'4 aaABb4 q.a.8*+A' getrerates language (G) = {aznb*' . a derivation rnay involve senwe tentia.ritr.bl .ngtrilgt:s' Here again therel ilnr rriany other eqrriva.r. . The clifferentre is etrl. we be often reqrrirc that the va.renot lirrcar. P) with produr:tions t.l forrn is replaced. 2.tsb.8 .ttutrr. 4.lfrrrmswith more thaln tlrrc variable.rntil ()hrrpter I befbre w(t (tirrr auswer this qrltlstiott' T Derivqtions Leftmost Rightmost ond In contextfreegrirfirilrarsthat a.3. L the that this gra.5} .id to be leftmost if irr each step tlrt: leftmost varitr. 3.44.riabltrs replaced in a specific order.5.. 5.bleis replaced.1 CournxrFRnn GR.rs. If in each step the rigltttnost va.ble in the sententia. A +aaA. Io. have a chtlice Trlkc for exampletht: grarrrrnar are repla.ced.> 0].5.8.ppliexl.sy sr:c to rz ) 0. in the order in which variiltrles G : ({A. In order to show which producliott is a.mrnar trt is ea. To rem6vt: suclt ilrelevant factors. it is rrot so easy to sexlif there are any lint:irr oiles' We will have to wait r. irr contrast to Example 5.c'Rs 129 incluclessur:hstrirrgs as (0) ancl 0 () () and is itr fa'ct tht: sct of all lrroperlv nestr:rlparetrthesisstructtrrt:s lbr the colrllnorl prtlgratntnitrg la. M A rlcrivatiou is sa.ntl 4 aaAB4 naB S uaBb4 aab \ s 4 . S + AB. we call thc derivatiou rightmost. Conuidcrrrow the two dt:rivatiotrs s 1.Ab aab.irely in tlrc order in which t}r: productiols arc aplllied. rn. Brrt.tiorrs rtot only yield tht: sarne sentent:tl but use exactly thc sarne procluctiqrrs. In srrch(iases.ta i.we have numbcred the productiotrs and written the appropriate mrrnber on the + syrttbol. F\'om this we see that the two deriva.lcrrtgralrtlrla.
re tertninals. Atr ordcred tree is a derivation tree for G if ancl orrly if it has the following propcrtics.. In a derivation tree. Figrrre 5.5.beled with the lcft sides of productiotrs arrcl irr which the children of a node rcpresent its corresporrdirrg right sides. latrtrlerd and ending in leaves that ir.1 ).AB. . For example.bABc. Bcginning with the root.hBhB abAbB+ abbBhhB abbbbB o'bbbb is a lefturostrlcrivationof the string abltbb. The followirtg tlcfinition makes this trotiorr precise.7. Let G = (V. A derivation tree is irrr orclered tree itr which rxrdes are la..A rightmost derivationof thc samestriug is + + S + aAB + aA + u. A +hRh.n. Al.ffi.\ \_1/ WnsicJer the gI'aIIuIIar wit} prochrctions S +a.lil.P) be a c:orrtr:xtfieegramlnar.hBh abAh+ abbtsbb ahhhh I Trees Derivotion A second way of showing derivat.iotrs.\.indt:pcndent of the order irr whitlh prodttctiotrs arc usud. [Rlnfii.Nil$. a derivatiorr tree shows how each variable is replaced in thc durivation. is by a derivation tree.Chopter 5 Cor'rlrjx'l'F nr:r: Lnrvc:uaclns Figure 5. fi __+ Then + + + S + uAB + a. 1 shows part of a tlcrivation tree representirrgthc prodnction A o.tf.schildren consistiug of the symbols ou the right side with the start syntbol of that productiorr. a uode labeled with a variable occurring on the left side of a production ha.fii 'f.
+ u 1 u 2 . omittirrg itrry . .is sirid to be the yield of thc tree. fr + Thc tree in Figure 5.nhave no other children. 3 .\'s encoutrtered.Gnnurutnns 5. Consicler grarnmar with procluctions S +aAB. always ta.sir. The yielld of the second trtrtr.n.la.belfrom T U {I}.3 is a deriva.W$I thc G. while tlrc tree in Figure 5. tttrexplorttd branch.t is. a2.2 is a partial derivatiqn trce for G. 4. A + bBb. Tlre descriptive term Icft to right ca. 4 A tree that has properties 13.trcl frotn V. and its chiklrt:rr are Iabeled (from ltrft to right) o. Every leaf has a la. Every inttrrior vettex (a vertex whic:h is not a leaf) ha. is a sentential form of G. partial derivation tree.1 CoNTEXTFnnn 131 l .intr. A leaf lahtllcxl\ Itas no siblings.9. 2.. Every lcaf has a label fiom V U 7'U {I} is said to be a.first rnarlrrer. a n .king thc lefttrtost. and 5. The string of syrnbols obtained by reading the leaves of the trct: frotn Ieft to right.llrclductiou of the ftrrrn A .. The string abBbB. The root is latreled . 5.$u\WWNNWW.nhe givt:rrit precisemeaning.. Tht: yield is the string of trlrrninals in the ordt:r they are ettcoutrteredwhetr the tree is traversed in a depth.' . then P must conta. If a vertex has labc:l A € V.tiotttree.. a vertex with a t:hiltl labeled A ca. thrr. rtlA. I . but itr which I docs rrot rrecessarily holrl and in which property 2 is replacecl by: 2a.e.1. abbbbis a sentenceof I (G). which is tlrc vield of the first tree.
3. this explicitness is a great helJr in making argurnetrts.2 Figrre 5. Also. Now any ?rrderivable in n * I steps .rmethat for every sentential form derivablc in n.S.lNc.' of G whose yield is ir. Proof: First we show that frrr every sentential fortn of I (G) there is a corrcsponding partial derivatittn tree.Chopter 5 CournxrFnnn L. We do this by indrrction on the number of stcps in the derivation.9 r u.(V. tlrcrr thc yield of fs is a senterrtial fbrm of G. we note that the clainred result is true for every scntential form clerivable irr one step. if t6.the yield of trny derivation tree is irr L (G).T. there is a corrcsponding partial derivation tree. As a basis. thongh.3 qnd Derivotion Forms Trees Between Sententiol Relqtion Derivation treeu give a very explicit and easily comprehended tlescription of a derivation. is atry partirr. First. Conversely. Assr. steps. Then fbr every € I (G). Like transition graphs ftrr {inite automata. 't . Since S + u implics that there is a production . this follows imrntldiately from Definition 5.l derrivationtree for CJwlxrsc root is labelecl5.tt)AcHs Figure 5.Let G .'. tlNrre exists a derivation tr*. we must establish the connection between derivntions and derivatiott trctts. P) be a contextfree gralilnar.
leftmost and a sce rightmost rlerivation (fbr cleta.arrcl W E rAy +'IA1A2"'Q. 1 .2. flowevcr' once wo have a.bbaba the grammar in Example 5. aE ( v u 7 ' ) * . We will loave this a"stlrr exercise.t the yielcl ()f every derivatio[ tr:eeis irr l/ (G). Exercise 24 at the erl(l of this scction). . we get rr.. showirrg that the latrguage giverr is gerrerated bv the gra.partial derivation tree we therefilre claim that the rlr. Since a clcrivatiorr trr:c is also a.2. rcsult is true In a similar vein.tiotr. l)erivtltitlrr trees are ablt: to represent atry derivation. By itrdrrctiorr. }rut we havet rrot cla. gap itl the preceding discussion. I Dcrivation tretls show whic*r productitlrrs are userl irr o[ta'ining a sentclrx:e. l)raw the dcrivatiorr Lrcc corresponcling to the dcrivatiorr ilr Example 5. partial derivatiorr tree whosc leaves a. W 5.Gnaultarr$ 5. we r:arrshow that t:very partia.1.!J: rAL V tlT.mtnar. Conrplete tire argumerrts in F)xample 5.. in rr.2 regular? Cornplete the proof in Theorcrn 5. for Give a derivation tree for w : ahhltu. wc catt alwaYs get a leftmost clerivatiott by thinkirrg of the trce as having been brrilt irr such a. with yield r&rfl"'amA: for all sententirrl forms. Is the language irr Exatnple 5. reflet:ting the fa.lwaysexpantled lirst. it follows that c:verysententlcin I (G) is the yield of some derivation trcrr:of G anrl tha..u. firw details. au gfus(:rvation whidr allows lltJ to close tr. A € V .titlrr.steps. wc are Iecl to thr: rrot surprisirrg result that any ?tr€ I (G) has a. aud sint:t:the gramrnar tnust have llroduction A + a1a2' ' '(trrrLr we see that bv expanding thc leaf labeklrl A.ils. way that thc leftmost variable in thc tree was rr. n . By cle{initionr any u E L(G) has a dcrivation.imefl that it a.l derivation trr:c represents some scrrtential fbrrn. Since try tlte iucluctivc assunrption there is a partial derivatiorr tree with yield :rA'g.1 by showirrg that the yieltl of cvery partial c{erivation tree with root .9 is a serrtetrtial form of G' s.but do rrot give tlx: order of tlx:ir applica.lsohad a leftmost or rightrrrost derivtr.1 CoNTEXTI"IIIJE 133 must be such that S I r A y . Fillirrg iu a.re terninals.1. Use the derivation ttee to find a leftmost derivation' show that the grarnrnar irr Example 5. derivatiotl tree. .4 does in far:t generate the language describerl irr Equation 5.ct that this tlrtler is irrelqvattL. 3.
\L).{ T r e { a . fe) = head.. /'\  ( L2/ Lct L: {a"hn: n > 0}.. (w)] (e) I : (h) f: 9' /\\ (10) {a"h"'ck. S (b) Show that . W . Show that the following.. 1 r r " < 3 n } S (*) I : {tl e {a. Given a contextfree grammax G for a language tr. w h e r e . For the definition of head. Find a contextfree grammar for head(tr).b. b}+ . .h I n + rn) {a"b*ch:k > 3}.l Lz is contextfree. \ .tt) {a. b}* : n^ (w) t' nu (w)} ( f ) . rn ) {a^b ch :rl:m or rn !.bl" : no (. lul : e l. L .17n + 3} ffi (b). (ut) I n. show how one can create from G a grammar I so that l. language is contextfree.mcuacps b f! Find corrtextfreegrammars for the following languages(with rr.ul: z} show that the complement of the language in Example 8. rn ) 0) (u) I : {anb* : rt.nInzL} (c)I: {a^b*:nl2rn} ( d ) . L : *r5' {uorur. i rt. L: { a ' b * : 2 n .Lh is corrtextfrec for a"trygiven A ) 1.q. 'i s a n y p r e f i x o fu r } r r (e) I : {w e {o. L: (e) L: Find contextfree grarnrrrarsfor the following languages(with n.L is the language in Exercise 7(a) above.b} forthelanguage p= {anu^u?b : w E *11. TndacontextfreegrammarforE: \/ x. b \ * : n o ( u ) > n 6 ( u ) . Show that Lt J Lz is a contextfree language. (a) I: (b).> 0..h} ffi {a"6*o"n: n : rn or m I k} {anb*tk:h:nIm) {a'ob*ck:n+2m:t+) {a"b*ck : k : ln *l} W (f) f : {w e [a.n > 1].see Exercise 18. *.. ) 0. (c) Show that Z and L* are contextfree.Lr be the language in Exercise B(a) antt .h>o). {o.I : (c) tr: ( d ) .1 is contextfree.. 14.I:{u. 13.Lz the Ianguage in Exercise g(d)./ \J (a) Show tl::.c]* : nn (tu) + nr.134 Chopter 5 ConrrnxrFRnr L. where .w): 2nt. Let . o.rr."h"'. Section 4.1.(ur)+ t}.
1 Cot'lrnxrFRer:ClR. Show that the cor[plefirent of t]re latrguage in Exercisc 8(h) is cotrtcxt*free. tr} . say 0 and []. properly nestetl strings irr this situatiort are ([ ])..b. BSb. Civc a verhal tlescription of the language gerreratcd by this gralrlmar' 4ti9)orrsicter \_. Give arr algorithm for finding sudr derivations from .ivcly.. Irrtuil. .nnrlunRs 135 16.l. c}. then every u E L (G) has a lcftmost V ancl riglrtnost clerivation.b} and V: {A. Show a tlerivation tree for the string aabbbbwit'h the grarntnar g +.* that thc langrrage : {wicll)z i'u. ffi for the set of all regular exJrressions on the Find a contextfrec grallllllar that carr generate all thc production rules for contextfreegrammars with T: {a.C}' ( 24.b}.tt)2e {o. A + ttB.\. give a t:ontcxtfree gramrnar for ge[erating all properly nested parelrtnescs.fa*"""^" a derivatiotr tree.B. with }j : {n.*. A * bBiiltr. 18'. is contextfree. I. show that thc striilg gau. (tt ll) t()l' but not (Dl o' ((ll Using vour clefirrition. Consider the tlerivation tree below.'inrl a simplc graurrrrar G for which this is the clerivation tree of the strirtg of Thcn find two ntore serrtettccs I (G)' rr:1ntrb' what one rrright mean bv properly rrested parenthesis stnrctures in/ Uu)n"ntte volving two kincls of pareflthescs. w aabbabba is rrot in the larrguaE. hrouo that if G is a contextfi'ee grammar.c generatecl by this 2O.5.. t urit}.rrrrrt"a." the grarnrnar with prorluctions S * aaB. fi+Aa.. ' L fuf) Sir. Fincl a rrrrrtextfree glalnlnar alphabet {a.481.
Thus. we have serrtential forms tlmt can be derivcd by applying a single production.ll this the exhaustive search parsing rnethod. It rnay be that sorrrc of these senterrtial fbrms can be rtrjected orr the grounds that ur c&n never bc derived from them.ec: f'rom the root down. we are also concerned with the analytical sidrl of thtr grammar: giverr rr. Let G : (V. we will ca. This gives us a sct of sentential forms.5 irr one step. 2 6 . Ihe tcrm parsing describes finding a se(luenceof productions by which a w ( L (G) is derived.P) bc a contextfree gramrnar such that every one of its productiotrs is of the form . An algorithm that can tell rrs whether'ru is irr z (G) is a nrernbershipalgoritlrrrr.5. On each subsequent rr)und. If norrc of these rcuult in a rrratch with tu.136 Chopter 5 CournxrFRr:n Larqcu.c. In t:itscsof practical applications. Show that the dcrivation tree for anv ?rl € I (G) has a height h such that I o g i ' l5 h'< q + u. F'ind a lirtear grammar for the larrgua. we will have on each round a set of prnsibJe sentential fbrms.tion Lr. in which we irpply all applicable prod'ctions to the leftmost variable of every r.S.string tu of tclrmina.ls. we warrt to know whether ot rrot ru is in L(G). we go to the next round. the nrethod will eventually give a leftmost derivation of tt. leftrnost) derivations arrd see wlrcther any of thern rntr. . we can parse it in a rather obvious fashion. which we can viuw ir.tion of ru. then it rrlrst have a leftmost derivation of flnite lerrgth. It is a ffrrnr of topdown parsing.t can be derived ftom . we may want to find a deriva. Given ergr&rnmar G. Porsing ond Membership civen a string ru in r (G). with lul : h ) 1. arrd so on. If u € L (G).sthe c:tirrstnrctionof a dcriva. l ond Ambiguity {ffiffi Porsing We have scl firr c:oncentrated orr the generative aspects of grammars.4 * u. Specifically.q.ns 2.vethe sentential fbrms that carr be derivecl in two steps. some of thcm possibly leading to ru. we start at round one by looking at all productions of the fbrm S+JDr finding all r tha. For referrlnr:e below. we again take all leftmost variables and apply all possible productions. After the first rt)und. wtr svstematically construct all possible (say. but in general.T.1J. after the second round wtl ha.gein Example 5.tchru. If so. we studied the sc:t of strings that c:anbe derived usirrg (J.
But even wfien effic:itlrrclvis a setxxrdary issue.9.9.9+bSaS. 5+55+5. I Exhaustive search ltarsiug ha. severalof thesectr.ahl)is in the larrguage gencrated by the gratnlnar rrnder consideration. be Again. we fiud thc actual targct string from the sequenc(] S+aSb+aaSbb+aabb. there is tl rrtore pertirrrlrrt objection.5.rr removetlfrorn contentirlrr. 3. l I tlre tr. This is certaitrly the cilse in .9+ bSa.\.oundtwo thetr yields selntctttial S =+ 55'+ S. ftrr from further <:orrsideration obvious The last two of theser:irnbe removexl forms reason$. fiom senteltial fqrm 2 we get the additional sentcrrtialfonns S+aSb+tr. Sirnilarly. S+aSb+aaSbb. it is possible tirat it never tt:rmiuates for strings not in L(G). the which are otrtainedbv re:placing lcftrnost S in serrtentialform l with all applicable substitutes.rsirrgis requirtxl. . 2.On the rrcxt round.ysparses au € L (G). S+55+aSb.5. S + aSb. Tlrerefore e. While the rrrelhod alwir. 5+aSb+ab.9 + .R.nd string ru : aabb.no Atr.95. S+$.+5 5 l o 5 b l b 5 a . The most obvioqs orrtl is its terliolsless.s serious fla.q. .. 5 +.lrrIctlIrv tBT S .2 Pnnstllc . it is rrot to be rrsed where erflicietrtpa.95b.ws.Round tlrregivesus 1. 4. S+ab'b+u'bSab.
P) is a contextfree gralrrilrirr which does not have any rules of the fornr O*tr.7. If wtl examine Exarnple 5.rsingis relativrly TIte probklrn of nonterminatiorr of exhaustive $c:irrt:h elasvto overcornc if we restrict thc: forrn that the grirrrlma. It generates the language irt Exnmple 5. Supptrse that G : (V. Since neither the lerreth of a senterrtial form nor the rrtrmber of . so that we canttot tell *r.sily when to stop.rsingmet. ftll any u € X*.l the previous cxir.ct. This is clear becauser length of the senteutial ftrrm grows by at lea*stone syrnbol in each rountl.Itorlr:ilrr be made into an algorithm whiclt. pa. ot a tells us that nrr parsing is possible.r ca.h tirere are two types of produc. As wc will see in the rrcxt chtr.pter.hl ba satisfies thc given requirerncnts. those of tlxl firrnt . where A. For each sentential forrrt. B e V.7 without thc empty string. Irr fh. cither pr:oduces parsing of trt. consider both its length and the number of terminal synrbols.then we have rttarry f'ewer dilTiculties.rildinto it $ome way of sloppirrg.this restriction does not irffcr:t the power of the resulting gratnrna.tion$we wanl l.9. The gr*tnrnirr S' *+ 55 laSbl bSu la. sentential forrns indefinitely urrlcsswe br. Thetr tltt+ cxhillrstive seart:hlla. After lruj rounds we have either produ<:eda.parsing or we know that w { L(G). If we do Irot producbiorrs. this prorhrction can be ustld to clecrease scntcntia.o nrlc out. we see that the difticulty cotn(ts tiom the productions thc length of successivc ^5'* tr. A* R.4 + . Given any Iil € latb]+. Proof..4 + B. Each step in the derivation increases at lcir"st one of these.witlt m : a.mple.Chopter 5 CoFrrnxrFnnn LRwcjunoes the method will go on producing tria.l fornts.hb.7.\ as well as those of the ftrrrn .n have. have arry srrr'. the exhaustive seitrch parsing metlxrd will althc ways termirrate in no more than lrul rounds.rs irr ir. The idea in this cxample catr be gtrneralized and rnade into a theoreur for contextfiee languages in genera.ny signilicarrt wav.l.
If we restrict our$clves to leftmost derivations.rnethod as a linear time parsing .l:tion . A rnethod in which the work rises with the third power of the length tlf the string.u cannot bc genertr. while better than an e.rrstivesearch parsing is vcry inefficient irr most cases. brrt all of them are sufficiently conrplicatcd that we ca. The const.nrJso on.2) is only a. no precise gt:rreral resrrlt ca.nnot involve more than ? lul rounds.n 2 lul rorruds.pirrsing rnethod which takes tiue proportional to the krrgth of the strirrg.8 we will take this question up agnirr briefly.t. In section 6.practical observation shrws that exha. onc reasorr for not pursuirrg this in cletail is that evcn these algolithms are unsatisfactory.r. a.nrrot everr descritre thern wittrout developing some additiorral results.n be ester. we obsclved that parsing ca. of rxrurse. I While the tlxhaustive search tnethod gives a theoretical guarantee that parsing can rr. a derivation carrnot involve rlore thtr. and often the rrumber of sentcntial forms is rnuch snraller.lforms are generated diffcrs from case to case. at wlfch ti're we either havera succerssful parsing or .l forms cirnnot exceed M = l F l +l P l ' + .2 ) This indic.n be fountl in Harrison 1gz8 and Hopcrofb rrnd Ullrnan 1979.xponential algorithttt.tes that the work for t:xhaustive seatrh parsirrg may grow cxponentially with the le. antl a corlpiler hased on it woultl neetl an excessive amount of time to parse (lverr & nroderately long program.5. + e f t * t l (5. In the proof of Theortlrn s.rrrrrar. bound. makirrg the cost of the mcthocl prohibitive.n algorithrn that parses any ut E L(G) itr a nrrurber of steps liroportiorral to lrirls. the total numtrer of sententiir. is still cluitc iuefficient. Exactly how many sr:ntentia.Equation (5. Frlr every contextfree grarrrrnar there exists a. its practical usefirlness is limitecl becausc the number of sentential ftrrrns generated try it may be excr:ssivelvlarge. What we would Jikc to havc is a. .lwaysbe done. .tedby the gra. w() can have no more thnn jPl scntential forms after one rountl.ngth of the string.2 Pansruc ANDAMBrcurrtr 1SS tertninal symbols can excced lrul. wc refer to such a. therefore. tlut we can put nome rough upper bounrls cln it.blished. More detnils ca. Neverthcless.f more eflicient parsing methods for contextfree gralllmar$ is a crlnrplicated mattcr that belongs to a course rlrr compilers. no ilrore than lpl? sentential fclrurs aJter the set:ond round. There arer severtrl known rnethods to at:hieve tliis. We wjll rrot pursue it here excr:pt for sorne isolated results.
Next.tifii ltiffiffifiFil. We see frorn this that eaclt step produces one tertrinal syrnbol arrtl hence the whole process must be t. we substitlte for the variable 41 . look at the exharrstive sea. and any pair (A..a". * € V*.r$ are quite restrictive. where A € V.. and starting with a1 olr the right.S __+ ald S + a. As we will see in the next section.. a) occurs at most once in P' The grammar g + c. we rnust have S 1a4azBtBz"'Az"'A^. a E T. Illaily featlres of comtnon programming languages can be described by sgrarrrmars' If G is arr $grammar. they are of some interest. The gra.cns algorithm. but sinr:e again there is at most one r:hoice.q.SS. specialcases' ]. we do not know any lindar time parsing methods for contextin free languages general.a) occurs in thc two prodnctiolE .9lbS$lc is an sgrammar. S .fi.'A*. the derivntion rmrst begin with S + a1AyA2. P) is said to be a simple grammar or $grammar if all its productions are of the form A +ar.140 5 Chopter CorurpxrFRuu Lerucu. (G) can be parsed with an effort proportional to ltul To see this..rchmethod ancl the string rl : tt1Q.7. then any string trr in f.mmar g + a.omplettld in tro more that ltul steps' . but important.2.H ir A contextfree gra. aS I While sgramma.mmar G : (V.but such algorithms can be found for restricted..9lb55la$Slc is not an $grarnmar becarrse the pair (^9. since there can be at rnost one rule with S on the left.
rvhere there should be urly one interpretation of each statement.aSblS^91I. we s&y "the" derivation tree because of the possibility derivtrtion tree rather than that a number of different derivation trees may exist. where it is tolerated and dealt with in a varicty of ways.9 .2 PARSINcaNn AMeIcuIrY 141 Ambiguity in Grommqrsond Longuoges On the basis of our argument we can claim that given any w e L(G). Figure 5. unamlliguous fbrtn.is ambigrrous.4.4 . A contextfree grammar G is said to be ambiguous if there exists some ut E L(G) that has at least two distinct derivatiou trees' Alternatively. ambiguitv rnust be rerrtoved when possible. The graurmar in Example 5. with productions .5.lent.4. Often w(] can achieve this by rewriting the grammar in an equiva. _ l Ambiguity is a common feature of natrual languages. "4" exhaustive search parsing will produce a clerivation tree for tu. ambiglitv implies the t:xistence of two or more leftmost or rightmost derivations. This situation is referretl to as ambiguity. In programming languages. The setttence aaltb has the trvo derivation lrees shown in Figure 5.
esfora*b*c. For instatrce. E . b . * . ( . IIowevr:r. (b) Wonsider the grammarG: (y. and productions E1.5 Two dcrivatirxr tre. 1alblc.5(a) asthe Correct parsing as it indicates thal b*r: is a subexpressionto be evaluated before perfortring the addition. as is done in programmilg marlrells. * T : { a . EElE.I}.this resohrtion is cornpletely outside the grammar. .ke Figrrre 5. ff + E. I b*c ha^stwo differtlrrt derivatitln tree5. as shown itr Figure 5.tesa.n Chopter 5 ConrnxrFRnu l.(E).T'E. restri(:ted subset of arithrnetic expressions lbr C and.5. The EI&IluIIar is arnbigutnts. Tlre strings (a+b)+c and q.. c . I One way tq resolve ttre ambiguity is.nncuacns Figure 5. It is better to rewrite the gramntar so that only one parsing is ptlsuihle. to associatelrrecedencenrles with tlttl operalors * and *' Since + norrrra'llY has higher preccdencetharr *.P) with v = {E. ) }. Puq(rilllike programming Innguages.rE. the string u. we would ta..rb + c arc in L(G)' It is easy to see that this grarnmar getrera.
*qfiplo S. givt:rr contextfree granrrrrar is amtliguous or whether two given txrntextfree grarnrnars are equivalent are very difficult to answer.lrercrrr'rv 143 Figure 5. In somr: instanrxls. t t Ttr rewritethc.mhiguity is in tire langrrage.lblr:.blcs. T+T*F.6 $.7. I o.mple 5.n equivalent unambiguous grammiir. r In the foregoing examplrr the amhiguity came from ttre grammar in the sensc that it crruld be removed by finding a. the questiorrsof whether a. No other tt derivation tree is possible for this string: the grammar is urrarnbiguous.tr'.ce productiorrs thc with E+7.grarnmtlrin Exa.5. T ' .+F .. In fact. we will latr:r show thrrt there are no general algorithms by which thesc questions r:iut always be resolverd.1} and repla. taking V as {ll. . F +I.but. F + (E). E*:E*7.11.? Pansruc Arun $.11we introducencw varia. It also is cquivalent to the grrrrnrnar in Exarnple 5. A derivation tree of tlrc sentenrxr lb + c is shown in Figure S. this is rrot possitrle becausc the a. It is not too hard to justify these claims in this speciflc instarrce.6. irr general. howevcr.
lt tloes of clerivirtions. tltett L is snicl to be una'rnbiguotts' If t:verY Branlrlrtrr that geueratt:s L is aurlriguous.llffl'''r' Exomplo 5. tire first puttitrg a rtlstrictiotr orr the uutnber for does thcl seune bts il.51 A .sy sitow.:flll::lil llH.52 atrd llrtr by arr a.'sa.r ambiguorrrisiltce tlrtl stritrg A''bncn has two distinct o1e strrrting with s =* 5r. tht: pther with.l 1lrff i.. clA.13 The languagt: L .bEr:ll..:]"". is The grarrrrrra.l .trcl while the scctlnr.ty ['1attd L2 for other tronarnbiguoltsgraIIIIna.L44 Chopter 5 CorirnxrFR.t L: by L1 is generir.aSzlB..lfl.I'"I..ff . 51.r:r:Luqcuaclli ni. Brrt in sotne: exist Sorrr() hi1vecolflicting requirerncnts.\.rS it.. thcn the languirgtl is called inherently ambigrrous' rwi sor'': th il. somewha.a. Thcrr L is gette*r. course not follow I'rom this that L is inhererrtlv a'mbiguous a's there rnight w.teclby the t:ombiuatiorr of these two grarntnars with the additional produt:tiorr ii 51lS2.{a"bn t:"'} u {atbtttctt'} t It is a.na. and .logous Sz.Abl.ri ifliif 11Spfi g1v611 If L is a cont(rxthee langurrgtl tbr which thcre exists an rtn1mbiguotls Brtt'mrnilr.terJ LtU Lz. A ftlw tries will bts.."'If.nd c's. is to That tr is cxrntextfree ea. is an irrhcrently anrbigrrous conttlxtfiee la[gllage.5'+ sz.:tious grermtnar with start syrrrbtll ... of a.t clifficult tniltter even to exhiit art inherently aIII with rz alcl rn nonnegativr. Notice tha.L2 is rlur. B .
D } . In general.12.a. using the grammar irr Exarnple 5. Show that the folkrwing gramlnar is amhiguous. uniq'ely.l'r. A rigorous argulnent) though.rr :."lt" : ru > 1}. Use the exhaustive scarch parsirrg rrrethod to pa.r of ll.t sBr'alnlnaris unambigrrous. for a 2. Show that the grammar in Exarnple 5.u. S * A E*b.a. one proof cilrr be founcl in Flarrison 1978. Give the dcrivation tree for (((a "f b) + c)) * o f t). ffi Give an unambiguous grammar that gerrerates thc sct of all regular express i o n s6 1 X : { a .b}} is not inhererrtty ambigu 16. L 10. S + aSbS lbSasl A L4.5.l be neederl to parse any string ru in this language? . W 15. } fbr L9 show thal sno. Show that thc grammar in Example 5. 17. Slxrw that the grammar in Exarrrple 1. Construct an urrarrrbiguous grammar equivalent to the grammar in Exercise 6.P) be an ssrarnmar. : {a''. 11. Is it possihle for a regular grammar to be arnbiguous? tt li:* 13.rse ttre string abbbbhbwith the gramrnar in Exanrple lr. ffi B.il.ru 2}. {ffi 7. Firrl arr sgramrnarfor L : {. is quitrl teclurical.n./l ancl l7l. Sltow that thc following grarrrrnar is ambiguous. E. how rnarry routcls wil. Show that a regular languagc cannot be inherentlv arnbiguorrs. l'ind an sgrarrilnar l.S. that the larrguage7: {uu..4 is ambiguous.rtt b).Il:tu E {a.13 is ambiguous. Clive an expression for the maximurrr sizc of P in term. but that the lnnguage denoted bv it is not. alAa. 6. ABlaaB. r Gl "u.t arr sEirarnmar tr (a.5 is unambiguous. Let G: (y.? Fansrrvc+ Auucurly arun l4E tluickly cortvince you of the imllossibility of combining thgsc requiremelts itr a single set of rules that c:over the case n : m.
s.5 2 0 . ir.mmar. aqcl so o1.12 miglrt appear irr BNF a*s I (erpressi.ssion) (term) .re important in achieving this. Thg symbols * ancl + are tetgrirtals. The basic problclll }rtlrttis to define a. This forrn is in essence are bqt the appearallceis diffcrr:rrt.. Let CJ: (y. to this chapter. Prove the following tesult.Termilal symbols are writtr:rr illso uses subsicliary syrnbols such as l. tiorts of Theorem 5. But othcrwise there irre no significant dilleretrccs bctwtlen the two notations' Ma. For exarrrplel.T.me. P) lre a cofltextfree Btailrrrlar iil wlrich every A e V oc(ilrft on the left side of at urost o1c productiol' (J is urrarrrloiguorrs.ges tend to use more t:xltlit. m Thert 5.r:kets.nt) i. BNF. As with most other latrguages. we rrefil c:ontextfiee langnages to model more cortrplic:attxl turpects.ngua. regular lalguagels irrc u$ed in the recognition of certain sitrtpltl ltirttl'rns but as we arguecl in the irrttoduction which occur in prograrrrrning la.f state.2. It is traditiorral irr writing orr pr()gfammirrg languages lo usc ir rxrnvention f'or specifyilg gIaIIIInars calltlrl the BachusNaur form or the sarne as the notatiorr wc havtt used here.r Example 5. BNF brir. Both rcgrtltr.mtning language are susceptiblc to rlclinition lry restricted forrns of txlntext'fiee glammars.f (erprt:ssi. but ::: is usetl instearl clf lir.ria. wc (:irrl define a programming latrguage by a grir.rxlin the construction of itlterpreters nrrclcompilers fbr them.r and cotrtextfrec lirrrguirgt.: (/ar:tnr) (term.ncl reliable trauslation prograllls. rmrt:h in the way we have done.on) ::: (terrnl  (t:nytre.) + (/octor) .146 Chopter 5 CorurnxrFnnn LeNcuacns 1 8 .bles enclosedin triarrgular withorrt any special marking. In BNF. The symbol I is used +' RNF as an alterrrator iL'Jin our rrotatiotl. (terrn) .sthe starting point for the lar. As wtl hirvtt $een. Thus.ngrrages.programming precisely arld to use this clefinitiorr a. Itascirl ifthenelse statetnent can be tlclfirrtld as (i. Fistl a grammar equivalent to that in Exa'rurple which satisfies the trrrxli Grummorsdnd ContextFree $ffiffilii P r o g r o m m i nL o n g u o g e s g One of tlte rnost imporlant uses of tlrtl tlxxrry of fbrmal languagesis in the clefinition of programmitrg languagcsa. va.'str.ngrr:lge writirrg of c:ffir:icrit a. Slxrw that the grammar in llxar.ny parts of a Pascallikc progra.on) (the'nclause)\elseclause) ' ::: .rrplc 1.it va.riable clescriptious of programrning icleltifiers to make the irrtcrrt of the production explicit. in tlre grirrnmtr.14 is unambiguous' 1 9 .
rxlrnpiler rnuch easier. we see that this looku liktr arr sgranunar production. For Itascal. AII other term$ are variables which still have to be defirrcd. We see here a reasorr why we use keywords in prograrrntitrg languages.blc to recogtrize attd remove arnbiguities. Unftrrtunatelg not all features of. yet allow us to pilrst: irr linear time. What we need for this purpose are algorithms for cletecting attd removing ambiguities in cx)rrtr:xtfreegrammars and ftrr tleciding whether or not a contextfree ltr. whic:h have the ability to express tlrt: less obvious features of a prograrrrrning larrguage.r typical programtning language (jaII be trxpressedby an sgramrna. Unfortunately. irnptlssible in tlte most general scIISe'as we will see later' of Those a^spert:ts a programming language which can be modelul by a However. extensive use has bcen tnade of what are called LL arrd LR grammars. it contextfieel grallrlrlar are usuallv rerfilrrcd to as its syntax' is rrortnally the case that rrot all programs which tr. Irr connection with this. the issue of ambiguity take$ orr added signifl(jarr(ic. Thtl rules for (erpress'ion) abovtl irre rtot of this type. but also make the work of a. the usual BNF definition alkrws constructs such a.3 Cox'r'ErxrFn.s 1)Qrr111 : r'eal. n ::3. The variable (if statement) on the left is always associated with the terrnirral z/ on the right. a naive approach catr easily introdrrce a. The question then arisesfor what grammatical rrrles we calr pertnit and still parse cflicierrtly' In compilers. r.sitrple rnatter.2. Wtr will briefly touch to on this topic in Chapter 6. theutr irre very difficult tasks. Keywrlrd$ not otily provide some visual structurqr that can guide the reader of a prclgrarn. A related question is whether a language is or is not irrherently ambiguous. tnrt for our purposes it sufficers realize that $rrch granulrars exist and have brlcrr widely studied. z '.4. For this reason such a statement is easily arrd efficietrtly parsed. ancl much of it is beyond thc scope of our discussion.mbiguity in the gra.nn L47 Here the keyword z/ is a terminal syrrrbol.'irtteqer u&T 'r i ?n.r.rt:syrrtactically correct in this serlseare in fact acceptatrleprogram$. so that parsing hecomeslt:ssobvious.Laucuacns GnalrlreRs ANDPROGRAMMING 5. This is not a. Ttl avoid such rrristakes we must he a.mmar.nguirgeis inherently ambiguous. otherwise a program mav yield very different results wherr processed by different compilers or rurr on different systems.[egel".crccptirble a Pascal compilt:r. The specification of tr prograrrlllring language must bcl urrarnbiguous. such iLT . to Neither of tht:stltwo cotrstructs is a. sitrcethey "etrnirrtcger variable cannrlt bc assigtred violate otherr constraints. If we check this against Definititlrr 5. As Exarrtple 5'11 shows.
Nothing Prograrrrrrrirrg cxlrrt:ise contextfree gramilrars exists ftlr tIrc Specifica'tion a. pa." this kind of rule is part of progra.ltics are a courplicatctl rnatter.Chopter 5 CorurnxrFRnn Lamcuecns a real value.nent) undefinccl)' 3. Find examples of fcatures of c that <:annot be described by contextfree gtamIIIATS.ngrra. It is atr orrgging concern both i1 prograrnrrrirrg larrgrtages and itr formal latrguage thexrry to find effective rnetho.s as elegattt atrtl of prograrnrrring lnngrrage semantics. aud Colsequerrtly some semantic features ilrav bc poqrly definecl or ambiguotts.s to clo with how we ittterpret thtr rntrir. .r[rrrar t]rat shows the rela.tion between a Pascal program ancl its srrbprogrants.ningof a. Several methods have beel proposedr btrt none of them have }ee1 as uttivcrsnlly accepted and as successfulfor sernarrtir:tlcfinition as contextfree languages havtl bt''en for svntax. sinrxr it ha.rticular corNtruct.te. 2. Give a BNH dcfinition fbr the Pa^rcalwhile statement (leavilg t]re general concept (sfc. Give a definition of ttre correct form of the ifelse stateuent in C' 6 .ge sema. Give a BNF Hril. 1' Give a complete dcfinition of \ex'pressiora)for Pascal. 5 .Is ftrr tle{ining prograrnming lauguage scrtrilntics.mming latrgua'gesenlalltics. 4 . lrr. Give a BNF definititxr of a FOttiI'RAN do statement.
Bccausc of this.pterwel strrdy $evrtrill trrr. corrrplelter ancl in facl.tisfies certa. A 1orrnal filrrn is orrt: tlurt. \A/tl alstl irrvt:stigtrtc rrorrnal forrrrs f'rlr cxirrtr:xtfr(xi grarrrrrrars.4 .t sa. [n this r:ha.mrna.tiorrs arrtl sutlstituticxrs thu.A and .lthough rr:strir:tc:rl. we must atttlrxl to st)rnt:te:t:lrrrit:irl rnirtters. a. However.neqrriva.t will bc rrscfirl irr srrbsr:tlrrclrrt rlisrlrssiorrs. we saw the corlelielce of certairt restrictiorts olr gralrulratical forlm.r into a.3.ypractical and theoretical uses. We itrtroduce two of the tnost useful of these.nsftlrmir. is a detrirnent il sorne argumenls.S i m p l i f i c q t i oo f n ContextFree G r q m m q r sq n d Normo Forms I efore we carr study context*free languagesin grealer clepth. In rnany irrsta.nces. 149 .lent one tha. An immecliate applicalion of the chonrsky normal forrn to parsing is giveri in ser:tion 6.B made the arguments easier. elirninating rules of the forn A .Both ltave tnatr.l. In'I'heoretr b.is llroad errough so that. is it desirablettl place evell lror(l stringcnt rustrir:tirlnsrlrr tht: grirrrrrrriir. a1y glirrrlrrlirr hlm itrr txrivirlertt trorrrtirlforrrr versiorr. Thc tlcfirrition of ir contexlfree grirrrrrrriil irnpOses ntr rtlstrit:tiorr whatstltlver oil the right side of a freedonris rrot necessary. . the chomsky norrnal form ancl the Greibach normal form. llrorlur:tirirr. we Ileed to look irt rnethoclsfor' lrirnsfonning an arlritrary corltextfree gta.inrestrir:titins on its f'rrrrn.
given any contextfree {I} G.they will be usedmany times in later discussions' Grommors ffiffi Methodsfor Tronsforming we first raise an issue that is somewhat of a nuisance with grammars and Ianguages in general: the presence of the empty string.T.{I}. there is a method for obtaining d such that L (e) : . Suppose that P contains a production of the forrn A +r1Br2.L be any contextfree language. there is no difference between contextfree languages that include . What we meaJl by it is tlte removal of certain types of undesirable procluctions. but we will use it nevertheless. tnaking Ss the start variable. and adding to P the productions 5o. looking only at languages that do not contain .n make for .\ and those that do not. as we see from the following considerations' Let . For the rest of this chapter.In doing so. We prefer to remove it from consideration altogether. and let G: (V.4 and B are different variables and that B+atlazl"'la" . unless otherwise stated. Therefore any nontrivial conclusion we c8. Let G : (V.rpl. we will not define the tenn s'implffication preci$ely.The various conclusions significant. P) be a contextfree grammar. Rule A Useful Substitution Many rules goverrr gerrerating equivalent grarnmars by means of substitutions. and it is often necessary to give it special attention. (G) (see Exercise 13 at the end of this section).S. we do not lose generality.tes. Assume that . Consequently.P) be a contextfree grammar for L .t..\free languages.L. ttre process does not necessarily result in an actual reduction of the number of rules. we will restrict our discussion to .lcnrroN CournxrFnEE The somewhattedious na.96.tureof the material in this chapter lies in the fact that many of the argumentsare manipulative and give little intuitive insiglrt.L. The empty string plays a rather singular role iil many theorems and proofs.I 150 FoRMS AND GRAMMARS NORMAL or.91tr.this technical aspect is relatively unimportant are and can be read casually. {I} grammar will almost cortainly transi'er to . Then the grarnmar we obtain by adding to V the new variable . * genera.S.. AIso.\. For our purposes. for all practical purposes. Here we give one that is very useful for simplifying grammars in various way.T. 6 Chopter Slr.
that S46u. so that e S 4c tr.F) U. lry induction on the number of times the production is applied. n Then (6. If this derivation does not involve the production (6. can strowthat if w € L(t). But with granrmar d we can get S 16 u1Au246 u1fr1!in2u2. tr 1e.. (E\.1) .1 is a simpleand quite intuitive substitutionrrrle. . € L ( G )'. A production A . Thus S 4c urAuz 46 u1r1Bfrzuz +G utixr. is *secJ again later.t h e n .1) is used.6. we completing the proof. e t ew Theorem6.1 MerHoos poR TRnl.ntBrz can be eliminated from a f{rammar if we put in its prace tnu" w e L(G).1).gjr:zuz. The subscript on the derivation sign * is used here to distinguish between derivations with different grammars.(*) =r(G) Proof: Supposethat .S. we lose nothing by assuming that this is done immediately (seeExercise 17 at the end of this section).ntBrz from P. The B so introduced eventually has to be replaced. we can repeat the argument.r. and adding to it A + rarrz lnraznzl.the grammarin which F is constructed deleting by A . I T h e r e f o r1 f. \ ./ By similar reasoning.r. It follows thrrrr. lrtanrz. Thus we can reach the sarne sentential form with G and d.rsFoRMrNGGnelrlreRs LEt is the set of all productions irr P vrhich have B as the left side. then obviously S36. .rr L(G). If it does.T. then look at the derivation the first time (6. Let E : (V. .
B + ahbAlb. In this result. FOr example. in the grAmrn1l wlxlse errtire production set is Sa ^ 9 b. While A can occur itr a string derived frorn 5. \ l A .MS aNl Chopter6 SrurunrC. P) with productions A +a. B} . l AaA. atrd the correspotrdittg derivatiotr A+aaA+aaabbc ln (j.IRL the set of procluctions in which B is replaced by all strings it derives in one step. using the suggestedsubstitution for the variable B.AlabBc. the variable B and its a^lsociatedprgductions Notice that. Retnoving this production leaves the Ialguage unaffected and is a simplification by any definition' . The case when A : B is partially addressed in Exercises 22 and 23 at the end of this section. are still in the grammar even though they can no longer play a part in any clerivatiorr. the protluctiorr .t nxqmpli Consider G : ({A.lrlgN OFCOlrrnxrli'Run GrtRMH.A4 aaabBc4 aaabbc in G. We will see shortly how ulch unnece$$ary productions can be removed from a gralrfnar. I _ Productions Removing Useless Onn invariably wants to remove prodlctions from a glammar that can ne. B + ahbAlb.b.bltAclabbc. irr this ca*se. c} . we get the grammar d with productions A +alaaAl aba. this can never lead to a sentence. o. A.r d is equivalent to G.152 Fc.Io. The string aaabbchasthe derivation A+ aa.5 + A Clearly playS nrl l()ler a$ 4 CAnnqt be trarrsfortrtxl into a terminal string.luu.laRfi NOR\. The new gramma.R.ver take llart itt any derivatitln. it is necessary that A and B be different variables.
aCb.T. let us look at another example. A. A production is useless it involvesarrv useless if variable. BbA. The casejust mentioned is of this kind. A variablethat is rrot usefulis called useless.rlrnns lEB Let G : (V.P).bA.6.5.P) be a ctlntextfree granrmar. A aAlA. Before we present the general case and the corresponding theorem. In words. y in (V U7'). In a grammar with start symbol S arrd productions 5iA.'l Mr:'r'sor$ FoR TRANSFoRMTNG Gn.a variableis rrseful and only if it occurs if irt at least one derivation. I This example illustrates the two reason$why a variable is useless: either because it cannot be reached fiom the start symbol or because it canrrot derive a terminal string. with P consisting of C ..S. .a.aSlAlC. E iinx. [xornpl+ s'2 A variable may be u$e]essbecause there is no way of getting a terminal string from it. A variable A e V is saicl to be useful if and only if thenr is at least one ?rr€ tr (G) such that S# rAy 1w.rr.ble may be rneless is shown in the next grarnmar. * (V. b}. Another reason a varia.7.d$$frqirbisli* liminate uselesssvmbols atrd productions from G S . where v : {s. thert: is rro way we can achieve S # nBy. the variable B is uselessand so is the production B . Although B ciln derive a terminal string. (62) witlr r.cI and ? : {4. A procedure for removing useless variables and productions is based on reco6pizing these two situations. A+at B + aa. B.
5 . . P ) b e a context*free grammar.A} .this argument cannot be made S So cloes. terminalsT : {a}. For this. \ / variatrlesor productions. Dependency graphs are a way of visualizing complex relationships arrd are found in many applications. B}. with an edge between vertices C and D if and ontv if there is a productiotr form C * rDy. becarrse + A for c..9. The formalization of this process Ieads to a general construction and the corresponding theorem. For contextfree grammars' a dependency graph hasits vertices labeled with variables. Fonnrs Figure 6. the variables A and B belong to this set. I L e t G : ( V .S'A. are led to the grammar Gr with variablesyl : {. In our case.i.F)tnutdo*..aSlA. However. A+0.fr. Figgre 6.s. F) with fr : {S. A +a. andproductions S + aSlA. 7 . Removing it and the affected productions and terminals..1 shows that B is useless. Baa. A dependerrcy graph for Vr is shown in Figure 6. s. we can draw a dependency graph for the variables. *# a. Then there exists an not contain any useless equivalent gra*rrrur I : (t. we are led to the fi. we identiff the set of variablesthat carrlead to a terminal r*ring. A variable is useful only if there is a path from the vertex labeled s to the vertex labeled with that variable.Chopter 6 SlvtplIllcRtloN oF CourpxrFREE GRAMMARSAND NoRMAr.1. i : {o}. Next we want to eliminate the variables that cannot be reached from the start variable. thus iderrtifying it as useless.nal answer e : (t . Removing c and its corresponding we productions.1 @ First. A Becarrse + a and B r oG. and productions S .
Thc remaining issue is wlrether every.4 $ rr : ah.2).l ..4 f'or which.5.t with all irr in yr U ?.Lo€T* is possible.2 will thr+n be added t. . we get the lina. 2.1 Merlrops non TRarqsFonMrNc GBenruens lEF Proof: The grammar I can be gr. all variables at level a * :l will be rr. wc draw the va. It is eq'ally crear that if A e vr.evelr 2 .riablcsA for whir*r A4. of twtr parts. In the fir'st paxt we rxrnstlLrct an intt:r.terwhile therc a. tht:rr A 4 ru e 7* is a possitrle cleriva. Hence A will cventually bt: adcledto [.4 . 1. The algorithnr cilnnot terminir.T2.l rltlrivation trcc correspondirrg to that dcrivation (Figure 6.reva.riable dept:rrdencygraph ftrr Gl and from it find all rtnz' .r'ly this prornclure termi'*tes.'rr.ncratecl from G by an algoritirur consisting.ls. The third Lirne throrrgh step 2.l answer t fro* G1.epeatthe ftrllowing step rrrrtil no rnore variables are arlclecl v.I1 (yl u ").tionv/ith Gr. to lbr every A e V for which l. prorluction of thc forrn A adrl A to I/1. and so on.T.1 will bel aclded to l/1 on the first pa"ss through step 2 of the algorithtr.6. in ck*r.so cvery variablc Aa at level A .rldecl.Level. cxlrrsiderany such . Set V1 t.. Ttr see this.k* 1l as all thr: productions irr P whose syrrrbols are rr. v1 on the sex. Figure 6.s the in trcc that are nrt yet in I. 3.4 ancl look at the partia.tt fr.2 \.P1) such that V1 contirirrsonly va. R. tht:re are only terrnina.riable.ondpass through step 2. Irr thn seconclpart of the constructiorr. is acrcled v1 before the to procedure tclrrninates. The steps irr the algoritirm irre. has a. Ta.&1 Levcl.'.{ ) .mediate graqrrrrar G : 1 (V.riirlile at level a . At level A. Any va.
t is sornetirrrcsunclesiratrleis otre in which the right sicleis the ernpty string.tiotr A+A is uossible is crallerinullable.r56 nnn Chopter6 SrN{1'1.riilr"rr i1t.cr. G does not contain any trseless procluctions.ic. gralrllrlirr of the folrn Arry producliorr of a.\prorluctious cillr .fi .Jf7 o u r s f iF fll'ili!. frorn G by tlrt: rtlrnoval of prcitlucltious.ble for which the deriver. yut have sornt: the .ched va.f.riable Set. Wtl t:an also elirnina.te tr.\production. .rrr. The rcsult is ttleglarlrrllirru:\v ' A t*.irringtr..s. * : i { . (ri3) A. Irr suclt cascls. Putting the two rt:srtlts sr tha.1 t .' proclrrc:tions'we all $iuce the c:orrstructionof d retains A a.\llrocJucliorisor: nullir.rrcl arc ttqrtivaleut' I 1.()o.tr. / ^ syilrtlols or Becaust:6f the colstnrr:tion.i?.oesrrot cicclnrin sotrrc rrseful prothrction.seq*cntly 1. graurrrrilr tnay gt:ritlrate a lauguir'ge not clonta.roN OFCoNttlx'trli'ttEECIRAMMARS Non.ted Irave everyttring ueeded ttl rnake the tlt:rivation S t6 nAy 11u ru.lprc.nr"1f.t c:aunot be rtler. Auy variir. d ir t:orrstructecl g p.(e) e LG). [rt: renroved. { t f" Removing A'Productions One kincl of llroc]uction tha. Also.l assoclia...ny ternrirurl that d.riables tha.rethe proth. These artl removed frorn the va.cotrtextfi'ee AA A is calledir. for each tu E L (fJ) wtr have a clerivation SltAylu.lral Fottr. we see that G a..ltlt:vtlriables..rctions involving them.F\. it$ a. r ? + $ i. The grirrnma.t F ^ \ / G togethr:r.t$ frotrt 5'.
Irr rnore grlrreralsituations. 51 + a51blA. 2.\protluctions can ht: rnade in a. we look at all productions in P of the fbrrrr ul+aDrfft". similar. T}re . > i}. wc put into F tn. Onc:c the stlt V7lr hirs been founcl. 2 wherc each nt.n be removed after addirrg new prtiductions obtairred by srrbstitutittg A for 51 wherc it occurs on thc right. 'Io .4142.\ not in L ((.l otre.1.rerepltr.i are botlt nullable.\. .tes Afreelanguage {a. and onc iu which both ri and 11 a.'oR TRAN$FoRMTNC. . usirrg the fbllowing strllls.mple.Az.ble variables of G.. substitutions for .$rblab' We can ea"silyshow that this n()w gramrrrar genr:rates the samr: language:as the clrigina.t:ed with ..\..'.An art:in !fu. ca.( V U ?.n.if z1 and r:.6. I Let G be arrv r:otttextfiee gr:r..\production the 51 + ).nb" : n. tirere will he oue prorluction in F with fir r()placecl v/ith . marrner. 'fhen there cxists an eqrlivalcnt gramrnar I having uo . prrt A into H1y.\.1 Ml:'rHops l. Fbr all productions A +. Repeat l. ro. A.rnmar with . a.41. where .nt procluction ir.. For t:ach srrr:hprocluction of P.. This grammir. althougtr rnore r:ornplicil. put B irtto l/r.rrrl1.cing nullable variables witlr all Fcrl exa. A in all pousible corrrbina.ted. Doing this we get the gr:Irrimar $'5r aS'rblab..rgenera. For all nroductions fi _.s th<lsegenerated lry repla..tions. .).s 167 S n aStb. 1.he following step until no firrther vrrriables ir. Proof: We lirst find tire set U1s of all mrlla.readrlt:d to V. we are ready to construct F. one irr which ri is rcplaced with A. Gnelrrverr.\productions.swell ar.frm.
i iiil.4..ble. D + tl.Cl A . The argumerrt that this grauunnr d is cqrrivalerrt to G is straiglrtftrrwarcl atrtl will bo left to the rearlor.la.qltoN r)r' CoNrnx'r'lrnEE Tht:re is onc exception: if all r.\. fl. D*d.the prodrrctiotr A + . . B .\productionsequiverleut the gratrruilr defitrtldby S .ABaC.tion. I grarrrma.158 AND GRAMMAR.wc get AaClAB alaC lAalE o. S .BC.2. where.ABaClEn..vs Chopter6 Slrrrurlc.BlClBC.r to Firrcla t:rintextf'ree without . we lirrrl that the rurllablevariablesare A.initien'0.rernrlla. E.3.i$fi'f.fii Arry production of a.blA. . folkrwing the secondstep of the construt.. contcxtfree grilmrrrirr of lhc fbrm A >B.\ is rrot Put into P. '1 . Therr. C. ('+ D. B €V is called a unitproduction. prrlrfilctious in which both sides irre a single variable art: at tiurcs uncltlsirahle. in Fhomthc first stcp of tlrc coustntctiorr Theorern6.sI NoRMAI Fon. I RemovingUnitProduciions 'fheorent As we see from 6. C +Dl. Bb.
Let G : (V.and we needonly considerA + B.f .To get arourrdthis. whereB + A1lyzl. we use the substitution rule discussed in Theorem6.A can be removed from the grammar without effect. Then there existsa contextfree grammar e: (t.F\ that doesnot / have any unitprocluctionsa"urlthat is equivalent\toG.B witlt Autlazl"'1a". Proofr Obviously.1 directly with rr : nz = .rRs 159 To removeunitproductions.7. ly.i. . Note that sinceB .4)... I .4 and B are different variablerl. U4. rroneof the yacan be a single variable. so that no unitproductions are createdby the last step. for all A and B satisfying (6. The new grammur d ir generated by first putting into F ail nonunit productions of P. it may seemthat we carr use Theorem6.At first sight. P) be any contextfoee grammarwithout .lgl ' . where . the utritproductionsare not rentoved.\productions.!r. But this will not alwayswork.4) holds whenever there is a walk between .\ to replace A.1.5. all variablesB such that t4 n.S. in the specialcase A. As the construction the next theoremshows. for each. then (6.4. we add to P A . (6 4) We can do this by dtawing a dependency graph with an edge (C. we first Iind.any unitproductiorrof the form A . Next.this carrbe in done if we proceedwith somecare.U* is the set of all rulesin F with B on the left.1 Msrnons noR TRehrsFoRMrNGGRerr.4 anci B. is takerrfrom F.rr\l. D) whenever the gtammar has a unitprodtrction C + D. To showthat the resulting grarnmaris equivalentto the original one we can follow the sameline of reasoningas in Theorenr6.B.y t l a t l" ' 1 a " .1.
Hence. The dependencygraph lbr the utritproductionr is given in Figure 6.B and the a*ssociated productions useless I We carr put all tltese results together to show that gramma"rsfor contextfree language$ can be made free of useless productions. B * albc' grammar to obtain. Aprocluctions.4.9 + .theequivalent 5 .\productiorr$. Then there exists a contextfree grarnmal that generates .L and that does not have any useless productions.S + E. the new rules S . and unitproductions.160 Chopter 6 SturlrnceuoN oF ConrexrFnnE GRAMMARSAND NoRMAL Fonns Figure 6. fi+bb. Note that the removalof the unitproductions has made .3.L be a contextfree language that does not coutain A. we seefrom it that . or unitproductions. + B albblbc. A + albc.3 o @[] .a lbclbblAa. :J G x o m p l6 .we add to the origJinal nonunit prodrrctiotrs ff+Aa. 6 i Remrtveall unitproductions from S * AtrlB. B _ Albb.4 4 B.alhclbh. n +bb. . Let . . A + alhclB. and . A + albblbc. B + A.
4 removethese kinds of productiorts trrrn.using both the original and the modified grammar.+G: tlr. Theorem6. The only point that needsconsideration in is that the removal of one tyJre of production may introduce productions of another type.1 by showing that . A + aaa. 6. Removeunitproductions productions 3. are equivalent.the procedurefor removing .3.rtlra. In Example 6. I  1" Cornplete the proof of Theorem 6. Also. for example. ffi .aNsFoRMING given in Theorems Proof: The proceclures 6.4 requiresthat the grammar ha. Show that the two Erammars S + abABlba. RemoveAproductions 2. and the theorem is uroved.we can renroveall undesirableproductions using the following sequence steps: of 1.Gn.ve Aproductions.l B . show a derivation tree for the string ababbbac.2.But note that the removalof unitproductions no (Exercise at the end of this section).ns 161 6.aAlbh and S + ahAaAlabAbblba.1.q.1. Mnrsons roR TR.. Removeu$eless The result will therr have none of these productions.and doesrrot createtrproduc:tions 15 productions does not create Aproductionsor urritthc removal of useless productions(Exercise16 at thr: end of this sectiorr). A + o.5 :+d rlj implies D . . Therefore. and 6.an.\productions can createnew unitproductions.
4bA.rtoNoF CoNroxrFREE GRAMMARS NoRMALFoRMS AND 4. B + b6. 7.\. Completethe proof of Theorem6.3 to removeA*productionsfrom the grammar in Example 5.ddd. C . Eliminate all . B*Aa. BAAWhat Ianguagedoes this grammar generate? productions from 6.rnce. In Theorem 6. CB. What languagedoes this grammar generate? ffi L Eliminate all unit productions from the grammar in Exercise7. 11.3.4. A * aBlA. and all Aproductions S .L62 Chopter6 Slupr. from the grammar all uselessproductions.1. A*I.\productions from SAaBlaaB.aSlAB.aAlaBB.cCD. Use the construction in Theorem 6. Eliminate useless g + alaAlBlC. D . why is it necessaryto assumethat A and B are difierent variables? productions from the grammax 5.\. What languagedoes the resulting grammar generate'l . Eliminate all useless S . A + aaAl. L Remove all unitproductions. 10.41. B + bBlbbC. 12. Completethe proof of Theorem6.4.
What can you say about the removal of . Show that if we apply the construction in Theorem 6. Let G be a grammar without Aproductions.2 for the removal of useless 'l:pfoductions.6.1 Mer'sons FoR TRANSFoRMTNG GnertlreRs 163 1 3 .T.\productions irrtroduces previously norrexistent rrrritproductions. 2O. we obtain a new grammar d such r h a r z ( S ): r ( G ) . Show that the construction of Theorem 6. If not. Oonsider the procedure suggested irr Theorerrr 6.P) has a production of where tr. Justify the claim made in the proof of Theorem 6. but possibly with somc unitproductions. then the removal of useless prodrrctions by the construction of Theorem 6.4 does not then introduce any.\productions and no urritprodurtions.{ . 19.. first eliminating va"riables that cannot be reached frorrt S. therr renroving those that do not yield a terminal string. ffi 1 7 . B+x. (V. r8. Reverse the order of the two parts. 1 4 . wc might the tcnn sirnplificatiorr precisely bv introducing the 'Ihis of a grarlimar. Suppose that a contextfree gramrnar G: the form A+fry. Does the new procedure still work correctly? If so. where "B { V.e. then the resulting grammar is equivalent to the original one.ve it.S. U . ffi 1 5 . It is possible to define concept of complexity of them is through the For example. Prove that if this rule is replaccd by A+ By. use (G) compt. one length of all the strings giving the productiorr rules. prt.\productions and unitproductions? . l/ € (y U ")t. Give an example of a situation in which the removal of .3.2 does not introduce any such productions.nity : A+uEP E {1 + lul} . give a counterexample.L (G). Show that if a grammar has no . 1 6 . Suppose that G is a contextfree grammar for which I € . can be done in rrrany ways.1 that the variable B can be replaced as soon as it appears.\productions. Show that the rernoval of useless productions always reduces the complexity in this sense.
. t . Show that the grarnmar ohtained try rcplating these productions with A * 'ltrlZth... A contcxtfree gramrrrar G is said to be rrrinimal for a giveu languagc . into two disjoint subsets A+ and AnllA*rl'.lZnt. 1 u ..164 GRenuaRs nNo Noti[4A. . z: Z + xil'tiZ. . *24.P) be a contextfree grammar.r 23.2..\ B . 1.s. placing all productit.. : 1 . Use the result of the preceding A + Au.r t .tlp: ?1. i :1..t11 plc that the rernoval of useless produt:tions does not necessarily protluce a minimal gramnrar. to exercise rewrite the grarrrma.!t..1.rr'r'ruxrFR.n1.riablc (say.r7i are in (Ill?').T. Prove the following resrrlt. rhentr(c):r(d).F). ? . fr '* rr. .2.L if (G) < torn7tlerZtlv (d) for arry d gcnetating f..laEcl. 'i : 7.!!rlazl ..'lA*". Consider the where Z f V and F is obtained try rc(u J{Z}. where fii. Let G : (y. Lct the set of productiorrs involving the variahle.4 on the lcft lre divided into two disjoint subsets A+ntAlr. wher'e .4 is not a suIfix of any :qi. . . Prove the following courrterpartof Exercise22..tAl .. s t l a z.. A *. ffi *ZZ..n't.. .T.2.".rrs that have A on the left by A + yil11iZ. Show by examr:omplexi. /).l*"A and A + .L FoRMs Chopter 6 SrllplrrrcATroN oF C<. .n' is etlrivalent to the original grarnmar... gramrnar d: but A is not a prefix of any gl.l. Divide the set ef productions whose left sides arc soilre given va...Bblbc so that it no longerhas prtxluctionsof the forur A + A:r or IJ * Br.S..
B. ltr particular. br:c:ause their widt: usefhlrress. A is rrotl both proclucticxts5 Dt:{irtition 6.4.2 Two lr.l form if all production$ at'e of thc form A _. a.ptl a is in 7'.6.a violate the conrlitions of I AA5 attcl zt  . Ci are irr I.ASIAAS. g'.tponrenl Nonna. One instance of tltis is the Chomsky normal form.l Fonus 165 iffiffi Two lmportont Normol Forms Tht:re are nrany kinrls of norrnal forrns we cir.$AIb iu iti Chornsky norrnal forrn. uR' funllfi .rrestahlish fot coutextfi'ee grirrrrrrlars.r() can ask that thc: string rln the right of a productiotr consisLof no trtore tirirtr two symbols. wc s.BC.rrlook ftlr is one in whit:ir Lhe mrrnber of strictly liurited. havtl beetr studof itxl extentively.ftftilrltiuw A contcxtfree grarrlrrlirr is in Chornskv rtorrna.'. S Ala'tt' u. fhe gralrllnilr fi .. OI A _+ ill whcre l.Sone of these. Wc considr:r'lwo of thettr bricfly. A . Chomsky NormqlForm One kirxl of nrlrrrral forrn we cir.ymholson thc riglrt of a prodrrc:tiotla.45'la.
5) we put into Pt the production A .166 Chopfer 6 Stlrpllrtc. For each production of P in the form (6.replacingthem with newly introduced variables. .P) with ^ # L(G) has an equivaAny contextfree lent grammare : (?. (6 I'r) where each 14 is a symbol either in V or T. Step1: Construct a grammar G1 : (V1 T.+C t C z ' ' ' C * . .rroN oF CourFxtFnEE GRAMMARSAND NoRMAL FoRMS grammar G : (V.Pl) from G by consideringall ..1 that (6 7) l'(Gr) : L(G). Proof: Because Theorem 6.' is in V. productionsin P in the form A+ntrz"'frn.syconsequenceof Theorem 6. introduce new variables Bo for each a € ?. we can assume of without loss of generality tlrat G has no Aproductionsand no unitproductions. At the end of this step we have a grammer G1 all of whose productionshave the form Aa. For every Bo we also put into Pr the production Ba + a' This part of the algorithm removesall terminaJsfrom productions whose right sidehaslength grea. where Ct = rt if z.q. F) in Chomskynormal form.T. In this case. ancl Ca: Bo if ra : s. The construction of G will be done in two steps. If n > 2.Cn. where Cl € Vr.i. (6 ri) A .CrCz. If z : I then trr must be a.ter than one. s. S.5. It is an ea...S. put the production into P1. terminal since we have no unitproductions.
Dt * CzDz. Obviously.6. we introduce new variablesBo. Ba.. As requiredby the constructionof Theorem 6. put into P the productions and A > CtDr.. fi. r ffi Sxnttpl+.7)with n:2 into F.D2. (") : LG).fi Convertthe gramrnarwith productions S + ABa. we introduce additional variablesto reducethe First we put length of the right sidesof the productions where necessary. Bn*b. We will leave this to the reader.? Two lMponrerur'Nonrranr.BoBoBa. Fon.6) as well as all the productiorrsof the form (6. For n j 2. and usethe algorithm to get S . : Dnz + CntCn.. (. B + AB. A + aab. to Chomsky normal form.Ms 167 Step 2: In the secondstep.. the gralnmar doesnot have any trproduction$or any unitproductions. Repeatecl of applicatiorrs Theorem6. A . so that \ ./' .B.6.ABB.. B + Ac. In Step 1. all productions of the form (6.the resulting grammar d ir itr Chomskynormal form.. Bo '' a.d). Ba+c' . we introduce D1 new variables .1 will showthat f (Gr) : . This somewhat informal argument can easily be made more precise.
but on the positions in which tcrminals ir.Iittle complicated and not very tra. Bu. Dt . Bc+c' I Greibqch NormolForm Another usefrrl grammatical fotrn is the Greibach normal form.tnns NoRMALF onlrs In thc secondstep. Nevertheless. constructirrg a grarnmar irr Clreibach normal form ertlrivalerrt to a given contextfree grammar is tedious.q'rroN ColrnxrIrrtnn GR.r is comrtron to both Greiba.!. We therefore deal with this rnatter very briefly. Bt.)occur at most once. Greibach trormal forrn has marry theorctical arrd practica.ch normal forrn and sgrammar$i but Greibach normal form does not carry the restriction that the pair (4. Here we put restrictions not on the lcngth of the riglrt sides of a prodrrction.ffi tN A txrntextfree grarnmar is sa. D2 + RnB6. we irrtroduce additional variables to get tlrc first twrr producLiorrsinto normal forrn and we get the fina.nd variirhles carl appear. +b.l result SADr.id to be in Grcibach rrormal forrn if all productions havtl the forrrr ! + o. This additional fret:dom gives Greitra. . a.ii1qniy.+vrr.Iandr€V* If we compare this with D<:fi.4..l conseqrrences. A + BoD2. a Q. Arguments justifying Greibach normal ftrrm are a.ch norrnal forur a generality not by lrossessed sgrammars. Sirrrilarly.nition 5.nelf'l.ri.AND oF' Chopter6 Srrururlc. wherca€.fr.BRo.nspartrnt. . we $ee that the form A * o. p +ABt.
However. S+AB.FoRrrs 169 If a gramrnar is not in Greibach normal form. A + aAlbBlb. respectively. I In general. from . we may be able to rewrite it in this form with sorne of the techniques encrluntered above. Here are two simple examples. A+d. B 'b is not in Greibach rrormal form.6.4 and B that are essentially synonyms for a and b. which is in Greibach normal form. B+b.1. we immediately get the equivalent grammar S + aABlbBBlbB. A . into Greibach normal form. which is in Greibach normal form. tltough. We introduce new variables . rreither the conversion of a given grammar to Greibach normal form nor the proof that this can always be done are $imple matter$. Here we carl use a device similar to the one introduced in the construction of Chomsky normal form. Bb. We introduce Greibach normal form here because it will simplify the technical discussion of an important result in the next chapter. Substituting for the terminals with their a"ssociatedvariables leads to the equivalent grammar S +aBSBlaA.10 Convert the grammar SabSblaa.2 Two IturoRrnrqr NoRl"ler. I Exoilrple 6.aAlbBlb. using the substitution given by Theorenr 6. However.
Show that there exists a grammax6 : (V. s u c ht h a t . Show that there is an equivalent grammar in Chomsky normal form with no more than (k . I . A + bABl. S. Provide the details of the proof of Theorem 6.13). 1 . B . there existsan equivalent Bramrrar d in Greibachnormal form. Convert the gramma.Chopter SrupllnrcauoN ColrruxrFnpnGnq.ABIaB A . Transform the grammar with productions S * abAB. P) be any contextfreegrammar without any trproductions or unit*productions. w h e r eo E T . A abAlb into Chomsky normal 4. into Chomsky normal form.. Ttansform the gramma..r S form.6. ffi 6 . aSaAlA. . m .BAalAlA. see Example 3. Let Z be any linea^rlanguage not containing . For everycontextfreegrammar G with ^ # L (G).aabll BbbA into Chomsky normal form.P) all of whoseproductions have one of the forrns A+aB. Draw the dependencygraph for the grammar in Exercise4. .ns NoRMAl 6 FoRtvrs AND oF a conceptualviewpoirrt. Let G : (y. Let k be the maximum number of symbolson the right of any production in P.T.1) lPl * lTl production rules.= r ( G ) . B E V . Ae. A . ?.S. we only quote the following generalresult without proof. 3. Convert the grammar S + ttSblab into Chomsky normal form. . L A linear languageis one for which there exists a linear grammar (for a definition.tr. Greilrach normal form plays no further role in our so discussion.r S .uve.\. !+Ba. 5.
BC.B. . or * . 12.C €V.rNoRMAL Fon.P) S + a. ItAb W aSblbSola. A + bABC. there exists an ecluivalenf grammar irr t.where a e ? and tr € yU {.B. Corrvert tl:regraurnar S irrto Greibar:h rrorma.U? 15. *16. ShowthatfrrrevervcontextfrcegrarrrtrarG:(V.2 lI'Wo Ir..L(G). e + aBCt irrto twostarrdard forrn.'I'wo$tandard ffi witli P givena. Convert the following grarrrrilar into Greibach norrnal form. B b.C}.. Convert the 6grammar S + ABblo. S + rt.94.{n.b}.l + o. 13.an.rrdard form. Convert the grarnrnar g + ab loSlaa5 into Greibach norrnal form.B.S. AaB.lb ffi f<rmr is generall for any contextlrce grarrrmar G with A { ..6. wlrcre € X U {I}. Convertthe grammar G: ({S.1 tr. A.Sblab. Can every lincar grar[mar be convcrted to a fbrm irr which all productiorrs Iook like A .prs 171 9.rpoRrAru. A_+e1 wher'eA.rhave the forrn A + a. A contextfree grammar is said to be in twostandard forur if a.T.S.BC.wosta.CeUandae?. a 10. 14. A+ B into Greilrach norrrral form. Prove this.P)thereisaneqrrivalent one irr which all pr'64u"tr1on.ttern .A.ll production mles satisfy the following pa.l forrn.s uaAlB. 11.
llthe F1r. V z z . . Assume that we trave a grarnmar G : (V.'en. Therefortt.j .2 . attd srt ott. V2r.... h < r . P) in Chornsky llorlllill forrn and a string 'til:eraz. The algorithrn works only if the grarnrrrar is itr Chomsky rrormal forrn aucl sutr:eeds by breaking one pr()hlem into a $equell(xl of stnallt:r ones irl the following wir. Ttl r:ontiurrc.i I U An instrrrx:tion t]rc indices in (ti..W . Clearly. . ..tuaRS At'ro Non. a.itned. D.nclonly if tltere is a.Chopter 6 Stltrurlc. II. S. r r d 1 ' . .2. C t r m p u L e t : t .}. r.i C {A: ABC. withB 4r1* v.r.a'r'loN oF CON:rEXTFRljtl GRnr. .i.c€Vr+r. arxl T. 1 " a 1f o r s o m e f u w i t h i < h . . tr/r*1. 3 .. We are lr()w in a position to justify the this clrrirn. produt:tion A+ BC. . Vil call llt+ computed for all I < z < rz by inspection of u arxl the prtlrluctions tlf the grarrrmar. V 2 t . Cocke. The algorithm wt: will describe here is t. . prothtr. . . V .if we prot:ced in tirtl secluetrctl l .fter its originators J.t A €Viif To cornpute Vl. wt: cla. ancl only if G contains a.wit}rBEHtr. notice tlrnt for j > i. F ORl{s iiffiffiffitil A MembershipAlgorithm for ContextF r e eG r o m m q r s * In Ssrtion 5.r}.y.. u E lt (G) if and orrly if S e Vr". Ctimpute Vrz. A derives ruii if a. r ...tr.T'..lvlel. Kasarni.without any elaboration.S) LE {r. 2. We definesubstrings wij:(ri"'ajt and sulnetsoI U vi : {e Ev :A5 u. I n t r t h e r w o r d s . C t r m p u t eV r . . .lled CYK algorithm. .ij: a .. Vlrnger..rbserve drrction A .rt l..ai.8) shows that it catr bc used to t:ompute o[ rr. (6. that mcmbership and parsing erlqorithttm f'or cotrtextfiee grarnmars exist that retpire approximately ltol'' steps to parse a string trr. V n .
(rz 1.xrFnnnClHe.Next.. R} .R } .0. {A} .rn.sky lolrng.wc irlso havt: yz:: {. { 5 .tns L7B Ultqftn$f$ld. Eir. ntitic:c Lhat exa. (_.V r r : { 5 . Thc CYI( algorithru. that i*.! +B(. R + ABlb. tinrlwc haveH23: {S'. V 4 a= ={ E } .irarsirtg rnetlxltl. V:s : {S.' t/rr} . V12 is crrtllt.:r sl.i. .riablesthat irurric:rliI ately rk:rive a. Y r r : { B } Now we use (6. r:arr l_re it qrnvcrted iulo ir.: . e strtlre reclrtircrl righl sirleis AB. similarly. l / z r : { . l'lr: {. 4 } ..I/rr : {5.ridcof a prorluction whosu righl sicle is A. terms irr (ti. Iir see thrr.l/+o: {A} ..heeva. alc r1r:r.r. RI . Sirice ur22: e. A _ BBla.4.t rriost ll.dditiorrsto keep trac:k of how tlrc elertrentsof l/1.rrrtl. With sorlc langrra.dctt:rrrlinesrncrnbershipfor ir. the sct c:onsists a. B E Vtt. { 5 . 8 } .inrtxl rt:srrlLlirilows. Since V11= {A} ancl V22: {A}.lrrntiorr a. V t t : { A } . First rxltrl Lhi'rl'ur1 .4} at:ci.ll variables t}iiit occnr of ott Lhe lefl .y.3 A MnlrnnnsHlr Ar.chinvtilvr:s 1. B € Vn. V r .gt: generatecJ lw 5AR. A straiglitftrrward argrrrnt:ltalongtlu:srlIinesthrlu gives Vn: : Yr3 Vr+: V ' n: s ot h a t u t e L ( G ) . a grarnrnal iu flhonr.8}. V4 : {4.riy gr:leraterl lr.r Fon {loFl'r. I fr. have to lltl cotrtl:rtted.l. of so tlrc cla.t the CYK rnemlrershill algoritlrrrr lecluiles ()n.ctly rr. E r : { .J frirrn.'Il'.ivecl. Dcttlrmine whetlrrrr the string In: tlul Hratnutar (rabbbis irr the langua.Vt+: {l} .4}.gc a. a } .e1rs.colr. B l . as clrrst:r'illccl her:e.1) /2 sets of V.. O t V22} . Since tlrt:re ale nrltre. } _tB(i.r.8) to get I4z : {A : . : { B } .$)..iso V11 is thc scl of all vir.
re in the language generated by the gtarnmar irr Examplc 6'11' of Examplc 6.774 Chopter 6 Srrvtt. Usc the approach crnployed in llxcrcise 2 to show how the CYK mcmbership algorithm can be trrade itrto a parsing metlxrrl. * * 4 ..Ilri GRAMMARSAND NoRMAI Fonlrs 1 Use tlre CYK algorithm to dcterrrrirte whethcr ihe strings nabb. using tlre gratnrrrar 3 . ffi 2 . IIse the rcsult in Exercise ll to write a cotnputer prograln firr Jrarsing with any contcxtfree gra.mfllat in Chomsky norural form. Usc the CYK algorithm to find a parsing of the strirrg aab.and obtibb a.r. .rprcetloNol CorvrrxrFn. aabba.11.
Pu hdown s Automoto he description of cnntextfree languages by means of contextfree gramrrrars is convenicrrt. we have seern. The next question is whether there is a class of automata that can be associated with contextfree langua. when scanning a string from the language 7 : {a"b. Sinc:en is unboundrld.whereas the recognitiorr of a contextfree language may reqrrire storing an unbounded amount of information. we understand that this is because finite autotrara have strictly finite rrrernories. allowing unbounded storage that is restricted tci operating likr a stack. We want a machinc that can count without limit. This suggests that we might try a stack as a storage mechanism. we need more tharr unlimited cxrurrting ability: wc rreed the ahility to store and match a sequence of symbols in reverse ortler. we rnust also corrrrt the nrrmber of a's.finite automata cannot recognize all contextfiee As languages. But as we see from other examples.st not only check that all a's precede the first b.s. This gives us a (lass of machines called pushdown automata (pda)' 175 . su()has {wwR}. this counting cannot be done with a finite memory. Intuitively.ge.. as illustrated by the use of BNF in programming language dcfinition. we mu. For example. I n } 0}.
we explore the connection between pushdown autornata and contextfree languages. Since this is an important family for the treatment of programming language$. forming a proper subset of the contextfree languages. Each move of the control unit is determitred by the current input symbol as well as by the symbol currently on top of the stack. we conclude ttre chapter with a brief introduction to the grammars associated with deterntittistit: contextfree languages.1 Input file In this chapter.Chopter 7 PusHooww Aurorr. We first show that if we allow pushdown arttomata to act nondeterministically. The result of the. Each move of the control unit reads a symbol from the input file.move is a new state of the control unit and a change in the top of the stack.1. while at the same time changing the contents of the stack through the usual stack operations. But we will also see that here there is no longer an equivalence between the deterministic and nondeter* ministic versions.tre Figure 7. Aufomoton Definitionof q Pushdown Formalizing this intuitive notion gives us a precise definition of a pushdown automaton. the deterministic contextfree languages. In our discussion.seccepters. . M A N o n d e l e r m i n i s tP u s h d o w n u t o m o t o ic A schematic representation of a pu$hdown automaton is given in Figure 7. The class of deterministic pushdowrr automata defines a new family of languages.r. we get a class of automata that accepts exactly the family of contextfree languages. we will restrict ourselves to pushdown automata acting a.
nsitirirr. the requirernetrt that the rarrge of d be a finitrl subset is ner:essarybeca. irrdicating that a move that does not collsurne an inprrt symbol is possible.iirqnri. t . the input svurbol read is a.finite suhscts of Q x f* is thc transilion functirin. Finallv. I . sta.k.. Suppose the set of trir. Q x (E U {A}) x l' . . I' is a flnite set of syrnbols ca. In our rrltation we &ssurnethat the instrrtion of a string into a.MrNrsrIcPrJsurrowruAuroltnrA 177 . d ) . where q is the next state tlf the control unit trnd r is a string whiclt is pnt on tcill of the stack irr place of the singlc syttrbol there before. irnd the symbol on top of the stack is b. tiris choice must be restricted to a firrite set of possibilitics. 1 . no m()v()is possible if thrr stack is emptv. b ) : { ( .irRR.1 NowlnrER.7.use I x f* is atr infinite set ir.stack is done symbol by symbol.llerlthe stack alphabet. The irrgutnentsof d a.rrdtherefore ha"r irrfirrite subsets.fil. stir.t:ksymbol. q 3 .and the currrlrrt symbol on toJ) of the stac. The re:sult is a set of ptlirs (q. We will call such ir rrlove a . Qo€ Q is the initirrl state of the corrtrol unit. \ } . then one of two thirrgs can ha.a .\. t r . . The ulrnplicated formnl appearance of the drlrrrairr and range of d rnerits a closer o'xamitration..tesof the control unit. 1 r . X is the input alphrrbet. ____________'Emunrpl+ f .l. F e Q is the sct of linal states. While an npda may have several choit:cs for its move$. or (2) the u)ntlol unit goes irito state qs with the symbol b rcuroved frotn the top of tht' stack.rting at the riglrt end of the string..rrsitionrules of arr npda crrntains d ( .rethc current state of the control unit. q o 1 z 1 F) whcre Q is a finite set of internal sta. c If at any time the control unit is in statc q1.lll lr A nondeterministic pushdown accepter (npda) is defineld the septry trrple M : ( Q .\tra. 8 . tht: current input syrnlrol. Note also thnt d is defined so that it needs a.$lr"q.z). Note that thc second argument of d may be . d .ppen: (1) the t:otrtrol unit gJoes irrto state {2 ancl t}re string cd replirces b on top of t}re stack. r €  is the stack start symbol.
b.0):{(q3. What can we say about the action of this automaton? First. claim. The crucial transitions are {I=.t } p : { a . 1 :) { ( q 2 . \ ) } . l r .1 ) : { ( q 2 .I)}. These two steps count the mrmber of a's and match that count against the number of b's. Of course.1) = {(q1. Q t . which removes a 1 when a b is encourrtered.$. 1 . Q z .. 1 ): { ( q 1 . we rnight say that the rrpda accepts the above language. . b . l ) : { ( q 2 ' . there is no entry given for d(qo.0). 1 .:0 ) ( q 3 ... which adds a 1 to the stack when an o. .} z:0.A. is read. ] d ( q t . . This assures that no b precedes the last a. t r . 1 1 ) . q. 6 ( q r . 6 ( q r . The control unit is in state (1 until the first b is encountered at which time it goes into state q2. I .. The interpretation of this is the same that we used for nondeterministic finite autoutata: an unspecified transition is to the null set arrd represents qdead configuratiqllfor the npda. After analyzing the remaining transitions. we see that the npda will end in the final state q3 if and only if the input string is itt the language 7. d ( q 0 . a .10).a. d(qz.{anb":n>0}u{"} As an analogy with finite automata. For instance.o){(q'.di : d (sor.178 Chopter 7 PusHnowN AuroMlt'rA ffi $xomple 7. I ) } . before making $uch a.\)}. and . \ ) }. r : { 0 .. r': {qr}. notice that transitions are not specified for all possible combinations of input and stack symbols.(q3.b } . and 6 ( q r . { /. we must define what we mean by an npda accepting a language.2 an Consider npda with Q : { q o .A ) } .'_ 6(h.11)}.
Here we need not even . i ( q o . On occasions rvhere several automata are under consideration we will use Far to emphasize that the move is made by the particular automaton M.7. thus (qt. .' is the unread part of the input string.F) be a nondeterministicpushdown automaton.2. Together these completely determine all the possible ways in which the npda can proceed. we introduce a convenient notation for de* scribing the successiveconfigurations of an npda during the processing of a string.. r ) i u @ . b ) . .t. 4 .U!) is possible if and only if i t ( q z . u ) . Let M: The language accepted by M is the set L(M): {. p e r .n/ld Jfn"J (5 {r. i ' Ht(dftfpls y. and u is the stack contents (with the leftmost symbol indicating the top of the stack) is called an instantaneoua description of a pushdown automaton. and the current contents of the stack.e F .80. i . A move from one instantaneous description to another will be denoted by ihe symbol F.S ii... .t .f.aw.a.y. i 1 .l i Construct an npda for the language L : {* e {a. } .tm$ilnlirtri ril. the language accepted by M is the set of all strings that can put M into a Iinal state at the end of the string. In words.r'ol u^rt L t' '+ j. The final stack content z is irrelevant to this definition of acceutance. a ) d ( q 1 . TheLonguoge Accepted o Pushdown by Automolon lllfim$. the unread part of the input string. . € Moves involving an arbitrary number of steps will be denoted Uy [.d. . the solution to this problem involves counting the number of a's and b's. which is easily done with a stack. The triplet .b!)  (qz. The relevant factors at any time are the current state of the control unit.b}* : no(w) : nu(ru)}.a . lc ) t. u e f . t i . w . As in Example 7.E.70/ JTdeF a where q is the state oJ_tlp_gqntrol uaiL tr.I NoNDETERMrNrsrrc PusHoowruAurouarn l7S To simplify the discussion. (8.
.I (qo. Brrt the lolcleterrlinistic rrature of the irutornatr)rr hclllls us with this. {0. a d ( q o .O ) : { ( q n .1]:f ''An apparent clifliculty with this suggcstiorr is th. the nprla. r) : {(q11. 0 0 ) }' b d ( q o . ftlr t:ourrting thel b's that a. For thc stx:tlnd pa. mtr. f ) : {(qn.I ) } .r \ we use the fact that the syrnbols are retrieved frorn a stack itr the reverse or<lerof their insertion.s a ( q s . {r1l}). .1 1 ) } .rc to trc rnirtchtxl irgairrst ats later. z ) : { ( q 1 1 . Tlte cotttplete solution is r1o. 0 ) : { ( r t 6 . 'W WMWWiN*t:tlrrstrtrtltilnrrpclilfilrtr. w h e r e . tr.Chopter 7 PusHnowu Auronere w6rry aout tlte order of the a's atrd b's.z}. Q : {. ir.b} . z)} . {&.re current input symbol with Lhe top of lhe stack.r.4z} y:{a.b.ge I. a b . In processing the string baotr.q.'\)} ' b d ( . a b ' a ) f l.r:k. 1. then pop olle counter symbol from the stack when a. ' I z )F ( q 0 . into lhe stack whenever an a is read. q o . . 2 .f . I ' ..i:i1. w]rttrc tl cnds ilrrd rirE sttlrts.tr. t) alrd hcrrt:t:thcl strirrg is ircx:epted. A s o l u t i o nt o t h e p r o b l e m i s g i v e u b y M : ( 8 . We can ll$e iI negiltivtl cotrntttr symbol.y 1.1(t. we r:ompi.rt. d (rJs. we push the rxlrrsgr:rrtivr:svrnbols rirr tiru sta. i ' ) .(qo.  : {a.lT*il:. F : {arl .( g o . The only difficulty with this is that if wtl there is a prefix of ur with more b's thiln r?. . #.b}.r. z) F (qr.nnpdrr. b is fbund.i.'it. a d ( q o .Qt.A'[: ({rlo. 0 4 )' } 1z)} .. with ri given as I . Wlren reacling the first part of the string. tirirt is. z) : {(qt. a d (qo. b a a b . . thtr npda correctly guesseswhere the rrriddle is artd switt:hes stirtes at that ptlirrt. b}} . tr. 6 . .0 to usr:. . . z} . z ) .\. 15. We ca1 iusert a counter symbol' say 0.t wtt rltt rrttt know tfie rniddle of the strirrg.ke$the move. cotrtinuitrg as lotrg as the ?iil: ix. 1 ) : { ( q 6 . : ( p \unun : ur € {ct. z. j n .cx:tlptirrgt}rtllirrrgrri'r.. 8 .:lT:'"'*ni[::r:]:. sa. will not find a. But this is easy to fix.0z) F (qg. r .b.
7.1 NolrnnrERMrNrsrrc PusHlowl Arlrouara
181
The transition firnction can be visualized as having several parts: a set to push zr orr the stir.r:k,
\ / ) I
d (so, a) : o, : 6 (qo,b,a) d (so, b) : o, (t(,10,1r,: b)
d (Qo,a, ): z d ( q o , b ,z ) :
J""sh
aa)} , {(so, b(z)} {((t(r, , ab)} , {(qs, {(qo,bb)},
{ ( q g ,a e ) } . { ( q o ,b r ) } ,
{ )
a set to guesstlre middle of the string, where the npcla switches frorn state ttl 9r1 qt '*d ( q o , ,a ) : { ( q t , a ) } , A d ( q o ,t r ,b ) : { ( q t , b ) } , a set to uatch zrrRagainst the contents of the stack, 6(h,a,0) : {(q1,I)}, : { d (,1r,1r,1,) (qt,I)} , and fina,lly d ( q t , A , , ) : { ( q z ,r ) } , to recognizea successfulmatch. The sequence of moves in a,ccepting abba is (qo,abba,z) t (qo,Itaa, az)  (Qo, baz) ba, F ( q 1 , b a , b u z ) l( t 1 1 , o . , n F )( g r , A , . z ) ( q 2 , . e ) . 2 F The nondeterministic alternative fbr loca,ting the miclclle of the string is taken at the third move. At that stage, the pcla har the instir,ntaneo's descriptions (qo,ba,baa) a'rl ha,stwo choices fbr its next rnt)ve. c)rre is to uue d(gs,b,b) = {(qo,bb) ancl make the move i (q6,bu,rt, baz)  (qo,a, bbaz), the secondis the one used above, namely d(q0,,\,b) : {(qr,b)}. Only the latter leads to acceptanceof the irrput.
I
Aurol\lArA Chopter 7 PusHnowtr
1 . Find a pda with fcwer than four states that accepts the same language as the pda in Exarnple 7.2.
D,
Prove that the Jrda in Exarnple ?.4 rloes not accept any string not in {tutrtF}'
ffi
fD C.,nrt.lrct nptla's that atxrept the followirig regular languages' (a) Lr : L(aaa.b) (b) IIr : L(aa,h"aba) ({ the urrion of trr an<l Lz
(d) Lt  Lz
on languages X : {4, b,c}' fD Construct npda's that acceptthe following (a)L={**bt*,rz>0} W (b) .L : {tnctu'r : ru e {n,,b}* } : (") f.L {.a^h*c'+*: n, } 0, > 0} (.1f : {o"^h+*c*:n}Q,m}1} ) (*) t: r: { a 3 b ' o c ' oz > 0 } W
(f)r:{a*b*:nlnz/3n}
( (S)r : {u : no (*) : rru t,) + t} ( I t ) . L : { u t : n o ( u t ): t r , 1  1 y
(i) {* '. n,, (to) + nr, (u) : Tr.. (tu)}
(j) .L : {'u:2no (trr) { n;, 1tr,)< 3n" (ta)} (k) .L = fw : no (tr,) < na (tr')] )<rrrstrtrt:tarr nlrda that ac:t:eptstfue language 7 : {anb* : n' } Q,r1 f v,ltt. Firrrl arr nprla on 5 : {n,,b, c} t}rat accepts t}re language L : i t t r tl ' t r t € { r r ,b } * , r u , I * { } z {zntcutz '
?, Filtt arr rrpda for'the cotrcatetratiotrof tr(a*) and the language irr Exercise 6. @ fitta an trpda lbr the language 7 = la'b(ab)" b (ba)" : n > 0].
9. Is it possiblc to lincl a dfa that acccpts the sarne language as the pda M : (Iqo, qr]1 {a,,1,}, {t} , qo,{qr }) , ,
71 NoNnerERMrNIsTrc PusHpownr AurouArR
1BB
with
d ( q o ,a , z ) = { ( q r , e ) } , d (qo,tr,e) : {(So,e)} , 6(qr,tt,z): {(q',r)}, d ( q r , b , " ) = { ( q o ,z ) } ? f f i 1O. What language is accepted by the pda M : UrIr, qt i qil qli 4+, qsj , {a, b} , {0, r, a} , qo,{qsl) ,
with , d ( q o b ,z ) : { ( q 1 ,1 z ) } , : 6 ( q t , b , 1 ) { ( q ' ,r r ) } , d ( q r ,o , 1 ) : { ( s . , } ) } , d ( q . ra , 1 ) : { ( q n , ) } , , f 6 (q+, z) : {(g+,z) , (gs, u, z)l'l
/+\
1.,
is try S1,/Wtrat language accepted the npda M = ({qo,$,gzl ,{a,b},{o,b,a}, 6,Qo, {qz}) with transitions z,
6 ( q o , a ,z ) : { ( S r , n ) , ( g 2 , . \ ) } , d ( q r , b ,a ) : { ( q t , b ) } , d(qr,b,b): {(q,,0)}, 6(qr,a,b): {(Sr,A)} ffi 12. What language is accepted by the npda in Example 7.8 if we use F : {qo,q'}? 13. What language is accepted by the npda in Exercise 11 above if we use F : {qrt,$,qz)? 14. Find an npdalvith no n:rorethan two internal states that accepts tlre language
tr(aa*ba*).ffi
15. suppose that in Example 7.2 we replace the given vaJue of d (g:1, A,0) with
,5 tr,0): {(qn, . (sz, A)}
_ (rel What is the languageacceptedby this new pda? we can define a restricted npda as one that can increa,$e length of the the stack by at most one symbol in each move, changing Definition ?.1 so that
d : Q x (Xu {I}) x  + ?Qx(rrufu{r}). The irrterpretation of t,his is that the range of d consists of sets of pairs of the ftrrnr (q6,ab) , (qt, a,), or (Q,,A). Show that for every npda M there exists such
a restrictetlnpda fr suchthat L (M) : L (fr)
ffi
184
AuroMArA 7 Chopter PusunowN
dD 
is O" alternative to Definition ?.2 for languageacceptance to require the stack the end of the input string is reached. Formally, an npda to be empty when
M is said to accept the language N (M) by empty stack if
N ( M ) : { t u . E . , ( q o*,, r ) i * 1 p ; , r ) } ,
L
J
where p is any element in Q, Show that this notion is effectively equivalent to Definition ?,?, in the sense that for any npda M there exists an npda fr such that L(M) = t (fr), atrd vice versa'
M
A P u s h d o w n u t o m o t oo n d C o n t e x t F r e eL o n g u o g e s
In the examples of the previous section, we saw that pushdown automata exist for $ome of the familiar contextfree languages. This is no accident. There iS a general relation between contextfree languages and nondeterministic pushdown accepters that is established in the next two major results. We will show that for every contextfree language there is an npda that accepts it, and conversely, that the language accepted by any npda is contextfree.
1 "1d "* 7{ttilrtl+jh
")t)F
[onguoges Automolqfor Conlext'Free Pushdown
We first show that for every contextfree language there is an npda that accepts it. The underlying idea is to corEtruct arr npda that can, in some way, cerry out a leftmost derivation of any string in the language' To simplifii the argument a little, we assume that the language is generated by a grammar in Greibach normal form. The pda we are about to construct will represent the derivation by keeping the va.riables in the right part of the sentential form on its stack, while the left part, consisting entirely of terminals, is identical with the input read. We begin by putting the start symbol on the stack. Aft'er that, to simulate the application of a production A + afrt we must have the variable A orr top of the stack and the terminal o as the input symbol. The variable on the stack is rerrroved and replaced by the variable string r. What d should be to arhieve this is easy to see. Before we present the general argument, let us look at a simple example.
.P*gnrnlS.,fir,5
Construct a pda that accepts the languagegeneratedby grammar with productions 5  oSbblo,.
7.2 PusHoowrr Aurordaraeuu CoNrnxrFRnrLANGuAcEs IBE
We first transform the frammar into Greibach normal fbrm, changingthe productions to 5'+ a1Ala, n + bB, B +b. The corresponding automatonwill havethreestates{go,qr, gz}, with initial state gs and final state q2. First, the start symbol ,9 is put on the stack by d (qu, ") : {(qr,S")} . A, The production 5 + aSA will be si*rlate4 tr lbq]p4gby_fg*oViqg g
to* th* tto,"k o.tt.l "o['lo."it g it o.ith
Similarly, the rule ,5  a should cause the pda to read an a w.h:ilesimply removing S. Thus, the two prodrrctions are represented in the pda by d ( q t , a , S ) : { ( s r , 5 , 4 ) , ( q 1 ,A ) } . In an analogous manner, the other productions give
6 ( q r , b , A:) { ( s r , ) } , B
6 (h,b,B) : {(qt,f)} . The appearance of the stack start symbol on top of the stack signals the completion of the derivation and the pda is put into its final state by d ( g r , A , z ) : { ( q 2 ,I ) } . The construttion of this example can be adapted to other cases,leading to a general result. I
For any contextfree language Z, there exists an npda II such that L: L(M).
Proof: If tr is a Afree contextfree language, there exists a contextfree grammar in Greibach normal forrn for it. Let G : (V,T,S,P) be such a grarnmar. We then construct an npda which simulates leftmost derivations in this grarnmar. As suggested, the simulation will be done so that the unprocessed part of the sentential fornr is in the stack, while the terminal prefi.x of any sentential form matches the corresponding prefix of the input string.
Chopter 7 PussoowN Au'roMA'I'A
Spar:ificallg the npda, will be M : ({qo,{r,{l},T,V l{z},d,rlo, z,{gl}), whetrez I V. Note that the input alphabet oI M is identical with the set of terrninals of G and that the stack alphabet <ffitains the set of variables of the grammar. The transition functiotr will include A d ( q o , , a ) : { ( q 1 , 5 a ) },
( 7l )
so that after the first move of M, the stack contains the start symbol ,9 of the derivation. (The stack start symbol a is a marker to allow us to detect the end of the derivation,) In addition, the set of transition rules is such that
, ( r n , u )€ 6 ( q 1 , aA ) ,
wllenever
(7.2)
A
au
is in P. This reads itrput a and removesthe variable A from the stack, tlte transitiorrsthat allow the replacingit with u. In this way it generates pda to simulate all derivations. Finally, we trave
d ( q r ,A , z ) : { q t , z ) } ,
( 73 )
to get M into a final state. To show that M accepts any ru € L(G), consider the partial leftmost derivation
'**.::::
.";i:o';rf
A*
If M is to simulate this derivation, then a"fterreadirtg ar&z" '4r,, the stack must contain AtAz' ' ' A^. To take the next step in the derivation, G rnust have a production A1 + ttBl , .' Bp. But the construction is such that then M has a transition rrile in which ( q r , B r . ' . E r ) € d ( q r ,b , A r ) , so that the stack now contains Br ' ' ' BnAz'" An after having r€fld a1a2' ' 'anbA simple induction a,rgument on the number of steps in the derivatiorr then shows that if
E+u,
7.2 Pusnnown Auroua.rA ANDCor{rnxrFnEE LANCUAoE$
fgz
then ( c 1 1 , u t , s a ) ( q 1 , . \z ) . t , Using (7.1) and (7.3) we have ( q p , ' r r . , , F ) g r ,w , s z ) i ( n r , ^ , a ) F ( q 1 ,. \ , . e ), .z( so that L(C) e L(M). To prove that .t (IUI) e LG),let u € L(M).Thern by delinition
( q o , r uz ) f k t y , A , u 1 . , Brrt thcrc is orrly orle wfly to get fiom qg to tlt and orily one way from q1 to {1. I'herefore, we must }rirvt: ( q r , * , 5 e ) i ( q 1 , ,z ) . A Now let us write ut = eje,zil;i. " a* Then the first step irr ( q 7 , a L a z a 1 " . . . a n , S z1 q 1 , , 1 , 2 ; )i must be a, rule of thc forrn (7.2) to get .' ( q y ,q a 2 a 3 . , . e n , S a ) F ( q t , a , z f l : r . a , , 7 . 4 2 ). r, Rr.rtthen thc grarnrnar has a rule of the fllrrn S + aturt so tha,t 5 + a 1 t i ,.1 Repeating this, writirrg 'LtL: Auz, we have a ( e 1 , u , 2 t't,;,1 , r , A u 2 z )  ( q 1 u , 1 . . e , , , , , u 3 I t 2,l ) , .1 irnplying that, A + u2rn is in the grailrnrar rrncl tha,t, S # a,1u2'u3'u2. This mir.kr:sit quite clear a,t any poirrt the slack contents (cxcludirrg z) are idt:nticirl with the unnra,tcheNl liirrt of the sentential forn, so that (7.4) implics 5 # a142.',ar,. In conseqrren<:c, L(M) g , (G), completirrg the proof if the larrgrra,gcr cloes not txrntairr .\. If A e L, we add to the constmcted npda the transitiorr d ( q o ,) , z ) : { k t , z ) } so thnt the empty string is a,lsoac:r:cpted. r (2.4)
g " ) } and d ( q . B ): { ( .Az) F ( g 1 . o . t r ) } .a ) .(qt.a. Mg).tr)}. A ) \ . q. In addition to rules d ( q o .".d) : {(q1A)}.s ) : { ( q r . b cB C z ) . A Tltis rxlrresponds to the derivrrtion S + aA+ aaABC + aaaBC + aaabC 4 ua'ahc. F ( q 1 .  ( q 1 . Bb.l form. Since the grammar is already irr Greibach normal form. d(qr.ssumed that the grirmmar was iu Greibach rrorma.g)}' d ( q t . a b cA B C z ) . t r . . z) S z)  (q1. F (qr . we can use the constrrrr:tion in the previous theorerll irnrneditltely. z ) } ' the pda will also have tratrsition rules d ( s r .bc. It is not necessary to do this.a ) : { ( q r . : d(qr. 1 1 .A) : {(q'. . C z ) F (91''\' z) F ( q 7 . 6 (qr.bc.6 Consider the grammar SaA.b. A+ aABClbBla. Cc. . .t r . t Irr order to simplify the argurnents. aao.rfu. we carr rrrake a similar and only slightly more complicated constru<:tion . the proof in Theorerr 7.Chopter 7 Pussoowt'r AuroN4ArA Jfi rr.c .aabc. The sequence of moves mrrde by M in processing aaabc is (go.afl."il Exonple 7. b .S ) : { ( s ) .A) {(qt.1 ir.
tr) .ving properties I rr. Takirrg tltis as given. whelrc : r:a (qi. All transitions rmrst have the form 6(qt. BC) . eaq:h trove either iucrease$ decrcascs the stack content by n sirrgle or symtxrl. and of . For example.A): {c1.cr}.2 PusrrDowm Aurolrnrn nulr Clt)rurr:xtFn.1 is also true. we must first rnatch t.he ab in the inprrt agrrinst a similar string in the sta. Frrr procluctions of the form A + abCr.7. for productions of the f'orm A +Bx.t is. Tltese restrictions rniry appeer to tre very sev()rc)but they are not.(76) Tha. We leirvc thc details of the constnx:tiorr arrd tlrc associatedproof as an exerr:isr:.te rly that is entereclif and only if the stack is cmpty.EE LaNcuacns 189 from a genelral t:ontext*free gramma. we will assurnethat the npda in question meets the following recluirements: 1..l ftrrrrr. To keep the discussiorras sirnple as possible. Ilere we need to cxplore it further. The rrlrrstruction involved readily suggests itsulf: reverse the proce$$ in Theorctr 7... we want the senturrtial forur to represent thc content of the sttrck. This equivalence wiL. This rrreans that the contq:nt of the stack should be reflected in thc variable part of the sententirr.r:k anrl then replace A with Cz.c?.r. (7 lit) r:i : (qi.1 so that the grammar sirnulates the moves of the prJa. As statecl.rexpkrrtxl partiallv in fixercises 16 and 17 in Srxrtiorr7.1frrlrl the stack ancl replace it with Br.nd2. It has a single fina. But the configura.o. . ConiextFree Grqmmors Pushdown for Aufomolo The converseof Thtlclretn 7.rgurirents as an exercisel(st:c Exelcise 1tj at the elnd of this section). Quite a few details are needed to mir.1. 2. It can be shown thrrt for any npda there elxii*s an equivalent one ha. but coruume no input symbol. while the processed input is the trrrrnirrirl prefix of the sentential ftrrrn.. we remove :. we now rxrrrstruct a contextfree grarnrrrar ftrr tlx: lirrrguage accepted by tlxl rrllda..gairrwe will Jeavetht: a.kt:this work.titirr the npda also involves arr irrternal state..l sta. hrrt a.
then frorrr q. and thc rxrn$trrrctirtn we give here is a little tricky.5) involves arr immediate erasure of A.ted by the gralrrlrlar will be identica'l to the Iangua. Productionu of type (7. This is due to thc fir.Il o' elrd going frorrr state 91 to qr.1) u. Srrbsequently. but this docs not aff'ect the gratnmar.ti) generate thc set of rules ( (qiAqn)+ (q1Bq1) qCqn). Tlxr resulting variables tr.. 'oErelsing" successivestritrgs bv which it is replaced) are retnoverdfrom the stack.mmar. Suppose for the motnent that wc ctr. and if we choose (qorqt) a"rits start symbol.'. tlrtr grtrmmar will have a cotresporrtling production (t1iA(t.4.lrres Q. therr (qozq. Brrt this is exactly how the rrpdtr. we go frorn t1i t'tt q1. To construct a grarrllrrirr thnt satisfies these conditiorrs. w€lexamine the cliffererrt types of transitions that c*n trc ma.nd going here mealls that A atttl its cffttts (i.ll the trringing the symbol originally bckrw .z (creating an empty stat'Jr) while reading rr and going from (lo to qf.tling u a.e it with BC. ttr gA.ke on all possiblc va.ge accepted by thc npda.ge grarrrrrrar. 1 A if and only if the nptla orases from the stack whilc rtltr.e. This is true. where qy is thc single final state of the nuda.ct that to erase A wc {irst replac. 3 if and only if the npda nlmoves .Chopter 7 PusnrrowN AUToMATA this has to be remembered in tht: sontential form as well. It is hard to see how this can be done. Fina. to the top. In the last step. Since (7. gra.lly.i) + (t'.cceptedby the h. as start variable wt: tirke (qozqt).clehy the npda. If we calr firrtl srrt:h a. a.a. it may seem that we havc arlrltrd too much' as there may be solrle stat()it ql that cannot be reaclttxl lrom qi while erasing B.nfind a grammar whose variablt:s irrrr of the form (qiAqi) and whose prothtt:tions are such that (qiAq1) u. erasing C. a in whcrr: 96 nnd g1ta. erasing B. fiom state qi to state 4i. w}rile reading a. .ccepts'tu' Therefore.i Bqi are uselessvariirbkrs ilnd do not affect Lhe larrgtra. the lattguagt: genera.
b . From the first two transitiorrs we get the set of productions (sgaqo)a (qsAqg)(spaSo) (qo.7.A ) } . a .4gr)* lr. A ) } .tzqo) (s0Aq0)(so"qo)l(qnAqt) (qrzqo)l(qoAqz)(qrrsu) I (qoAq. 5 (qo. To satisfy the latter.but not 2. i (g1ag2) .\.r) lo (qnAqr)(sszsc). (q. . d ( q r .") : {(sz. 4 ): { ( q u . ilhe new set of traffi d (qo. .tt) I (soAqr) (qtzqt) . qt) (Sr".r) (qtrqz) . Thc last three transitions are of the form (7. + a (qsAqs)(s. we irrtroducea. 1 ): { ( q r .toArtr)(qraqr)l a(qoAsz) (qrrs.I ) } .a.A e ) } .raqt) (SoAqt)(Sraqr)l (sozqr) l" (r (qoAs.2 PussoowN AureMArA aruo Cortrux'r'FRpeLANcLTAcEs 191 Consider the lpda with transitions d (qo. qo) k. qo). (qszqz) + (s0Aq0)(qorqr) l(qoAqt) (qnzsz)l(qrArrr) (. (gu. a ) = { ( q 2 . . k . Az)} . 4 ) } . z ) : { ( q s . ))} . (qo"qz) a {qs Aq1) (qozqz)lo (qrAr1 (qr rqz)l r) a (qoAqz)(qzzqz)lo (qoAS.Zo)l lo il (qoAsz) (qr. a) : { (q6.4qr))(qo"qr) (So"qr) lo(.t) krzrlrtt)lo (qoAqr) (qtzqt) . (s0Aso)(so"qr) l(qoAsr) (qraqr)l (qoAqz)(qr". . A:) { ( S t . A .tr. then replaceit in thc . b . (ss"sr) .])(qtrqr) . 6 ( q o . (qn Aq=) (qt.) (q:aqo). z)} . z) : {(qs.4) : {(ql. + a (q0. (qzzqz) ' (qoAqo)(qorqt) l(qnAst) (sfiqz)l(srArt ) ktzzqs) (qoAqs)(qrrq*) . .. a 6 ( q .5) so that they yield the correspotrding productions (goAqs)a. o . d (gr. * A 6 ( .nrqr)  (soAs.r**t *ou*. A . d ( q r .new state 43 and an intermediate step in which we first rcmove the A from thc stack.A)} Usirrgqo 4qinitial stato aryl ga qqth* fi the npda satisfies txrndition 1 above. .
1. it can be applied to any pda whose transition rules satisfu the given conditions.ab. The first gi in the leftmost variable of every sentential form is the current state of the: pda. We use the suggested construction to get the grammar G : (V.7) and vice versa. x ) implies that (qiAq1) 3 u.A e f .z) F (q2"\' '\) ' The corresponding derivation with G is (qorqr) =+ a (q6Aq3)(qtrsz) + aa(62q2) + aa(qsAq1)(qzq21 + &eb(qrzqz) * aab. with successiveconfigurations (qq.d. A Xi) ( t i . while the sequenceof middle symbols is the same as the stack content. wheneverthe npda is such that the symbol A and its effectscan be removedfrom the stack while readirrgz and . The first part is to show that.2. (7. h .P). I Iffi language' u h: r(:r.S.{gl}) satisfies conditions 1 and 2 above. then.T.z)  (qo.Chopter 7 PusHpown Autouara The start variable will be (qorqz).S. Q i . u .b.z)  ( q o . This forms the basis for the proof of the general result.A z ) F (g1.The string aab is accepted by the pda. with ? : E and V consisting of elements of the form (acq.A.X.Lis a contextfree Proof: Asslrme that M : (8.f. Although the construction yields a rather complicated grammar.i). We will show that the grammar so obtained is such that for all e t . r r . X € l * . ? € E * . to The steps in the proof of the following theorem will be ea^sier understand if you notice the corresponderrce between the successiveinstantaneous descriptions of the pda and the sentential forms in the derivation. Az) F (q3.r)lforsoqenpdaM. u . Q ( q o . E .aah.
8) or (7.Similarly. We see from this that the sentential forms derived from (qAqi) define a sequenceof possible configurations of the npda by which (7.e) 6(qr. then the variable (StASi) can derive z. at least one of the variables on the right will be useless.1r. Prove that the pda in Example 7. I 1 . .10).A): {(si. Notice that (qiAq1) + a(qiBq1)(qfiqn) might be possible for some (qiBq) (qfiqn) for which there is no corresponding transition of the form (7.o.7. 1A . in that case. reading a.7) can be achieved. then there must be a corresponding transition ( 7. Show that the pda constructed in Example 7.5 accepts the language 7 : {an+7b2^ : rz Z 0}.)) whereby the A can be popped off the stack.2 PusHoowu Autorrrara . 1 ConsequentlyL(M) : I(G). with the control unit going from state qt\o qj. For the converse. Bd put on. Using the corresponding transition for the npda (78) we see that the A ca. We only need an induction on the number of moves to make this precise. consider a $ingle step in the derivation such as (rlrAqx)+ a(q1Bq1)(q. the argument given holds. r .n be removed from the stack.5 accepts the string aaabbbb that is in the language generated by the given grammar. .A)} (7. If we now apply the conclusion to ( q o .ul ConrnxrF'Rpp L. a ) i ( q .crrcuacps f03 going from state qa to qi. A ) . if (qiAqi) + a.Cq*) . This is not hard to see since the grammar was explicitly constructed to do this. But.q. For all sentential forms Ieading to a terminal string. we $ee that this can be so if and onlv if (qszqy) w.
ffi 5.b}. Construct an npda corresponding to the Brammar S + aABBlaAA. AA)} . In Example 7.9Slab. S . B + bBBlA.1 to find an npda for the language Example 7.5.L (M): l. b. For every npda M.2 imply the following. P ) . d ( s o . L 7 . d (so. e) = {(ge.{a. (E L: tr"O an npda with two statesfor the language {a'ob*+r: n. l r } .Chopter 7 PusHoown AuroNrar. there existsan npda frwith at urost three states.Bqn.suchthat. 1 8 . A) = {(qo.1. W Construct an rrpda that accepts the language generated by the grammar ^9 aS. Show that the npda in Example 7.7.z}. Show how the number of states of ff in the aboveexercisecan be reducedto two. 7 . Use the construction in Theorem 7.d. 6 .o. Construct art rrpda that will accept the languagegeneratedby the gramrnar s G : ( { S . with transitions 6 (qo.1 and 7. {A.7 accepts I (aab). t o .{0. ffi 8 . Section 7. 1 6 . \ \ / ff/ nina a contextfree grammar that generates the language accepted by the npda M : ({qo. {qr}). )C"tt*ttrrct an npda that accepts the language generated by the gramma^t S + asbblaab. A ) } . Give a construction by which an arbitra"ry contextfree grammax can be used in the proof of Theorem 7.2. Show that the grammar in Example ?. ffi 12. Show that for every npda there exists an equivalent one satisfying conditions L and ? in the preamble to Theorem 7. > 0}' ffi L0. A ) = { ( q ' . Find an npda with two states that arcepts 7: {a*bz* I n I 1}.1. (if) . 11.and (q6agr) are useless' 14. Give full details of the proof of Theorem 7.o .c. 13. / e. A . A u aBBla. A } . Aa)} . Showthat Theorems7. w i t h p r o d u c t i o nS + A A l a . show that the variables (+o.Qr}. .7 generates the language L (oab).2.s.S A I b .
the orrly criterion for determirrism is that at all times at rnost one possible rnove exists. a.il.(MJ.1rathertharyb E. lffim. b) conttrins at rnost one eleinent. no inJlutconsuming alternative is available.1.ffif.ig be. .fl \1 # It is interestirrg to note the differt:rrce between this definitiorr and the corresponding definition of a detcrrrrinistic finite automaton" The do4ra. $ornc: transitions of a. This does not affect the definition. Ar'lrr DptpRL{rNISTrc Corurnx:rFRHH L. l.3 Dprnn. at most one mov{} carr be made. undefined. =l.Nc. for every g E Q. The lirst of these conditions simplv requires that for any givcn input symbol and any stack top.chieved a modification rr.caPrtc"we want to {gtain )transitions. subject to the restrictions ttrat.I.F r eL o n g u o g e s r e A deterministic pushdown accepter (dpda) is a pushdown automatorr that neverha^r choicein its move. bv of Definition 7. d (g.L is said to be a deterministic contextfree only if there exists a dpda "lb"sn'ch"tlrat t.il i A puslrdown automaton M : (Q.11 Dcfinition 74 language if and A language . gf_!h!_Eg+q4g4 funqlio" iu still asin Definition7. 1.clptla may be to the empty set. tust be empty for evcry c € E.7.1. AIso. qo)zt F) is said to be deterministic if it is anautomaton as defineldirr Definition 7. The second condition is that when a Amove is possible for some configuration. Since the top of the stack playsE role in determining the ncxt move.q.vINISTrc Pusunowrq Aurclnrnra. the presenCbof . This can be ir. so there may be dead rxrrrfigurations. that is.e E X U {I} and b € f.uacns Pushdown Automqtoond ffiffi Deterministic D e t e r m i n i s t iC o n t e x t .\transitions does not automatically imply nondeterminism. d. # .finrnqffi"!.
1 i) . that ilrc not deterministic.{qs}) b . lr}.stlrtl givcn lir.lent dpda.ta..40. and d (qo. o .l 't : 10} 6 kto.) : { (s' .3. l ( q ' .tr.r.d.ngrta.b. Thc npder.b.\. .196 Chopter 7 PusIItowt't Aurtltrlare Exomple 7.1 obvirlrs moclification of tlrc argrrment that L1is a r:ontextfreelatrguirgrr shows that . does rxrt irnplv tha't the language {trlurR} itsolf is nondetertrtinistir:. \ . } . A. This..rrtlthe next exalrrplc wo see tha.since there is tlte possi}rility of irrr cqrriva.l f l f r{ . :) { ( q r . o) : {(q.\ \ . of course. satisfics the conditions of llelirritiorr 7.zl0) and :n.ge.4 anrl is therefore deterrninistic. nol ecluivak:nt. The language L = I'tl) Lz .Qz}. d (116. Brrt it is knowu that tlxr lir.{o.u 1 (J"\ r''.u.r. : ({qo.t . r ) } .>0].4. It accept.Qt. 6(qr.o) {(sr.L2 is also contextfrtx:. ) : { ( q 1t.. is a deteruritristic with {0. a)} violate cotrditiorr 2 of Definition 7..I'he ptlir.t1. 9 Lt:t L1=la"b". There are contextf'reela.nguages E x o m p l e7 . a) = {(qe.n"l 1" i . I there is ttot dett:rrninistic because Look now elt lixample 7./}. in cout. d (qz.I)}..ro.l(4r.L + ' l ' t . Fl'otn tiris a.t. .8 The language 7:{a"b":n>0} M t:orrtcxtfiee language. o. : .1) {(qr./o.. aa)} a. .ngrtage is indeed rxrt detrlrrninistic.rastttr deterministir: irrrrlnondeterrninistit:llrulidowtr automata art: finite irutorrra.i ) } .
grarrlnlar G : (Yr U VzU {5} . Q nI Then consider f r : with (d.q.:l DETr. sint:c the pdn" has either to match tlrre b or two aga'inst each a.P2) be contextfreegrarn that I and rrrarssuchthat Lr : L(G) and 1. Lct G1 : (Vr.c.ontcxtfree language.52. 40.ybe enterecl rrfter M has reacl atb"'. .rrinitial choice whetlter the irrput is in L1 }.tthis point.ut. 9 1 1 5 2 } .S1.7.q.T.. the llrocess thir.frce. F) with Q : t q o 'Q t . There is rro informirtion availa. If . w}rtrre P:h U P z U { . Accepting this.'"bZ" Figure 7.z. brrt the details of generatesLl)L2. rrnd so has to rnake ir.la nnp DprnnulNls'l'I(i ColtrnxrFRnn 7. ' .} F':Pu{fi'eeEFlt.. X.. But .tlPda M for L.T.F) . Therrl is always tlte possibility of a completely clifferent a. This will follow from a ge.P1) attd G2 : (Vz. d.E.rgumentwill be clefirrrecl 1. behincl the constnrction is to add to the control rtttit of M a The icfura part irr whiclt tratrsitions c:ausedbv the iuput symbol b are replacxxl sirrrilar with similrrr ones fttr input c.8. .z. This new part of the control utrit mar. . rr formal argurnent ftrllows. 5.eginning of the strirrg or in .. we sec that' the a. .ncasily be "d* plultffiHtTHis point.2: L(Gil' If we assume the V2 are disioirrt and tha.lRMrNrsTrc LAI'rGu. hrrt cloesnot prove anythirrg.then L: t ' l ) { a " h " c n : r z> 0 } wouldle a c. we show the la'tter try constructitrg a.2 describesthe construction graphically.pproach that avoids nrr itritial crhoice' 'lb but it turns orrt that therc is not. sint:tl the second part rtlsponds to c'iu the stlrrre wav as the flrst part cloesto b"'. now also accepts (trrbrrcn.rrd..ncral is contextfreetu'J !+g{:t}H be prcin sentecl the ncxt chaptcr. Of courstl. . this sort of argumelt is basecl9rr a partit:ular algorithrn we havtl in mind. Let M : (8. . cxrrrrlrinitrg two.t recognizes Q. 8 : a u {ao.gtl' This seerns rea$onable.tlus to the r:tlrrect conitlcture. but ca. frornd by irrcluding attdfr constructed F ( 0 r A . f .duF..t S # U U V2. is context.L wcre a dett:rrninistic rxlrrtextfreelangua'ge..n npda M fbr tr. ar. . shorrld be fairly clear a. given a.Lz.. s:) { ( f l ys )} .'Ihis until Chapter 8.bleat the t_ry which the choice.PuSHDowN Aurorvt.9. it rnay letr.L is not a derterministic contextfrce langua.P).L is indeed nondetertninistic' see tlis we first establisli the folkrwing claim.r.ns 197 well. ir. b* marle deterministically. then. .
r) (fr.Chopter 7 PusHnowruAtJ'rouara Figure 7.lsobe true that ( e o . I . ( q j .o that i at is contextfree.tr. im zr. .But we will show in the next chapter (Uxirnpte 8. BecauseM is cleterministic.It re. " b r n .u ) . b n r l f * . .s € 1.t l r ) ..1) that i is not contextfree. a so that for it to accept unbz" we nrust further have ( q r .! C) Addition Control unit of. it must a. But then. h " .mains be shownthat no strings ottrer to than those irr . this is considerecl severalexercises in the end of this secbion. .s: {(qi. l'or M to accept anb" wE must have ( q o a n b n r ) i * ( g oA . . for some qj E F. i: . Therefore. by r:onstruction (ti. . . I It :r{ q f r+'' sl for all ql e 4 s_ Il. et EQ. and € : 8(ir.ru € l*. ) t * ( q i . .ccept a.44 i.b. with q4 € F.c".The conclusionis that L : t (fr).nb"cn. r ) . .u)} . u)} for all ) 6 (qo. so that M wille.r.our assumptionthat L is a deterministic ctlntextfreelanguagemust be false. A .L are acceptedby M.s) {(fr.
Lr is deterministic contextfree and lz is regular. but that the language in the example is nevertheless determirristic.rg.L : {u e {a.(u) f n6 (ur)) is a deterrninistic contextfree 1 2 . : n > 1} U {a} in Exarnple 7. b}. Show that fif i. 7 : {a*b D. the closely related language 7 : {wwR : w E {a.l stto* that if . E***ple 1 4 .2 deterministic? ffi Show that the pushdown automaton in Example 7. {a*bz": rz > 0} is a deterministic contextfree 2} is deterrninistic.b}.B DnrnnvrNrsTlc Pussoowu AurouRrn . Is the languag.. L: in=rno. Give a. I s t h e l a n g u a g e7 : {anh* in:n7 G. ffi "\ (ro.Lr flLs is a deterministic contextfreelanguage.*:k) {a.. * the language{ucwT : w E {a.9 does not accept a"bznch with fr > 0.b}. Show that 7 : {6nlt't" : rn I n* 3 .B statement plausible' fiJ. . 7. .NoDnTTRMINISTICCoNr:sxtFnEE LANGUAGES 199 I . Srro*that .L* is a deterministic contextfree language.9 does not accept anb ck fot k I n. Show that [4itt E*u*ple 1 3 . or n: m}2} deterministic? 8 ...} is known to be nondeterministic.b.*g" irz/ 'J ri) V show that under the conditionsof Exercise16. Show that 7: language.Z.rguments that rnake thi.9 does not accept any string not in . Give reasons why one might conjecture that the following language is not deterministic. show that every regular language is a deterministic contextfree language. Is the languag.3 is not deterministic.. ffi n.} deterministic? ffi 1O. Ianguage. For the Ianguage L irr Exercise 1. ffi LtJ Lz is deterministic Iu. show that.then the * contextfree. .. 6 . Show alstr that it does not accept a"b*ca unless 'trl: n or m:2n. Show that fr} in E***ple 7. cir" an example of a deterministic contextfree language whose reverse is not deterministic. 7 : {a"bn : n } 1} U {b} deterministic? 4 .cr.q.L (a"bc)' 7. while the language in Exercise I is deterrninistic.
$a. we can achieve our goal. Herc we enter a topic important irr the study of contpilers. we can easily write a computer progralrr frrr it. so thcre is no choice to be made.ke the sgrammars introdtrtnd in Definition 5.4. while developing a senterrtiir. . We scan the input ru from lefl to right. sfl. Sittce there may be . . I " *+* a 1   oo. But in this case there is only one surh rule. To get the next symbol of thc sentential forrn matched against the next symhol in u.tment.3.lfbrm whose terminal prefix matches the prefix of ur up to thc currently scanne. Suppose that to : rul?{rzand tlnt we have developed the sentential form tulAu. From the discussion thcre. We can see this intuitively by viewing the pushdown autorrraton as a parsing device. we use the approach illustrated in Figure 7. ta. For the sake of discussion. attempting to find the leftmost derivation of a particrrla. Since there is no backtracking involved. For a general r:ontextfree grammar.Chopter PusrruowN 7 Aurou. the string ru does not belong to the language. To purrrue this. let us see what grarnmars might be srritable for the description of deterministic contextfrec languages. We will provide only a brief introduction to some irrrportant results.ct that they catt be parsed efficiently. If there is such a rule.\transitions involved. we would likc to know exactly which produr:tion rule is to be applied at each step.4 * ay in the grammar. this is not the case. Supposr:we are parsing topdown.3 tt1 d2 d3 d1 t t . The question then is wltether there are grilmmars that allow us to do this. r.r sentence. ref'erring the reader to books ort r:ompilers for a more thorouglt trea. Figure 7. Sentential form Matched part Yet to be matched .ue ContextFree ffiffi Grommorsfor Deterministic Longuoges* The importarr(:c of deterministit: <:ontextfree larrgrrageslies in thc fir. Input ro 4 .rutsus on lhe right track nevertheless. we cannot immediatcly claim that this will yield a lineartime parser. the parsing can proceed. we simply look at the leftmost symbol of ur2. . To proceed in matehing consecutive uymboftl. . As first casc. If there is no rule . but sorrrtlwhat peripheral to our interests. This would avoid backtracking and give us an efficient parser. but it J. it is clear that at every stage in the parsing we know exactly which protlrrction has to be applied. and we rrray expect thtrt it will work t:fliciently.dsynrbol. but if the form of the grammar is restricted.
1 The grammar S .4. the first . The grarnma. we mu$t apply the production S .rst ca*se.9laSbl ab generates the positive closure of the language in Example 7. but it is an LL gralnmar.10 The grammar S + oSblab is not an sgrarnma.ris an L. we have available two possible productions 5 r . by looking at a limited part of the input (consisting of the scannod symbol plus a finite number of symbols following it).1. and the second a b.L(ft) grammar if we can uniquely identifu the r:orrect production. Suppose we n()w use a lookahead and consider the first two symbols.4 Gnelru.LL gramrrlax we still have the property that we carr. In a similar fashion.S. must start with S . The scanned symbol does rrot tell us which is the right one. If the first is an a.t stands for the fact that the input is scarrned from left to right. The grammar is thereforc not an LL(z) grammar. we can see that .ab.FoR DorenMlr. To start.r.Rs 201 Although sgrammar$ are useful.q.rucuacns 7.:ad"of the next ft .bbab.aSb. Does this allow us to make the right decision? The &nrJweris still no.n LL grammar. The above is an oxarnple of an LL (2) grarrrmar. The term LL is standard usage irr books on c:ompilers. predict exactly which production rule must be used. Exomplo 7. thcy are too restrictive to capture all aspects of the syntax of programming languages. Otherwise.r is rrot an LL (k) grammar for any h.In the fi. One type of grammar is called a. we Iook at two consecutivesymbols of the input string. but the concept is rrrore general.q. this is the language of properly nested parerrthesis structures. since what we have seen could be a prefix of a number of we strings. I We say that a grirmlrra. As remarked in Example 5. given the currently scanned symbol and a "lookahe. including both aabb or ao. Iook at the derivation of strings of length greater than two. . In an . Every sgrarnmar is an LL grammar.I syrnbols. finding tlnt they arc aa.9.9 and S + aSb.10. while in the secorrd it is necessary to use 5 * 55. To see why this is so. the second tr indicates that leftmost derivations are constructed. In order to determine which production is to be applied. the rule S +aSb must be used. Exornpla 7.u$uc CorrrextFnnn L. Wc tieed to generalize the idea so that it becomes ntore powerfirl without losirrg its essential property for parsing.
when the syrnbol is b or if we are at the end of the string. The final result 5'6 SaSbS aSb.7. To see this. If for every pair of leftmost derivations Sl Sl 1 wtArt 41il1!!1n1 w1'*2.'21 lur3l is less than fr. then h is replaced by the smaller of these.we must u$e S . We corrclude our discrrssion with such a definition.\ is an L. When the input uymbol examined is r:r. Rewriting the gramrrrar avoids this dilliculty. the equality of the ft leftmost symbols of ur2 and tl3 € or implies lJt * Uzt then G is said to be an LL(h) grammar. We can find an . We see that we never have any choice.Chopter 7 Pusnnowr*lAuroN.aSbS.Lgrarrtrnar nearly equivtrlent to the original grammar.?t)z.) . (If lu. Then + S + aSbS + ab.SlA f While this itrfcrrmal description of trtr grammars is adequate for understanding simple exarnples. wtAxz 1trt1y2r2 l wrtts. witlr zr1. S.9 + q. we need a more precise definition if any rigorous results are to be develoned.hq. This observation about the grammar does not imply that thc language is not deterministic or that no . IliMl Let G : (V.'u)B 7*.L^Lgrammar equivaletrt to the original grammar. yet the gralnmar requires an immediate decision. P) be a contextfree grammar. The difliculty lies in the fact that we c&nnot predict how many repetitions of the trasic pattern a"b' there are until we get to tltc end of the string. is then arr . We fix this by introducirrg a new start variable 50 and a production to en$urethat some nonempty string is generated. there are always some situations that cannot be resolved.bS abab. consider the leftmost derivation of ru : aba6.baSbS+ a.L. But the problern is not yet completely nolved because the new grammar can generate the empty strirrg. we must use S + A.L grammar for the language if we alalyze the reason for the failure of the original grammar.IalA no matter how many lookahead symbols we have.LL grammar for it exists. The grammar g a$b^91.
2 .I'EXTFREE LANcuAcES 203 The definitiorr rnakes precise what has already becn irrdicated.assumirrg 5: (a) Z: (b).L. trut can bc viewed as constructing the derivn"tiorr tree fiorn the bottom up. 1 . tletermines whether or not G is an Z.tion is uniquely determined (as expressed bY gr : l/z).11 is an Z/: grailrnr&r and that it is equivalent to the original gramrrra"r.4 GRAMMARSl. ffi 3 .. Ctlrrsequently. The topir: of .c}.7.rrr. therc is itrterest in other. } 0} {a..rlv irnportant a. Sirow that the gramrnar frrr L : lw : n* (r) : rrr. But LLgranrmars are not srrflicietrtly genertr.L (abbb").l of material orr this subject that can bt: found in books orr compilers (e..ldrltcrrninistic grammirrs. {a. and rnany rrrrrrpilers have het:n written using . Show ttrat the strcond gramffrar in Exa. { a t t l 1 r r t .. .L (k) grarnuar. ffi i). (u)l . 4 . Construct an LLgraurnar for the languagc L (a*hu.Lpilrsers. If at nrry sta'gein thr: leftrnost derivatiorr (w1Ar) we know the next h symbols of the input.h.mple 7.L. then I (G) is a dcterministic contcxtfree languagc." {a'"tn+2rn':rr. Give.L gramnrar f<rr the language in Exercise 2. 9 .rnrnars. Ibr any given ft. 6 . Show that arry LL gramrrrar is unambiguorrs.13 is not arr tr.)tt.(u) < n6 (ur)} (e) f.g. Show that a dctcrnrinistic contcxtfree larrguageis never irrherently amhiguous. ffi L Let fJ be a corrtextfrcc granrrrar in Grcibach rrorrual form. A nrrrntrcr of progranuning larrguagescan bu defined by LL griilrrfilars. Hunter 1981) or trooks specifically devott:d to parsing mt:thods for fbrmal larrguages (such ns Aho and Ullman 1972). rrroregenera.LL gramma. There is a great dur. Show that if G is an /.b.rether socalled LR grir." (w) + nu (ut) t' n.[ (A) Srammar. Describc an algorithm whi<.L gramrnar.(trr)) given in Examplc 1.L gramrnar$ for the lbllowing larrguages. : 70. m .}fl. li'irrd ari 1. : {tu : n.0 } W } tzlt*7'n+"n: n. = {u : n.nt t)rre it the study of courpilers. wlrich also allow efficierntparsing.'oR Dnrnnlrrtrrsuc CON.. the next step in the cleriva.l deal with &ll dertt:rrninisticcontcxtfree lanto guages.rs is an importa.trz>1} (") fI : (d) Z. r t t " r : n Z 0 . Particula.
.
On the one hand. there are broader language families of which contextfrce larrguages are a special case. On the other hand.o Properties f ContextFree Longuqges he family of cotrtextfree Ianguages occupies a central position in a hierarclty of formal languages. and structrrral results such as pumpirrg letnmas' These all provide us with a mear$ of understanding relations between the different families as well ir. we inveutigate characteristic properties of the various fatrilies. (:orrtextfree languages include important but restricted language farnilies such as regular and deterntinistic contextfiee languages. As in Chapter 4. algorithms for determirring properties of members of the family. 205 . wc look at closure urrder a variety of operatiorts. To study the relationship between langtage families and to exhibit their similarities arrd differences.sfor classifying specifit: languages iu an appropriate category.
'uu"x:'!'z E L1 /R d. " .. Since the lrumber of va.the other for a restrir:ted type of contextcgntextfree languages free language. Frrrthermorer in the .r.nd lr vl> 1.2.ndsome sufficienbly long path in from the root to a lca.2) er.' r. . 1. .ll strings of termitrals. thcrtr rmrst be some va.c a. Then there exists some positive integer rn such that atty ut F L with lrul > ?n calr be clccotrposed as (8. . Correspotrdittgto the derivation tree irr Figrrre 8.tedby the graInIIIaI arrcl tlre therefore in L.I fbr all i : 0.Therefbre. c a t t be gerrera.ria. 1 .f.r l : 0 .l. we have the derivation Sl uAz l uuAgz A'u. since L is infinite. * . a l l t h e s t r i n g s u ' u i . Proof: Consider the language L . 2 . Flom the above we see u t l r a t A 3 u A y a n d A 5 . .{. or Aproductions. Consider ttow sut'h a high derivation tre. I t I AI and z art: tr. lJince the Iength of thc string on the right sidc of any production is bounded.\}. say by k. ard assurne that we h*ve for it rr grarnmar G without utritproductions.. sur:h that (83) .. there exist artritrarily long derivatiorrs and corresponding derivation trecs of arbitrary height.Chopter 8 Pnopnnuns oF CoNTlJxrFnnn LeNcuacns ffiM Two PumpingLemmos The pumping lemtna girrcnitt lThe. .. This is known asthe pumpirrg lemma for contextfree languages.1. [onguoges A PumpingLemmofor Conlexl'Free Let .unrtrz.hles G is finite. one for irr gonertr. wlrert.1) with (8. .1. . ' g i z .L be an infinite r:ontextfiee language.Iiable that repeats on this pathr as shtrwn sclrematically irr Figure 8.oEm ?37is an effectivctool for showing Similar pumpitrg lrtmmas are knowu art: not n that we will discuss twtl strt:tr resrrlts. the Iengtlr of tltc clerivation of any'ur € tr must be at least lwl lh.
8,1 Two PuuPtllc LetuN,Ias
207
Figure 8.1 Derivation trec lbr a long string.
+
\A)
X
/
\
derivations A I uAy and A 4 ,r, *e calr assuilte that no variable repeats (otherwise, we just use the repeatitrg variable as A). Therefcrre,the lengths of the strings u, fi, and y depend orrly on the productions of the granrmar anrl carr be bounded independently of ur so that (8.2) holds. Finally, since there are no unit productions and no.\ productions, u and y cannot both be ernpty strings, giving (8.3). This completes the argument that (8,1) to (8.a) hold. I
This pumping lemtra is useful in showing that ar,larrguage does not belong to the family of contextfree languages. Its application is typical of pumping lemrpas irr generall they are used negatively to show that a given Ianguage does not belong to some family. As irr Theorem 4.8, the correct But argument can be visualized as a game against an]Urlellig€@!. now the rulos make it a little more difficrrlt for us. For regular languages, the substringzE\hose length is bounded by rn starts at the left end of zu. Therefore*tliEiubstring gr that can be pumped is within rn syrrrbols of the beginning of ur. For contextfree language$, we otily have a bound on luryl. The substring u that precedesura carrbe arbitrarily long. This gives additional freedom to the adver$ary, rnaking arguments involving Theorem 8.1 a little more complicated.
Show that the language [,:{u,nbncn:rz>0} is not contextfree.
Chopter I
PRopnnrtns on ConrnxrFREE Lancuacns

lT*.,/ ),g,,,,' ,".'dir' i''r*lir'lr+r
lHi*: . Dc.t :,!rt,r;rH,H"l;:T""Tl:'i:,ffiT:i lffiJlil,':,,';*#il1iJ *Y, L umpinglemma.Thattris ,
suchasthe con$truction from someother argurnent, must conre contextfree
of a contextfree granrrrra,r. The argument also justifies a claim made in Example 7'9 and allows us
,  .r r q rr Lr{ a E I
which is Once the adversary lrqs choserrrn,,we.pick jl: ttli"efq3id! in tr. The adversary now hasEETffilcltoices. If he choosel urq to contain only a,'s,then the pumped string will obviously not be in ,L. If he chooses 4 slriqg rnntaining a,n equal number of a's and b's, then the pumped string k #;.,ot L* generated, and again we have [[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[fJ*ith generaterl a string not i1 .L. In fact, the only way the adversary could stop us froilI wirrrring is to pick ufra so that u3rhas the same number of a's, bts, and cts. But this is not possible becauseof rrstriction (8.?). Therefore, L is not conte4!free. If we try the sarne argutrrerrt on the language 7  {a"b"}, we fail, as we must, since the language is contextfree. If we pick any string in tr, such as u i : e , " ' b " ' , t h e a d v e r s a r y c a t r p i c k r , : t t r ka n d g r  b A , N o w , n o m a t t e r w } r a t
to closea gap in that examPle.Tlq
L: {a"b'} {a^bz^}  {anb"cn}
is not ctrntextfree. The string artlbrnctrl is irr Z, but the pumped result is not.
T
irixUd$lle,raigi
Consider the language
L  {ww : tu
:glt]
Although this language appears to be very similar to the contextfree language of Example 5.1, it is not contextfree. Consider the string a*lfna*b*, There are many ways in which the adversary can now pick uzg, but for all of them we have a winning countermove. For example, for the choice in Figure 8.2, we call use i : 0 to get a string of the form akYa*b*,h < rn or i < m, which is rrot in .L. For other choices by the adversary similar arguments can be made. We conclude that ,L is not contextfree'
8,1 Two Pulrerrvc Lnrvruns
209
Figure 8.2
a...ab...ha...ab...h
F_i_]___+
ilv fr! E
,
nul
n_st
r'eilr rtr
{a"t:n>0}'/
l_p
c.t
is not contextfree, In Example 4.11 we showed that this language rg11g!_tgg!g:. However, for a language over an alphahet with a single symbol, t[eEEfrttle clifference between Theorem S.l a,nd the prrmping lemtra for regular languages. In either case, the strings to bt' 1'r'rped corrsist entirely of ots, and whatever new string can he gcncrated by Theorern 8.1 can also be generated by Theorem 4.8. Therafore, we can use essentially the same arguments as in Example 4.11 to show that .L is not contextfree.
r
t
.\ r ,,,1* *.lo*
fr,#sthplqS,,4 Showthat the language
<J
,
I '1
r^ bb 'trra lrrrcroao6a
t
, 5lf.
is not contextfree. Given rn in Theorem 8.1, wc pick as our strirrgitr,r"'6n,)The adversary now has several choices. The only one that requiris mffh thought is tho
q4ovrn_illFigur* S.?..$q!11g11!jry will yield glr€wstriue;v/ith ,on=q lg + \i  I)kr a'sJand + U I)hz b's.lIf the adversary takesk, I 0, W E;FO@TIffi i:0.Sinrer (*  hr)'g (*  1)' = n 7 , 2 m * r 2 < rrf  lq, the result is not in tr. If the opponerrt picks A1 : 0, frz I 0 or h * 0, hz : 0, then again with i : 0, the purnpedstring is not in ,L. We can concludefiom this that .L is rrot a contextfreelanguage. I
210
Chopter I
PRopeRrtrs or Corutnxl'FREELRwcu.+cps
Figure 8.3
# n  j_E#
u i l l r l
]
d . . .d . . . a . . . a h ' . ' h ." b . . ,b z
A PumpingLemmofor Lineqrlonguoges
We previouslymade a distinction betweenlinear and qog}llrj3rg4textfreg grammaxs.We now maftea similar distinction betweenlanguages.
IIM
A contextfree language .L is said to be linear if there exists a linear contextfree grammar G such that .L : L(G).
. ,1, C'l
Clearly, every linear languageis contextfree,but we have not yet esis tablished whethet or not the converse true
tl ,.''ri.',r )),,.*
I
r?> 0) is a linear language' A linear Brarnmar for it is given in Example 1.10. The grarnmargiven in Example 1.12for the languageL : {.; no (tu) : nr,(tr)} is not linear, so the secondlanguageis linear' not necessarilY I _
Of course, just because a specific gramma,r is not linear does not imply that the language generated by it is not linea,r. If we want to prove that a language is not linear, we mu$t show that there exists no equivalent linear grammar. We approach this in the usual wa5 establishing structural properties for linear languages, then showing that some contextfree languages do not have a required propertY. Let L be an infinite linear language. Then there exists some positive integer ?n, such that any w €. L) with lurl ! m can be decomposedas w:uafrllz with
luuyzl 1nt',
lual>r,
(8 5) (8 6)
8.1 Two Purr.rprr{c Lnlrues
ZLt
suchthat uu'ny'z € L, (,$7)
f o r a l l z : 0 , 1 ,2 . . . . . Note that the conclusions of this theorem differ from those of rheorem 8.1, since (8.2) is replaced by (8 s). This implies that trie strings u and 3r to be pumped must now be located within rn synrbols of the leff and right ends of ru, respectively. The middlc strirrg z can be of arbitrary length. Proof: our reasoning follows the proof of rheorem g.l. since the language is linear, there exists soure linear grammar G for it. To use the argumerrt in Theorern 8.1, we also need to claim that G contains nei urritproductions arld no .\productions. An examination of the proofs of Theorem 6.8 and Theorern 6.4 wilt show that removing Aproductions ancl unitproductions does not destroy the linearity of the gra,mmar. we ca.rrtherefore assurre that G has the required property. consider now the derivation tree as shown in Figure g.l. Because the grammar is linear, variables can appear only on the path from s to the first A, on the path from the first A to the second one, and on the path from the second .4 to sorre leaf of the tree. Since there are only a finite number of va"riablers the path frorn 5 to the first 4, and sirrce each of on these gerreratesa finite number of terminals, u a'd z rnust be bounded. By a similar argurnent, u attd y are bounded, so (g.b) follows. The rest of the argument is as in Thcorern 8.1. I
llH$$$si$.Niiil rhuru,.g,,us*
is not linear. To show this, assume that the langua6Jeis linrrar and apply Theorem 8.2 to the string
,uJ : arnbzrrl(f,ttr,
Inequality (8.7) shows that in this cas. thc strirrgs ut Dt a must a,llconsist at entirely of a's. If we pump this str'ing, we get orrt,lk62nzont+t, with either h > 1 or I ) t, a result that is not in tr. This contracliction of Theorern g.Z proves that the languageris rrot linear. I
F
2r2
Chopter I Pnt)pr:nrrr:soR Cot't:rsxrFRenLeNcuecns This exatnple answers the general qtrestion ra,iscd on thtt relation beof tweel the f"tr,milies contextfree and linear languages' Tlttl farnilv of lineflr is Iangrrelges a, proper subset of the family of contextfree larrguagtrs.
1. Use reasonirrg similar to that in Exarnple 4.1I to give a complete proof that the language in Examplc 8.3 is not cotrtextIree. that the langrrage L : {a* I rz is a prime trutnber} is not t:orrtextfree. @Stto* 3, Show that L : {trr.utrttt : w E {a,b} } is not a cotrtextfi'ee larrguage W 4. Show that .L : {ur e {tl, b, c}" r n2"(*) + "3 (*) : til (to)} is not contextfrcc, 5. Is the language 7  {anb* : n * 2*} corrtextfree? 6. Show that the language I: q (3) {o"' , " } 0} is not coutextfree'
Show that the following languagcs on X = {a, b, c} u,re not contextfree.
( a )I :
{a*ll ; " <.i'} m
(b) I : {alhi :" > (j  1)'} (c)L:{anbich:k:jn\ (d) /,: {a'obich:k>n,f >i}
( e ) L : { a ; " h ic k : n I j , n ! h S . i } (f) .L : {ru : n,,,(tu) < nu (tu) < n.. (tr)} (S) (e) f  {w: no(w) lrr't,('u,): n" ('u')} (h) Z, : : {u € {4, b,t:}* : n,,.(zr)* nr (tu) 2n" (tu)}'
,/
)
,{\
!dr)
Deternrinc whether ol not the following languagcs are cotrtextfree (a) Z,: {a'f"uuHa't: n,} 0,u e {a'b}} ' (h) L:,{a"'bia^H (c) 1, : : ? r} 0 , i l 0 i W
> 0} {a'ltirtib" : tr.> 0,j
(tt) L : {a"H akht I n + i t k + I} (c,) L = {anbi ahlsl,r, a k, J < I} j} (f) r: {u"'b"t! ,," < g. I1 Theprenr 8.I, find a bound for rrz in tertns of the properties of the grammar G. m nctcrmi*e whetrrer .r *.t t6e followi.g language is cnntextfree. ffi
^ '{ ,
i e [, = {.,w1ttn2 ,u]r1,ttz {4, b} ,wt I wzl
8.2 CLosuRePRoreRrlns ewo Dncrsrol Ar,conrrHMs FoR CournxrFnpn LaNcuecns
213
b q
drb V QH
Sfro* that the language7 : {a'ob"a"ob'* n 2 0,m > 0} is contextfree ; but not linear. Show that the following languageis not lirrear.
t,: {w : n*(w) > nu(.)} W is
1 3 . S h o w t h a t t h e l a n g u a g eL : contextfree, but not linear. \' D (4.
{w € {a,b,c}*:rro (w)+na(*):rr"(u,)}
 1} islirrear. {a"bt :i 1,n<2i 15. Determine whether or not the language in Example 5.1? is linear. ffi O e t e r m i n e w h e t h e r o r n o t t h e l a n g u a g e7 = 16. In Theorem 8.2, find a bound on m in tenns of the properties of the glarrrrrrar G.
17, Justify the claim made in Theorem 8.2 that for any linear langrrage (not rxrntaining A) there exists a lirtear grarrtrna,r without }productiorrs arrd unitproductions. L8, Consider the set of all strings af h, wherc a and b are positive decimal integers such that u I b. The set of strings therr represerrts all possible tlecirnal fractions. Determine whctircr or not this is a contextfrec languagc.
* 19, Show that the complement of the language in Exercise 6 is not contextfree.
\t
@
t* the followinglanguage contextfree?
L: {o""' : rz and m, are prime numbers} W
C l o s u r eP r o p e r t i e s n d D e c i s i o nA l g o r i t h m s o L for ConfextFree onguoger
In Chapter 4 we looked at closure under certain operations and algorithms to decide on the properties of the farnily of regular languages. On the whole, the questions raised there had easy arrswers. When we a,sk the same questions about contextfree la,nguages, encounter more difficulties. First, we closure properties that hold fbr regular,rlalnguilges do not always hold fbr contextfree larrguages.When they do, the arguments needed to prove them are often quite cornplicated. Seconcl, many intuitively simple and important questions about srntextf'ree larrgrragesc:artrrottrtt arrswered. This statement rrray seerrrat first surprising and will need to be elaborated as we proceed. In this section, we provide only a sample of some of the most important results.
longuoges Closure ContextFree of
is The familv of contextfierela,ngrrages closed und(lr union, ctltrcaterratiorr, arrd starclosure,
G+ : (Yr u Yzu {50}. Wt: c:arra^$$urne without loss of generality that the sets V1 and Vz <rredisioint. P:t: Pr U Pz U {5r * 5rl5z}.gain Sa is a new variable and P+: Pt U Pz U {5i SrSz} . Suppose for instance that u € trr. Nr:xt. u'I'2. Cotsider rrow the larrguage I (Gr). r.2L4 Chopter 8 Pnornnuss ol' CoNrexrFRen LANcuAGEs Proof: Let trr and L2 be two contextfree languages generated by the c o n t e x t .93. AIso. then tu must be in. ? r .10) is used first.8) rmrst trr: the first step of the derivation. Alternatively. .9) (8. But it is ca^sy see that to L(Gs) : Lru Lz.. ancl V1 and V2 are disjoint.} . so that L (Gz) is a contextfree languagc. together with an alternative starting production that allows us tr) us(l orro or the other grammars. P3) .f ' r eg r a m m r r n J r : ( V r . if (8.L2 and it follows that . Here a.Sa. P 1 ) a n d G z : ( V z .il'.2. Hence trr must be in L1. Suppose(8.I0) (8. 63 is a contextfree grammar. More precisely. P 2 ) . rxlnsider . The productions of Gs are all the prochrt:ti{rrrrt G1 a.L (G3) is the rrnion of L1 a.nd 1. A similar argument can be made for u € L2. wlrere S'3 is ir variirtrlc rrot in V U Vz.9) is used. e e G r spcctivcly. Then S:t+Sr4ru is ir possiblc derivation in grammar G:J.rrcl of G2. Obviously.Pa). S r .. the derivation SrSru can involve productions in P1 only. 1f w e L (G3) therr either 53 + ^91 or 5h + 5'z (8. T z . gerrerated by the grammar Gr : (H u V2 U {S. S 2 . Since sentential ftrrrns durivcd fiom Sr have variables in I{. u 72.
Proof: Corrsider the two languages L 1 : { a n b " c n ': n Z 0 .3. which we have already shown not to be contextfree. r n } 0 } and L2: { a " b * c * i n . Then I ( G s ): L ( G t ) * . so it is contextfree by Theorem 8.5155l. Finally.L1 and "t2 are contextfiee. r and The family of contextfree languages is not closed under intersection and complementation. concatenation. Thus we have shownthat the farnily of contextfreelanguages closed is under union. . a grammar for L1 is S * Sr. consider (Gs) with I Gs : (VLU {Ss} .U.en LANGUAcES 215 Then L ( G + ) : L ( G r )L ( G r ) follows easily.7.2 ClosunE PRopERTTES DucrsroN AlcoRrrHlrs AND FoR CoNTEXTFn. For instance. starclosure.8. There are several ways one can show that . Thus. 5z + c5zlA' Alternativelg we note that tr1 is the concatenation of two contextfree languages. P5) where 5'5is a new variable and Ps: Pt U {S5. the family of contextfree languages is not closcd under intcrsection. . 5r i a5rblA.55.1.5'2.r n > 0 } . } 0 . But L1 t L2: : {unbncn n } 0}.
In this. p r )r ) e 6 ( ( q a .8. p r ) .po. then the right side of the above expression would be a contextfree language for any contextfree L1 and L2. action of M1 and M2: whenever a symbol is read from the input string.Fz) be a dfa that accepts. Proof: Let M1 = (8. * t': I\x Ezt and define d such that o . r While the intersection of two contextftee Ianguages may produce a Ianguage that is not contextfree. e d1(qa. we also require that if a : tr. that the intersection of two contextfree languages is not necessarily contextfree. a ) : p t . X.3 and the set identitv ::=. the states of M are labeled with pairs (U. Let . .t.: L 1 t 1 L 2 :L t U L z . ffi simultaneously executes the moves of M1 and M2.I..Chopler I PnornRuns on Cowrnxrl'Ree LnxcuRcps The second part of the theorem follows from Theorem 8. Consequently.L2' We construct a pushMz: qo. if and only if (gn.E. In other words.r language' Then L1 tt L2 is contextfree. the closure property holds if one of the languages is regular. representing the respectivestates . If the family of contextfree languages were closed under complementation.s. r) and 6 z( p i .L1 be a contextfiee language and tr2 be a regula.f. dr.dz. ( ( q r . then py : p7. 0o: (go'po). a. b ) . To this end we let e:exp. q0. b) .r. .F) which simulates the parallel down automaton ffi : (0. the family of contextfree languages is not closed under complementation. . But this contradicts what we have just Shown.pi).F"l) be an npda which accepts Lr and (P.
by the closure of regular languages under complementation and the closure of contextfree languages under regular intersection. " ) i . I . because trr is finite. P " ) .nlI00j is (:tlntr:xtfree.5. Therefore.q.. nd d* (Po. € . . Also. Then. n ) . I The property addressed by this theorem is called closure under regular intersection.7 Show that the language trla"h"rz)0.:{a"bn:n>0}1121. w . 7. dtlr q. It is a straiglrtforward inductiorr argument to show that ( ( q t " P t ' )' u ' "z ) i . that is.z ) . It is possible to prove this claim by constructing a pda or a contextfree granrrnar for the langrrage. the desired result fbllows. it is easy to see that.8.2 Closunn PRoRnnrlns . Exomplo 8. a string is acceptecl Uv fr if and only if it is accepted by Mr and M2. Let L t : { a l o o b l o o }. but the process is tedious. ( ( q ' .FID Dnctslol AlconrrHMs FoR ColrrexrFREE LANGUAGES 217 in which Mr and M2 (:arr be after reading a certain inprrt string. Becauseof the result of the theorem.Fr and p" € Fz if and only if ( q o . We can get a muclr treater argument with Theorem 8. we say that the family of contextfree languages is closed under regular intersection. ( g . if it is in tr (Mr) ) L (Mr) : Ltfi Lz. Therefore.w) : p". This closure property is sometime$ useful for simplifying arguments in connection with speciflc languages. it is regular.
Some additional results are explored in the exercisesat the end of this section.then tr (G) is ernpty. The pumping lemma can be used for this. there exists an algotithm Giverr a contextfree grammar 6: for deciding whether or not . if not. Proof: We assume that G contains no . b.. changes have to be Proofr For simplicity.6. (". If '5 is found to be useless.P)' there exists an algorithm Given a contextfree grammar 5: fbr determining whether or not I (G) is infinite. Wtr conclude that tr is not contextfree. .Slight made in the argumerrt if this is not so. For the purpose of this discussion.T.L (G) is empty. we a$$ume that the Ianguage is described by its grarllma. But we alreadv know that this is not so. but agaiu we can get a tnuclt shorter argurnerrt urting closure under regular intorsection. no unitproductions. * b * c * ): f a ' b " c ' : r z l 0 ] would also be contextfree. Suppose the grammar has a repeating variable in the sense that there exists some A e V fbr which there is a derivation Al rAs. Then L f t L ( a . Longuoges Properties ContextFree of SomeDecidqble By putting together Theorems 5.rucuRcns W[*Blii$l'tt. a^$sume that.Chopter I PnornRTrnson CorcrnxrFREEL. Suppose that . I (VrTr^S.P). and no useless symbols.^9.\N showrhat the languase L : {* € {cl. then . c}* : no (*) : n.q.L were contextfree.)} is not contextfree.\productions.L (G) contains at least one element. We use the algorithm for removing uselesssymbols and productions.2 and 6. (V. I Closure properties of langrtages play an important rolc in the theory of fclrrnal languages and many more clt$ure properties for contextftas languages can be established. we have already established the existence of a rnembership algorithm for contextfree languages' This is of course an essertial feature of any language family useful in practice' Other simple properties of corrtextfiee languages can also be determined. (tr) : t.\ # L(G).r.
4.7.(.Au 3 ux'''Ay"u 3 ur"zynu is possible for all n.But it trrrrrs rnrt that there is rrclsuc:tr we do uot have the ttx:hrrical tnachiner:vfirr ltrtlllerly clefinirrgtiro rnt:arritrg of "there is no ir. so thrr.rierblc. cycle is ir repeatiug one. Sintxl A is tteither nullable nor ir.te rr. grilrnrnitr ltas wc a repeating va. But then S 1u.L1 in .n a.ritlblos in sucir a way tha. As itr I'heoretn 4.8 contcxtfree? ffi 'lheorem B.iAlcoRt't'HMSFoR.t7cilnnot bc sitttultatreously emptv. R) whenever thcre is a corresponding production A +:nB.lgorilhn.rrtltto unitproductions. 1 .$srrrned have no Aproductions ir. Show that this larrguage is litrcar Consider the language .rhirs ir repeating va.t I (fJ) is infinite. Is thc conrplernent of the languagc in llxamJrle 8. Corr$e quently.!t. we rreed only to cletermine whclthcr the gramtnar ha. where. Since we now have an algorithm fbr rlu:itlittg whether tt.sstturt: rcpeating varia'bles.ble r:an cver repeat.2 Cl.cyt:lo. I Sornewhatsurprisitrgly. the gra. I'his can be done sirnply by drawing a depenrlerrc:vgraph for the va. ?.other sirllllt: properLiesof contextf'rtlclanguages rrr(: rrot so easily clealt with.riahlcrif rrrrd otily if the clepelncltlncy graph ha... to get a. u.sa. wtl rniglrt look for a.'oltrnxrFnnE La. we have SluAuSut and A1z.osuRril zfg to Since (J i$ ir.L (G) is finite. This is an important poirrt to which we will retrrrn lirter. 'I'hen any varia. then the length of irny derivation is boundecl by lI/1.mmir.NcuncF:s 8. Iarrguage. rrscless svmbol. In that t:itsc. and a trre irt 7'*. r and .r.n tllgoritltrn for detertniuing wlxrtlrcr I (G) is finite.lgtr the sarntl ritlrrrr to deternrine whcthcr two contextfree grilrrrrnarsgenera. have an algorithm frrr rlctertnining whetlrcr tlr ttot I (C) is infirritc. If no varia.Pnopnnrrns eNr Dncrsror.lgoritlrtn.t therc is att edge (A." but its intrritivt: rncatrilg is clea. I'hus. For the mtlrntllrt.trle that is rrt' Lhe base of a.
L be a rleterministic contextfree language and defirre a new languagc L1 : lw : aw € L. therr .Lr arrd . 7 : {tu e {4. W Show that if . Givc the cletails of the inductive argurnent in Thcorcm 8. b}.Lr is lirrear and 1. . 15' Cive an example of a cotrtextfree language whose complernent is not contextfree.220 Chopter I Pnopnnuns ol CoultLxr!'nnn LANGUAGES 3. Slxrw that the following language is contextfree.n is not a multiple of 5}iscontext 18.tu does not cttrrtain a substring aab} Is the farnily of deterministic corrtextfree languages closed under homornorphisrn'i 19' 20. Show that the family of urtarrrbiguons contextfree languages is not closed under urrion. for any giverr trntextfree grarnmar G.[s it necessarily true that . . 14. Slxrw that the family of contextfree languages is closecl under reversal.t (ur). Showthatthelarrguage : 7 free.2 is regular." ('u) : rr. 21.: n. 23.Lr is a tleterrninistic corrtextfree language'l 17. Which of the languagc faurilies we have discussed are not closed under reversal? 7. I' Show that the farnily of deterministic contextfree languages is closed urrder regular difference. *13.r .L1. that is. 12. Show that the family of litreat languages is closed under union. then /. ffi 16.5. 9. Show that the farnily of unarnbigrrous contcxtfree languages is not closecl under interset:tion. ffi 1O. Let . brrt is closed under regular difference. r:arr determine whethcr or not A e I (G).L. show that therc cxists an algorithm to dctermine whether the language gerrcrated by some contextfree gratrrrnar contains any words of length less than some givetr llumtrer rr. but not closcd unrler concatenation.Ls is regula. Give an algorithrn which. {S 22. Show that the family of linear larrgrragesis not closed under intersection.L2is a linear language. Let Lr be a contextfree language arrd . Show that there exists an algorithm to deterrrrine whether or not .: be regular. {anb": n } 0.a € X]. if . Show that the family of deterministic contcxtfree larrguages is not closed utrtler union and intersectiorr.. 11. Show that the farnily of contextfree languages is not closed untler difierence in gerteral.Lz is contextfree. 5.L2 have a cornrnon elerrrent. ffi 6. 4' Show that the family of linear lartgrrages is closed under hornorrrorphism.r. Show that the family of t:ontextfree languages is closerl under homornorphism.Lr is rrrntext*free and .
This prompts rrs to look beyond contextfree {**}. we have encountered $Qmefundamental ideas. we have the more powerful pushdown automaton. in particular the concepts of regular and contextfree languages and their association with finite autontata and pushdown accepters. that pushdown automa. we have a finite automaton. Ianguages and investigJatehow one might define new Ianguage families that picture of an inclrrde these examples. we can expect to disCover even m()re powerful language families if we give the automaton more flexible storage. If we compare flnite see that the nature of the temporary storage creates the difference between them. a. we automaton. Extrapolating from thiS observation. For example.re limited in scope' This was made clear in the last chapter. To do so. where our results showeclthat some simple languages. we return to the SEeneral arrtornata with pushdown automata. If there is no storage. 22r . We also saw that contextfree languages. and therefore.ta are more powerful than finite automata. while firndamental to tha study of progranmfng languages. if the storage is a stack. Our study has revealed that the regular languages form a proper subset of the contextfree languages. such as {atb"c"t} and are not contextfrtle.Turing M o chi n e s n the foregoing discussion.
tion 1. It is more instructivcl to ask a more arntritirlrs question and rxlnsider how far the concept of arr arrtomaton can btr ptrshed.t ca. To deviate slightly frorn the general schcme of chaptcr 1. .t can traval right or left orr the tape and tha.apesused in actual curnputers.n read arxl write a single symhol on each nrove. thrne stacks. which maintains thirt any urmputational llrocess. The discussibnr:rrlminatesin the Turing thesis.rrtomaton and tlrrough it a new langrrirge family? Tlis approach raises a lirrge number of qucstions. This array extcnds indefinitely in both directiorrs and is thercfirre capable tlf holdirrg an urilirnited amourrt of infbrmation. whilu the mechanisrn of a. in turn.Thring rrrachine'sstorage is actually quite sirnple. The infornmtion can be read irrrd changed irr any order. to a precise elefinition of the idca of a nrechanicalor algoritlunir: computation. t:elnbe done on a T\rring machine. It cart lle visualized as a single.3. each of which can holtl tr single syrnbol. eac:hof which is caJrrrbleof holding one symbol. Definition q Turing of Mochine A Trrring rnat:hineis an automaton whose ternporary storage is a tape. irr the general scheme of Figure 1.mentalcorrr:r:pt a Turing of machine and. rhis telpe is divided into cells.or $orneother storage device? Does each utorage device tlcfine a new kind of ir. we will see later that this modificatiorr of our general model in sec. whatcver input and output is necessarvwill be done orr the machirrc's tape. a queu(]. rnost of whit:h turn out to bc rrrrinteresting. onedirrrensional array of cells.such as those carried out by presentday cornprrters. T h eS t o n d o r d u r i n g o c h i n e T M Although we carr cnvision a variety of automata with complex and sophisticaterdstorage deviuru. the concept is tlroad enough to c:oververy complcx processes. Next wc errgrrethat. We bcgin our study with a fbrmal defirrition of a. we rrsrxl two stacks. the autornatorr that we use as a Thring machirre will have neither an input filc nor any special output rnu:hanism. but we leave them out becausrl the resulting automaton is a little easier to rkrscribe. what carr we $ay about the rnost powerful of arrtomata and thc limits of computatiorr? This leads to the funda. a. Ttrring rnar:hine.2 is of little r:onsequence. thel develop sorne feeling fbr whtrt is involved by tloing some simplc programs. Associated with the tape is a readwrite head tha.lJsuch a storage device a tape hecauseit is analogousto the magnetic t. we will ca. T[ring rrrachine is quite rudimentary. we could retairr the input file arrd a specilic outprrt mechanisrn without affecting any of the corx:lusions we artl rrbout to draw.222 ChopterI Tunlnc MACHTNES what wtlrrld happen if.
qo E I is the initial stntc.1 Reedwrite head A dirrgrani giving an intrritivtr visualization of a T\rrirrg rnachiue is shown irr Figure 9.. In the rlclinition of a Ttuing rnachine.rtiirl futrction on Q x f. The result is a new stattl of tlte control unit. d is the transition function. that is. ! e f is a special symbol called the blank.1 rnakes the notion prccise' A T\rrirrg tnachine M is derfirrtxlby M : ( Q . f . I)efinition 9. I is a finite set of symbols called the tape alphabet.1.R}. Blanks are ruled out a.i l . Thritrg tnachine opcrates. F C Q is the set of firral states. . not including the blank.1 THn Srer"rrrARlrTuR.srrtrsetof the tape alphabct. d is a pa.r irrput for reasons that will become apparelrt shortly. that the input alphabet is a. a new tape sYmbol. where Q is the set of intcrnal states.9.q 6 . The arguments of d are the current state of the control unit and the current tape symbol being read. The transition funt:tiorr d is defined as d:QxfQxfx{.Inc MacntNn 223 Figurc 9. X is the input nlphabet. ( t . Irr gerreral. X .1. its interpretatiott gives the prirrciple by which a. F ) .we assumethat E ! f * {n}.
which has a finite llrelnorYr and in its tape. this is possible because d is a partiat function.2 showsthe situation before and after ttNr move causedby tlx: . it has a secondary storage of rrnlimited capacity. but this is not so. It has a processing unit. A Ttrring maclfne is said to halt whenever it reaches a configuration for which d is not defiiled. how this computer acts. so the T[rring machine will ha]t whenever it enters a final state. Tlte transition function d defines ttprogram" of the machine.o) : (91.n cs. we will assume that rro transitions are defined for any final state.f ii*tnele +. the conterrts of any cell on the tape may be cxamined and changeclmany times' Eventuallv. which we achieve in a Tlrring machine bv putting it into a halt state. to ctrange the state of thc control. During this process. I . and a move symbol.rry ttut ttre very limited: it can $ensea symbol on its tape and use the result to decide what to do next' Tltc onlv actions the machine can perform are to rewrite the current symbol. and to move the readwritc head.It) . It then gotls through a sequenceof steps controlled by the transition lunction d. the automaton stait$ in the given initial state with sorne informatiorr on the tape. r.The move syurbttl indicates whether the readwrite head moves left or right one cell after the new symbol ha^sbeen written ott the tape. This small instflrction set may seem inadequate lor doing complicated things. flnternal state7o f Internal smte fI I l'l'l'l l'l'Fl(b) which replaces the old orre. In fact.Chopter 9 TunIuc MACIIINE$ Figure [.lll. We can think tlf a Thring machilre as a rather simple computer. T\rring machines are quite powerfirl in principle.i (qo. and wc often call it the As always. L rtr R.2 The situation (a) before the move and (b) after the In()ve.rJ. the whole process may ternrinate. Tlrg instructiols that sut:h a c6mputer ca.1 Figrrre 9.
b .rlwrite head will replace the a with a b. h't(HtilFld f t Consider the Tirring machine delined by Q : {qo.fi. tlte applicable transition rule is d (go. d (qo. .a.3 of A seqrrerrr:e moves. ft) .3 shows several stages of the process for a simple initial configuration. n.o) : (go.r. The machine will remairr in state q0. ) . r : { q r }' ano d (qo.a. L ) .ffin#Ili.. R d (qo.b' ft)' Therefore the rea. b d (qo. E.q.but let F be empty. 5 : { a .. a) : (qo.b.b.o) : (qo.b) : (go.tnRRo 226 Figurc 9. s4ren the machine errcounters the first blank.L). E. then halt in final state q1' Figure 9. then move right on the tape.b.il) : (qs. but b's will not be modified.tr}.n) : (qr.Hi$" Take Q. tI (qr. If this T\rring machine is started in state qs with the symbol a under the readwrite head. tr) .1 Tnn Sr. E) . d ( s r .o) .b } . it v/ill move left one cell. Define dbv d (qo.(q1. b ): ( q o . I as definedin the previousexample.(q1. ) . f:{a. ft) .L) . Any subsequent a will also be replaced with a b. ft) . d (sr.b.qr} . tr.!) : (sr. H. d (qo.TunInG MlcnIbrn 9. I .
with the readwrite head moving alternately right then left. As an analogy with programmirtg tcrminology. To exhibit the configurations of a Ttrring machine. 'fhere These corrvnntions were cltosen primarily for the convenience of subse* qrrent discussiorr. This is an instalxjc of a Ttrring rlachine that does rrot halt. k + 1 o' " ' is the instantaneou$ description of a machine in sta..4.. is no special irrput file.t the T\rring machine is in an infinite loop. while q defines the state of the corrtrol rrnit.with the readwrite head on the a. and the positiorr tlf the readwritc head. The Turirrg machine is detcrministic in thc sensethat d defines at most one move for ea. This convention is chosen so that . I Since omr can make several different definitions of a Tirring machine. whatever the initial information on its tape. The machine then reads the a. will run forevet..tiln show the tape contents. The machine goes back ittto state {q and t}tc readwrite head moves left. The symbols et. We a"ssumethat at the initial time the tape has some specified r:ontent. We will use the notation in which frtIrz or ' ' a t f l z ' ' ' a k . Its next strrte is {1 and the readwrite head moves right. we uso the idea of an irutantaneous description. 2. but making no rnodifications to the tape.chconfiguration. there is no special output device.. it is worthwhile to nummarize thc main features of our model. 1. vrhich we will call a standard T\ring machine. we say tha. Any configuration is completely determined by the crrrrent state of the control unit.Chopter TuRrNc Macmbru$ Chopter 9 Tunrr. The T\rring machine hils a tape that is unbounded in both directions. trnd the sequence of moves stilrts again.te q with the tape depictetl in Figure 9. allowing any number of left and rigltt moves. we will look at other version$ of Ttrring nratrhines and discrtss their relation to our standard model. some or all of the contents of the tape may be viewed as output. Suppose that the tape initially corrtainsc. h o . we cirn trace a typical ctr.. the conterrts of the tape.se. We are rlow track exactly in tlre original state. 3.rc To see what happens here. It is dear from this that the maclfrrc.b. In Chapter 10..r q 0 . Similarly. so thai it is now over the b. Whenever tlte machine halts. This symbol is also rt:ird and Ieft unchanged. brrt does not change it. Some of this rnay be considered input.
nothtrr will tre denoted by F. A move fron one conliguration to a. Subscripts. T .3 correspond to the sequenceof instantaThe picrtrrrc:s hqya.ttl is q1.1 TnE Srlrun. The unspet:ified part of the ttrpe is assumed to <xlrrtain all bla.ru indicates that the readwrite herr. llfi i:il 9.e.nks.Et{frr$Sl'$ The action of thc T[ring machinc irr Figure 9.qRn 227 Figure 9.nrllt:ft of the readwritc head.ntl are not shown elxplicitly in the irrstiurtaneousdescriptiotr. bqlbneous dt:scriptions Qoaa.Tuntuc MncHttqn 9.1bq(raI bbqptrF bqlb qoilill bqLb. the blank syrubol rtray appear in the irrstantaneous desr:ription. Thus.q2d.however. arxl the reaclwrite head is on tlte c. II d (s1.4 the positiorr of the readwrite head is over the ccll contaitting thc sytnbol immediately followirrg q. bbqsl..ir.$ .tlis on the cell to the immecliate krft of the first symbol of to and that this cell coutains a blank. rxlrrrrally such blanks are irreleva'nt ir. I'he instantnrrcous description givcs otily a finite amtlrrrt of informatitlrr to the right a.il drawn in Figrrrc:9. the tape t:orrtainsabcd. The symtrol F has the u$rral neaning of irrr arbitrary nutnber of mtles. then the rnove abqlctl I abe. For examplc. If the positiorr of blanks is rc:levant to the clisr:rrssion. R) .3 can be representul by rl11{r0. such as F14. is ntacle whenever the internal sta. the insta'nttrrrtxrus description qE. r:) : (q2. are used iu argumtlnts to rlistinguish betweort several ma'crhirrcs. $fisrhpls S. I ll.
.*n) = (qz.of M. ) . 1 r . special notation for it.chines. indicating tha. F) be a .rious observations jrrst macle in a forrna.o) is unrlcfined.nystring a1 . . starting frorn the initial l:ontiguratior :t. firnclarnentirl role in the dist:ussion of 'I\rrirrg ma. g I and h € Q. A .1 Q 1 0 r i 0 . The sequc:rrr:e couligtrof ra.Chopter 9 Chopter Tunltrtc MncHrlrns For frrrther discussion.rQiIzI tltqiauz firr any qj arr(l a. 1 ro r ) : ( q z .?F co.t.ant with n. is possible if ancl only if b. b . tr) . carrnot escape.lconfigura.lway.ici halt sta. Let M : (8. fbr which 6(qi.rting rfli:x2 iI to it:.lJshorvsthe possibility that a T\rring machitrewill never hall. H d ( . fronr sorncinitia.it is convenient to summarize the: vir. is ir.Tlrtlrr ir. t t +'t t L n  & 1 " ' qTukrbAp'r1 "' A. A move { 1 1' " f i .kLqruht&t1+t.1qfr!1the tnar:hirrel never halts.tionsleaclirrgto a halt statc will be called a computation. A move A1 "' " t l . d.so w() u$e a. proceeclingin an crrdlessloop I'rrlm which it. X. 'fhrirrg ma. .r q I A h A h + l is possible if and only if . Qo.chine. This situatior plays a. n. d (qr. We will reprt:sent it by x:lqf.ninstantalttx)lliJ descriptiott a.l. Iixample 0. .tion M is sa. .
fitg6l. It savs Defirrition 9.ruis written on the tape with blalks on eitfier sicle.#.3 tells us what tnust happetr when ut e l'(M)' rrotfiing abqrrt t]re outcorrrefor any ot]rer iuput. If. n1qyn2 sorne t 'l'his rlerfirritionirrdir:atesLha.d. The reason for excluding trlanks fiom t]re iuput rrow becomesqlcar: it assurosus that a. rro matter how ma.lt. : (q1.t. t t 1 . bra.tthr: irrput .qo. the Ttrring rnachine entets a fina.:.nhappen: the rrrachinecan halt in a norilitral sta.i E : {0.w. or 6rre of two thirrgs ca. we terrrrinate anrl irccept the string. tlten tl is ctlrrsideredto be accepted. I t.n easy exercise irr Turing rnin:hiile prograrrttrting. tr) d (qo.pt:. te strirrg is not irr r (0i].ffit*.) n.ntl trever ha. The rnachine is started in the irritial state q1 with the retr. ) ' F for I i z.ny hlrrtrks it str.. 1}. This is a.$. Io keep track of thc r:omputtrtiotr. If we reaclr a blank withtlut eucorrntering alything hrrt 0. W]reu u is trol itr L(M). Without this could not lirnit the relgionin which it must look for convention.tcr M does it can errter an infinite loop a. qptt..1 THn Sr^r'lDnn. 0:) ( q 0 . Any string for whit:h not halt is by clefinitiott trot in 1.nkirrput somcwltere else orr the ta.f.F) accepted by M is b e a . arrclwe halt itt a tronlirral state.).tlwritehead positioned on the leftrrtost symhol of tu. T \ r r i t r g m a c h i r r c Then thc language L (M): ( * qy t r . As trir. (M)' accepts tht: lauguage dcnoted Fbr . r .chirre the inprrt.cketecl hlanks ou the riglrt and left. after a sequellce of tnovels. t.nksfillirrg out ther urtused portions. we contimre by movirrg right. € f .wo interrral sta'tesQ: {qo'{t} and one: final stirtc F: {qr} are sufticicrrt.l TuRIlrcj MacnItln 229 AccePters Mochines Lqnguoge os Turing Ttrring machintls can be vitrwed as accepttlrs in the fblklwing sense..llthe itrput is rcstricted to a welldefined by region of thc tape. If it is.it coukl ntlver be strrtr that thelrt:was not sorntl nonbla. wr: reacl each syrnbol anrl tlheck thnt it is a 0. Let M : (Q.9. . 0 . If the input txrntains a i any*fiere.E.f. design a T[ring rnar:hinethat by the regular exPressiou00*. with bla.nsition functiotr we carr take d ( s . the rna. A string tu is written orr the ta.pe. r . starting nt the lcft end of theri1put. fi) .Sr.'€ X+ .l state atld halts.
1) is undefined.qt1qz1qBtq4} 1 F:{qn}' g:{a.r . The set .r7. b ): ( q z . .i Exorrrple For E : {a. T}avelirrg back and forth this way' we ma.3.L.y ) : ( g r .chineaJsohalts in a Iina.ceit with a. it is possible. say y. we go left again to the leftrnost a. design a Thring machine that ar:r:eprs 7:{a. If after some time no c.t a conrplete solution for which Q: {qo. the head will move to ther right. we check it off by replacing it with some symhol.r. a d (gr. R ) . and thc concept is ea^sy understand.nr. Note thirt the 'rhrirrg ma. and so on.  : { a . since illrring mac]rincs have a primitive instrrrr:tion set. replrr.nsitions can be broken into several parts. indicating that an a has beru successfullypaired with a b. The tra. working orrt the cletails.rh":n}1}.n r. d ( q r .cingit with a y.nytime a 1 is read.u . y . which in turrr is checked off by replacing it with another symbol. f t ) . Intuitively. to as the next exarnrrlcu illustratc. then causcrJ the readwrite heatl to travel right to the first b.tch each . b .230 Chopterg Trrurrc MACHTNEs As long irs a 0 appears rrnder the re^dwrite head. r :y . rcpla.b}. rerplacestlrc leftmost a with an lr. then move to thc leftmost b and replrrce it with .:n cumbersome on a Turing machine.l state if utarted in stnte {s on a blank. say z. a) : (qr.with a correspondirrg b. t]re computations that we carl pregram ea"sily in a higlxlr level lailguaEle ere oftr. the rnachine will halt in the nonlinal state q()' $ince d (. we could interllret this as ilt:ceptance of A. L ) . ft) . . T The recogrrition of morr: complicated languagesis more difficult. thc:n the strirrg must be irr . starting at the Ieftmost a. we arrive a. the machine crrtersa state qz. when the y is written. a. ) : ( q t . .'s or b's remain. b}. q. i ( q o . we solve the problern in the following fashion. We then Iet tlxl readwritc head travcl right to fincl the leftrnost b. still. E } . d ( q r . If at a. but for ttxlhnical rea^s()ns ernpty string is not includecl the irr Definition 9. After that.
a . When an r is encountered. d (Sr. 6 (qz.. the machine will have carried out the paftial computation . b i * q o o . This can be done by 6 ( q o . ' ' a h h ' ' 't i * * q s " ' ( r A Y' b .1 Tsn SrannaRn TuntNc MaoHItIp 231 The next set of transitions reverses the direction utttil an z is encountered. a ..tr. ) .9. . indicating that the matching processis being carried out properly. The particular input aabb gives the following successiveinstantaneous descriptions araabbl rqlabbl raqlbbl nq2ayb tsq2raybl rqsaybl nrtnyb I myqlbl rnq2yyl ng2ryy I r:xqoyu rryqy I myqsJ I I rnyyZqal."a . the readwrite head travels left with the machine in state q2. After two passes' we will have completed the partial cornputation " q o a o .!) : (ga. and b. It will halt becauseno transition is specified for this case. When looking for the Ieftmost a. d ( s z . the rewriting continues this way.) R 5 (qz. ' . Other input not in the language will also lead to a nonfinal halting state (see Exercise 3 at the end of this section). For examplc.a ) : ( q z . R. If we input a string not in the language. if we give the machine a string a'b*. a y h" ' b . E) .To termirrate.L ) . ' and so on. instead of finding an a it will find a 37. When the ir4lut is a string a'b'. stop ping only when there are no more o'$ to be erased.tclted with a single b.a. and returns control to the initial state. r t b b.a): (q:t. q o e . a )= ( Q : t . . so that a single a has been mtr. wittr rL 7 ffit the machine will trventually encounter a blank in state q1.L ) . d ( q z . a final check is made to see if all a's and b's have been replaced (to detect input where an a follows a b). repositions thqt readwrite head over the leftmost a.n) : (qo. the direction is reversed to get the a' But now. E) . the computation will halt in a nonfinal state.g ) : ( q z . After one pass tlrrough this part of the computation. a . r We are now back in thc initial state q0. ready to dcal with the next c.
At the conclusiorrof the computation. for sornc finrr. Turing Mochines Tronsducers os We have had little reason so far to study transducersl in language theory. accepters are quite adequate. T\rring machines ate tttlt only interesting as language accepters. The inprrt for a computation will be all the nonblank symbols on the tape at the initial time. the output will be whatever is then on the tape. u. .r. But its we will shortly see. a"s well arwith some not in L. provided that QywI L. they provide us with a simple abstract model for digital comlrrrter$in general. respectively. but straightforward exercise. t}tey r:an he acceptedby I'uring rnachineswith very similar structufesI Otte ctlnt:hrsion we can draw frorn this example is that a Thring machine can recognize sorrre langrrages that are not contrlxtfree. If we want to rnodel comprrters using T\rring rrrachines. and c by replacing them in order lty r.MACHrFres At this point the T\rring machine halts in a final state.nsducer M as arr implementation of a furrction / defined by fr=f(w). We matclr each a.mple9.:{a"bncn:n > 1}.spect more closely.r Qyfr. We leave it as a somewhat lengthy. You are urged to traccl this program with several more strirrgs in L. b.lly a simple exterrsion of the previous exarnple. Thus. we can view a Tirring machine tra. At the end.we have to look at this er. The ideas userl to Exrr. uo the string aabb is accepted.t Tirring machines are rn()re powerful than pushdown arrtomata. a first indication th. Although conceptua.7 are easily carrierl over to this case. Since the primary purpose of a conrputrlr is to transform input into orrtput.lstate q7. a. Notice that even though {o"bn} is a txrntextfree language and {anb"c"} is not. writing the actual progralrr is tedious. I i Design a T\rrirrg macrhinethat accepts 7. we check that all original symbols have been rewritten.282 Chopter 9 Tunrr'rc. it acts as a transducer.
are T\rringcomputable. or just A function / with clomain D is uaid to be Turingcomputable tnachine M : (Q.ItoTunINc M. After the computation. f .$. so that the addition amounts to nothing more than the coalescing of the two .Plil. such as addition and arithmetic comparison.i'l$fimmir."nted by ur (r) e {1}+.rt tty looking at some simple operations.tcsItqn 233 inn. we will use unary notatiol in which aly positivc irrteger " i. all the common mitthematical firrrctions. F) computable if there exists some Ttuirrg such that QowluWf@t). (r) where q1 is a final state. no matter how cofirplicated.!l THn SrAFtrrA. As we will shortly claim. we first have to choclsesome convention for represerrting positive integers. constructing a prograrn for this is relatively simple' All we need to rlo is to move the separating 0 to the right end of u (g). ru (r + U) will be on the tape followed by a single 0. . with the readwrite head on the leftmost symbol of trr(z). design a T\rri1g machirre that computes nlA. srrch that (r)l lur : z. qo.1.' € l). For sirnplicity. eyeF.E. We sta. we thcrefore want to design a Ttrring machine for performing the compltation qour 0t.*pr". Given two pgsitive integers l' and y.t. (s)i qp (*+ s) 0. and the reaclwrite head will be positioned at the left end of the result. we rnust also cler:idehow r and 3r are placed orr the tape irritially and how their sum is to appear at thc end of the ctlrttputation. for all z. d. We will assume that ro (r) and tu (37)are on the tape in unary rrotation' separated by a single 0.
.1 . with Q : { s o . The resulting programs ara much shorter and simpler than if we had used another representation.l (s. 0 . To achieve this. a fact that is remembered by putting the machirrc into state q1. is needed to remove this at the end of the r:omputation.1.chine that copiesstrings of I's.?0. 1. : (qr. 5( q z . q n 1 1 1 0 1F 1 q 6 1 1 0 1l 1 1 l q s 1 0 1 1 1 1 1 q 0 0 1 1 1 * F F 1 1 1 l q 1 1 F 1 1 1 1 1 q 1F 1 1 1 1 1 1 9 1 t r 1 F 1 1 1 1 1 q 2F 1 1 1 1 q 3 1 0 1 i qstrl11110 qa111110.1) : (qo. ft) . q z t q J ) q 4 } . 1 ): ( r 1 r . tr) tr. d. qs. d (Ss. ) . other basic operations are copying strings and simple comparisons. 1) : (r1:1. Exotnple fl"10 Designa Thring ma.1 ) : ( r l l .X. d ( q r . we construct M : (Q. Z . F : {qn}. These can also be done easily on a Ttuing machirre. I Adding numbers is one of the furrdamental operations of any comprrter.0.i (. E. for any ?u e {1}* . R d ( q r . F). such as binary or decinnl.234 Chopter 9 'Itltrrruc MncnrNns strirrgs.i q. Note that in moving the 0 right we temporrrrily create an extra l. . [ ) . L ) . L ) . one that plays a part in the synthesis of more complicatecl instructions. : (s+.1 ) : k t t .. although cumbersorrrc for practical computations. F utrary notation. ft) . iB) This can be seen from the sequen(re instantaneous descriptions for adding of 1 1 1t o 1 1 . The transition d (qr. 1 . l. ft) . 0) d ( S r .t r ) : ( q z . find a machinethat perftlrms the corrrputation tyl'. More preciselv. . is very convenierrt f'or programrrring T\rring machines.j.q t ..1ww.
n d ( S z .t Iirst. we irnplemcnt thc fblklwirtg itttuitive process: 1 . t r ) .t will halt irr ar. 1 . Clorrir.1. Il) . a.nd that Iirral state q.1 ) : ( q 1 . 1) : (gz. tr) where q3 is the only final stirtc. .D.1r. ft) . ryprr' + if n Z A.r7 two positive integers represented in utrary notation. . z ): ( q 2 .rr (r)0.Hrrue 235 To solve the problem. f t ) . d (so. Tlavel to l. the mat:hint: is to perform the computation (z)0. r. 4. (E) t qrut(r) 0. . l ( q r . Rtllllaclc:c:very 1 lry an r. t r ) . Firrd thc riglrtrnost r and replace it with l. More specilically. Tlrc c:orrrputation perfbrmeldin this case is qsll F rrgeI F tru(en I rqln F r l g 2 n F r q 1 1 1F q 1 r l 1 F 1q211 11s21 111q2tr F F F 119i11 lq111F F s 11 1 1 1F q 1 t r 1 1 1 1 q 3 1 1 1 1 . ft) . (E)  q.5(Sr.l (. (y) qn. (y) .teqn iI n { g. A T\rrirrg rnachine version of this is d (qu. if r < A.9. . n) !. )( S r . will lralt in a nonlinal sta.1 .1 THn Sremoann TuRrr'rcMrc. ) : ( r 1 r1 . . Tlfs trtay be a little hard to see a. . R.et r a. : (gr. struct a TLrring urachine tha. r) .c1lr:irt Stcps 2 and 3 until there are no more tr's. if r j 91. 3 .ce the pr()grailr with Lhe sirnple string 11.w (r) Oru .he right end of the current nonblank regiorr arrd create a 1 there.rrd.l) = ({0. F T trt: I. so let us tra. : (q3.
whrlrc of alternate instrut:tion strea. we match each l on the left of the dividing 0 with tlrr: I on the right. aaabbbb.. signal to enter the state gr. Insteacl of rnatr:hing tz's and b's.m]m. Tltis t:xample ma.kesthe irnporlant point tha.. 4.... In thc first case.rn rn. we carr rrrrethe idea.e minor rnodiflcations. The cornplt:tc progra. deperrdirrg whether u > E or A > fi. we will have on the taoe eithc:r tyr.m fbr this is straightforward and is left as arr exercisc. Design a Turing rnachirre with no morc than three states that acccpts the language L(a(al b)). Ihis can bc usecla.in Example 9.236 Chopter 9 Tunrug MecnrNns To solve this problem. ?. and shoulcl produce ar^s outprrt thc result of the rrrmprrtatiorr.b}.t a.. We use this to get into thc other state qr. deprlrrdirrgon the outcome of an arithmetic operation.n inititl. Such a simulator should accept as input the tlescription of any Ttrring tttacltirre.\rring rnachitre irt Example 9.ve heerr replaced. At the end of the matching. Assrrrnethat X: {a. we still find a 1 on the right when all I's on the lcft her.msare entered. Is thcrc any input for which the Thring trachirre irr Example 9. we encounter the blank rrt the right of the working space. Deterrrrirre wha.sa.. together with a.ke decisions based otr aritlrrntltic cnmparisons.7 goes into at irrfinite loop? .7 with sott.Thring machine carr bt: progralrurlc(l to tra.when we attempt to orr match arrotlrr:r 1.te machine'l ffi 3.21ltr.lllnn. t ** L.ge computers..l corrfiguration.7 does when preserrted witlr the inputs aba arcJ. Write a Turirrg rnachine simulator in sorne higherlevel programming larrgrrage.t the 'I. This kind of simplc dtxlision is comrnon in the rrrachirxrlangua.. Is it possi}rleto do this with a twogta. Irr the second case.
if ru : e.. write out d irr rrrrnplete detail. R) . (a) /(u):3a (u) / (t.i (qz. 6./ (r) : .e.C) .b.I THr.*e. } 0 } l (h) I: For each problerrr. 0. l .cceptsthe ianguage L:{tntr. Design a Turing machinc that fincls thc micldle of a string of cvcn lcngth. .b}. { q s } ) w i t h ii (qo. Q o .9. ft) .. 1}+. .oz'. with o1 € X.ire lunction .. the X.. d . 8.t:rue{a. L Construct a Ttrring uachinc to compute t. "( u ) : n 6 ( t a ) } (e) /.ins any syrnbol other than 1? 7..ir and 3r positive irrtegers represerrtetl irr rrnary.b}+}. 'r > a x<A .: n .. r r r> 1 } {flitbzn:n> I} ( g ) f l l:l { a ' " I ) " t L " b " : r r .i (Sr.a^cetu+t.tez.b) : (sr. Il) .} : { u : n .*.R) . { a n b " " s " + ' n .{. .a) : (qs.: o {tu: ltol is a rnultiple f 3} (. ! } .qt. then check your answers by tracing several test exarrrples. 'I\lring machine Sper:ifically._t. .zrrj where c g f 'I\rring machines 11.i (qz.: lrrrl is even} ffi (") ..u. b) : (sr. } 0 .i S'rAlrr^.rc MaclHrlrn 237 5. What language is accepted by the rnachine M { a .nrr TuRrr.b .t) .qt}.. : (q2. (*) r:Z(obab) ffi (b) f : {rr. b) d (qr . b}.qz.. n) : (gs. Construct l\ring machines that will accept the fbllowing lirnguagestxr {o. b. Design to corlpute the following functions for . whereur € {0. What happens it Exarnple !1. : ({qo. d (r70.I0 if the string u conta.a) : (qr .r.1) I : {e'6"''n}I. .. (f) I. 10. {ffi slrorrltlprorlrrc:e fi:ataz. Design a T\rring machine that a.e.a.E. 'a): x '!. b.nf f m. a..
9 we had decided to represent r and gr in birrary.238 Chopier 9 Tunnrc MAcHTNEs (") f ( n .'s will ust: wc . in digital complrters. u ): 2 n . Design a T\rring machine with f : {0. n} that.10 goes through when presented with the it4lut 111.l 3 a ( d ) f (") : E. To demonstrate how'I\rring machines can be cornbined.Since. high level. la. Sketch how Exarnple 9.ctua. What happens when this machine is started with 1lt) on its tape? 1 5 . whereLfJ denotes cqualto f. We start with a highlevel description. ffi C o m b i n i n g u r i n gM o c h i n e s o r T f Complicoted osks T We have shown explicitly how some important operations fbund in all computers can be done otr a Thrirrg rna. Write a T[rrirrg rnac]rirre progranr fbr doing the indicated cornputation in this representation. Write out a complcte solution for Example 9.tiorrs carr also bc put together on a Thring machine. when startcd on any cell containing a blank or a 1.10 does in fact carry out the indicated t:ornptrtatiorr. Modify the definition so that languages that contain A mav be accepted. 1. block diagrams or pserrdocodeare the two alpproir.9 trlrltl be solvetl if r and g/ were represented iu decirnal.chine. Is it generally true that for any Thring rnachine. largest integer less tltarr or t2. Give the sequence of instantaneous descriptions that the Turirrg rnachirre in Example 9. Give convincing arguments that the T\rring mar:hirreirr Exarnple 9.chines$evcral wirys irt ir. Cornplete all the details irt Exatnple 9. lct us seer how thcstr basit: opura. there exists another one with only one final state that acccpts the same language? ffi 2 0 . 1"6. Definition 9. 1 8 .11.nguage then refine it successivelyr. we follow a practice common in prograrnming. strt:h primitive operations rrre the brriklirrg blot:ks for mclrc c:orrtplcxirrstructions. We can describe T\rring ma. L4. ffi 1 3 . 1 9 .? excludes the empty string frorn arry language ac:ceptedby a Ttrrirrg rrrachine.8.rntil the program is in the a. You may have noticed that all the examples in this scction had only one final state.l with which we are working.r:hr. Supposc that in Exarnple 9. if t is evert r r f f 1 $ (rdd rtl (e) / ( t ) : r m o d 5 (f) the /(*) : Lfrl. will halt if and only if its tape has a (l sornewhere on it. 1 7 .
5 J'@.9. What. with states indexed with . For the sake of discussiorr. The diagram shows that we first use a comparing nrachirre.11. for example is meant by saying that the conrparer serrds a start signal to the adder? There is nothing irr Definition 9. we will often use such highlevel.5. we encapsule computatiorrs in boxes whose function is described.11.y):nt'!t . asrrume that r and E are positive integers in unary representation. we implicitly claim thai they r:nn actually be constructed.4. arr erasirrgprogram is started that changesevery 1 to a blank.2 Cor"lerNlr'rc TURIr'tcMacnrNns pon.start signal to the adder. (E)  qa.rpl. In subsequent discussion$. using a Ttrring machine having states irrdexed with C. .ow(r) 0.11. blackdiagram representations of Thrirrg rnachines. Before we accept this highlevel view.9.0.. Nevertheless.9 and 9. The cornputation of / (r. gr)can be visualized at a high level hy means of the diagrarn in Figure 9. but whose interior details are not slrown. By using such boxes.t) rnost frt:qrrently in subsequent discussions. As a first example. For the eraser E. the cornparer serndsrr. which therr computes n*y.1 that offers that possibilitv. The computations to be done by C are qa. It is certainly quicker and clearer than the corresponding extensive set of d's.mple 9. For the adder. with the rest of the tapc blank. it can be done in a straightforward way. wc muut justiSr it. If n < 31. if x < s.IcArnn Tasxs 239 Figure 9.gur \ut (y) I nop*(r)Oru(s) . If not. The program for the cornparer d is written a. ifnlt!. we corr$tnrct a Ttrring machine having states indexed with E. qc. The value zero will be representedby 0. to determine whether or not r> y. suggested in Example 9. If so. we cornbine the machines in Exarnples9.ow(z)Oto (g). Design a Turing machirre that cxrmputes the function f(n. we u$e the idea in Example 9. In a block diagram.like that in Exa. Corr. (r) arrd + if r'P a.
1. While this description is not usahle on the cornputer.0 and qp. We first defi. for all c € f.a): (qio.c. it is to go into state {i without changing the tir. with the following interpretation.t o ] . s .b) : (qftg.rwcMAcHrNos If we take {. o t( n ) 0 . To implement this macroinstnrction requires several relatively obviorrs 'f\ring steps of a machine.R) d(qr. pseudocodeis a way of outlining a computation usirrg descriptive phrases whose meaning we claim to understand. f o r a l l q a€ Q a n d a l l b € f . E a^s T Anotlrt:r useful. If the Thring maclflle reads an a.t the relevant lowlevel code is substituted for each occurrence of the ma.a.ne the macroinstnrction in terms of tlte lower level language. One simple kind of pseudocode is exentplified by the idea of a macroinstmction. The computations performed by the adder will bt: ( u t t . we assume that we can translate it into the appropriate language when needed. L) d (qon.c) : (qi. 6(qt. we see that C starts eithcr A or . and that of the erraserE will be q n .pc content or moving the readwrite head. ( y ) i q r . the machine is to go into state 96 without changitrg anything. which is a singlestatement shorthand for a sequence ()f lov/er level statements.L) c) for all qt€Q.240 Chopter 9 Tun. In computer programming.b. This idea is very useful irr Ttrring machine programming. Thc states qio and {6s are new states. respectively. n ) O r ( g ) i q o .4 and E. : (qp.c.8.g as t}re initial states of . If tha syrnbol read is not an a.5. u The result is a single Thring machine that comhines tlrc actiotr of C. A. o w. y r u ( r * u ) 0 . We then use the macroinstructiorr irr a progralrr with the assumption tha. introduced to take care of complications arising from the fact that in a standard T\rrirrg machine the readwrite .ft) d (gio. Consider the macroinstruction i/ a then Qi else q*.t:rtr instruction. for all c e l. and indicated in Figure 9. r o . then regardless of its current state. highJevel view of T[uing1rnac]rirresis otre itrvolving pseudocode.
w(l rmurt be ir.croinstruction. For example.. where A will expect to firxl it.debefore entering the drlsirul statc qi or q4. A may call B frorn state {i.rerlrrNc Tunrr'rc MAcrrrNrils FoR CoMpllcAruu T. we would like to resurne prograrrr A itt sta"teq. B will use 7 to find its input. we carr divide the tape irrto several regiotrsas showrr in Figurtl 9..6 Region separator Workspace forr{ Workspace for. it writes the irrfrrrurirtionneededby B (e.9.ssinforrnation from A to B ancl vice versa. To solve the control transfer problern. To solve this.sxs 241 head changesposition in each rnove.with the readwrite head (which may have rnoved during B's operatiorr) irr its originrrl pla. we (:an replace macroinstructions with sutrprtr grarIIS.q. let rrs outline briefly how a 'I\rring rnachine carr bc used as a subprograrrr that t:arr bt: irrvoked repeatedly by another Thring machine. At other times. .When B is linished.ppensin a rerr. Irr this wiry. rr.ce. itt w}tich c:itscc:rlrrtrol shor.. aucl to assurethat the tetttporarily suspcndcd comprrtations of . Norrnillly.2 Corr. we want to change the state. Before A calls B.B .teof B. We let the head mrrve right. the argunents for B) on thc: tirpr: in some regicln 7. milcroinstruction is replaced by actual corlc a.bleto pa. the two programs can interact in the required fashion. Figtue 9. but put the machine into a stirtc (Ii1 $ qko. Note that this is very sittrilar to what actuirlly htr. The workspaur ftrr B is seprlrtlte fiom 7 and from the workspace for A. but they can also be used with TLrring rnac:hirr:s.4's current state. Subprograms are fundarnerrtal to highlcvcl prograrrrrning languages. but leave Lhereadwritt: hcir.rldreturn to this state.lrxrmputer when a subprograrn is called. Irr the ma. After transfer.ch occurrerrcci wlrcreiul er. say rnachine A in state q.' subprogram.llingprogram's configuration so the configuratiorr carr br: rtx:rcilterd on returtr frorn tlN. To rnake this plrrusible.t ctr.g.rl then pa*sses control to B by making a trarrsitiorr to the start sta.4 are not afftx:ted by the execrrtion of /1. I Goirrg a stt:p firrthtlr. Wlxlrr B is finished. so no irrterfertlrr(lc (jirn or:<:rrr.6.. subprogram is a single piece of code that is irrvokcxl repcirtt:dly whenever needed. . be able to r(l(:r{lntcA'u cxrnfigurationwhen it recovers control from B. Tliis indicates that a lefb move mrrst be ma. invokes rrtircltint: B. Ihis requires a new featurc: ther ability to store infornration on tlrt: r:a. it will return relevant results to region T.dwhere it is.
Design a Ttrring machine that multiplies two positive integers irr unary notation. while rather primitive in principle. the idea is simple enough that there should be no doubt that it can be dorre. Let u.n actual Ttrring machine program. provided that we know (in theory at least) how to translate this pseudocode into 8.7. 1. can be combined irr rnatry ways to make them quite powerful.Chopter 9 Tururuc MnctIIwns Figure 9. Our examples were not general a"rrd detailed enough for us to claim that we have proved anything. tiplicatiorr can then be visualized as a repeated copying of the rnultiplicand g for each 1 in the multiplier r. Find a 1 in r and replace it with atrother symbol a. Write out the complete solution to Example 9. Although this pseudocode is sketchy. Replace the leftmost 0 by 0y. I In spite of the descriptive nature of these examples' it is not too farfetched to conjecture that Ttrring machines. 2. whereby the string g is added the appropriate number of times to the partiallv cornputed product' The following pseudocodeshows the main steps of tlte process.14. .7 v We can now program T\uirtg rrrachinesin pseudocode. Replace all a's with I's.'Ja$rJlrrne The process of mulfinal tape contetrts are to be a. Repeat the following steps until r corrtairrs tto more I's. 1. brrt it should be plausible at this point that Tirring machines can do some quite complicated things.sindicated in Figure 9. A rrrultiplication machine can be constructed bv combining the ideas that the initial and we encourttered in adding and copying.
copiers.ruturt} {*r..qi).a. 'rrrrg {1}+ by f (wr. Establish a conventiorr for representing positive an{ legative integers in unary rrotation. with your conventiorr. (u) Z: (b) t: {.11. Therr use them for the solution.rc Tunnrc Mecumns FoR. or rnultipliers. rrketch the construction of a subtracter for t:omputing a .Provide a for T\rring rnachines that ar:cept the forIowing la'guages orr {a.. (b) /(n):r'1'. b}. (c)/(rz):2". 3 . where i is such that ltui j = max(lturl. > 1} (e) f : {an : n. the machine is to go into state ql..wz. t wr # wz: lurl : l. draw block diagrams for Thring nrarhines that computc the functions ( a )/ ( " ) : n ( n a 1 ) . For each problern.qt. (e) /(n) : p'!. 8. use a bhck diagram to sketch the implementation of a function / defined for all w1. otherwise it is to go into state 4i. "highlevel" rlescription 5. (d) /(n):.: n?. : {u"bn' : m. fu3l) if no two tu have the sarrre length.9.r. define a set . 7. and i:0 othcrwise. which indicates that the rnachine is to sea.f appropriate macroinstructions that you feel are reasonably easy to implement. W . w2. then sketch a method for addirrg and subtracting such numbcrs.r'r:hits tape to the right of the current position for the first occurrcnce of the syrnbol a. Sketch the constructiotr of a Ttrrirrg machine that can pcrform the adclition antl multiplication of positivc intcgers x ard g given in the rrsual decimal notation. sutrtracters.rrrrl} ffi (c) The complement of the language in part (a) (d). is a prime nurrrtrer} 6.luzf . CoMplrcArun Tasxs Z4B 2 . If an a is encountered befrrre a blank. using adders. comparers. ffi for all positive integers z. suggest a method frrr represerrting rational numbers on a Ttrring rrrar:hine.ws) : i.3y. Give an implementation of the macroinstruction searchright (. 4 .2 CortelNlr.
whikr it takes very little imaginatiott or ingenr. this section).nslattt a block diagram or pseudocode into the t:orrtlsporrdirtg error prone' Tlrring rnachirreprogram. but nrore complex procert$es wtlll. Thc discussionalso ilIustrates how prirnitive opera.'rliglrt srrspect that a'ftrring tnachine begins to ilpproar:h ir.8. clescribableby block diagrarrls or pseudocode. But doing so is unpleasant and diuouglrt to be avoided if possible. ailtl 9.utoma. In Example 9. T\rriug nachitres are equal itr power lo a typical digital computer? Htlw trtiulrl wtl .cha rea.tionu cilrr lxl t:orntrirredto solve nore cotnplex T\rring rnttclltirtescan lre cotnposed.b} that a.ccepts 1 0 . the best we t:tlrnprotnise' To see how we might achieve can do is to rea.nding. Iixamples 9. Wt: rrow face a dilenuna: we want to t:la. subprogrrur filr arxlthrlr.bkl philosophical issue' such a comprclmise.n he brrilt this wiry. Use the rlacroinstructiotr searchright irr Exercise I to creatc a Thring machine program that replaces the syrrrbol immediatcly to the left of the leftrrxrst a by a blank. Thesis Turing's 1flffiffitr Thc prtrcxling discussiou not only shown how a T\rrirrg rnachitre can be constnrr:ted from sinrpler parts. If the irrput cofltains no a.n section. irntl irdds Iittle to our understa. perform string rrarripulations. w(. Tht: irrstnrction set of a Turing mais <:hirrt: so restrictecl that any argument. in some $enst:. actually doing it is timtl t:orr$urnirrg. from the examplesin the prtrvitlrs We can rlrirw sornrlsirnple conclusions 'furing macllines appeal to be more pqwtrrful thir.sona. Somehow. soltrtitlrr. Ttrc first is that . stlt: Ext:rcise 2 at the etrd of pusltclowtr aul.10.rity to tra.:omment on this.I gram can act a. and how olle proproblems. wt: skctchcd the construction of a T[ring mat:hinc fbr a larrguage which is uot coutextfree ancl for which. To defilrrd such claitns against challenge. we sholld show the releva.11 s]row thal Tlrrilg machines can do sorne simple arithmetic optlrittittns. repla.ce the rightmost rrorrblaflk syrnbol byab.rrcl a way of carrying out a reasonably rigorous rlist:rrssiorrof T\rring machines without havilg to write lengthv.tyllir:irl t:ornputcr itr power. on E: {a. r:orrscqucrrtly. a.we would like to firrd tracting. Suppose we were to make tlle conjecture that. but also illrrutrat()sa llegative aspect of working with such lowlevel a.o[tata (for a.nt progrirm$ cxplicitly. how sever.244 Chopter 9 TunIwc MAcHINEs Use the macroinstruction in the previous exercise to design a Ttrring rnacltirte the language L(abab"a).r.ta. or proof for a nontrivial probletn is quite tedious. There is unfortutlately no completely satisfactqry way qf gt:ttirrg out of the predicament.im thilt T\rrirrg lllacllines can pcrfrrrrrt ttot only the sirnple operations fbr which wc have provided explicit ils prograllls. attcl tnake sotne simple compilrisons. 9. lro puslrdown au[omaton exists.wo ttrrrr to a sornewhat. lowltlvtll code.sa.9. Since very complex operations ca.
chinelanguage instruction set of a specrific computer and design a Tlrring machine that can perform all the instrr. but the evidence in its firvor is vr:ry strrlrrg. we raise the clucstiorras tti whcther this definition is sufficiently broad.lcornputer" arrrl tha. 3 .cceptingthe Ttrring thesis as the definition of a rnechanical comDrrtation' 1 . Some rlrgrrments for a.t r:.3 TuRrNc'sTnnsrs 245 def'endor refute such a hypotlxrsis'l To def'end it. Is it farreaching enough to covcr' cvcrythirrg wr) now do (a. If this were possible.t we have no lneans for rnaking a precistl rlcfirrition. we would hrrvr:trl define precisely the term . M.'ha. we crrrr *lsr approa.ndleave us no further aheacl than before.n$ can be perfbrtrr:d by some T\rring rna.rrexa.mple. wrr r'ight try lo find sorne proctxhrru filr which we ca. we might also takc the ma.sis rejecting the hypothesis.ation is tha.s been a.. This hypoihetis states tha. But rlo one has yct lrr:crr f'rrr able to prorluctl a r:ountt.5llcce. fact that all such tries have beerr the utrsuccessfulrnust btt takttn as circunrstantial evidence that it cannot be done.so it is important to keep in rnirrrl what Thrirrg's thesis is. If ue take this attitude attd regat'd the TLrlirrg thcsis sitrply as ir dclinitiotr. .chine.9. but frrr which we can show thirt rxr T\rrirrg ma.rctionsin the set. Arrythi'g tha.n a.sweeping sta. lrToorre. Every irrrlic. while (lvery . but it ought to be possible in prirrciplc if orrr hypothesis is correct.t Ttrring machines are in principle as powerfirl a"sarry corrrputer.r:hthe problem from the other side.l cligita. Arguments of this type lul A. Ihe 'rhring thesis is rrrorc protrxrrlyvierweda. Tlxr dirlir:rrlty lies in the fact thir.t we don't know exactly what is meant by "a typictr.definition of what constitutes a mechanical cornputirtiorr: ir cxlrnllrtatiorr is mechanical if and only if it can be performed by sorne Turirrg rnachirrc. Still. solvable by what we yet irrtuitivtly rxrnsidcr ar.nwrite a computer prograrrr." This would requirc sorne clthcr irllstrar:t model tr.$s this directiott would strcngthcn our convictjon of the truth irr rlf tIrc hypothesis.lgorithm.sa. Alternative moclels have beel proposed for nrechanical ctrrnputation.mechanicalrneans. Thin iu rr.n be done or any existing digital computer can also be dotre by a T\rrirrg rnir:hirrt:..bleto suggest a problern. It is not something that can be proved.nd r:onceivablv tright do in the future) with computers? An unequivocill "yes" is rxrt possible. for which a Thring rnachine program canrtot llr: writtcrr. brrt none of them are tnor:epowerful than the T\rring rnachine model.chinecan exist. Ttrring and others in the midlg30's to the celebrated coqiecture called the T\rring thesis. we could take a seqrrerrcc 6f increasingly more diflicult problerns arrd show how they are solvecl by sorne Tlrring machine. Tb do so.t any cornputation lhat can be carricrl orrt lly nxrchanical mea. it would not lead to a proof. we would hirvt: ir lrtr.tement. 2 . This would rrndoubtedly tax our patience.
ntly." or "thcr(l prove rigOrously sUch claims as tlxlllit:itly irrr algoritlrrrr for . The conclusions we draw f'rom it agrt:e with wltat we ktrow alrout real computers' and so far' all irtttrrrrpts to invalidate it have failed. MacHhrns Tlx. Ttring ma. repea. There is always the possibility that sorrreorrewill come up with a. thc likelihood of this happenilg seems to be very srnnll. we are in a positiou to give a precise rlcfirritirlrr of art algorithttt. To avoid prosprx:t$. t0 trurrstrtt<:t is rro algorithm. so we have some reason for it c:onsiclcrirrg a basic law of computer scieuce. In solne sense) Tbring's thesis pla. for example. Irr spite of its plausibility.tirlrrfclr TUringts thesis.chine progrirm tilIows us ttr "there exists a. Thring's thesis is still itn alJsrrmptiorr. and Thring's thesis ctr.l result contradicts a conclusion based qn thc laws. Specifically.tions. Slch litws t:arrrrot be proved to be true. which given as input any d € D on its tape. Hirvirrg accepted T\rritrg's thesis. Classical physics.w fiom thcrn irgrtlc with our experietrce ancl our ohserva.rse irrguulellts ale circurnstantial.q1 frlr all rJ € D.rrythirrgw() (titrr tl6 orr arrv txrrnputer carr also be dotre otr a Tirring tnachine.str. even relativelV simple problems is el vcrv lcrrgthy urrdertakirtg. This is the sitrra. lM" Arr algorithm for a function f : D + ft is a Ttrring machine M.1) e F. sorne Of t)trr $trllsctlttent discussions would have to be modifled signifir:ir.ihrrc to invalidate a Iaw stretrgthens our confideuce in it. rather. Howc)vcr. .R on its tape. Itr such atr eventu6lity. Iclentifying an algorithm with a. although they can possihly bc invalidatecl.n algorithm .nother definition that will accotrnt fttr somtl subtle situations not covered by T\rrirrg rnitchirrcs lnt wlfch still fall within the range of our intuitivc notiorr of rrrechatrical cotrtputatiotr.ys tlrt) rltrrrrt: role in cxrrnputer science as do the basis laws of physics and chemistry.vekrgical n(xx)ltitity.nnot trtl provrxl by therrr. wr: t:ilrr irllltc:itl to T\rrirrgts tltesis atrd clain that Sur:hunpletr. On the other hancl. we might begin to question their validity.t explain mur:h of tlrt: physical world. We accept them because the conclusions we dra. is based largelv on Newton'$ Ia'wsof r'cttitttr' Although we call them laws.t qodl nr . But viewitrg Turing's tllesis simply a"san arbitrary defirritigrr rlis$gs an impottant poinL.we can require thtr. they do not htr.terl fir. If an exPerimellta. eventually halts with the correct answer / (d) e ." However.UI (. tltey are plausible models tha.rrt ir. .246 Chopter 9 Turrtuc.
There are a number of enjoyable articles on Turirrg machines in the popular Iiterature. Skctch how the various instnrr:tions in this set could be carried out by a Ttrring machine. we stated at one point that Tlrring machines appear to he more powerful than pushdown autorrrata. This would easethe burderr of cxhibiting a. and we are irr rlrrnger of claiming the existence of nonexistent algorithms..cAmerican. E.l program" is well defined.Ttring machinet' in Defirdti'rr 9.5. it woukl seem that we can actually clairn that a Tlrring machine is more powetful. we could substitute ttPascal program" for . a. What irnlrortant factor is not takerr into account in this argurnent'l W ** 3.clear verbal description" is not. In the above discussion. This greatly simplifies the discussion.entifi. But this da. then write a brief review of it.lly. Actrrtr.lgorithmsconsiderably.. Hopcroft. While "Pir. May 1984. and that we can give cclnclise descriptions for sorne rather c{lmplex processes.rnguage. A good orre is a paper in Sci. ** 1' Conrrider the set of trar:hine langua. we will go t)ne step further and accept vurtrirl descriptions or block diagrams as algorithms on the a"ssumptiotr that we could write a T[rring machine prograrrr frrr them if we were challenged to do so. 2. titled t(T\rring Machines. Since the tape of a Turing rnachine can always be made to bchave like a stack.9. ." This paper talks about the irleas we have introduced here and also gives some of the historical context in which the work of Turing and others was done.3 Turumc's Tunsrs Conseqrrtlntly.st.rr.nger is more than offset by the fact that we can keep the discussion simple arrd intuitively clear. .swe have already done. but it obviously Ieaves us open to criticisrn. by J. Get a copy of this article and read it.ge instructions for a corrrputer of your choice. The reader who hirs any doubt of the validity of these clirims can dispel them by writing a suihrble program in some prograurrrrirrg l.
.
1 are possible. 249 . we exper:t that cromplicating the standard Thring rnachine try giving it a morc txrrnplex storage device will not have any cffrtct on the power of the automaton. If wc a<:r:erpt Tirring's thcsis. therefore. can be dorre by a sta.O t h e rM o d e l s f o Turing oc ines M h ur definition of a standard Tirring rnachine is not the only possible one. more complicated models. there are alternative definitions that could serve equally well. in a sensewe will define. if for no other teasoll than that arr explicit derrrorrstratiorr of the expected result will demonstrate the power of the T\rring machine and thereby increase our confidence in Ttrring's thesis. For example?we can consider T\rring machines with more than one tape or with tapes that extend in several dimensions.ndard model. to other. Any computation that can be perforrrred orr surlh a n{}w arrarrgement vrill still fhll under the category of a rnechanical corrrputation arrd. showing that the standarcl T\_rring machine is equivalent.er we look at several variations. In this chapl. The conclusions we can draw about the power of a lfuring machine are largJely independent of the specific structure chosen for it. It is nevertheless instructive to study rrrore complex models. Many variations on the basic model of Derfinition 9.
re can rnake use of the tape only in a restricted way. If f'trr everv tttrtomatort M1 in C1 there . these difference$ maY rrot have atry interestiug consequences.mily of rcgular larrguages. These have quite different definitions. but that automata.250 Chopter l0 O'rsun Molnls or Turut'rc MecHtlrns We will c:ortsidervariants that will be useful in srtbscqucrrt discussious. we follovr the precedence established for nfa's ald dfa's and define equivalence with respect to the ability to at)rxrpt larrguages.t different tirnt:u.ny Whenever we chlnge a dtrfinition. tigate whether these changes make a. in prepa. Extrapolating from of this. orl ffiffiffi Minor Voriotions the Turing T Mochine heme irr We first consider some relatively mirxlr {:}rilnge:s Defitrition 9. we must carefully state what is to be nnderstood by this equivalcrrcc.t of one uachine executing "reprodifferent programs a. These ar.tir. we irrtroduce a trew type of autotnata and raise the qgestion wlrt:thcr thcse new automata are in any real sensedifferent frorn those we have alreacly encounterecl. we carr defirre equivaletrce or nonecluivalence for classes automata in gerleral. Wlrat do we mean bY ir. Atrother issue that is not irnmediately resolved by Ttrring's thesis is tha.ssof automa. This is unexpected.re elqrrivakrrrtirr the serrsethat they both are iclentifled exactly with the fa. grarnmable" or Finally.ndalntlthcr'l Although there rnay be clear differences in their clefinitions. We have seen tln exampltl Of this in tlte case of cleterministic and nondeterministic Iinite arrtornata.ration firr lrrtcr cltapters' we look at linear bounded T\rring rnacltirres tltat have atr infinite tape. since T\rring's thesis covrlruonly rnechanical computatious and does not tlddress the clever guessing implicit in nondctt:rrnirrisrrt. of Equivolence Closses Aulomoto of Whenever we define equivalence for two automata or classes Of atrtornata.l clifferencebetweetr one cla.tr. Corrsider of two cla.sses automata C1 and dr. For the rest of this chapter. This leads to the idea of a "universal" T\tring rnilt:hirre.. we also look at nondeterministir: T\rrirrg rnachiiles and show that they are no more powerfrl tharr dclterrlilistic ones. but they a.ntlsst:ntiir. ilM Two automata are equivalent if they accept the same language.1 and invesdilltrrttntr: irr the general cotrcept.
but tltis does not affect anything if we can tell whitlh configuraltions of M a. Irr otht:r wrlrd$. There are ma. Turing Mqchines with q StoyOption Itr our clcfirrition of er. The r:onstruction of Theorem 2. ancl the two automata arc rxrivirlent.t err.tion M. We sav that another autorrraton ffi r:a.'s alndnfa's..}r qf t}rir.Hr. sta. Sometimesit is convenient to provide . we show that frlr t:vury mar:hine in one cla.nsimulate a corrrputrr.ton.o. we say that rt . . srr{.rrlrl have clone. thcrr uratters catr bc arrarrgrld so tha.rereleva. VARTATTONS .fr what M w. If M can sirnulate erely cotrprrtation of M. we often use the inrportant ter:hnique of simulation.the simulation is proper. If the converse also holds anrl fbr every M2 in C2 tlrcre is arr M in (J1 such that L (Mr) : L (Mr). rt should be clear that if M can sirnulate M. &r€ instantaneous descriptiorrs. of M may involve scvt:ral move$or ff.ted with a unique configuratiorr of M. wlxrre d0..testhis computation if it carries out a r:orrrllutir.d1.ny ways to establish thc equivalence of automata.0  Aadl Therr M siurula.nt.n determine from it exactly wlnt computations . tha. wc siry that C1 and Cz are equivalerrt..1 Mn'ron. For demonstrating ttrc equivalence in connection with T[rrirrgts machines.tion.\.t oN TuRrrqc Mec.un mimic the computatiorr of M irr the of following rrrarutlr.d1. sirnrrlate M.ch them is associa..ccept the same language.ss. given the corresponding starting configura. he the sequence of instantarreous dr:scriptions of the comprrtatiorr of M.there is a machine iu the second class capirtllc:of sirnrilatirrg it.he simulatiorr of ir singk: move rJi ts*d. we say that Cz is at lur^st as powerful as C1. the readwrite heacl rnust tnove either to thc riglrt or to tlrc lefl. Lt+t ico.. if we know the cotnputatiorr r.a..Hluo THnlrn 25L is an autorrraton M2 in C2 srrch that L (Mt) : L (Artz) . if ffi . Note that t.2 does this for clfir. the intermecliateconfigurations in 4 io 4*r rnay not corresporrd to arry r:onfigurrrtiong! M.t is d.t M ancl fr a.r.10. As long as we can deterrnine from the cornputatiorr of .tion analogous to that of M. To demonstrate the equivalenceof two classesof autotnata.+.rriedorrt hy M.ndard rhring machine.4 woukl havr: done. we ca. Let M btl an rrrrtomir.
F ).{o. If the move of M does not irrvolve the stayoption.5} with the interpretation that 5 signilies rlo IIIoveIIIeItt of the readwrite head.) . b .262 Chopier l0 O'r'ntn Montls oF TURINGMacntnns a third option. t . it is obvious that arry starrtlard T\rring machine can be simulated hy one with a stayttptiorr.nd F. to have the readwrite hcad stay itr place after rewriting the cell content. Frrr cach Stransitiotr 6 ( q r .cJ : .tr. h .X.a ) : ( qj . we ca.1. c.chtransition d ( q r . . fbrallcel.n.f.IetM : (8.R.rlateM. so that M can simr. b . o )( i j .^ 0 ((. Ieaving the tape t:ontents urtaltc:rcd. we put into f E(A. t}te simulating machine M does \ / the following. L o r R ) .) . \Qi . lhen M will make two moves: the first rewrites the syrnbol artd rttoves the readwrite heacl right. u ): ( q i .S ) . fio. be a TlrringtnaTo show thtt txtrrverse. It is reastlnallly otrvious that every cornputaLion of M llas a corresponding computation of M.d.F) chine with stayoption to be sirmrlated by a statrdard Ttrring machine M : ^\ //^ {O. d.tr or rt) . Prooft Since a Ttrring machine with stayoption is <:ltlarly iln extension of the standard model.1 by d:Qxf8xfx{tr.. esscntially identical to the move to be simulated.n define a Tiuing rrtachitre with stayoption by replacing ci in De{initittn 9. f . For each rntlvc of M. The class of Ttrring rnachines with stayoption is eqqivtrlent to the class of standard I\rring rnachines.b.. This option does not extcrxl the power of the automaton. the secorrtl rrrovesthe readwrite head left. a. ' f . Thus. tve put into E the corresponding transitions : E ( a .. the simllating machine perfbrms one nl()vc. r. The sinulating machine can be constructed by M hv defining li as fcrllows: For ea. If S is involvtxl in tlter rnove of M. R ) . .. = (Qj .
It is ttot difficult to see that this restriction does not affect the power of tlrtl tnachine. In thc picture.rd. we ufJetht: arrangenrent showtr in Figrrre 10. I{ere wc look at two srrctr trtodels.hrrt use o11e with a ta. we hervt:divided each cell of the tape into threN:parts.sedon this visualization.1). we make one rcrrtark otr the sttr.chuynrbol is composed of several parts.as shown in thc above theorem. in whic:h tlte tape symbols are triplets fiom sorrtesimpler alphabet. sitrce all we need to do is rnake I au a'lphir'bt:tirr which ea. calltxl tracks. to talk about the proces$precisely and prove thetlrtlrns discussiolr. Ba.1 (Figure 10.t a.pe svrlbol can be a compositc of characters rtltlrt:r than just a singlt: tlrre.1 as standa.rrguage in pserrdtltrrtle. artd the forma. Befrtrcl irrtroducing other rntldels.1.nda. 'fhring machitre urodels involvc a change of defirrition.ndir. are often crrrlllersonte.rr made rigorous.1 MrNoR V. We can visualize this a. hrrt such a view in no way extends Definitiott 9. Some variarrts tha. each crontainitrgotte memhtlr of the triplet. we keep ortr disc:rrssiou rather thtr. such an arrtomaton is sometirrrescalled a T[rring tttachine with multiple tracks. t'his Tlrring nrachine is otherwist: identical to otrr statrdarcl nrodel.rre9. which are sometime$ us(xl as the standartl defilition.rtlThring machirrc M by a machirre fr with a semiinfinitr: tape.1 that each ta.pc that is unbortndtxl only in one dircctiotr.we use tht: rrotiotr of sitnuabout equivalenr:c.but we generally mahe tto attempt to dcscribe everything in a rigortltrs and detailed wtr. To avttid this.reklss cottttnotr are explclred in the exerc:iscs at the end of this section.rd Ttrrirrg rnachine. bgt it should not he trirld to see how they r.1 Track1 Track2 Tiack 3 b t Simulation is a standard techniqut: for showing the equivalence of automata.2).chirttls descriptive. It is implicit in Definition 9. The simulations artt givetr only iu brotld be outlirre. This can be macle morc explicit by drnwirrg att expanded versiort of Figr. . The reader will find it instnrctive to sketch each simulatiotr in some highcr or level la. To sirrtulale a sta.3.y.In our suhsequernt latiol freqrrently.sa tapc that has a left bourtdary (Figure 10.tRIarIoNs oN THt TLIRINGMncHIun THntrae 25S Figure 10.a.lismwt: have described makes it possible.Complete simulatiorrs with'I\ring ma. otlter ther txluivaletrce with thc stattdard machint: ltas to be demonstrated. Tope with Semi'lnfinite Turing Mqchines Many authors do not consider thtr rnodel in Figlre 9.nirr theoremproof forrn. so Howcvcr.10. cxcept that no left move is permitttxl when the retrrlwrite head is at thc boundary.
Bcca. The simulatirrg machine will first rnove via tlic transition : (f ( E ( A . tlrc second to be usud on the lower one.use belongs Qu.re put on thc left boundary of the tape to facilitrrte switching frorn one trrlck to the othcr. the simulatirrg at machinesees(ff. @ . wlrerefi € Qu. c .slong as M's readwrite head is to the right of the refererrr:epoint.a):(qj. a$$rrmethat the rntr.chine has a tape with two tracks. and work on the lower track as M moves into thc left part of its tape.'d when workirrg otr tho upper track.ins the left part of M's tape in r{}verseorder. L ) . the position of the readwrite head at the start of the computation. special end markers ff a. Now.? Figure 10. say Qu and Q. we keep the irrfrlrmation to the right of some refercrrr:e point on M's ttrpe. only irrfrrrmation the rrpper to in fi trat:k is considered this point.L). Ref'ercncepoint /'' h a )q.chineto be simulated arrd the simulatirrg machine are irr the respective corrfigrrrations shown in Figure 10. On the upper M onli. r . For example. The distinction carr be made by partitioning the state set of M into two parts. b ) .4 (a) Mat:hine to bc simulaterJ. ff) Figure 10. for example.3  Treck 1 for risht nart of standard tepe" Tiack 2 for left part of stefldfid tape The simulating rnrr. The lower trirc:k conta. (b) Simulating machinc.c.: the flrst to be usr. { (a) .4 and that the move lo be simrrla. The refererrccpoint could be. M is programrned so that it will use information on the upper track only a.254 Chopter l0 Otnnn Monnls or TuHrNc Macnrruns Figure 10. ) )r .ted gerrcratedbv is d(qr.
F\rrtlter the rnachineis in a state from Q.c . # ) . If wc put the iuput fitc back into the picture. .1 we disr:arded the input file fbr reasons of simplicity. # ):) ( F i .nd details of the simulatiori are straightforward' Turing Mqchine TheOffLine The genera. tr.qn.6. what is currently re:ad from the inprrt file. A forrrral definition of an offline Ttuing machirre is easily made. .a ) : ( q i . We now expand on this claim. shownirr Figure 10.l )0t in state ti e Qu. ( # . A schematic representation of a'n offline machirrg is shown in Figrrre 10.1 Mrr. and what is seen try the readwritc head. each move is governed by the intt:rilal state. In Definition 9. L ) .lon V.pter I containcd an input file as wcll as temporary storage. but we will ltrave this as an exercise. ( # . claiming that this tnade no difference to the T\rring machinc concept.6 Readonlv input file . What we Figure 10. we get what is known as an offline Thring machine. putting it into the conliguration will work on the lower track. R ) .10.Now with fr € Q1. It rrcxt uses& transitiorr t ( t .Inuolts oN THtl TURIr'rc MnclHttrs Tsplrp 255 Figure 10. Itt such a machine.5.I tlefinition of an arrtomaton in Chir.5 Setltretceol conligurations in simulating d ( s r .
The syrnhol found in the cgrresponding cell on track 1 is remernbered by puttirrg the control unit of M i'to a state chosen for this purposo.6. The first track has the input. Then it carr proceed in the same v/ay as the standarrl machine..chmove of M requires a number of moves of _ffi. This irforrnation is again remembered by M with arr tlppropriate internal state.256 Chopter l0 Ornnn MoDFlr. Give a formal definition of a Ttrring rrrachine with a semiinfinite tape. of First. tlNr third representsthe ta. the second marks the position at which tho input is read. All that needs to be rlone by the simulating machine is to copy the input from thc input file to the tape. all four tracks of M's tgqe are modified to reflect the move of M. track 4 is searched for the position of the readwritc head of M. we rrow know that M is to do. Next.s rc:rrdwritehead. M scarches track 2 to lor:a.pe of M.ft end. and the fourth shows the position of M. Then .7 want to do briefly is to indicate why the classof offline Thrirrg machines is equivalent to the cla^ss standard machirrert. say tha le.s or. the readwrite head of M returns to the standard position for the simulation of the next move. 1. and with the relevant informatiorr rnarked by special end markers.7. Firrally.TuRnrcMACTTTNES Figure 10. Starting from some starrderrdposition. the behavior of any standard T\rring machine can bc simulated by some offline model. A standard machine can sirnulate the computation of an offline machirre try using the fourtrack arrangement slnwn in Figure 10. With the remembered input and the symbol on track !. The simulation of an offline rnachine M by a starrclarcl machine fr requircs a lengthier description. Next. Each of the frrrrr tracks of M plays a specific role in the simulatiorr. The simulation of ea. the tape contents shown represent the specific configuration of Figure 10.te the position at which the input file of M is read. In that picture.
rd T\rring machine. 6 . Suppose we make the requirement that a Ttrring marhine can halt only in a final state. Consider a version of the standa. This can be achieved by the restriction that if 6 (qr. 1 t . a) be defined for all pairs (g' a) with a €  and q t' F. the distance and direction of travel being one of the arguments of d.lso accepted by some standard machine.{a. tr or R) .q'cHIIIn 257 prove that the class of T\rring machines with semiinfinite tape is equivalent to the class of standard Turing machines. but also on the cells to the immediate right and left. then a and b must be different. Consider a Ttrring machine that. For examule 6 (qt. Suppose we make the restriction that a Tlrring machine must always write a symbol different from the one it reads.b.re made if the current tape svmbol is not one of a specified set.rd Ttrring machine in which transitions can depend not only on the cell directly under the readwrite head. that is.L or R). Give a precise definition of such an automaton and sketch a simrrlation of it by a standa. Consider a model of a Ttrring machine in which earh move permits the readwrite hearl to travel more than one cell to the left or right. that is.1 MINoR VlRhrIoNs THurae oN THE TuRINc M.R) . A nonerasing Tirring machine irr orre that cannot change a nonblank symbol to a blank. Consider a Ttrring machine that cannot write blarrks. b must be in f . E. but not both. Consider a T\rring machine with a different decision process in which transitions a.10. (a) Give a formal definition of such a machine' (b) Show that the cla^ss such machines is equivalent to the clags of of standartl T\rring machines. then a must be !. that is. Does this restrict the power of the Turing nrachine? 9 . Make a formal definition of such a machine. c.b. ffi i}. Give convincing argunrerrts that any language arcepted by an offline T\rring machine is d. Show that no generality is lost by making such a restriction. we ask that d (q. L or R) . s. on any particular mover cart either change the tape symbol or move the readwrite head.a) = (qj. Does this limitation automaton? ffi reduce the power of the 1 0 . b}) : (fi. then sketch its simulation by a standard T\rring machine.. Show how such a machine can simulate a standard Thring machine. if d (Sr. 4 .. ffi .a) : (qi .{tr}.o) = (qi. for all d (gn. Give a formal definition of arr offline Ttrring machine.
I.258 Chopter  0 Orrrnn Mor)ELS or.8). If n : r:urrent configuration $hown in Figure 10. But this is not the ca$(]rL$we now illustrate with two examples. gs.reas in Definition 0. we define an rztaperrrachine M : (Q. The formal delinition of a multitape T\rring machine goes beyond Definition 9. At the same time.n e.' TuRrr.F by a. E.E.8 Tapc 2 . where Q.1. Multitope Turing Mqchines A multitape Thring machine is a T\rring rrrar:hine with several tapes. L . For example.rc Mecruruns will allow the indicated movc if the currerrt tape symbol is neither a nor [r. Typically.l. R ) * is interpreted as follows. brrt where d:QxI"Qxlu x IL.1. F). then d ( q n .8. The transition rule carr be applied only if the rnachirrc is in state Qsand the first readwrite head seesarr a and the second a. with a specifies what lnppens on all the tapes. Forrrralize this conccpt and show that thisr modification is equivalent to a standard Turing rnachine. each with its own irrrlependently controlled readwrite head (Figure 10.n. l'he synrtrol on the first tape will then be replaced with an r and its readwrite head will move to the left. the svmbol on Figure 10. d. e ) : ( q r . . g0. ffi M T u r i n gM o c h i n e s i t h M o r e w C o m p l e xS t o r o g e The storagc device of a standard Thrirrg mrrchine is so simple that one might think it possible to gain power by using more complicated storage clevices.r .RI" 2. y . since it reqrrires a modified transition function.
that any standartl T\uing rnachine can he simulated by a multitap() orle. but corrceptually stra. M.q.9 o.ndartl Ttring machines. The simulatirrg singletape machirre will have f'ogr Figure 10. Cotrsider.2 Tunrt'tc.9. for example.10.yselect to run a. we arguc tltat any given multitelpe Thring machintl M cau be simrtlated hy a statrdard Ttrring trachine M and.iglrtftrrward. The second part of this clirirrr treedsno elabclrtrtion.tiorr of a machirrc with only one of its tapes doing useful work' mriltitrrpe machine by one with a single tape is a little more complicated.10.nges state to 91 and the machine goes irrto the new conliguration shown in Figure 10. Corr.cHIunswIrH MoRl. To show the equivalence between mrrltitape and sta.multitape 'Ihe simula. converscly. tr b Tapc I t v f Tape 2 the sccond tape is rewrittt+n as'g and the readwritc head moves right' The its control unit then chtr. since we can alwa. the twotape macldlle in the conliguratiotr depicted in Figlre 10.10 7 a lr t ) E "f h .tplnx Sronacn 259 Figure 10.
:1fl :illi.t a. there rs a. Wherr we rcat:h the end of tht: a.':i1?l..lHJil"'m.single 1 tnarkirrg the position of M's readwrite iread. exctlrt firr a.ll zeros.p[* ilCI. Wratever can trt: done of M can also be done on M.lso much the same.rticularly.1Tli:iJ'il1'#'JH. The represerrtir. Tfa.lHf.pernachine is similar to that ust:d in the sitttulation of a. We tlrcn read all t.n deternine wlxlther there are arr cqrral number of a's and b's without rcpeated backanrlforth movement of the readwrite head.l". the only differencebeing that thcrtl ilre more tapes to rxlnsicler. beginning of tlrc cornlltta. The nonblank part of the second track hrrs a. copying Lherrrrlrrto tape 2. The actual steps in the simulatiorr irre a.ponding configrrrirtion of M.nt to keep irr rnind the followirrg point. what la.n offline rnachine. Certainly.7. it takes a lot of writing.rre10. The fir:st track rcpresents the corrtcrrtsof ta.il'Hl pa.n initial strirrg e.11). wc c:ir.'fhe outlinn given for the simulation of offline tnachinescat:riesover ttl this casewith rrrirrrlrmodifications and suggests a. we clescribecl laborious thir.cks3 and 4 play a similar role for tirptr 2.tiou..11 makes it clcirr that.nguages be a. for the relevirnt txrnfigurations of M (that is.il'1il.ccepted.lJJi' t . While it is rrot difficult to nrake the corrstnrction precise.ge a {a"'b"}. In Exampleg. rnachines.11 trat:ks (Figr. When we c. can txqm. Figure 10.t'bttt is writtun on tape I at tlx.260 Chopier l0 Otrnn Moonls oF TLrRrNcMncnrrrps Figure 10.H':. thu computations of M given the appeara.*"'"IJ"ilili. It is irnporta. proceclure try wirich the transition function f "f fr can be constmr:tecJ fiom the transitirtn ftnction 6 of M. llrt this has no of beallng on the cottclusion.pe 1 of M.lte nts.'s.f M.we match tlrt: b's on tape 1 against the copied a's ou Lapc 2.. .l' Considertlte ltrngua.tionof a multitrrllc rnachine by a sirrgklta.rrr:e being lengthy and elaborattl..il#T:. This way.laim that l. the ones that have the indic:atedform).unique corrtli.
Ilrc MecHtut:s wIrH MoRn Colvtplnx SroRlGu 261 Remember that the various models of Thring ma.using some spt:c:ialsymbols showu itt thc picture' to delimit the fields. U .rrrlrrgetnettt.ttxl in Figurc 10. A ditlgrart1 of a. To simrrlate a move. T[e formal rJe:firritionof a. Mochines Turing Multidimensionol A multidirrtensiona.dclress.rrbe viewed as extcrrditrg infiniterly itr tnore tharr one dimensiotr. twodimensional Ttrring machitre involvt:s a transition function tI of the fbrm .chintl M and ttre readwritcl head of the simrrlatirrg ma.s that.mmirrg or auy othtlr efficiency mcasure we rnight consider.l a lumher of ways.10. not witlt respect to eilsc of progra. rcspectively. . 3) contains b is shown iil F'igure 10'13' Note can invcllve arbitrnrily large intcgers. in the twodimt:rrsioual ftr. the address track caunot we must rrstt a variable fieldsize a. This t:atr Lredone irr or acldrnsswith the r:clls of the twtrditneusiona. where [/ anrl D specify rnuvement of t]rc readwrits ]read up tr.12 Twodimensiond addressscheme .ck tape of the silrulating rnirchiue will use clne track to stortl cell conttlnts and the other one to kcep the a^rsociated a. First.shion indica.chinc ffi ut* u]{tw. Wc will return to this importtrrrt point in Chnpter 14.t:hinesare considered equivalerrt only with respect to their ability to do things.ck moclel depictcd in Figure 10. at tlre sta.12. L R . twtr dimensiotnl Turing rnachine is shown in Figure 10'12. we (iaII use the orr twotra.12.13.2) contains a. To simqlate this machitte on a stanrlrrrd't\rring machine. so orrclcornplica.tiorr:the cell a. In thc scheme of Figure 10.te ordering tape.tlrlress trse a fixedsizc field to sttlre addresstls. ancl cell (10.nd down. Instead.2 Turr. wc associa. fbr example. Thc twotra.I x I ' x { . the configrrration in whictr cell (1. oll corre$poncling cells. D } .the rcadLelt us assume write heaci of thrl twoclirnensional ma.l T\rrittg machine is otre in which the tape ca. ): Q x f . the simulating tlachine M first comprrtgs the addressof the tnll to Figure 10. tr.rt of the simulation of eat:hrrrove.
iu..in. unfbrtunately.chines is to lend cre'rhrirrg's dence to thesis by showing how seemingly rrrrlre complex situations cirn be sirnulated on a starrda. Only the counter symbol t:an be put on the stack or removed frorrr it. . a clueue. * 4. the number of rcquirecl states in Exercise b as far as you can (Hint: the smallest possible nurnber is ihree). Assume that sur:h a machifle is an onli[e machine. with the strirrg to be pror:essedplaced in thc queue prior to the start of the cornputation. describe the sirnrrlrrtions in just enough depth to show that the details can be worked out.rrnter is tr stack with an alpha. 'Ihen sh6w how such a r'achine ca' be simulated by a standartl r[rring rnachine.262 Chopter l0 Outtlrr MoDELSor. cach with a sing. given M. and the[ show how suc:h a machine t:an bc simulated with a standard Ttrring machine. In the exercises trelow. once tlrc trddressis computed. Give a formal definition of a rrlrltihead T\rring machine.tions are v()ry tedious irnd conceptually uninterr:sting. 'lot"R*.<lTrrring machirre can be visualized as a T\rring machirre with a single ta. a stack start symbol and a courrter symbol. trriu is a sitnple computation. ffi 2. Show that any T[rring machine carr be simulated using a counter autornaton with four counters.It which M is to rnove. The purpose of much of ortr discussiorr of Ttrring rna. 1.ightforwirrd construction fbr 44. A queue automaton is an a. Give a formal dcfinitiorr of a multihearlmultitape Turing rnachine. that is.utomatorr in which the temporary storage is . show how srrch a machine can bc sirrrulated wit.h a multitape rrrar:hine. then investigate its power in relation to Ttrring machines. irrdependent rearJwrite heads.ddressorr tra. 'I'uRrr*rc Mecrrrr{r:s Figure 10. A multihea.'nuional addrcss scheme.ck 2 and then changes the cell corrtents to account for tll rnove of M.. . it has rro input file. M Iinds the cell with this a. 3' Give a forrnal definition of a T\rring rnachine with a si'gle tape but multiple control units. Aga. using the twodirr*. detailed sirnuler. Give a formal definition of such an autorrraton.rd T\rrirrg machine. there is a stra.. ffi * 5' Sho* that for every il\rring tnat:hine there exists an equivalent standartl Ttrring uachirre with no rnore than six states.pe and a single control 'nit but with multiple. A counter automaton is a deterrrrinistic autorrraton with orre or more counters as storage.tr: * 7' A c.bet of exactly two symbols.le readwrite heatl.
b. We must look at the effect of noildeterminisrn itt more dertail if we wa.l (qo. (q2.1. The rloves qouaal bq$..c. the range of d is a set of possible transitions. R) .3 NONDETER. Since nortdeterminism involves an element of choice and so has a nonmechanistic flavor.1.c"hirre transitionsspecilied . L)} it is nondeterministic.function d:Qxf+2Qxrxtr'fi}. *fimrttplClq.& and qa1znal qzlcaa are both possible.a) : { (qr. I .mecannot be said of nondeterminism. showing that nondetermirristic behavior can be handled deterministit:ally. an appeal tg T[ring's thesis is inappropriate. atry of wlfch can be chosen by the machine. M N o n d e t e r m i n i s tTc r i n g o c h i n e s iu While Thring's thcsis makes it plausible that the specific tape structure is immaterial to the power 9f the T\rring machine.nt to argue that notrdeterminism adds nothing to the power of a Turing machine' Again we resort to simrrlation. show that everv cornputatiorr that can be done by a standard T\rring machine can be done by a multitape machine with a stayoption and at most two states. except that d is now a. Write out a detailecl prograrn for the computation in Example 10.MINISTIC IIURIIIIC MNCHII. A nondetermirtistic Thring rnachine is an lrrtomaton as given by Definition 9.lwayswhen nondetcrminism is involved. the sa.IES 263 g.I O. ffi 9. As a.il 6y has If a Thring ma.
the processcarrbe simulated by a deterministic Thring machirre.2 and its succ:e. Nondetermirrism can be vicwed as a deterministic backtrrrcking algorithm. To see hovr this can be done simply.te or to an infinite loop.rkersand so can always be found.ut i qq. these aJternatives are irrelevant. and a deterrninistic rrrar:hine can simulate a nondeterministic one as long as it can handle the bookkecping involved irr the backtrackirrg. A nondeterrninistic machin{} may have move$ available that Iead to a nonfinal $ta. whenever a new machine is to be created. we need to provide a dcterministic equivalent for tht: nondeterrninisrn. all wc ir.. the machine prt)duces as marry replicas as needed and givcs eaclt replica the task of carrying out one of the alternatives. activating new machines as needed. Each pair of horizontal tracks representsone machirrc. the hottom one for indicatirrg its internal state and the position of thc readwrite hczrd. To show that a nondeternrirristic Ttrring machine is no more powerful than a deterministic one. A nondetcrministic 'rhrirrg machine is sairl to accept u if there is any possibre sequence of moves such that t111. The simrilating machine searches aJl active tracksl they are bracketed with special rna. let us mrnsider an altcrnative view of nondeterminisrn. unlirnited replication is certainly not within the llower of presentday computers. Nevertheless. It then carries out the irrdicated moves.re interested in is the existerrce of some sequence of moves lcading to acceptance.14). We have already alluded to one.the top track conta.15 representsthe initial r:onfiguration of the machine in Example 10.reasonableoutline of the simulation. when more than orre rnove is possible. But. This view of rrondeterminisrn rnav seern partit:ularly nonmechanistic. as always with nondetcrminism. Quite a few dctails have to tre resolved befcrrewe can claim to have a.utomata are usually viewerl as accepters. but we will leave this to the reader.ruun Moonls or. Figure 10.r2.''IuRrrrc MACtrrNEts since it is not clear what role nontlcterminism plays i'computing function$.tions.14 Tape contents Internal state and position ofhead Tipe contents Internal state and position ofhead .$sor configurrr.264 Chopter l0 O. one which is ustlfirl in rnany arguments: a norrdrlterministic rnachine can be seen a$ one that has the ability to replicate itself whenever neces$ary. two new tracks are rrtarted with the appropriate information. nondeterninistic a. wiih q7 € F. Figure 10.ining the rrrachine'stape. Consider a Thring rnachine with a twodimensional tape (Figrrre 10.
b}+}. 2 . Discuss in detail the simrrlation of a nondeterministic Ttrring machine by a deterministic one. \ r U ' u t . Because of its irnportance. Contrast this with a deterministic solution. Write a simple pl'ogram for a nondeterministic the language 7 : {***"a i 'r1a1ue {a. 'I\rring machines and the class of nondetermirristic The class of determirristic 'I\rring machincs are equivalent. Itow artive machines are identified. Outline how one would write a program for a nondeterministic chine to arcept the langrrage L: Tltring ma  ( { . " 'a. W 4. Iated bv a deterrnirristic machine. Proof.TunInc Mncutt'lps 10." u : t r . Based on this simulation. Writc a progranr for a trondeterministic Ianguage T\rrirg machine that a. ) Ttrring machine that accepts 5. r I. we state this formally.bl+I. b}+ . . and how machines that halt are remove<l frorn further consideration.11'r Sirmrlation of a rrorrtletenninistic fitove. Use the: corrstruction suggested atrove to show that anY nondeterministic Ttrring rnachine can be simulated by a deterministic one.3 NoNDETERMINIsTIc 265 Figure 10. t €. Show how a twodimerrsional nondeterministic T\rring machine can be sirrru3. Irrdicate explicitly how new machines are created.ccepts the 7={ww:ue{a. our conclusion is that for every nondeterministic: Thring machine there exists an equivalent deterministic one. l"l } lgl} How would you solve this problenr deterministically? .
grr.1 is a special purpose cxrmprrter."L) might appear a.ffi Cortsider the fbllowing argurncrrt a. & 2 .x f .nyTtring rnachirre r:an be described cornpkrtcly with d only. etc..1 so that d : Q x (tU{A}) x l' x f * f i r r i t es u h s e t s f Q x l .. .. on the other hand. Similarly. a2 as 11. where a1 rcpresents the blank. Wc then select an encodirrg in which q1 is representedlry 1. .ton thal.s . uz) : (qz. without loss of gerrerality. qz is representedby 11. rI r with q1 thc irritial state. ir. A twostack autornaton is a nondeterministic puslxlown arrtomaton with two indepcndent stacks.universal T\ring machine.a3. . Desigtr a rrorrdeterministic Turing rnat:hirrethat accepts the language L: {a" : rr is rrot a prime uumber} .266 Chopter l0 OrHnn Monnls oF TrrHrNC. & r n } . . q2 the sirrgleIinal state. we modify Dcfinition 7. . string ?u. Consequently.chine is restricted to ca. and f : { a r.r.t MAcHTNES 6 . ffi * 7 . given ils input the descriptiorr of any T\rring machine Jl[ ir. assume that Q = qt. 1 1 .'s." This ohjection can be ovr:rrxlme by designirrg a rcprogrammahle Ttrring machirrc. is arr autclma. Once d is defitred.nd a. . vrith the arguments and result itt sornc prescribed sequencc. 1 0 1 1 0 1 1 01 0 1 0 . d(gt. For exa. and so on. a1 is encoded as 1.rrying out one partir:rrlrrr type of computation. Slrow that the class of twostar:k automata is equivalent to the class of T. Thring machines cannot be considered equivalent to gerrcral purposc digital computers. To construct such irn M. A rrnivt'rsill T\rring rnachine M. With the initial rrrrrl final state and the bltr.uring rrrachines.mple. T<r clefine such an autornaton.Qz. called a. tht: ma. Digital computers. arc guneral purpose machines that can be prograrnmed to do rliff'erent jobs at differerrt tirnes. .inst Thring's thesis: "A Ttrring machine as preuented in Definition 9.Q.rd way of descritring T\rring machines. The transition function is encoded actxrrding to this scherne.. o A movc dcpends orr the tops of the two stacks arrtl results in new values being pushed on these two stacks.nk defined by this corrvcntion. NN A Universol TuringMochine ffiffiffifif. . can simrrLlte the computatiorr of M on ro. we first choose a standa. . The symbol 0 will be rrsedas a separator btrtweenthe J. We rnay.
. but we can easily spot these. thirt cvtlry elertretrtof tire sel has some fitrite index. For infinite set$. sltowtt in l'igure l 0.4 A Lilrvllnsnl TuRrNc M. higher level la..44 Tape contents of ll4 It folklws frotr this thirt arry Thrirrg rrrachirrehas a fitrite ettcoditrg as a string trrr {0. A set is snirl to trc coutrtable if its eletnents with the positive integers.. Fclr cxilrnplt:.4. Wrr prcfer irrstead to appeal to hypothesis..Tz.ngua.lurriversal Turittg tnaciritre (see.t t]rc cxlrrtcrrts of ta. The in irnplerrxlrrtrltirlrr tlc:irrly catr be done usitrg solne programming la.. of far:t. 1}+.rrlrltititllt: milt:}rirrt:. thtl l)r()grilrrrsuggesteditr Exercise 1. the strong 00011). and Qualitz 1978).I)enrris.nd tapc 3 thc irrtertral state oI Il[. ta. By ca.. wtl tlxpect that it catr also be done by a stanclard T\rring machine.onettl_oncrxlrrt)tporrrlerrce this we mean that tht: clernerrtsof the set can be written in ${lme ordtlr. a.tion ir. Sone sets are finite. Wetarc tlrt:rrjustified in claiming the existerrceof a. Tape 2 will contain the tnpe cxrntentsof M.nbe put into a. 1 } and the stnx:trrrc of ir.. given any encodingof M. we carl clecodeit uniquely.s iruporta. we need to review some resrrltufrorn sct theory.pcs 2 arrd 3 to deternine the tape 1 to seewhat M would do in this configurtrtion of M. sirlr. A universal T\rring rnilr:hirrc M.T\uing trachine that. but the process is 'I\uing's rrninterersting. Section 9.16.t is therefbre ir proper urorlt:l firr ir gcneral purpose cotnputer.thc set of all even iutegerscau be written in therordcr 0.. .T:1.16 Control unit of M. It thc:rrcxrrrsults c:onliguratiorr.chinecan het rcprtlserrted by a string of 0's and l's ha.1 is a rea. so they a. M* looks first ir. we distinguisir trtrtwccrrsets that are countable ancl sets that are uncountable. thrlrr has itn irrprtt alplnbet tlrrrt inits cludes {0.nt irnplications.ttguage. It is within rea$en to txlnstnrr:t an actuar.g.c(:Hr'lr: 267 Figure 10. tapes 2 and 3 will be rnodified to reflect the resrilt of the rnove. givcrr any program. utriversal Ttrring machiue in a.pe I will keep a. call carry out the comprrttr.n encxrdeddofirritiorr of M.. Firrally. rrrrclthat.reof no concern.. Internal state of . The observation lhat every T.\rring ma. lrut most of the interestirrgst:ts (and languages)are infinite.tiorrssptrilied by that progl'an ancl tha.rr11 stl . For atry input M and u.Thereforc.10. Some strings will not represent any Turing machine (e...2.ge. for exiltrplt:. Dt:nrrirrg. Brrt bcfbre we explore these implications.lizir.
there is a. becaust:thtrrr fr worrld never a. This does not irnply tlnt thc stlt is uncountable. 'I'ake the set of all quotitlrrts of tlte fortrt ytf q. Tlxr stir.t is... occurs irr position n { 1.{#} . in such a way that any s in . Then an enurneratiorrproc:+ 'Ihrirrg dure for 5 is a rrrirc:hirxr thilt r:rr. ancl every elerrrent has sotrc place in the seqrrcrrc:t:. whenever q" is entercrl.17. and write dowtt the elerrrerrt thc rlrder encountereclfollowing the arrows. € S'. but there are lrlore corrrplic:irttxl cxilrrrllltis.17 Since any positive integer 2n.repositive integers.We call such a rnethod arl enumeration procedure. tlxr strirrg folbwing f rnust be in S. . # witlr 1llr1 I'* . e finite number of steps. This shorrldnot be too surprising. irr This eives us 1 1 2 1 2 t ' z 'T ' 3 ' i ' " Here the elerrtcrrt f..9 is prodrrt:rxl irr a.t the scheme depicted in Figure 10.ppear.268 ChopterI0 Ornpn Morrtlt.tesignifying rnernbershipirr S.n carry out the secluenceof steps qoni q"rr # ELi Q"'rz sz .bet E. clever way of ordering the set to show that it is in fact courrtablr:.sta.y seem counterirrtuitive. wlrt:rc p ilrxl q a. irr this r:asu. We see from this t:xirrnplc thir"t we can prove that a set is courrtabltl if we can produce a rnethotl try whir:h its elements can be written in sorrre $equence. Lrlok ir. Tht: set is therefore countable.. How should we order this set to show that it is courrtabk:'l Wc t:ilnnot write 1 1 1 I I't'3'4'. or:r:rrrsin the eiglrth place. the set is countable.tr:q. "." is rr. sorrre of whit:h ma.sol''l'unrlc MAcHTNES Figure 10. Let 5 be a set of strirrgs orr rromrtnlpha. tha. Since an proce$$1 can use a {lrurrnr:rationprocedure is sorne kind of rrrer:}rirrrit:al we Thring trracltirrtl rnork:l to define it formally..
tionprocedure tha.r:hirrtl coding.3 that any given string he listtNl a. r. any specific milr:hirx: will eventrrir.r:trrrrr Stcp 1. ca.TunlNc Macnrr'm 269 Not every set is countable. Tlrrirrg rnirchirre. in which wtl takc the lerrgth of thc strirrg as thc first criteriorr.10. Genera.n enumera.yin the order in which they would appea. tr. all worcls beginning with a a. 3. thus viola. NevertlNrluss.chT\rrirrg rna. b. finitc number of steps.chines. However. As we will have severa.u$e prochx:cs wcllrlc:firrcxlarrd it prtxlir:tabk: rtlsults. followed by an alphabetic ordering of all equallength strings. iln (:rnrrnerirtiorr llrttcedur{l crrrrrrot be called att algorithm. 1}* irr properl orcler.nenumeration prclcedurethrr. But when there ale an infinite number of a worcls. sa.If so.chines a. Let X : {a.pe in the form rr:tpircd by Definition 10. we (:an rrse il rxldificd rlrdrlr. The set of all T[rring nta. there are sorne uncountable sets. I  . Chcck the getreratecl string to see if it defines a.te when S is infinitel. ba.ritablewithout modifica. ab. the order used in dictionaries is not sr. write it on the ta.t prodrrcesits elermrrnts sornr:rlrrklr.r. since it will not termina. If not. Witlt this enProof: We ca. We can show t}rat the S : X+ is countable if we can find in n.bc:. is countable.lly lltl gtlrrt:rateclby this process. to Since every Tlrring rnilt:hirrc hirs a linite description. ignore the string.aae. bc{:ir.. It. usirrg 0 and 1.recorrntable.. $trir:tly speakirrg.hb. This is a.rin a dictiorra. I Arr irrrportant consequence of the above discussion is that T\rrirrg mir. ua.t protlut:us thc sequence d.f'tcrir.tethe next string irr {0.n errcodeeir. we will never reach b. As we will seein the next chapter. R.lt]xtug]rirrfirritc. cc.4 A Ur{rvnnsal.luses for sur:h an orclerrirrgwcl will crrll it the proper order. 2. In tr rlictiona. 1.3.ting the condition of Definition 10.. Instead.relisted befbre the stling b.it r:an bc c:onsirlcrcrl it rnerrrrirrgfrrlllrot:cus.ac.ry. c}. But any set for which an enurneration procedure exists is corrntable bccarrsethe enumeration gives the required sequence.ry. we thrlrr rxrrrstnr(:tthc following euunreralion procedure.tion. cb.
if we use a diffcrclrrt crrcxlrlirrg.s TtrRrN(+ oF The particular ordering of T\rring machines clepends on l. it is possible to limit it by restricting the way itr which tlrt: ta.ce.3. We catr also restrict thc tir. 4 r . Show that the Cartesian uroduct of a finite trurnber of countable sets is rrlrntable.92 arc also countallle. find a function / (tt) that gives for ea.chtl its index irr the propel ordering.chine.Je.j. 2 . This is of rro conserlrrcrr(i(lr howevcr.i. using the suggestcclmcthod. .1}t o..a r . I t ) .wc rmrtlt oxllrx:t a rlifferent orrklrirrg.pe rrsrr.1lc r:an br: rrsed. We have already seenan example of this with pushclowtrautornatir.tis restricted to being used like a stack. 7: {anb" : ri } 1} 6 . . Sketch an algorithm that exarnines a strirrg in {t). i ( g r. Give a complete encoding. Suppose that Sr and 5'r are coutrtable sets. A llrshdown automaton can be regarded as a..a r ) : (S. 1 . 3. d (93.a r ) : ( S r . gpi. we nright permit only a finite part of tirc tirlrrl to tlc ustxl as work $pa. ir. positive irrtegers. {h 4 ... Show that the set of all triplcts. a : ) : ( g r .L) . az. What rrrattcrs is thc cxisttlrrr:t:tlf sorntl rlrrlt:rirrg. tnpe thar. What is the irrrlex of ()tlr in Exercise 3? D. Slrow that then Sr U Sz and . (see . It can be shown that this leads us bir. Sketch a Turing rnachinc program that enumerates the set {0.ge other ways. 1 . ffi L 9 . a . (i. For Example 10. nondeterrninislic I'uring rrrachirrewith a.r:kto finite arrtomata. is countk able.rrd slxrws that the ordering itrrelf is utrimportant. fbr in tlxample.$ffiffi1f BoundedAufomqto Lineor While it is rrot possiblc tri qrxtendthe power of the standard 'I\rring machine by complicating the tirpc: stnrt:trrrt:. ) .7. irr proper Design a Ttrirtg rnat:hirre that emrmeraterr the following set in proper ordcr. for thc Ttrring rnachine with d ( g r .he encoding we use.270 Chopterl0 Mncslrur:s OurnR MoDEr. I} I to tleterrnine w}rether or not it represents an encoded 'I\rring rna.9r x .?.h) with .
'nc 27L Ext:rr:isc3 rr. In particular. i i l A string 'uris a.sin Definition 10. see Exercise I at the end of this section. has an unbounded ta.St*.rlsto rl rnore interesting situation: we allow the tnachine to use only that part of the tape occupied by tht: input.ttho end of this section)r $o we need not pursue this. Iinea.Qo.l symbols. F). A linear bounded automaton..l. and the readwrite head cannot rnove to the left of I or to the right of ]. the linear bounded automata (or lba). Rut there is ir wiry of lirnitirrg taptl usr: tha.srrhiectto the rerstric:tion that X rmrst r:orrtairr two spct:ial synrbols land ].ccepted a linea. The errd rrrarkers cannot be rewritten.pe. we restrict the usable part of the tape to exactly the cells taken by the input.t)1.rings Lhan for short ones.10.l rxrrrfigurilti<lrr thc T\rring of rnachine is given by the instarrtaneousdescriptiort qn[r]. it is not of known whrrtlxlr thc:y ir. we can envision the input as hracketed by two specia.1lffi A linear lrounded a.L).l) catr contain only elerrrents the forrn (qi..]. generating another class of machines. We sometimes say that the readwrite head "bounces" off the end markers.]) can contain only elementsof the of form (qi. a.$[iP'i. While one can deline deterministic lba's.d. Thus. the leftend marker ([) and the rightend marker (]).n2 € f*.rhorrndcd irutornirtorl is assumed to but essential he nondeterministic. tlrtl irritia. To enforce this.ninPii1. but how much of the tape can be used is a function of the input. Tlxl lirrrguirgcacccpted by the lba is the set Note tha.n. For ilrr irrput rl.q. of irll suc.rbounded autornaton if there is a possible by s(xllr()rrcc{lf movr:s qn i l*rqr*rl lrl fbr some et E F. more space is available for long input sl.2. suclt tlnt d (qr. This is not iust il rrrilttt)r of corrvetrience to tlxl rlisc:ussiorr lba's. 1t).rtl ecprivalent to the nondeterministic version.utomaton is a nondeterministic Tirring machine M : (8. . r. Iike a standard T\rring machine. For sorne exDloration of this.t in this clefinition a. and d(qi.l.E.5 LINITAR Brturunr:tr Au'r'oNl.harr:t:pttxl strings.t lt:ir.
We sketch the solution to point out one tacit implication of Dcfinition 10.Find a linear bounded autornaton that accepts the language L:{ont. For this problem. we dividc the rrurntrer of c..4. a more direct approach is srrggestedin Exercises 5 and 6 a. and the second track conta. I Thc last twcl cxilrnplt:s suggest thirt lincir.18). we still have to show that any contextfiee language ctr.. Ifthe input is in L. sirrce art atrount of scrirttlh spircc l)rol)ortiorral I ' i g r r r cl { ) .re invariably solvable by a littear borrnded aut{rilratorr.ilL4li l'he Ianguage [.:{u. We will do this later in a somewhat roundabout way. and the extra tracks can be used as scratch space.272 Chopterl0 Ornen Morrus on Tunrlc MAcsltlss Exanrplle.nltncn:z>1} is accepted by sorne linear bounded autouraton. if not.8.tl. The actual sohrtion is fairly sirrple. l. To prove srrch a conjecture.r bounded automata. so it can be carried out by a linear bounded automaton. Problems like Exarnple 10.insthe current divisor (Figure 10. This follows from the discussiorrin Example 9. at some point a nonzero remainder will a. One way to solve the problem is to divide the number of tl's successively by 2. IJsirrg the divisor on tlte secortcltrack. wc irrcrement the divisor by one. I 6xil. since neither of the languages are contextfiee. The first track contains the nurnber of a's lefb drrring the processof division.3..mF.4.rz}0}. The computation ouLlined there does not require spaceoutside the original input.ii$ti$. Tht: tapc of a linq+ilrboundtxl automatorr rnay be multitrack. say lry retrovirrg all syrnbols exccJrt those at multiples of tlx: clivisor. 1 8 .n be accepted by a linear bounded automaton. eventually there will be a single a lefl.5 a.rise. It is not so ea"Ey to rnake a conjecture orr tlrtl relatiorr betweerr Thrirtg rrtachirres arrd lirx:ar botrrrrlcrl arrtorrata.t$ orr the first track. and continue rmtil we either find a nonzero remainder or are left with a single a. until we can either accept or reject the string. After this.t the end of this section.. we can use a twotrack tape. are mortl powerful than pushdown automata.
( * ) .10. Show that such a machine is equivalent to a finite autornatot. L Give details for the solution of Exat4rle 10. is fixed for all inputrr.ss of lirrcar brlrnded arrtomata is less powerful than the class of unrestricted Thritrg tttachirres.rtt*. Show that for every contextfree language there exists an accepting pda. but require that the Thring machinc be deterministic.utomaton.L : {ai'o : n. we carl use Definition 10. Find a solution for Example 10. is not a prime number} (d) r: ( e )I = {ww.nnothe accepted by rrny linear hounded a. 4. L: { a ' " : n : n f . ffi 3 .b}. it is quite difficult to come up with a concrete and explicitly defined larrguage that r:rr.5 LINFTAR FJor. W Use thc observation in the abovc exercise to show that anv contextfree langrrage not containing A is accepted by some linear bounded automaton. try to fin<l rrolrrtions that are. it can use at most n extra cells fbr work space. assurrring thatE:{a. Fittd lirtear bountlerl automata for the following languages. but il dc:monutra. Are the sohrtions all deterministic linear bounded a.iruuutAurcruere 273 to the length of the input is availatrle. Consider an offline Turing machine in which the input can be read only once.n>1} W (f) f : {rururty't:. Find an lba fbr the cornplernent of the larrguage irr Exarrrple 10." : n is a prirne mrrnher} (") .b}+} {wn:w €{a.5 that does rrot require a rrecontl track as scratch space. Irr ftu:t.nr}l} (b) fI : {o. that the number ofsymbols in the stack never exceedsthe length ofthe input string try urore than .b}+} { D. arrd not rewritten.5.5.ure a. whete rt. 6 .tion of this requires a lot more work. .w e {a.4. rnovirrg left to rigtrt. Examine your solutions to Exercise 4.b}+. On its work tape. 2 . In Chapter 11 we will show that thc r:la.utornata? If not. such L To define a detenninistic linear bounded autornaton.
.
thtlsc grarnmilrs rlnd relgrrlrrra. This leacls to a . I'he proof is slrort itntl c:lt:garrt.The nontrivia. Our irrrrrrediirtcgoal will bc to cxa.tionwill be to look aL the relatiorr betweerrThrirrg rnachirrtlsirrrrl t:crtilin tyllt:s of grirmmar$ and to esta.crx. Arxltlrcr avenue of investiga. Ilrt norrr:r)n$tnx:tive.A Hierqrchy f o F o r m q lL q n g u o g e s ond Automqtq e ttow returtr our atterrtion tt) (lrr rnairr intelrc$t.tnplesthat actually allow us t. We will answer this queshy tiorr lirst by slxlwing thrrt tlxrrc a. Re'Ihritrg cause tnachiues can perforrn arry kirrtl of ir.r a.l questiorr is whether there are arzy languages that arc rxlt a.milies.nd r:ontextfiee gra.lgrlrithrnir: r:ornputation. lbr this rcas()rr.ed with tnircltirrcsilrrtl srlrncof thcir rostrit:tions. we expect to finrJ tira.remore la.t the family of languages a"ssociated with thern is rpitt: llroarl.t I ti . and gives little insight into the probleut.the study clf fbrrrral Ianguages.blish connection between a.mmars.nguages 'I\rring associat.re no Ttring rnachitres.clrine.ges. It int:hrrlcs not only regula.'pted some Tirring ma.but also the various exarnples we have crrcourrtereclthat lic outsidc tht:sc fir.ncJ contextfree langua.o iderttify oric suc:h lirrrguagc:.nguages than T\rring rnachines. so that thtlrc rmrst llt: sornt: Iarrgrrilgcs for which there a.wt: will cstalllisir tlx: cxistenrx: of latrguagestrot recogrrizablcLry Tlrring rnar:hinesthrrnrgh mrirc cxplicit exa.minethe la.
tll Leucuacns axn Autoraana hierir.king.tccvurythirrg so tlrrrt .ra. In cloing sor we rnust nrake the important distinction between Ianguagesfor which there exists an accepting Ttrring machine and klnguages for which there exists a. it rnay btl that the rrrachine halts irr a rrotrfinal state or that it never halts and goes into an infinite loop. w{: rnake t}re tacit assutnptiotr l}rat the languages adcl errepr. We can Lremore demanding irnd ask that Lhe rnachine tell us whether or not any given input is in its Ia. for la.ted with Thring rnachines.ges that do not inchrrlc thc tlrnpty strirrg.tc. This deflnition implies only that there exists a Ttring rnachine M.t accepts it.Ttrring ma. Recursive ond Recursively irilmmffiffiilt E n u m e r o b l[e n g u o g e s o We start with some terminology for the languages associa.chinetha.276 Chopter I I A HlrnnnclHv or FoRtr. To irvoitl havitrg to replrrase the definition or having to disr:lairncr.ngua. the first does trot irnply the second. A language1. It is a in trivia.ngua. do ttot contain A.Sorrresettheoretic cliagrarnsillustrate the relationships tx:twt:crr various language families clearly. many of th{r irrgurnetttsitt this clmpter are valid only Strictly spea.r not irr . such lhat. we have defined theur.memhership algorithm.ctthat T\rrirrg urac:hirrt:s.L. This restrictiotr arises as fiom the frr. rrnll:ssothcrwist: statetl.tncl cliscussecl this r:haptor. canuot accept the ernpty strirrg.\ is irrcluded.rc:hyof grilmrnars arrd tlrrough it to a method for classifying ltrnguerge f'rrrrrilics. fbr every ?0€ L. is said to be recursively enumerable if there exists a.ge. Ber:inrsea T\rrirrg rnat:hirrt: does not necussarilyhalt rlrr irrput that it rloes rrot accept. tyyut I y :I:tqII:21 with gy il finill ster. . but we will leave this to tire rerrder. Thc dtrlirritiorr says rrothirrg about what lnppetrs for 'rr.l matter to rtrstir.
the machirrc M.t cvery language frrr which a.sit stands. beciruseif sorne rui is rrot in 1.J ancl 11.. Then we ltlt M generate u1 on ur2 irnd let M executt: one rnove on ur2.1. a Ia. 'I'hese delirritions a. cvery string in . say M. clifferent way. rrrd sri on.Iriion A languirge tr on x is said to be recursive if there exists a 'rhrirrg machine M that ar:rx:pts L and tlut halts on every . Wc Iirst get fr to glenera. Trr make sure that this does not hrlppen...1 First move wl j uj Secondmove Third move . As wltir:h is modifled so that it writes strings orr its tape only if thcy are in L.ssociated with rtrring rnachines.nguageL. wc will eventually get a rna.2 .nenumeraticlrr pror:edure clxistsis recursively crurmerahle. We sirrrply cornpare the given irrput string agairrst successivestrirrgs gcriera. In olher word$. Flom this.2 givc: rr$ very little insiglrt irrto the na. Defitdtions ll. that follow rr. folklwod by the second ilr()vc on urr.r that M will never get into an infinitcr Ioop.llyproduced by M. let us sa/ '{t1. when startecl with ttj ou its tirltt:.11. We cannot use the erhoveargumerrt ir.nenurrrerationprocedure lbr cvery recursively enumera'ble languagc is not as easy to srlrl.L is cvcntrra..t rleternines mernbtirship iu a recursivtr la. and the processr. 'r'hat thcrr: is a.tr:h. finite numlrer of steps.te arrrl lt:t M execute one rrr()ve it.nd clo one stcp on ur3.they becourethr: irrprrt to M.L.ro in X+. may never halt rrrrrltirerefore never grlt to the strings irr r.. After this.tiorrprocedure. language is lrx:ursive. the third step on [r1. If a.1.1 R. It is easy to sec tha. lf 'ur E . the cornllrtation is perforrnetl irr a. then there cxists an easily cclrrstnrctedenuttterirtion procedure.r)2.ECURsrvrrt AND R. Wc first construct anotlrrtr lhring rnac:hirre.ture of either recursive or recur$ively enrurrerallkllanguages. The order of ltrlrftlrma.nceis depictcxl in Figure 11.rFirqflir$.lsoa. we gerrrlrate?lr3a. Since any ur e tr is generatecl by ffi ancl acceptecl by M in a. that gcrreratesall strings in X+ in proper orrler.ttilr:h narnes to languagc I'nmiliesa.nguage rrlcursive if and only if there exists a rncrnbership algorithur for is it.but shcd u2 Figure 11.nn be terminatecl. it is clca.ncunslvl:lv Exulannenlt Larvcuacns 277 I .ted by the errurnera. thesc strings are generatrlrl. the second step on ur2. Sr4lposethat M is rr T\uing machine thir.'i irr the enurneratiorr.
replace 0 with 1. with a 1 in position i if and only if sa is in while t.tR. that is.n 0 f 0 ) 0 0 1 t f 0 ] \*.2. The trew sequencerepreserrtssome element of 2D...et S : {"r. In the are 1L00. and vice versa. say t!...278 Chopter I I ANDAuToMATA A Hrnnancny o!' FoRMALLANcuAGEs no light on thc nature of represerrrtativelanguages in these families.en iu some ordcr. is represented 10101"" Clearly..t2..."r..ts. Thtrn any element f of 2" can be represented of by a sequenc:c 0's attd L's.} represcntcd by such a sequencc. But it camrot be f1 bet:ir.. In this ta. t t (t) \. the elerments resrrlt. Orte is very short and rrsesa very fundamentir.lly Enumeroble Longuoges Are Not Recursively Thqt We can establish the existence of latrguages that are rrot recursively enumerable in a variety of ways.2. For the same Figure 11. then its elements could be writl. "r..nite countirble set.. s3.hk:.. Then its powcrset 2* is not countable.use differs from fr through s1./j : .2 'l 0 r l t L l l t.. Proof: I..s6} is representedby 01100100.take the elements in the maitr diagonal. especia. arrd atry such sequentlc represents a unique clernent of 2s... that are not recursively emrmtlrable?tt While we will be able to supply some &nswers.. We "Are there languages are thercfore immediately faced with question such as "Are there lanthat are recursively orrurnerable but not recursive?" and guages. Let S tre an infi. describablt: sotttehow. and we could enter these into a table..nguagefarnilies we have tlrrcountered before...I and elegant result of matherrratics. and compklrnerrt each entry. For example. Suppose that 2s were countable.}. any eletrerrtof 25 can be by {rr... say f1 for some it i.we will not be able to produce very thc second one' explicit examples to illustrate these questions. the set {"r.. N6r do they tell rrs much about the rclationships between these languages or their conmlt:tion to the la. as shown in Figure 11. as the example in Figure11.so we get 0011.
I 1.ble. F.nguagesis courrttrble. is irr rnany ways unsatisfyirrg. In the next fcrw chapters. is called diagonalization. we investigate the conclusion rnore explicitly. For any nonempty x.eLH LANCIUAGES 275 rca$on it cannot tre f2. this irnplies that there rrlrst be some languageri on x thar arc rrot recursivelyenurnera. The argumerrt involves a variation orr the diagonalization theme. becausc it involves a rrranipulation of tlxr criagonal elements of a table. so that there must be sorne languages that ir. But the set of all T\rring rnachincs t:tr. In the next set of results. Theorem 11. I This kind of arflument. . As an irnrnediate consequrlnt:e this result. while it tells us of the existence of sorne larrgua. who used it to clernonsrrirre that thc uet of real nurnbers is not countable. the description of a larrguagc that is not recrrr$ively enumerable must be indirect.srrbsetof E*. there exist larrguagesthat are rrot recursively errurnerable.there are fcwer T\rring nrachineu than there are languages. or arry other fi. It is completely rronconstructive ancl. we can show that.ncuRsrvg AND R.nbe enumerated.1 tells us that the srrt of all languagrn on E is not countable. it gives us no ferrling at all for whirt these la.ssurnption lhat 2s is countable. By Exercisc 16 at the end of this section. Since X* is infinite.l is diagonalization in its purest form. 1 R.rurr. we will see a similar argurrrent in several contexts. so the set of all recursively enurnerable la. f3.ECUR. Proof: A language is a.srvgLy Er. Thercftrre the set of all Innguagesis exactly 2E.nguagesrnight look like. eilthough short arrd simple. Cantor. arrclevery such sutrset is a language. A longuoge Thotls Not Recursively Enumeroble Since tlvery language that carr he described irr ir direct algorithmic fashion can be acct:pted by a I\rring rnirc:hineand hence is recursively enurneratrle.it is possitrkl. Tltcre exists a recursively enumerable lirrrguage whose cornplenrerrt is not recursivsly enurnerable. Theorern 1l. Nevrlrtheless.c.ren.renot recursivcly enumerable. This contradiction crcates a logical impasse that can be rernoved orrly by throwirrg out the a.ges that are not rur:ursively enurnerable. I This proof. in some of serrue. The techniquc is attrilruted to the mathcrnatician G.
By (11.2) implies that ak 4 t' ltvt*1. Given a'. It is clcirr that the la. we first find i by counting the number of a's.2) Consider the strirrg aft. with its c:lements.. that simulates tlte action of M orr a'. if we assume tha.. this set is t.ourrtable. L is recursively emrmerable' I  . For each i ) 1. By Theorem 10. The combirred effect of this is a Ttrring tnachine that accepts every o. But thett from (11. To complete the proof of the theorem a$ $tated' we must still show that .. there Conversely.swe will show. (11.L. strrrting from the assumption that . the computatign carried out by M. If a' is in tr. and consider the set of all Ttrring machirreswith this input alpha. If this is tro.L is recrrrsively enumerablc.1) well definctl but. tlre string a'.. We now cotrsider a ncw language tr dcfined as follows. which is rr.L if arrd oilly if ad E L(Mt). then there must be some Ttuirtg machine. Finally. and we must t:onclude that olr assumption that Z is rccursively enumerable is false. is ttot recursively enumerable. ke L ( M i l . qirch recursively etrumerable larrguage on X.L or in Z? Suppose that ak eZ.3. We can use fOr this the knowrt enumeration procedure for T\rring machittes.280 oF Chopter  1 A Htunnrrc:IIY FoRMAI Leucu'q. a. The contradiction is inescapablc. there is some T\rring machirrt: that accelrts it. for is an associated recursively enumerable lirrrguage L(Mt).M2. sucltrthat T: t lttt*1 (1r. since the Btatement u' € L (Mr)' and ltettce a' E tr' firust either btl true or fhlsc. we give its description along with a" to a universal T\rrirrg rnachine M. is in.2) this implies that a .For cach Tirring rlachile Mi.. say Mp. therr oo # L and (11.' e tr' Therefore.L is well defiiled.t ak is in . T: { o o' a i 4 L ( M .so wc can associatc arr orcler Mt. Is it in .cus. Conversely. we corlsidet the complcment of L.Iso We will show this bv srtrtradiction. by Definition 11. will eventually halt.bet.}. But (11.1) we get that ak eT.1.L is recqrsively enumerable. We therr use the enumeration procedure for I\ring machirres to firrcl Mr.rrguage . Next.twn AuroMere Proof: Let E : {o.1) now irnplies that o *f T . ) } .
If a lattgutlge tr and its complernt:nt Z are both recursivrlly enumerable. throrrgh (11. There exists a recursively enumerable languaga that is not recursive. respectively. Suppose now we are given any ?r. Z is llroperly defined.in. If they are not thc same. wc: next let M generate u2.renot identical. and consequently lloth are recursively errrrnrerable. Proof.srrbsidia..€ E+. If we need to continue. a welldeflned Ianguage that is not recursively enumerable. Then there exists a membership algorithm for it.ngrrage of Theorerrr 11. Thiu is not to say that there is arr easy.qcns ZBI The proof of this theorern cxplicitly exhibits. If the rrmtching string is procluced by M. For tho (ionverse. in tr. the family of re:cursive languages is a prolrer subset of the family of recrrrsivelyenurnerablelangrrages.L is rcr. we need do so in a rather roundabout way. that is. I : ... and so on.s enumeratiorr procedures for L and Z.ECTTRSIVELy _ErruunR.fr2. The first will produce rily. but its complement is not. giving us thr: krokedfor example.3 is in the first but not in the second family. I Flom this. we show there are some languagersthat are recursivrlly enumerable but not recursive.4.1 RFrcuRSrvE ANDR. Sirrce any recursive language is recursivcly enumerable.. we Iet M gerrerate fi1 and compare again. so eventually we will get n match. intriltive interpretation of Z.3. But this brxxrmesa member$hip algorithm fbr Z by simply corrrplt:rnenting its conclusion. sively enumerable. by Theorem 11. Therefbre Z is recursive. Aga.L. it is not recursive. it would be difficult to exhibit more than a ferw trivial members of this language. then M gerrcrate fi2. we cxrnclude directly that the family of recursively enurrrertr.then there exist TLrring machirres M and M that scrve a. If.milyof recursive larrguages a.{.lrr$ive.. zrrbelongs to L. . Arry ru € X+ will be gerreratgd either by M or . Nevertheless.4. We first letff generate ?x1and compare it with ru.1). Proof: If tr and Lare both recursively enunrerable.anln LRrucu. Coruider the la. The process is a membership algorithrn for both L and . then both languagesare recursive. The language tr irr Tlrrx>rem11.11. assume that tr is recursive. so they are both recursive. We begin by establishirrg tr.then Z is also rccursive.rlt2. Thereftrre.blelanguages and tlx: f'rr. A Longuoge Thotls Recursively Enumeroble But Not Recursive Next. thc uecond fr1.ry result. in L. otherwisc it is in Z. This language is recur1. the proof is completed.
Prove that the cornplcment of a t:otrtextfrce language rnust be recursive. Choosc a particula. Show that the fatrily of recursive lartguages is closed under union and intersection. ffi 7.L is recursive. and Sr C 52.r' encoding for T\rring rrrachines. 2. 18. Show that. 52 a sct that is not countable. Wc cotrclude from this that there tr.roirrdeed welldefincd latrguages fbr which one cantrot crrrrstruct a memhcrship algorithrn.282 Chopter I I Ln'lqcuecnsnNrr A(r'ro\'Iann orr A Hrun. Show that .qnc. Suggest arr etrumeration procedure fbr L+. Let L bc a finite larrguage.Lr necessarilv rccursivelv ernrrtetable. show that in fact Sr .Lr is recursively enumerable.9r be a countable set. Show that then . its complement carrrrot be rer:ursive. L4. Prove that the set of all real mrrtrbers is uot countable. and with it. is it necessarily true that 1. Show that the family of recrrrsively enumerable languages is closed under union. Suppose l. Let . 16. Let L bc a contextfree language.+ is also recursive? W L5. Show that the set of all irrational mrrnbers is not countable' ffi ffi . 13. is recursively enumerable 5.hat L is such that there cxists a'firring rnachine that emrmerates the elements of tr in l)roper ordcr. 12. Show that S'r must then r:orrtairran infinite number of elements that are not in.1 fail when S is firrite'l 19.Lz . Isr the family of recursive larrguages closed under concatcuation? 11. If L is ret:ursive. L S[ow that the farrrilies of recursively emrrnerable and recursive larrguages are closed under reversal. find one clement of the larrguagc Z ir't Theorerrr I1. Slxrw that l+ arrd suggcst a. ffi 3. if a language is not recursively enumerable. Why does the a.n entrrttetation procedure ftrr it. 4.Sr' 17.Sr cannot be countable.3. Is the family of recursivcly enumerable languages closed urrder intcrsection? 8. Let Lr be recursive and L2 recursively cnurnerable.r'gumcnt in Theorerrr 11. Prove that the set of all languages that are trot recursively enurnerablc is not countable.Ilv FoRIrnAt. In Exercise 16. Show that this rneans that . 1. 10. 6.
We show this in two parts.2 UNRESTRTCTED GRaMunRs 283 U n r e s t r i c t eG r s m m d r s d Trl investigate thc connection betweerr recursively enumerable languages and grarnrnars. the flrst is quite straightforward.11. we . In fact. a.t S+u..{tilli l'iirj A grammar G : are of the form (V.t is.1 the production rules were allowed to take any form. tha. ur is derived irr orre step. Proof: The grarnnrar irr cffer:t defines a procedurr: frlr enumerating all strings in the language systt:matically. Since the set of the productions of the grammar is finite. unrestricted grrrmmars are nruch rnore powerfirl than restricted forrns like the regular and contr)xtfiee grammars we have strrdied so far.L such tha. iiilffifr.ffi.n be on the left or riglrt. thcrc will he a finite number of srrr:hstrings.' (:ernlist all ur in . we return to the genarnl definition of a grarnrnilr in Chapter 1.ble.ffi$ffi. If we take the gcrreral form and itttposc no restrictiorn. Next. Any rrurrrber of varia. unrestricted grarrrrnars txlrrespond to the largest faurily of languages so we ca. There is orrly one restriction: . where z is in (V U ?)+ ancl u is in (y U 7). but the second involvcs n lengthy constructiorr. For exarnple. Anv language getreratcrl by tln unrestricted grirrrrrnirr is recursively enurnerir. but various rcstrictions were later made to get specilic grirmmar types. As wc will see.fi{. In an unrestri<:tedgrammar.T. rrnrestricted gralrrrnars gcnerate exactly the farnily clf recrrrsively enurnerablc languages. wc gert rrnrestricted grammals.bles and tertrinerls ca.P) is called unrestricted if all the produt:tions U+U. In Definition 1. essentially no conditions are irrrposedon the productions.t is. w(.S.sthe left side of a prodrrction.n hope to recognizc try mechanical rneansl thir.nd these can occur in ilny order.\ is not allowtrd a.
but its itnplemeutation bet:tttresttotatioually cumbersotne. and so on. To irc:trieve irr broad outline.4) 'Ihis is not hard to do. we construct a grarrrrrlarwhich' ftrr all tu satisfying (11.F) prodnce a grammar G such that .5 carr derive qou for all ur e X+. The grammar generates strings by a welldefined algorithmic process. have atr enumeration procedure for the langrrngtl. and want to We are given a Ttrring mat:hineM : (Q.the grammar.f. we describe how any Tlrring machine can be mimickccl by an uurestricted grarrrmar.84 Chopter I I A HtnneRcHy or.uel Lancueons AND AUToMATA Iist all w itt L tha.r this string into the origirral'u.a) is possibleif and only if (11. Since the computatiorr of the T[rring machine carr be described by the sequerrce instantancous clescriptions of qou I rqfA.3).n.3) we will try to arrangerit so tltat the corresponding grarnmar has the property that Qxu tf'I:Qy!.t can he dcrived itr two steps .d.chine. tratrsforttts 3 .9+r+tr.mmars is rrot surprising. (11.L (G): L (M)' The idea bchind the constrrrctiorr is relatively simple. To show the converse. We can simulate these derivations on a. .3) holds. wltat is more difficult to if and only if (11. Tltring rnachine ancl. Wherr a string rqly with qy € lr is gerrerated. r This pnrt of the correspondenccbetweett recursively emrmerable langueges and urrrestricted gra.3) holds. therefbre. has thr: following properties: 1 . (11. Hence it is recursivelycrrutnerable.?.E.4) and wha't we rea. so tlte derivations can he dclnc on a T\rring ma. S4u' this. namelv.llywirrrt. see is how to rnake the connection hetw(:t:rr(11. 2 .go. .' !'on. (11.
11. Thc first is saved. for cirdr trarrsitirlrr . 1 0 ) .ndVoi6f'or all a € XU {n}.ncl while V. ( rr . " ): ( q j .fheseprocluctions allow the glarrunar Lo gerreratean encoded version of any string q0urwitlr a.4). we prrt into the gra. M t:tttr:rs a firral stattl.t" VpiqVoa.l (qu. Therefore. h € I. q e I'.5) Tlrc thirrl step in the a. L ) of M. WdVrtq. (i1. we introduce variables Vo6 ir. R) productions of M. p € EU {l}. we include productions WiA + et ( 11 . For cirr.rolvooo.2 UrunnsrRrcrED Gnalruans 285 Tlrt: rxlrnlllete $equen(ie derivations is then of $ 4 qorrl xq. If in thc st:t:ttrrrlstc:p. lbr the second sttp. which is saved in lhe first indices of the V's. for every ei E F... thc statc: r.'. for irll n.tr.6) (11.VrrSlSYnll?l 7' TV.mmtr. we inchrdein G VpqVo. Tlx: first step irr (11. s ) ( 1 1 .tionis the troublesome one.yy 1w.rringthe secondstep? We solve this Iw enurding strings so that the coded version originally has two copies of u. d . 'Io produce two copies of tr and to harrrllrl thc statt: syurllol of M (which eventually has to be removed by the grammar). the grammar erases everything except the saved'ur.ry number of leading and trailing blarrks. while the second is used in the steps in (11. '.Vprt . the gratrrrnar tnust thert get rid of everyLhingexcept ur.7) for a. How can the grrrmmar remember u if it is rnodified dr.n arbitra. When a fitral corrliguriltiorr is tlrrtcred. .5) can berac:hieved the encoded fbrm) by (in 5 .r W. and all rl such that qa e Q.)e for all a. q e f. p e }JU {tr}. : kt.}r d ( q 0 .lla e E. (11.1..bove deriva.J .i . The variable ydb ()rr(j()(l()tr tlrt'two syrnbols rr.i6 encodesa and b as well as b.
Carefully check each step in the exa. wc rmrst first rrsc (11.qo. R) d (qo..se when M moves outside thnt pa. CQ. (11. tr) !. F) uea Turingmachine n.14) which acct:1lts tlrt: strirrg na.x. To rnirkc things work irr this casc. r) .nrlal Lnr'{cuncos awn Auttlltane f'rrrnll a € X U {tr}.by a cWt Wtc for all a.\. S =+ 5'Vll =+ TVaa +'lV""Vaa 4 VoooVooVll.. wtl first us(rnrlc$ of the form (11. We need one more specrial E .:{a.13).mpleto see what the variorrs prodrrctions do antl whv thclv itrt: rrecrlcd.tr}. I'he Iast sentential fornr is the starting point for the part of the derivatiorr that mimics the cornputation of the'I\rring machine.lqprrrn. c€ EU{tr}.kescare of the ca.o) . f . The extraneous blanks are removed at the erd by (11.tncIIY oF Fon. Clonsirler rrow tlte tlttrnuutittiorr esaal aesal aaesl F aqla[ (11. (11. which tltcrr causes rewritirrgirr the rest. T}ris createst}re first terrninal in the string.a. with qr} I : {qo. 1. a(.286 Chopter I 1 A HInn.b} . Ttl tklrivc this strirrg with (J.rt of the ta.. Tlris rttachirre acccptsL(uu*).. representing all the tape region used.7) to generate E.(q11. )J : {rr. 1 2 ) prodrr:tion lre f.b.pe rlr:t:rrpicd tly thc input 'ur. t' : {qt} ' itncl d (so.6) and (11. d. lliHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH mt M : (Q.7) to get the appropriate starting string.6) rrnrl (11. . The following example illustrates this complicated construction.. b e f.ri) (11 . : (gr. .13) This last production ta.!. It contains the original .
Proof. By an induction on ttre rmmberof steps. always rernembering the itrititrl input. and of Vo*l/tr0tr WruVaa corning frtlur (11. The sequenceof the other indices is 0aan. Finallv. 5 .srV11. aa0!. then e ( r )+ e ( y ) . whiclt arc spcr:ifir:insta.. a1aE.8).z * Voolltr0tr..13) is the basis of the proof of thc following result. Then the next steps in the derivatiorr are V..I1. T For every recursively enurnerable languagc. (11.9). The construction described suarantees that if rla.6) to (11. whercc (z) denotes encoding a string according the givenconventhe of to tion. The sequenceof first indices remains the same. aOa[. Next.') " (g) ..10)to (11. Thc cxrnstnrction describedin (11.14). we apply VoooVoo WoWoo.we cirrrthen showthat e (So?.L : L(G). such that . and VoooV.2 UNRnsTRrcTED GRarrrlrans 287 input aaE in the sequellce tlf first indices and the initial instantarreous description esaal in the remaining irrdiccs.uVaa VouVoooVVa * )VooVooVlon + VooV. there exists an unrestricted grammar G.nces (11. which is equivalent to the sequenceof irrstarrtanrxlrsdesr:riptionsin (11..V.L.13) are used itr tlte last steps Vu"VtuVaa 4 VooaVag + V""al + aal + aa.0.
In Example 11. unre withA€Y.\. Construct a Turirrg rrrachineftrr tr (01 (01)). Give the details of the proof of Theorern 11.4 +. then investigate how such gratnma.7.ssor:iirtt:rl etrumerablelauguages.A aa 2 . Thev show that wittr utrresl.t we can gcrrt:rate every possible sta'rting t:ttnfiguratiorr and that tu is properly rerxlrrstnrctt'd if and only if M enters ir lirrirl whir:h arc rrot too difficult.hlishwhat we set out to clo. Formalize this concept.1.288 Chopter I I Arttr Atlr'ovane A Hrnnnncnv or Fon.7. with the family of rec:rrrsivcly 1. 6 . 5 . Consider a variation oII gl'alnmars in which the starting point for arry derivation can be a finite set of strings.releft irs iilr cxerconfiguration. ffi 4 .rs telate to the unrestricted grarrrfirars we havc usccl here.q. Give a derivation fbr 0101 using the resulting gramnrar. What difficulties woultl arise if we allowed thc empty string as the left sicle of a production in an unrestrictcd grammar? 3 . t . cise. I These two theorems esta.rsis idt:rrtitlirl the family of langua. thcn find an unrestrictetl grarnrtrar for it using the coustruction in llheorem 11. prove that the constructerJ grarrlrrrar cannot generate anv sentence with a b in it.ils. L. rather tharr a single variable. Slxrw that fbr everv unrestrictcd grammar there exists an etpivalent stricted grammarr a.Ncuacns if and only if q1uri Y' Wc also rnust show tha.a$rb bB .ll of whose protluctioris have the fortn 'u+at with z. m .ltnl.gesa. or .ricted gra'mma. a. The cleta. What language does the utrrestrictcd grammar Jj + 5'rB '5r . u € (Y U ?)+ and lzl < lul.bbbB aStb ' clerive? ffi B .
sour Dcfinition 1. there is an equivalent grammar of the other typc.v(vu7)..3.r$ ha. whic:h we irrtrorhrt:edirr Ser<:tion 10.ge.'$$fr.5.nguer.3 Corur'r:x'rSnrqsrrrvn Clnauuans .rir:ty of "srlrncwtrilt restricted" grammitr$ t:tr. Show that the conclusion of Hxercise 7 still holds if wc add the further conditiorrs zl < 2 and l"l < 2. C o n t e x t .fi ..S e n s i t i v e o m or s Gr m ond Longuoges Bctween the restricted.ype. These grarrrrrrirrs gtlrrcrilte lrr..lirrc:itrtrourrdcxlirutornata. in the sensc that for cvcrv grammar of one t.q. the cotttcxtst:rrsitivtl grilmma.rup Larucuacns 289 8.5.\tthitffi tl.ted with a.l' A grammar G: (V. a E (V U T)+ ancl l z l5 l s l . a.mmarsancl the general.'s rr.nd a€(vu7)". among the ones thtlt rlo.ve relcc'ived considerahle a..ffi.d€(vu"). this alternate definitiorr. The diffctcncc is that hcrc thc lcft side rnust have at least orre variable. where . the productions of In atr utrrestrictetl Hrarnrnar are rerruired to be of the form T+U.'I'.I1.P) tions are of the forrn is said to Lrecontextsensitive if all produc where tr.ttention. 9. Somc authors give a definitiot of urrtestrictetl grarnrnars that is not qrrite the same a.nbe defined. contextfiee gra. (il.ssrlr:ia. unrestricted grarrlrnar$r ir grt:at vir.15) . Show that this alternate definition is llasir:ally the sarne as the one we use. N$ffi$fl N. restricted clirrs of T\rring macltirres. Not all c:irscsyit:ltl irrttlrcstirrg rrnultu.
s u c ht h a t . Yet. but it cau tre shown (see. Salomaa 1973) that all such grarlmars can be rewrittt:rr irr a rrormal form in which all productions are of the form xAY + YPY.L : L ( G ) o r L : . While we use the terminology arising from this particular interpretation. and we will rely entirely on Dtllirritiorr 11. if A language .290 Lauc.gefamily with the sarnc rrarrle. thc forrn itself is of little interest to us here. so that a contextsensitive gra.4 implies that r + A is not allowerl. every contr)xtfrrxl larrguage without .4.4. wo' reirrtroduce the empty strittg.wclHy FoRrr. Definiiion 11.I. it is noncontracting. (G)U {A}. for cxarrrple.mmar can never gernr:rittc a lirnguagtl containitrg the empty string. By inclrrding thc ernpty strirtg itr the definition of a contextsensitive langrtage (but not in the grammar). is said to be contcrxtst:rrsitivc thcre exists a contextsensitive g r i r m r n ? r(rJ .\ can be generated by a special caseof a contextsensitivc grarnrrlar. say by otre in Chomsky or Greibach normal ftrrrn.It is less obvious why such grammar$ should be called rxlrrtcxtserrsitive. in the sense that thc lcngth of successivesentential forms can never decrea^sc.uacns eNn Aurol"tn'r'A or Chopter I I A Hrnn. we can claim that thc frrmily of corrtextfree languages is a subset of the family of contextsensitivc languages. both of which satisfy the conditions of Definition 11. qnd LineorBoundedAutomotq ContextSensilive Longuoges witlt As thc terrrrirrologysuggests.tel This definition shows clearly one asper:t of this type of grammar. This is equivalent to saying that tlre prodrrr:tion Atu cirn be applied only in the situation where 4 occur$ irr a context of ihe string z on the Ieft ancl the string 3yon tlrc riglrt. In this definition.contextsensitivegrammar$ are a^ssociated a latrgua. .
tttfble t.ngrragein the a. there exists sorne lirx:irr brlrnded a. we see that the farrrily of rxrrrtcxtfree ltrnguages is a proper subset of the farnily of contextsensitive larrguagcs.q. Hm6.rqu LeNcuecns 291 . Iinear bounded autorna.ge.lirrear hounded automaton.rrgutr. Ihe solutiorr clIectively rrses the varia. it carr trclvirlwrxl as a.bA.4 is created orr thc luft. S + aAbc+ abAc+ abBbct.tthclaAbc:.rnpleswill show that.henthere exists a corrtcxtsc:nsitivt: grammar fbr r{A}.ngua.ton.r is S .t.\. + aRbbcc + aaAbbcc+ aubAbt:c + aabbAcc+ aabbBbtcc + aab Bbbccc+ aaB bbbccc.In.11. Tlrc pror:essis very similar to the way one might program a Thring machine to accept tlrc lir. then finding atr equivalerrt grarnrnar frrr it. We can see how this works by looking at a derivation of o:lblcil. Ab .One sut:h gramma. Frlr every crrntextsensitive language . If L is contexLsensilive. It therr sends the messenger B back to the Ieft in order to create the corresporrdinga. + aaabbbccc. Often the solution is rnost cirsily obtained by starting with a Ttrring machine progra. trirvels to the right to the first c.2 I The language L : {artfirlsn: n > 1} is tr.utomaton M such that tr : L(M\. Proof. the corresllorxlirrg T\rring machine ha. The linear bounded automaton will have . A ftlw oxa.2 also shows that it is not an easy rnatter to find a contextstlrrsitivc grirmmerr even for relatively simple examples. At: + fi6"". . whenever the language is contextserrsitive. We slrow this by exhibiting a contextsensitive grirrrrilrirr fbr thel la. We show that derivations in t}ris grarrurrarcan be sinrulatecl trv a.spredictable space requirementsl in particular. bB +Rb. Arr .bove example is not contextfree. contextsensitive language. . aB +aalaaA.L not including .ge tr. where it creates another b arrd r:. Exarlple 11.3 ColrrlxrSnNsrrrvr Gnntttrrtns. I Sirrrxl tlxl lir.bles A and B as rnessengers.
But this production can be omitted. since we can claim that the cclrrect productiorr carr always be guessed and that no unprodrtctive alterrnativcs havc to be pursued. Proof: The coustruction here is similar to tha. and look a. that is.292 Chopter I I A HreRRRcHy FoRMAI Lnmcuacns eno AurouAlA oF two tracks.t origirrrrlly occupied by ur.7. it can be done bv a linear bounded automaton. The grammar obtir. then thcrc r:xists a contextsensitive grammar that generates tr. It is necessaryorrly when the Thring machine moves outside the bounds of the original irrput.lincar bounded automatotr is. . by clefinitiotr. cotnpleting the argument. contextProof: Consider the contextsensitive la. We know that l*il5 l"i+rl. Every r:rlrrtext*setrsitivelattguage L is recursive.t a. I If a language tr is acc:eptcrl by sorne litrear bounded automaton M.derivation of ro 5+11 =+r?'+"'+rn+ut' We can a.6 can be carried out without using space except tha.10 follows easily frorn tlfs. the othrtr r:clrrtainitrgthe sentential forms derived r. A key point of this argumetrt is that no possible sentential fbrm can have lcngth greater than lul.9 tclls us that cvt:ry cotttext*sensitivelatrguageis accepted by somtr T\rrirrg rnirr:hitre:arrtl is tlterefore recursively enunerable.ssumewithout any loss of generality that nll serrtcrrtial forrns in a single derivation are diff'erent. thirt is.13).i t' x. nondeterministic.ineclhy thc corrstructiorr without this umrecessary produr:tion is nttncorttractittg.$sociated sensitive gramrnar G.nguage. which is not the case here. Theorem 11. The crux of orrr argrrment is that the numbcr of steps in arry tlerivation is a bounded firnction of kpl. Another point to notice is that a. Therefore. r Between Recursive Contextsensitive Relotion ond [onguoges Thrxrrr:m 11.t in Thcorern 11. *. This is rrct:cs$aryitt the argument. one containing the input string ur. All productions generated in Theorcm ll.L with an tr. the computation described in Theorctn 11. tr*4.rsing G.i for all i t' j.7 are rroncontractitrg except (11.
definesa contextsensitive {u. with trl : nL(lrl) u bor. l This for all j. Thcrcfore:.'1.blc f'orm V : {Vo. A given string ui rrlay ntlt define a contextsensitive gramrnar.} .vtr. h () : 61:t6.ritr..b}. the representatiorr is itrvertible itr the sense that. The cxrly tlfttg we need to add is that there exist some rn. set of the We t:tlrr r$() a convention in which each grammar has tr. there are only a finite number of these. Proof: Conuider tlrtr set of all contextsensitive grammars en 7: {a.11.L. givcrr arry such string. any contextsernsitive grarntrtar call be represented rrniqucly by a string f'rom I ((0110).urded function lV u 7l nnd lz.t most ont: corrtextsensitive grarrrrrrarcorresponding to it. otherwise it is not. We check all derivations of lcrrgth up to l'ulrn (l'rrrl).. I There exists a recursive langurr. call tho grirrnrrritrGi.. Furthermore.)}.):s1+6' h'(V):01i+50' Thus. so we ca'nwrite strings in tlre order uJltlrzt etc.ls 293 because G is noncontracting.Vt.grlthat is not contextsensitive. of follows becausethe finiteness of lV U ?l implies that thcrc are only a linite number of strings of a. there is a. . l .I r n . i< l r i 1 .. h (b): o1zo..3 CoNTExrSnmsrrrvn GnavveRs ANDLANcuAcr. we defiue a' la'nguageL tly L: grammar (J4 alrd *.i: trr4 . Let us introduce a proper orcleringon {0. # L (G. Since the set of productions of G is finitc. if it does. To this string wtl now apply the hotnomorphisnt ft'(a) : 616. Evcry contextsensitive grammar is completely spet:ifitrl by its productionsl we can think of them as written au a single strirrg l U 1+ AtlfZ + UZl .given length. lz(. then u € L. such that l z .. Next. This observation giveu rrs imrncdiately a mernbership algorithrn for . 1}+. + Jlrrr.). If any of them give ur. dcperrdirrg otily otr G atrd 'ur.Vz.the length of a derivation of w E L i s a t m o s t l r u l r n( l r r r l ) . ' .
then ruaf L.L. art: ir. (a) I: { t n : n * ( ' u r ): 7 r . then u. 1.ndcontextsensitive languages orr one harrd.testhat linear bounded automata are indeed less powerful than TLrrirrg urachines.grammar.t.. As various examples show. ) : n .t) is rer:ursive. and pushdown autornilta ilnd contextfree la.L is ttot mrntoxtsr:nsitive. Contextfree languages. If it were. t The result in Theorern 11. We must therefore conclude that .s. wu see that anv Ianguage accepterl by ir. > 1] {ww. * 1 .r*L tlrat trri ( L(Gj). beirrg gr:nerated by contextfree grarnmars.srrhset of the contextsensitive languages. (u) {an+rbnr" 1:ri}1}. ( t u ) } (b) t : {w t no (r) : rro (u.bekrrrgsto. Find contextserrsitive fJrammars for the following languages.11 indica. It f'ollows from the same resrrlt that linear bounded automata are rrrore powr)rfirl than pushdown automata. Larucuncus ANDAuroMNr'A. lf *" .. Elut .. so wtr Irave a corrtrrrrlir:tion.10to find orrt if rui € I'(Gt).L (Gj). is well defined and is in fact recursive.w €{a.L.lgorithm. ( u .ui # L(Gi).nguageson the othcr.and we can use the membership algorithm of Theorem 11. Show that thc family of corrtextserrsitive languages is closcd under union.if we assurnethat. tlrcrr by clefinitionuj is not in. then by definition wi € L and we havr: irnother contradiction. : { t t " h " a . then L((. Because of the esserrtial equivalence of linear bounded arrtorrrattr tr. 4. I. Conversely. since they accept only a proper subset of the recursive languagt:s. ( b ) Z . suclr that L : L (Gi) We ca. If not. there would exist sornrr rl.rn.nthen a.xtsensitive.b}+} ffi * 2 .')< n.llrshdown automaton is also accepted by stlrne linea.nguagesaccepted by sorntl Iinear bounded automata for which there are no pushdown automata.r bouncled automaton. we construct a membership a. (ur)} 3. ffi . > 1 } (c) I: (d)l.. Show that the farnily of contextsensitivc languages is closed under reversal. check it to scc if it defines a contextsensitive we grammar G.skif uri is in . Given ?o?.re la. If the strirrg rlclrlsdcfine a.L is rrtlt rxlrrto.254 Chopter I I A HtnnencHv or FoRvel. But L: L (Gi).: rn {e. If it is not. Tcrseethis.rtfir'txrt4rrt ] l. z " :n . but that there a. they arc ir proper suhset.'incl contextsensitive granlmars for the following {anguages.
Type 0 languages are those generated by unrestricted gramurars.3 shows the original Chomsky hierarchy. give explicit bounds for m. show that therc cxists a contcxtfrcc grammar for the language 7 : {uul i 1t. We have also met several other language families that can be fitted into this picture. Type I consists of the contextsensitive languages.ra. type ? consists of the As contextfree la.miliesr:rr. in Theorem 11.11.t the extended hierarchy shown in Figurc 11.4. arrlong them the recursively enr.3 7. we a. altirough thcir relatiorrships do trot always ltave Lite neatly trestecl structure of Figures 11. (Ln. Figure 11.t11t. each larrguage farnily of type i is a proper sulrset of the family of type i . and recursive la.ges (I'nac).hle languages (Lnu). fa.L6. the corrtextsensitive la. Figurc I I. that is.n defined tr.ndtheir pltx:c:irr Figrtrc 11.1.ngrrages and the regrrlar lilngrrargers Orre way of exhibitirrg the relatiornhip between these farnilies is by the Chomsky hierarchy. This original tenninology lns persisted and one firrds frequent refererrcesto it. Noarn Chorrrsky. provided an initial classification into four language types.ngr. type 0 [o type 3. the relationships are not cornpletely unclerstood. b}+} ffi E The Hierorchy {ffiffiffiffiffiChomsky We have now encountered a nurnber of language families.4. as a function of lrrrl and lv u71. Without explicitly constructing it. tho rxlrrtcxtfieer (.nguages and type 3 consists of the regular la. but the numeric types are actually different rlarnes for the language families we have stuclied.10.nc). Frrr m.3) exhibits the relalionship clearly.nguages. In sorne instances. a founder of fonnal language theory.{a.3 and 11.4 Other la. the recursively enumerable languages.nguages (Zcs).ge he sturlirxl. Including the families of deterministic contextfree languages(L nc p) . A diagrarrr (Figure 11.' .ngua. lir.4 'I'HE CHor"tsxv HIURARC:HY 295 5 .rrivea.rmera.'p). we have seen.
but not linear. This indicates that the relationship between regular.5.296 Chopfer I I A HrpRaRcuyol FoRraer. linear. f l'igrrre 11. determirristic corrtextfree. but not deterministic.5 ..4 ll\]i$'$.: {a"bn} U {a'b?'} is linear. On the other hand. and rrondeterministic contextfree languagesis as shown in Figure 11.Lnmcuacns enn AuroMArA Figure 11.ilWil$\$ language we havepreviously introd'cedthe contextfree L: { w : n " ( w ) : n 6( u ) } and shown that it is deterministic. the language 7.
there is tro easy answer. Collect examples given irr this book that dernotrstratc that all the sub. To slrmmarizc.n unresolved issue. we established a hierarchy of languages and classified autornata.re more powerfirl thtln lirrcar botutded autouata.set relatiorrs rlepit:tetl irr Figure 11.n pushdown inrtomata. ir.ter. propcr subset of the contextsensitivela.mple I1. $ec:tiorr10.11. . 2 Firxl two exarrrples(excluding the onc in Example 11. it is not known whether thc tarnily of lattguages is accepted by deterministic linea. Firrtl two exalrrples (excluding thc one in Exa. At this time. I . w() have explort:d thcl rclationships betweeu several langurlgclfarnilies arrd their associatedautomata. In doing so. Ttrring machines a.rsxv Hlpn.At thc bottorn of the hierarchy are finite accepters.5.r bounded automrr.with which we begarr our study. by their power ir^slanguage accepters.3) of languages that are Iirrear but rrot deterministic contextfrcc. We introduced thu trrrx:cl1ltof a deterministic linear boundecl automaton in Exercise 8.an.crrv 257 There is still a.4 are indeed propcr ones.ngtrages. deterministic contextfree but not linear.4 Tsr: Clsorr. These in turn are more powerful tha. We can now ask the question we asked in r:onnection with other autorrrata: What role does nondeterminism plav here'l Unftlrtrrrratclly. 3 .3) of larrguagesthat are .
.
it would not be very interestingl norrrec:ursive: lirrrgrur. w() now look at what they cannot do.ges have little pra.ctical value.lked a. We first define the concept of decidability and computability to pin rlown wha.nv a.bout what Thring nrachincs r:iur rlo.ted(but nol yet proved) that we there exists no algoritlmr to detc:rrrirut whether a contextfree grammar is rrnambiguous. Sorneof tlxr results ca.meabout quite sirnply. if a langurrge is nonrecursive. hirve sta.mrning languages.lgorithmsfor the solution of ccrtain problems. We then look at several classical prcltrk:msof this tvpe. FYornthis follow a nuuber of rrllatrxl prohlems fbr T\rring machines and recursively 299 .t we mean when we say that sorrxlthing cannot be done by a. Now we rnake more explicit what we mean by this claim.crtical significancein the study of progra. we hitvc c:lnimed on severa. If this were a.L i mt s o f A l g o r i t h i c i m p Com ulqtion rrvirrg trr. then by definitiorr thurc is no membership algoritlrrn ftrr it. Although Thring's tlxnis letadsus to believe that thcre are f'ew limitations to the power of a T\rring ma. alrrong thern tlx: wellknown halting problern for Thring rnachirres.chine.ll there was to this issuc. For r:xa"rnplc. Thrirrg urar:hine. This question is clearly of prrr. But the problem goesdeeper.loccasions that there coukl rrot uxist a.
mwe will understir. WInt is more plrter that wc carr now cottstruct or even pla.ssify a. sitrlrkl "yr)$" r)r o'rr).indornitirr is saicl if to be c:omputrr. l'or example.in." 'I'he probletn is to it is fa. Hcrc we find quite a few importa.t we do not know what the correct' . sirrce the allswer is either single instance of a problem is ir.nt ltrobletns for which. This may seenrlike a fa. the set of all contextfree gramma. we talk about a problem being decidable or undecidable.f rlrr pnrt of its dorrtairr. ond Computobility Decidobility In Definitirlrr 9. becausethis ma. there is iln rrnrk:rlyirrgtlorrrain. with irrt apparetrt possibility of an a'lgorithmir: solution.Tlrrirrg rnac]rine t]ral computes the frrnction on ther wtxlltl of its dotttaitr.t a. We say thtlt a prublc:rnis decidable if there exists a. thc:rc are Ito algorithms.ges. fhe fact tha. but we call the function comprrtable only if there is a. each of which must he either trtrtt or false.4.geI. T S o m eP r o b l e m s h o t C o n n o fB e S o l v e db y Turing ochines M The argurnent that the power of met:hilnir:irl txrrnputations is limited is not surprising. After this. for others G. when we cla. In the first cirtrcra TLrrirrg rnachitre lhat always answers gives ttxl rxlrr(x:t irr$wer. Specifically." lrr this case. wc look at sotne questions relnting to contextfree langua. we rnust be clear on wira.ya.city of arty cotnforerstx:.rt which are known to lltl rrttsolvableby atry computer.chinetha.mbigrrorrs. ftrrrt:tiorr rrs cortrputable or not cornputable.lwirys "ttue" true or fa. certrr. lem rnay be decidable on some rlomairr llrt rrclt ort atrother.bltl there exisls a Tlrring ma.t computcs t]re value tlf / frlr all arguments in its doma.t its rlomairr is.coRt'ruMIC CoMpurATIoN cnumerable languages.lse.flectthe:c:onrlhrsiotr. a rkx:idablc. Our conceru her:e will be the sorncwhat sirnplilicd setting where the result of a comlrutatiotr is a. We see fron this that.bility results. br.rs. A firnctirin is rrrx:rlrnputableif no such 'I\rring machine exists.tement "Fer il t:ontt:xtfrtxl graIIIInaI' Fur sttrtreG this is true. wc rntrst always The probkrrow whal the domain is.brrt cleir. the langua.300 Chopter l2 Lrurr$ oH Ar. we consider the sta. Intuitively wc krrlw that many vague and speculative qutrstiorrs require special insiglrt and reasotritrg well heyond the captr.rlywc rnust have otte or tlte other.cetiollrtrn$w()rr tnrt it ernphasizes atr ittrporlatrt point.nda set of relatecl staterrents.lways answers "firlse" is appropriate. filr Whetr we state decidability or: undecida. By a proble.Tirring macirine that gives tlxr txrrrcct irnrJw()r t:very statetnetrt in the domain of the problem. Them miry be a Turitrg rnachine that can compute . wrr stated that a function / on a.r. while in the second case one tha. decide whelthrlrthrl stirtcrrtetrtis true for any G we are given. Again.lse. unfortunir. (G) is rr.tr:ly.rsibly is that thcrc arc questiotrs that can be irrterestirrg to computer scientists clearly and simply stated.
T\rring machine fuI : (Q.whcn startecl in Lhe iritia. perform rr.nv ?rA. We will imsllme tha. F).u). But as wc now show.:11'w if . we rrskwhelther M applied to . Sirrrply stated.string that describesa.l.yr. A.rlrrect answer for any M arrrl ur by perfbrming sorne arrirlysison the machine's clesclilltion a.ll ru.urhalts. a. TheTuring Mochine HoltingProblem Wt: begin with sotne problurns tha. What we need is an algoritlrrn tha.ttersis that tlxlrc exists some'Ihring machinc that cloesgive the corr(x:t re$ponse. br:(:ir.t have sornr: historica.fiinnrnmtmW. starting point frlr clevelopinglater results.lting problem.l configuration {6ro. given thu rlt:scription of an arbitrirry M and tl.nclru are encoded as a string of 0's irrrrl l'$.t'ur. It rnrry simply be a caseof a.d. 'fhe dorrrairrof this problem is to bc tir. no such algorithm exists. are both fitral states of ff .c:tionof M on tu. Thring machine M irrrd rr.ke a specific tlcfinition of whal. then no matter how long we wait.t if M applirxl to zr does not halt. r'Rfi$.it is convenient to hilvr: ir.we ma.If M en[ers an irrfinite loop.nd a. )_1.t rr. for this rcilrJorr. Here q. Tlrc bestknown of thr:sr:is the Ttrring rnachirrc: halting problem.bhreviated way of talkirrg aborrt the problem. tha[ is.ven ey TuruNc.tion.ninput 'ur.q a.tthe same tirne givc: rrs a.Ni'.nclthe input. precise idea of wlra.1 Solrn Pn. M. n.otl. wlfch firr a.ur.does . We cannot firrrl the iruiwer by simulating tlrc a.rr rrl u'. For subsequent discussirirr. w() (iirrrrrever be sure Lhat M is irr ftrct in a loop.ssuggestedirr Secltirln10.what rna. arrrl g".4.t we mean by tlu: ha.cHtvt.rand ?r.he cornltrrta.kenas the set of all l\ring machirrcs ir.tr' applied Lo .8Nllili Let up1 be ir.t can detennine thc r.f. A solution of the halting problt:rn is a T\rring machine H. r1o.l signific:ance arrrl thir. anrl qo'tlt It) i. halts or docs not iralt.rrse there is no limit orr the length of l.or simply (M. .'u bc a strirrg in M's alphabet.rlernis: given the description of a.12.. Q(Ju.4.c:omputation that t:vcntrrerlly halts? Using irrr tr.c.s 301 alnwcr is makes tro differerrcc.. llcrforms the computatiorr i *rqo*r. we arc looking for a single T\rring marchinethat. atrcl let. say by perfbrming it ol a urrivt:rsill Thring machine. will predict wltctlNlr or not the cornputatiori of M applied to to rvill hrrlt.vcry long cornputation.EMS THAT Carvruor Bu Sol. the prol. we statc:rlsrlrnewhat looselv above.
2 wc warrt to colrvey that the trtt.iglrtforward..'!!2.chine ff . {ri or qr.tl w^ill htr.1 .1. f if M appliecl to tl does mrt halt.nbe visualized }ry a block ditr... s&y. irr situations where fI retit(lhtl${rr alrd iralts. Proof. in srrt:hrl way that lhe tape remains unchanged. thrrt solves the halting protrlcrn. regardlcssof the tape syntbol.gra.. that there exists a.ddeclstates itr Figure 12.c+orrtrtIr.!!2. We imsurtte the contrary.14ru. The input to ff will be tlte string u. As reqrrired by Definitiotr 12. With thc a.tIc Thurc cloes not exist any Ttrrirrg rnachiue Fl that bchirves as required by Definition 12. if M is startsl irr state {s with input rit14tr.nsitiorrs tretween state qo atrd tlrtl rrtlw states q* and q6 are to be rnade. qow u) i rt. and (1.grarrtis to indicate that. Next.lt in ttnt: of two correspottding firrrrl states.. givctr any wMwl the T\rrirrg machine.lt with either a yes or no arswer. wermodifu fI to produce a. namttlv.halts. if M applied io 'u. the rnodified machine fll will t:rrter att infinite loop.t n lltQ.rn like Figure 12. Figure 12.. Thc wtw this is done is stra.2. The haltirrg problem is therefbrc urrdecidable. The siluation ca.U't M if M applied to ru dot:s rrot lrrrJt. TIte intetrt of this dia.n algorithtn. if M applied to ur halts.lt in state q. it will eventually ha. The: rcxpiretnent is then thtr. We achieve this trv askittg that 11 ha.t. Comparing iI and II' we see tha.t.*. we want ff to operate according to thc following rules: Qowtwuisngyn2.T\rring rnitt:hirreiI' with the strrtctrrrtl showrr irr Figure 12. Formirllv.Ar. or 4r. and qDwMu. the action of 11' is descrihed by qilt)Nt'tu irr.1.302 Chopter l2 CoururNt'lol Lrlurs or. and consequently sorne I\rring ma.1.
This is clearlynorsense. not hard to see tlnt these somewhat arbitrarily chosen conditions play only a minor role in the argument.often we can tell by an analysis of M and ur whether or rrot the T[rring machine will halt.F is applied to fi. We have tied the problem to a specific definition for the sake of the discussion. if fr applied to fi halts.rCaruruorBp Solvno ny TuRrrrrcMacnrNns 803 Figure 12. It is.q. and qofr n AtenUz. I One rnay ohject to Definition 12.. in additiorr to being the description of . if JI.identifyirrg M with Il. say. Flom the ir. ending in its^irritial state qs. After that. ri. It is important to keep irr mind what Theorem 12. to solve the halting problem. "I1 had to start and end in very specific configurations./ applied to w71halts.l says.also can be used as input string.. and that essentially the same reasoning could be used with a"rryother starting and ending configurations.1 SoMEPRoBLEMS Tg. we get * gsfi l6 oo. however. it behaves exactly like I/'.bove. i if fr applied fi does halt.12. Ff. Now fI is a Ttrring machine. must be false. Tlfs string. Then the ar:tion of fI is such that qow*t l ii Q11wx4u)14 *. What . but this does not affect the conclusion. 1}*. and hence the assumption of the decidability of the halting problem.[y'. It does not preclude solving the haltirrg problem for specific cases. and qowm ifi qouMlilM f p Afl.1. since we required that. contradiction to not The tells us that our assumption of the existence of 11. so it has a description in {0.2 FYom f/'we construct arrother Ttrring machine f/. if M applild to wp1 does not halt. We can therefore legitimately ask what would happen if . This new rnachine takes as input ur14 and copies it.Az.
L be a recursivelv errurlerable languageon E.[.'no. d.304 Chopter  2 LIvIrs oF AlcloRl'rHr. w € E+. then (lvety recursively crrurnerable language would be recursivc.mplesto illustrattr this idea. Iet . that is. so it will eventually tttll us whether tu is irr .1a. Consequentlv.5 is a consequenceof the fact that the halting problettt and the memtrtlrship problem ftrr recursively enumerable languages artl rrearly identical. The argumerrts for proving Theorem 12. q is ever entered .L. illustrates the very important teN:lrrrique reductiorr' We say that problem A is reduced to a problem B if the decidability of A follows a. The stateentry problern is as follows. The proofs of Theorems 11. 40.F1says 2. whereas in the membership problcrrr we do. making L recursive. if we ktrow that A is urxlecidable.tIcCotueurrrlom the therlrem says is thrrt this caunot always be done. 1. and let M be a Ttrring mac:hine that accepts . This problem is urrdecidable. the halting protrlcrn is undecidable. If H says "ycs. But M rnust halt. solves the halting problem." then apply M to u. t:ttrrrrecting the halting problem to thc ntembership of probletn.roclosely related. Let us do a few exa. '. thrrt the halting problem is undecidable.2. from the decidabilitv of B. Giverr atry Ttrring machine M : whether or not the state (8. Apply H to w14tt.L or not' This constitrrtcs a trtembership algorithm. we carr conclude that B is also undecidable.t catr make a correct decision for all toy and ur.5 (via Theorem 11. The1.3) and J. thtlre is no algorithrtr thrr. t The sitrplicity with whit:h the halting problerm can be obtaincd from Theorem 11. <lecide wherr M is applied to tr. both being a version of diagonalization. X. Let 11 be the T\rring machine that 'We cotrstruct from this the following proceclure. f . The only different:c is that i1 the haltirrg problem we do trot distinguish between haltirrg in a final and rrotrfinal state." thett by definition rrr is not in . The corx:lusion of the theorem is actua.If . But we: already know that there are recrrrsively enumerable lattguages that Arc rtot recursive. Proof: To seethis. F) and arry q € Q. Problem Another lo Reducing Undecidqble One The above argument. The contradiction implies that I1 cannot exist.L.lly inrplied irr previous results as the following argument shows' If the haltirrg problem werq tlecidable.1 vrcre given becarrsc they are classical anrl of historical interest.
)..12. To show how this rtxhrr:tion is accomplished.. we use the algorithm for solving the blanktape halting problem. If A rr. Given a Ttrring urachirrc M. ..R ) .*) does not halt. ru). This is undecidable. srrch an algorithm clearly exists.nswers yes.u. .q. we chrr. that is. We first rxtnstruct from M a new rnirr:hine Mthat starts with a blank tape.1. If A says no.will halt on a blank tape if arrd only if M halts on ru.ny M and .11. it cloes$o beca. the starteg is entered.2 is summarized in Figure 12. After that. We can do this simply by looking at tlxl transition function d of M.teentry problem is decidable gives us an algorithm fbr the halting protrlcur. M. Tltus. the assumption that tlrt'sta. Supposenow that the blanktapo halting problem were decidablt:. I The construr:tion in the argurnents of thesc two exelmples illustrates an approach cornurorrirr cstnblishing undecidability results.crrrlus 305 Trr reduce the halting problem to the stateentry problem. If M halts. we first modify M to get M irr urrcha way that M halts irr state q if and only if M halts. writcs ur on it. suppose that we have an algorithm A thrrt solvesthe stateentry problem.lts. l ( q n a ) : ( q .u..use some d(qi. For exanlple.JrJume exists. a . The conclusiorr tr:lls us whether M applied to tl will halt. Given any (M. where q is a Iinal state. then positions itsclf in ir.. tlrcrr (M.' M and some tu. Clearly M. thon apply the blanktape haltirrg problern algorithm to it.regiven sornt. assumc tha. we first us(] an rrlgorithm that In transforms (M. The constnrc:tion in Example 12. an algorithrn for the blarrktape haltirrg problem can be converted into an algorithm for the haltirrg problem.il The blanktape halting problem is anotlrcr problem to which the halting problem catr be reducerl.t we a. which wrr a*. acts like M. that diagrarn.nge every such undefined d to .w) irtto M. A block diagram often helps us visualize the process. Since this can be done for any M and ur.l SoME PRoBLEtMsTHnr Calrlror Bn Solvnt ey Tunrlrc M. Because the halting problem is undecidable. We apply the stateentry algorithn eto ('rt. the same must be true for the blarrktape halting problem. determine whether or not M halts if started with a blank tape.rr) is rmrlefined. T Bxfiilnpls t fi. Sirrcc the latter is known to be urrdercidable. the stateentry problcrn must also be undecidable. configuratiorr qoil). But this is impossible.. given a.rrr) ha. To get fr. then (M. and we can rruncludethat A cannot exist. Putting the two together yields an algorithm for the haltirrg problem. we first construct Mr. Next. We could then uscrit to solve the halting prohlem.
This function.llltl funclions we must Iook a little firrther.vrlorrly final sta.lfurtctions to to stx: if they are comprrter.ttcrnpts to preclict thtr bchavior of T[rring rrrirr:irines.pplietl to a blank ta'pe htrlts irr rrd more than rzr nrovcs. hlarft tape makes mtlrtl tltatt ?Iumoves. We cau loclk ir. All we hirvc to do for this is to hirvr: M count its moves arrd termiuate when this r:outtt exceeds rn.he fr. for exarrrple rnachines that ha.nd ritrrge. Of a.sstlciated with tr.rrurn's Figure 12. Iet trs rrtake sure that .rnctitlrr question.f (n) is dclirred for all rz. Most exa. This is because Q arxl 1' are finite' so t)iras a finite domain a.ktrthe Iargest to give 'l\rring Tirkc irny ma.chinethat does halt will execrrttl a certaiu number of rntlvesl of these. 1.chine M and positive numbcr ttl. Assrurlc rlow ihat / (tz) is <xlrnputable by some T\rrirtg I (n) .ttcr will always ha.reorrlY a finite number of T\rring machines with n statcs. ir true or false ilnlJwcr.lt with one of two answers: M a. so for exermplcs of uucotnprrta. Every ma. we expect that functions t:rrcoutttered prar:titlal circurnstanceswill he crtrnputable. 1}.rnllltls of uncomputablc ftrtrctions are a. Some of the zstate tnachineswill rxrt halt when started with a blank tape.lso at rnore genertr. It is easy ttr modify M to produce M in sttch a way that the la. but they do not enter the definition of /. or M applied to ir. as it turns out. we f'olltlw the established trcthocl and rerhrr:c the halting problcrrr (or any other prtlblern knowu to be rrntlecidable) in to the problern of computitrg t. is not cxlrnputable. I'his in turn implics that there are only a linite rrurnber of different dts and Lherefore a firrite rrurnber of difl'ercrnt rlstate T'uring machinrrs.306 Chopter l? ou AlcoRlrHMI(t CoMPUTATIoN I. do so. Let f : {0. Noticg first that there a.Il of the nstate machintls.tesand therefore tnake no mov()lt. Before we set out to dt:rnonstrate this.5 Algorithrn ftrr halting problerri. there are some that always halt. Considor t}rc function / (n.) whose value is the mtrxirrrutn rrumber of moves thilt catr be made by any zstate Ttrring milchirre that halts when started with a blattk tapc. that i$. n}. A decision problem is cffectively a' fitnctiorr with a ra'nge {0. we ttr.blc. Becatrscof Tbritrg's itr thesis.
1 to require that qo. Show that even in this restrir:ted setting the problem is unrler:idable.rrrnr. lirst we compute /(lQl).nktape halts or rloesnot halt in lcss tha.This tells us whether M applied to a.r MAcHlNns sOz Figure 12. We carr then put fr ancl F together as shovrrrin Figure 12.chine 4 .then trtx:tr. We say that such a problern is decidable iffor every u there exists a (possibly different) algorithnr that detcrmines whether or not (M.t. R. we ask for an algorithrn that gives thc corlect answer lbr arry M and u. the irnplicirtion is that M nevc:r ha./ iu not cornputable. for exarnple. by looking for an algorithm that works for all .trla. depertding on whether M applied to rl ha. Irr the genera.blc.ltsor not. 3 .12..universal T\rring ma. g. tu) halts. If we find thal . Mdoes not halt in rn steps rrtirr:hineF. detcrrnine whether or not thc symbol a is ever writtcn when M is appliecl to 2.te of M. qow*tw i I o.. W .ncla clescription of ffi is givt:n to a. where Q is thc ster. Thus of we have a solutiorr to the blank tape hrrlting problem. We can relax this gencrality.4.rrr € X+.lhalting problern. Show that the following problerrr is undecida.fapplied to a blank tapc makes rnore tharr /(lQl) rnoves.ke if it is to halt. . a E f. I I' Dcscribe in detail how H in Theorern I2.1 sorvrsPRonlnvs T'rrn*rcaNNor Bp solvnn nv Tun.1 to show that this diflererx:e in the clcfinit..o.4.il. The value wc get is then usecl as rrr ttl r:onstruct fr as outlinecl.chinefor extx:rrtiorr. r'he irnpossibility of tlrc conclusion forces rrs to accept that .u q.n / (lQl) steps. a.4 Algorithrn Iirr blanktape halting problern.ximum number of moves that . M. Given any T\rring ma. and.iot does rrot affect the proof in any significarrt way.n bc modified to prodrrce Ii' Suppose we r:hangeDefinition 12.41 }lrt only a single .\4 r:arr ma.lts. Thiu tells us the set mir.use the definitiorr of /.eexarnirre proof the of Thcoretr 12.lxc.tu.1 ca.
Give an expression for rn(rr. ffi L How is the conclusion of Exercise 7 affected if Ml is a finitc automaton? 9 . Show thatr thc problern of determining whether or rrot niu u is undecidable.tIc show that there is no algorithrn to decide whether or not an arbitrary Tirring machine halts on all input. 14. ffi Considcr the set of all tlstatc Trrring machirtes with tape alphabet  : {0.2. in e$$errce. . givetr a pda as in l)efinition 7. I. !} and let b (n) be the maxirrrurn nurnber of tape cells exatnined by any rzstate Ttrring rnachine that halts when started with a blank tape.ble' brrt not recursive.lrages. give the values of / (1) and / (2)' 1 2 .308 Chopter l2 Coururun'tol.) is not t:onrputable. Show that this set is recutsively cnumera. Show that there is no algorithrrr for dcciding if arry two Ttrring machines M1 arrtl Mr accept the sarne language. Itrvariably.GoH. In Example 12. Show that b (rr. As we now show. Deterrnirre whethcr or not the following statement is true: Arry problem whose dornain is finite is decidahle. 1. Corrsider the questiorr: "Does a Turing trarhine iu the course of a computation revisit the starting cell (i. can we always predict whether or not the automatorr will halt on input Trr? 1 0 . that.3. Recursively erlrrnerable languages are so general any question we arrk about them is undecidablet. 1 3 .tIrson Al. the t:ell under the readwrite head at the beginning of the cornputation)?" Is this a decidable questiorr'i D. but rather is the gcrreral rule. 7 . ffi 1 1 .t there is no membership algorithm for recrrrsivtrly tlrrurnerable languages. Let M be any T\rring tnachinc and r arxl gr two possible instantaneous descriptions of it. 1 5 .l LItr. Is the halting problcm solvable for deterministic pushdown automata. Lct I' : {0.). 1ti. that is.rrguages. Lct B be the set of all Ttrring machines that halt when started with a blank tape. Thc lack of an algorithm to decide on some propertv is not an exceptionirl state of affairs for recrrrtrivcly enumerable lan. 6 . t}re nutnber of distinct Tbring machines with this f. there is little we caII say about these la.rrttr. ffi ffi f P U n d e c i d o b l e r o b l e m so r R e c u r s i v e l y o E n u m e r o b l[e n g u o g e s We have cletermined thrr.D}.e. Slxrw that the problem of tletermining whether a T\rring machine halts orr arry input is undecidable.
.it checksits it saved input and accepts it if and only if it is ur. carr alwtr.rr.y Er*rurrrnRasr. then we carr prrt ? and A together as shown in Figure 12.t L(M".If we let 7 denote an algorithm by which we gerrcrateG.7.r tells us whether or rrot rt is in L(M).5 Mernbership algorithm. Figure 12.rallyclear is that. Eqr. we would have a mernbership algorithm for any recursivcly cmrmera. M first savesits inprrt on some special part of its tape..44 arrd .ted problem "L(G):0" i s n o t d e c i d a . whcrrerver enters a final sta. If sur:h ir T\rring machine existed. Clearly.I Figure 12. Let G bc an unrestricted grailrmar.t there exists an algorithm A for deciding wlrt:tlrcr or not tr(G) : E.): L(M) n {zr}.. srrch tha. Usirrg Theorern 11. Suppose we are given a Thring rrra(:hineM and some strirrg zr.u. thc rxlnstruction leading frorn M and tl to G. Assurne now tha. We can modify M as folkrws.ysbe done.E LaNcuncns 309 whetr we ask a question about recursively enurrerirble languages.ECUR$IvRr.blishedresult. whethcr or not Then the problem of determinirrg L(G\: O is undecidable. 2 UNDECTDABLTI PRonlnus FoR R.5 is a T\rring rnac:hine whiclr for any .te.nindication of tlre seneral situatiorr.) is nonernpty if and only If w e L(M). b l e . Wu conclude therefore that the sta. Proof: We will reduce the menrbership problem for recursivcly enumerable languages to this problem. we tlrcn construct a corrcsponding grammar G. creating for r:irr:hur a machine M. we find that there is sorneway of reducing the halting problern to this question. We give here sonre examples to show how this is done and frorrr thcso examllles derive a.ble language.5.12.L(G. irr dirs:t contradiction to a prcviously esta. We carr do this by changing d in a simple way. Then. L(G) not empty ueL(M) eL(M) .
4.6 . ur) halts. thgp fr will tralt in a final state. M either accepts the infinite language X* or the finite Ianguage 0. b}.*).tIc Lct M be any Ttring machirre. If M halts in any configrrrrrtion. we can construct a solutiorr to the halting problcm as shown in Figure 12. First. Therefore. the original mat:hirre is modified so that the new machine ff first generates tr on its tirpe.chir. using atrd some otherwise urrused space. To show this.sked. If so (M. In other words. the the newly crcer. then frwiil not halt either a. except that whert M reaches a halting configuration.teclur moves rrrade by M after it has written u on its tape are the salne as would have been tnade by M had it started in the origirral configuration gnrr. For this. all input is accepted hy MThis can be done by having any halting configurtrtiorr go to a final state. M will rea. ye use exactly the sarnc approach as in Theorerrr 12. (fr) is finitc. Therefore no algoritlrrn for deciding whether or not tr (M) is finite can exist. Sccorrrl. it will be modilied to accept the two strings a and b.u) tloes not halt. is imrrraterial. thetr performs the sarrrecomputations as M.ncl will accept nothing' In other words.rc clnttged so that if any one is reached. the halting states other of M a.4. From M we construct arrProof: Consider the Blting problem (M. the problem . We can change the nature of the problem without signifitnntly affecting the argrrrnent. If we trow assulne the existence of an algorithm A that tells us whether or not .I'r'ulr. if (M.310 Chopter I2 Coututnrlot't LIuIrs oF Al.6. I IE Notice that irr the proof of Theorem 12.L(M) contains two different strings of the samc length" is undecidable.narnely "Is L (M) finite'i". T\rritrg machine M that does the following.colt. tht: irritial input is saved and at the Figure 12. final state for all input. Show that for an arbitrrrry Thritrg machiue M with X : {4. the specific nature of the question a. Then the qrrestion of whether or not L (M) is finite is undecidable.
Thcse questions."rrr I2. Show that the two problems mentionetl at the end of the preceding section. I In exactly the sarrrc ilIirnner. ffi a Slrow Show that the question in Exercisc 6 is undecidable for any fixed G2. Thus. ffi . M will accept two strings of equal lcngth. ffi Show that the problem 4 . This tlreorem states that any nontrivial property of a recursivcly enumerable latrguir. and Gr any regular grammar. A general resrilt fbrma.rgument then procxrercls as irr Theorem 12.w) hirlts.pLer.L (M) contains any string of length five.L (G) : f (G)R? 6 . L lbr an unrestrictecl grammar G.ra. Let G be any unrestricted grarnmar.rs non R. . show that the question "Is tr (G) : L (G)*?" is undecidablc. " L (Mt) C L (Mz)" is rrndecidable.cceptingonly these two strirrgs. LeL Mt and Mz be arbit. The adjective ((nontrivial" refers to a property possessedlly sorrre but not all recursively enumeratlle Ianguages. Let G be arry rrnrestricted grarnmar. The rest of thc ir.redwith a arxl h.f) regular?" without affecting the argurnent essentially. 1 . as long a^rtr (G2) is not empty.L (M) is regular. we can substitute other questiorrs slch as "Does L (M) contairr tr. otherwise fr wiil accept rtothing.[f in Th"o. Show in detail how the rrrachine .4 is constructed.A precise statetnent arrd a. Does there exist an algorithrn fbr determining whether or rrot .proof of Rice's theorem can be fourrd in Hopcroft arrd llllman (1979).r'y T\rrirrg machines.4. as well ars similar tluestions. are undecidable.L (G)E is recursively enumerable? 5 .rrystring of length five?" or "Is L (.geis undecidable. Does there exist an algorithm for determining whether or rrot . (b) .lizing this is known as Rice's theorem. are all undecirlrlble. a.12. namely (a) . if (M. that the problcrn .rcuAces Bl1 errtl of the cornputatiorr compa. Let Gr be any urrrestricted gramrnar.ncuRsrvr:uy ENulrnRanl.L(Gr)n L(Gz): is urxlecidable.4.2 UwpBcTDABLE PRonlu. Argue (a) frorn Rice's Theorem and (b) from first principles.
1} and take A and B as : r u r : 1 1 ..312 Chopter l2 LItr.nt to study atrd therefbre make the argurrrerrts easier. These intermcdiate results fbllow from the rrndr:r:idability of the lrtlltirtg problem. say A: Ld B : UtrI)2r. The Post corrcspottdence prohlem t:itrt be stated as follows.7 1 ul u2 I .rlz : 001.. 0 0 v2 "J I .h.ue:001..rI)r1 u)Lt.us:011 Figure 12. whether or not there exists a. Post correspondencesohrtion (PCsolution) for pair (A.l interelst. there exists a PCsolution as Figure 12. If we take ?01 Ul = 00.i... But irr rrratry instanrxrs it is cumbersomrr ttl work with the hrr.S Let x: {0...ltirrg problem dircctly.PCsolutiotr' Eifrrlplq t{. and it is corrvcrrient to establish $ome intermediate rcsults that bridge the gap hetwecrr the halting probkrrn and other prohltrrns. brrt are more closcly related to the protrlerrrs we wa.)1r.7 sltows. particularly in tlrc area of contextfitNl languages. t . such tltat U ) i ' t . B) if there is a nonelmpty sequenceof integers i. for any (A...u2:11. : 1000 urs :0. One srrchintermediate result is the Post correspondence problem.?s:1I For this case. The Post correspondence probk:rn is to devise an algoritlun that will tell us.1 u.LDzr. t " i' u k . Given two sequence$of z stritrgs on som{} alp}rabet E.B). ' : UiUi''' 1. ? r 2 1 0 0 r u t r: 1 1 1 ur:111.tIrs oF ALcoRI'I'UMIC CoMPUTATIoN Problem fiWilffiThe PostCorrespondence The undecidability of the halting problem has many (toruequencesof prirctica.Il)n we say thnt there exists a.
.tely obvious. With these..f. The Post prrlllklm is therefore undecidable. to it. I In sptx:ific insta.rt. we break it into two parts. BbC.rndecidable. Cl . the first elernt:rrts of the se an MPCsohrtion.i.f. For tlrt: sa. We want to claim eventually that rp e L (G) if and only if the sets A artd B txinstructed in this way have an MPCsohrtion.restion under irll r:ircumstances.b. .} .I2.lgorithm for deciding if arr artritrrlrv pair (4. the string .oeLEM 313 there cannol. B) permits a PCsolutiorr. be arry PCsohrtion simply becausc a"ny string composed of tlltlments of ..8. This problem is a. then there is also a ltCsohrtion.9 =+ is to be taken as'u1 irrrrl tIrc string F as u1.tJ*Ti.Ia. Itr tlrc first pa. we iutroduce the followirrg r:onstruction. Iet us illustrate it with rr simpkt example.:K'J#liJJT Exom$lf I f. but the converseis not true. Wcrwill demonstra. In Figurc 12.". Wc ua. " In the modified ltost corrt:sllorrdence problern.lsor. The rrrotliliod Post correspondence problern is to devise an a.ll THr'. the membership problem for rrlcursively enumerable languages.Posr CoRRnspoNDENCD Pn.8. we create thc pa. Ttr this end. Brrt in general.keof clarity. The order of the rest rlf the strings is immaterial. Supposewe are giverr arr unlestrictcrl grammar G: (V. R) admits an MPCsolution.ythat the pair (A. that no such solution can cxist.'l'.5.. corresponderrce To show this is a sourcwhat lengthy process. such that 'W1'Wi'Ui "tttk = U11)i1)i'''U1x.. as we did abt)vc.xTIffi ilTlil'll.aABblBbb. AC + uat:. Since this is perhaps trot irnrntrditr. we introduce the modified Post correspondence problem.4 will be lorrger thnn the corresponding strirrg frrlrrr B. ilirr. B) as shown in Figure 12.d Let G : ({A. B) has a modilied Post correspondencesolutiott (MPCsolution) if there exists a sequrrnr:e of intrlgursi.P) arrrl a targr:t string ur. there is tro algoritlun ftrr dor:iding this qr. ()r we may be able to argue.5.F.te the undecidability of thc modified Post correspotrdence problerrn by reducing a known undecidablc llrotrlem. B. c.ir (4.fr.nceswe rray be abk: to show by explicit constructiorr that a llair (4..:fi ". P) with productiorrs S .
in P } D and take ut : aaac.rrdB obtained from the suggested is con$tructiolr are given in Figure 12.e.8 d F. The integers above and below the derivation string show the indices for z. The sequencesA a.e. Figurc 12.C \ B Fis a symbol not in FU 7' a foreveryaET v tl for everyVreV E JroE E is a symbol not in I/ U ? ti x.s A B C s s s E d'{Bb Bhh U ailt j addcb Bb AC .(Lc in.L(G) and lus a derivation S+aABb+aAC+a.c.' and u.314 Chopter l2 LIIrllrs ol AlcoRtrHMIc CoMPUTATIoN Figure 12. *1.wltere the first two steps in the derivation are shown. The string ru : a. used to create the string. for every *.10.ralleled by an MPCsolution with the corrstructed sets can be seen in Figure 12. respectively. vi I FS+ a tj t F tl 2 3 + 5 6 7 8 9 10 11 72 13 l4 b t A B c . How this derivation is pa.a.9 T u.9.
T. Let (A.ngua.ges the rnodified Post correspondence problem and to thereby dernonstrrrtethe undecidability of the lattcr.P) be any unrestricted gra.pspoNt)ENcE Pnoer. lending us to the secondstring in the partial derivatiorr..rur. we u$e tr10.mmar. Irr this instance. we is always longer than the corresponding string a1uiaj.12.rtreduce the rrretrbership problem for recursively enurnerabkl lrr.n.'16.linductive argument based orr the outlined reirsonine..a t s = "10 il14 b v12 + ilt^ v C '13 E .2 l vg Exarnine Figurc 12. The only exception is the last step. so we must sta. .3 THE Posr Con.7 *10 1J). indicate tho lines along which the rrext rcsult is established.11.8. Tho c:onstruction. We want to construct an MPCsolution.ppening.10 r:a.rt with tu1. we ca.. I Let G : (V.10 f wl %to s ?10 a d B b il! ur F . f. B) permits an MPCsolution if arrd only if w e L (G).S +.11 F vI 1 u1o a i D ur4 *z s a @r2 ur4 u2 a il u. I Witlt this result. B) be the correspondencepair c:t)rr$tructecl fiom G and trr be the process exhibited in Figure 12. together witlr the example.t and that the first is exactly one step ahead in the derivation.refully to see wlnt is htr..ir ('{. Proof: The proof involves a forrrrir. with u any string irr T+ . The complete MPCsolutiorr is shown in Figure 12. Therr thc pir.nlr S15 Figure 12.this brings irr'rr. so to rnatch it we have to use u1s or u11.. Looking at several rnore $teprJl see that the string u.ur1. B il'rz b = rt C vz Figure 12. where uo rnlmt he applied to let the ustring catch up.S. ^ u2 tus u12 4 s ?10 d n il. This string contains 5. We will omit the details. that is.
and B : 1. li : zi : wilQu)i2Q''' LDi. Lti... Proof: Given arry urrrestricted gra. By Theorem 12.mmar G: (y.n. We can then cttnstruct atr algorithm for the merrrbership problern of G as sketched in Figure 12. wt: construct the sets A and B as suggestctlabove. We must therefore c:clrrclude probletn' r algorithm for deciding thc rrrodified Post correspondorrt:tt With this prelimina. Proof. gri is creir.2.nrs on Al(loRIrttNatc CotrrlpurerloN Figure 12. defincd as follows.ry work. We argttc that if the Post corresponden(icl)roblem were decidable.tUn*tt D : z o t Z t t .5.' does not. l. .nrl rni : lu.1'ttn. a. wc are rlow ready to prove the Ptlst corresporrdenceproblem in its originirl ftrrm.t. .P) and ur € T+. r'1: lua In words.s16 Chopter l2 LItr. respectively.B) Iras an MPCsolution if and only if w e L(G). .. The Post corresponclenr:cprotrlern is undecidable. .4 and B as in Figure 12.\)i1 on lrorrrealphabet X. ..12.ttxlfrorn trri by appending I to cac:h l.n..T. ueL(G) vte L(G) The modified Post correspondenceprohlem is undecidable. but a membership algorithttr for that there ca.8..S.. .Aft. arrd ?i. now we assume that the modifiecl Post correspondencleprobSuppclsc'l Itlrr is decidable.. where ru4i irrrtl u4i denote the jth letter of u).r algorithm. thc lrair (A.i$. An algorithm fbr constrttctitrg A front B from G and tu clearly exists.12 Membershil. A tu* we Srrppose are given sequence$ : 'u)71Lu2r.il. the modified Post correspotrdence problcm woulcl be decidable. We then introdu(le rrew symbols g and $ and thtl rrtlw sequeilces C:Ao..ai1Qui2Q. Construct... tZ n l t .nnot be arry G and u. For i : 1.
with f . ( . B) irave a PCsohrtion? Does it havc an MPCsolution'l ffi Provitle the rletails of the nroof of 'l'heorcm l.2.. 0 0 1 .1' . .+1* #li.3 THE Posr ConnnspoNDEN(iEPRosr. Q1.t this inrplies ' W 1 U l . such a solution must have g/0orl Lhe left rurd Un1l olr tlrtl riglrt and stt must look like ' 'Qwtt(.urra 317 Figure 12. t algorithtrt for decidirig thc Post rxlrrosp{nclence 1 . .'rof u. To cornplete the deflnition of d and D.4.rJv tltx:idesthe modified Post correspotrcletrce problcrn. u t . We Assurne now that tlrrl Post r:orrespondence r:an then construct the rnachineshown irr Figunl 12. 'Qw.+I $r f() * 3lr 2". Q E . I 1 . Bcr:arr$el of the placernent of q arrd $.2 L u . . This mar:hineclea.13. Does the pair 1 } (4.1 1 1 . L e t A : { 0 0 1 . . q \ : g z t 1 1 t l u 1.n N{PCsolution. we see tha. j ' ' " l i l 1 r ': U1'11. 1 1 1 . . Brrt tlrc rnorlifiexlPost prohlern is undecidable. problem is deciclable. t . Ignoring tlte charactcrs r/ arrd li. while aa is obtainecl by prefixing each chirrnr:tr. r  . . .13 MPC algorithrn. 0 1 0 } . consequently.12. MPCsolution No MPCsolution charat:ter. B) pcrmits a. We can trrrrr tht: ilrgrrrnerrt rlrclund to show that if there is an MI'Csolution for (4. we camrot havel arr cortesprlrrrlcrrtxr problerrr. B) therr tlierc is a PCsolrrtion for the pair (C.5. Consider rrow the pnir ((J. 1 0 1 a n d I J : { 0 1 . D). i .i''' Ulrj so that thc pair (.1111Qw12' . and suppose it has a PCsolution. D). we tako A 0: g Y r ) : 3/n.
'5.. a 2 . &jilil .'ur) over some alphabet E. .t1r)tr.. t {or. . that is.re undecidable. {wiwi .1J)2.I. P . (a) There is an MPCsolution if there is a scqucnce of integers srrch "'utNwT = uiuj "'utrur. j 6. . We illustrate this with a few seler:tedre$rrlts.B) on a singlclctter alphabet. ffi 4. . aiai.i ... there is an algorithm that can decide whcthcr or not (4.l Now look at the contextfree gramma..&r}flX:O. Show that for lll : 1. n } . a n }. The corrcspondence pair (4..4.. (wt. B) is sairJ to have arr euen PCsolut'ion if and only if there exists a nonempty seqlren(:eof everr integers 'i. j.Show that the problem of deciding whether or not j. B) hasa PCsolution ftrr arry given (.. an arbitrary pair (..k such that wnwj . .4. new set of distirtct syttrbols & I r & 2 r .ion is undecidable.or.qrtox 3.3L8 Chopier l2 Lrvrrs or..wk : Itn'u . ffi that tuatrri (b) There is an MPCsolution if there is a $equenceof irrtegers such that ultrr2zriur "' u)k :'Dt'Dz'tti'uj'' "D hr. for Undecidoble Problems ContextFree ffiilffiffi [onguoges The Post correspondence problem is tr convenittrrt tool for studying undecidable qrrestions f'rrr txrntcxtf'ree larrguages. and qlnsider thtt twcl larrguages Lt1 : and Lp : luitt.r G : ( { S .u2. B) has an even PCsolut. Suppose we restrict the domain of the Post correspondenceproblem to include only alphabcts with cxactly two symbols. . 'r... S r a . .. } J U { a 1 .. AlcoRrrHurc Conrur. S ) 5 .. There exists no algorithm ftrr decidirtg wltetlter any given contextfree grammar is amhiguous.wtuth&ka4.... the Post correspondeuce problem is decidable. .ndB : Proof: Consider two sequences strings A: of (ut. 1)luke.ka. . . Choose a.a r . . Is the resulting correspondcncc problem decidable? 5. S U C hh a t . Show that the ftrlkrwing rnodilications of the Post correspondence protrlerrr a. .
S n } .ncuncns 3L9 wltere the set of productions P is the union of the two subsets: the first set P4 corrsists of SSa. G n : ( { S . 12. if G is ambigrrourr it mrrst be because there is a ur for which there are two derivations J :* Jg ) wi5nai L (Gs). 1.4 UuoncrDABLE PRonlnlrs FoR. But since there is no algorithm for the Post correspondence we conclude that the ambiguity problem is undecidahle. . If there existed an algorithm for solving the ambiguity problem.. Sn + uiSnailuiat. if G is anbiguous. . I .if G is unambiguous... Similarly. then its derivation with grammar Ga rrrust have started with 5 + u)isai.q. we could adapt it to solve the Post correspondence problem as shown in Figure problem. S a } . P e .n. . S ) and 'i: I. It is easy to see that Ga arrd G6 tly themselves rlre rrnamhiguorrs. then the Post correspondenceproblem cannot have a solution. Conversely. S.q. .2. ... Now take G a : ( { S . we can tell at any later stage which rule has to be applied. Thus.& .12. Then clearly Ln:L(Gn). a 2 .CoN'rEXrFRueL. .. the second set P6 has tltc lrro<1rr<:tions S __+ Sr. Consequently. then the Post correspondence problenr with the pair (4. S ) .... i : I. a n } . P n .n. wi]eatlwiai.. B) has a solution..L (G) ends with aa. If a given string in . Ln: and L(G): Ler Ln. ^ 4 * wiwi " 'wtrak " 'eiai : 111 and 5 =+ ^9gg ui]Bai 4 + uiui ' ' 'ukek'''aiai : v1. I \ ) { a 1 . a 2 . X U { a 1. .14.2.
14 PC algorithm. or if two different specificatiots of a language are in fact equivalent.320 Chopter l2 Coupuranron Lluns or Ar. Then the pair (A. That there are many undecidable problems connected with contextfiexr languages seems surprising at first and shows that there are limitations ttt computations in an area in which we might be tempted to try an algoritlrrrric approach. . Srrppostr common element. ' a j a . ' u t r a k ' . that is S a 3 t r t i t u. But the results that have beerr established tell us that this is not possible. We conclude that . Keep in mirrd that this does not rule out the possibility that there may he ways of getting the answer for specific . whilc otlters are more easily solved by establishing different intermediatc rc'sults first (see for example Exercises 6 and 7 at the end of this section). This reduction Drovesthe theorem.i' . r There is a variety of other known results along these lines. B) has a PCsolution. Somtr of them can be reduced to the Post correspondence problem. i .rrrrrurc Figure 12.8. and it would tra a waste of titne to Iook for an algorithm for either of these tasks. We will not give the arguments here.4. but poirrt to sorrreadditional results in the exercises. i and S 6 1 u i u i . u k e k" ' e j a .c+oR. B) hasa PCsolution. it would be helpful if we corrld tell if a programrnirrg language defined in BNF is ambiguous.L (Ga)n L(Gu) is nonempty if and only if (. Converscly. For example. then L (Ge) and I (G6) cantrot have a common element.if the pair does not have a PCsohrtion. . There exists no algorithm for decidirrg whcther or not r (Gr) L(G2): n n for arbitrary contextfrcc grarrllllars G1 and G2. defined Proof: Take as G1 the gralnlnar Ga and as G2 the Hrammar Gr a*s that tr(Ga) and L(G") have a in the proof of Theorem 12.
the problem "L (Gr) n L (Gz) is contextfree" is undecidable.. * 8 . xn." [.12. * 6.8 is regular.. * 5. then it must be empty. such that (a) the set of all valid rnmprrtations is /. For any such comnutation gourF fll F 12 F .e by therrrselves are unambiguous. there are invariably some situations for which it will breah tlowrr.XT[..4 UNDECTDABLE PnonlnrulspoR. Show that for every M we carr cortstruct three contextfree gramrnars Gr .rr. What the urrdccidirbility results tell us is that thert: is no txrmpletely general algorithm and that no matter how many dilTerent cases a rncthod r:an handle. Gz.f  . Let M be any Tlrring rrrachine. * 7 . I. Use this to show that the problem "I (CJ) is regula. We can as$lrme without loss of generality that everv computation involvcs an even number of rnoves.(Gr)c t(Gr) is trnrler:idahlefor contextfree grammars Gr. * 4. Use the results of the above exerciseto show that "L (G) : X*" is undecidable over the domain of all contextfree grarnrnars G.'l Ihis is called a valid computation. Gr.rJ is urxler:idable for contextfree G.Cor'r.8 that Ga and G. frrr arbitrary contextfree grammars Gr and G2. Show that the problem "lt C L (G)" is undecida]rle. Let Gr be a contextfree grarnmar and Gz a regular gramnrar. the complement of the set of valitl t:ornprttatiorts) is I (Gl). Is the problem I ( G r )n L ( G z ) : o decidable? . Show that. Let Lr be a regular larrguage arrd G a contextfree grammar. Prove the claim made in Theorem 12. Show that thc problem of detertrining whether or not * ? 1.'REE LeNcuecns 321 casesor perhaps even most interesting ones. Show that if thc languagc L(Ge)nL(Ge) in Theorerrr12. GB..ff. (h) the rret of all invalid comprrtations (that is. F :r. t 3. (Gr ) n L (Gr). we can then construct the strins qot F *fl F rc.
rrrnrrc CoNrpurerroN * L Let Gr anrl G2 be grammars with Gl regular. (b) when G: is contextfree. (c) when Gz is regular? .on.c.322 Chopter l2 Lrurrs or Ar. Is the problern L(Gr) = L(Gz) decidable wherr (a) Ge is unrestricted.
T\rring's work was published in 1936. Eventually.sed the discipline entered an era of explosivr:grclwth. The results that were fourrtl slrrxl light not only on the concept of a mecha. In fact. To understand what Thring was trying to do.t tha.l r:ir. other rnodels hirvt: llccrr proposed.lcrrlus Newton and bv Leibniz in the seveuteenth and eiglrteenth c:etrturic:s. No t:rlrnrn(:r(:ilrl rxlrnprrterswere availahle a. we must brielly look at tlxr statc of ma.t tirne. M. At variotrs times. some of which at flrst glance seernedto tre raclically different f'rorn Thring machines. thev a. A mrmber of and 323 . Much of the pioneering work in this area was tlorrr:in the period hetween 1!130 and 1940 and a nurnber of rnathernaticiirrrs.l goal was not to provicle a fotrrrhis da. Although T\rring's ideas eventually becarnc vcry importattt irr cornputr:r sr:iunr:e. contribuled to it.A. however. With the discovery of diffr:re:rrtiill irrrtl integra.whole. all thr: rnorlcls were fbund to be equivalent.t that time. but orr rnatlxlrnaticu ar a. lhe whole idca had bccn rxrnsideredonly in a very pt:ripherir. Ttrring among them.tion for the study of digital r:ornputcrs.l way.re not tlre only ones.thematics inr:rea. irrtelrcst in ma.O t h e rM o d e l s of Computqtion Ithough T\rrirrg rnar:hinersa. origina.re the most general models of cornputation we carr corrstnrt:t.nical cornputation.thematicsa..
Frlr sorne tirntr it was hoped that consistent and complete systomtr ftrr rrll of tnalhernatics could be devised thereby opening the door to rigorous but cotrtpletely mechanical theorem proving.324 Chopter l3 Motnls on Clolrlru. then shown to be ftr. different areas were studied.rt with er. Itowever. aud atrything cleriveclfrom an inconsistent $y$torrrwoultl tltl cotrtrary to all we agree on. Rut this hope wrs rlir.t]rcrrratic:s t]te crrd of the nineteenth century.tiorr carr be checkeditr a routitre atrd completely mechanicalway. tvpit:a. irrrd lortg ancl involved proofs have heen published and fbr. Tlrrl first was that the system should be consistent.lse irrxlthtlr ctpirlly valid argumeut.nbe seen casilytt irrrd ttit follows from this.t. at !'indirrg such lortnal systetns was a ma. Ttr see why this was llecessary. By the end of the nirxlteenth century.refulapproach. But to those demanding cornplete reliability. since it is possiblc to overlook things.ble.irnsis rrrade.tical knowledge in the proce$$.rgrt.nypropositiorr cxpressible in the syst.ct to a. then iL is considered false.jor goa.isely definecl nrlels for logical inference and cleduction.t tlrc rt:sult is true. use faulty hidden assumptions. By this wc rrx:irrrtltat tirere should trot be atry proposition tha.rnclerr*one()uiJ orrly after a considerable amount of time. A secoud concern wils whctlrtlr a systern is complete. This krd to a concern with rigor in reasoning and a consequent examiniltiorr of the foundations of rna. Thc: nrlr:s rnust bel such Lhat the correctnessof their a. In his famous Incornpleteness Theorem.rart(ltt O'r'truR." Such phrases a. the body of mathcrnatical knowledge hacl become quite la. Ihe arguments throw light on the subim:t attd at least increaseour confidencethir. if challengedto do so.n be proved to bc tnxl by orre sequellce of steps.thema. this type of reasoning is a. A propositiorr is irr considered proven true if we can derive it fiom the ar. Gcidel showed that tlnt is. by which we mea. a. We sta. called axioms.r sct of assutned givetrs.lse. this is very dangerous.Ir that a. by Consistency is indispcnsablc irr rnathernatics. circlt of whiclt takes us from one proven fa.t ca.nrl sigrrificatrt aclvauceswere made in alrnost all of them. they are unacceptrr.re (pnv()rrtiorritl. One alternative to such "skippv" rnathtlrlrirtitls is to formalize as fa.pplictr.nother. irrxl what one Ineans by them is thtr.l proof itr just about every book and ptr. Of course. The rules tlre of used in a $eqrr()n(:o stclls.rison that required a more ca. Whetrever we see arguments like if this. we cannot help but wonclelr tlrtl llroof we are given is indeed correct. Often there is no way of telling. Two concerns immediately arose.ptrr dealittg with mathematical subjects. prec. Godel.eurcan be provecl to be true or fh.txxrptcd of by most rnirtherrraticians.l of mer. it rnust couany interesting consistent systeiurmrrst tltl irrrxlrrrlrlctel .shtxl by the work of K.xiorrrs a Iirrittl sequeltce of logical steps. or make wrong infi:rcnt:tls. Becarrse pra. A sequenceof plausiblc cla. considtlr what is involved in tr. and as possible.t:ticalliruitatiorrs.inlerspersedwith phraseslike "it ca. Matitettnticiatrs also had becromeurrfficienlly sophisticated to recognizc'that sorne logical difficulties had a. one rxrrrldgivt: rrtclredetailed reasoniug. If thr: propositiorr corrflicts with another proposition that t:iln b(: llroved to be true.
y$irr whidr furrctions can be definelcl. S.llyr:a. Itost.1 R. called thc range of the function.rite firrrt:tiorrsarrtl Post systernsca. while otht:rs are less colnmolt.t gives iln explicit nrethod for thtlil cotrrputation.rrrust be eqrrivaient.t havc br:crr sttrdied. so that sta.ntl Post systems.t there are futrctiotrs that cannot bc cxprcssed itt atry way tha. This is very broad and generrnlirrrd itnmeditltis association. It also irnplies lhal there is a. This thesis statcs that all possible models of ulrnputir.l rrxrtlels of computation were esta. Sorrte of thetn we use frecluently.vt: bccrr proposed. have diverse origins.nd Qualitz (1978).ncunsrvn !uncrroFls 3?5 tain soure unprovable propositions.thematit:stxltrld be tnade precise with mechanically verifiable proof's. gives $trong e:virlt:trce powerful models cart lle fcrutrd. Gcidel's work left rrnanswurudthc tluestiorr of whether the unprova. called the domain of tlx: firrrc:tiotr. Prorrrirrc:rrtirIIIoIrg them were tht: rccursive futrctious of Church and Kleene a. and the combined notion is sotrretines called the thesis. if they are sufficiently hroa. ilhe spirit of this observation is genera. There is a wealth of materia. as well as others tha.As surrrmilrizcd in Section 1. unicluevalue in a.referritrg the reader to other ref'ercnctlsfbr detail.tementu thcrc was still sotrtehope that most of ma.tion. I'he clainr is tif t:tltrrsevery closely related to Thring's tk:sis. It was this problettr Lhat T\rring and other mathematicians of the time.hle rxnrld sornt:ltowbe distinguished from the provable oncs.tl. a. C.trelcrlcrrt of otre a st:t. In tiris cha'pter we briefly review some of the idea. Recursive 1gffiffifi$ffiffi Functions Tlrc rnrrcept of a function is firndeurcrrtalto rnuch of nta.blished.l here thnt wc carrrtot cover.1.ll fnmiliar wilh functional rrotatiorr irr wlticlt we write expro$sions like + I Ui: rz2 l. ately raisesthe questiorrof how we can explicitly reprtrst)rrt There are many wa. We are a. We will give onlv il vtlry bricf presrlrrtatiorr. Gtjdel'u rnvolutiotrary conclusion was published in 1931. but there arc rnarry other such systems tha.nbe firrrnd irr accessibleaccount of recursiver Denning.flnction is a rulc tlnt assigusto a. Brrt it was evetttually found tha't thev wcrel all equivalent in their power to c:itrry ottt cornputations.sthat arosc out of tltese studies.1i1. Kleene.t ha. a variety of forrnir. addressed. Dennis.lkxlChurchts thesis. arrtl E. A qr.rticrrlarlyA. .thematit:s. prr. In order to studv the tprcstiott. Church. whilc not provable. while a good disclnsiorr of vitrious other rewriting systems is given in Salornair(1973) and Salomaa (1985). It provides a genertll prirrciple for algorithmic: ChurchTlrring that uo more cornputation and. a. The utodels of computation we study here.ntltlrt:rset.ninherent limitation itr lhis atrd thir.
xt in sequence to r.326 Chopter l3 Orrrnn. . we start with the basic: firnctions: 1 . 3 . we will consider only functiorrs of one or two variables. P n ( n 1 . and therr add one. in the rmual notation. Primitive through recursion.. 2 . and h. that is. or I x I. s (z) : z * 1. gz. a ) ) frorn defined functions gL)gz.u)).h.r z ) : r k . Tlre zero function z (r) :0.. we take the domain to be the set of all integers.yrrof building more complicated functions from these: 1. a ): l t ( n ( * . In this setting. 2. hy which we construct ( I @ . There are two wa. and whose range is irr 1. we must first clarify what the permissible forms are: for this we irrtrodrrce some basic functions. If.we can compute its valucs in a strictly mechanical fashion. whose dornairr is t ither I. MoDEr. we know the relation bctween the elements of its domain atrd its rartgc). Since ma. . Sirrce the function is defined irr this explicit wa.r from defined functiorrs gt.s(r). can it be expressed in such a functiorral fclrm? To answer the question.y.v). Composition. we also rmrst specify its domain.ny very complicated furrctions (:an be specified this way. Primitive Recursive Funclions To keep the discussiorr sirnple.i.a ) . (n. Thct projector functions whose value is the integer ne. the set of all nonnegative integers. s z r .i.Tll . To cornplcte the definition of /. for exarrrplc. multiply that value by itself.soF CoMpuTATroN Tlfs dcfinrls the function / by means of a recipe for its computation: given any value for the a. h : I.rgrrment n. If a function is dcfined (that is. then the range of / will be some subset of the set of positive integers. by which a function can be defined recursively :'. The successor function. we rnay well ask to what extent the notatiorr is universal.'. together with nrles for building from thenr solne rnort) cxlmplicated ones.. for all r € 1.2.
I Substraction is not rpite so obvious.! ! : .lltxl thc: monus.rlcrloNs 327 We illustratc how this works by showirrg how the lrasicoperatiotu of integer arithrnetic can be constructed in this fashion.the uerxrndstr:p is arr applicatiott of pritnitive recursion.tl).lly. ta. adrl.g l f x : 7 A . iu which h is itlrrntilicd with Lheatld. I ) + I ) : ( a d d ( 3 . defined by ttdd(n. To adcl 2 and 3.1. A kind of subtra(:tiorr is dcfined from usual subtraction by : t .. we apply these rules successively: a d d ( 3 . Additiorr of integers r and E can be implernented wiih the function add(r. ( r n u l ' Ln . it defines subtraction so that its range is 1.1 Rncunsrvr Fr.0) : . First.13. (r.r. 2 : a d d(. and 92(r.3/ + 1) : add (r. Forma. :t"u:0ifc<E' The operator .ccorrnt thnt rrcgativtl nutrrbers are not perrnitted iu our system. t :.is sornetirrxlu cir.3 .king into rr. 0 ) 1 )+ I + :(3+1)+1 :4*1:5. we cirrr rr()w dt:fintl rrrrrltiplication by mult(r. I Using the add function defined in Example 13. Y + 1 ) : a d d ( n . we rnust deflne it.t/) + 1. .g) is the projector ftnction Pt\r. Y ) ) .y). function. m u l t ( r .0):0.
A firnction is called primitive recursive if and only if it can be constructed from the ba*sicfirnction$ s? . and lr are total functions.3): :  pred. It follows from this that everv primitive recursive function is a total function on I or I x 1. Conrpureuorr Now we defirre the predecessor function P r e d ( 0 ): 0 ' Ttred(Y*I):'c. by successive composition and primitive recursion.l .t. and from it. then / defined by composition and primitive recursion is also a total function.irfr. gz.e. we reduce the proposition by applying the definitions a rrumber of tinres: subtr(5. 1)) ) ed pr ed(pred (pred(subtr (5.(subtr (5. we can define integer division.d(B) _. . i'r.t/ + 1) : pred(subtr (*.0)) )) pred(ered(ered(5))) .. fil i. With the algebraic operations precisely defined. the subtracting function subtr (r. and very complex computations built from the simple ones. T In much the same way. We call firnctions that can be constructed in such a manner primitive recursive. but we will leave the denronstration of it as an exerci$e.pk. Note that lf h.328 Chopter  3 Ornon Monnls or. other rnore complicated ones can now be constructed.0) : r. we see that the basic arithmetic operation$ are all constructible by the elementary processesdescribed. If we accept this as given.il$' lrr . y)) .. subtr (r.pred(pred$)) : pre.2)) pr ed(ytr (subtr (5. To prove that 5*3 : 2.
Therefore. Proof: By the argument of the previous theorem. We will give here one example that looks quite simple.and define a firnction . We next construct a function g defined as s ( i ) : f i ( i )+ \ i : 1.. Such strings can be encoded and arranged in standald order..its the following argument shows.rz... We can then write all functions in some order. and most common functions are primitive recursive. function in . Then there is some Proof: Every primitive recursive function can be described by a finite string that indicates how it is defined. but equally clearly.. g is well defined and is therefore in F. Then there is some function in C that is not primitive recursive. the demonstration that it is not primitive recursive iu quite lengthy. g differs from every fi in the diagonal position. f2. Let f' denote the set of all frrnctions from I to I.. Let C be the set of all total c:omputable furrctiorrs frorn 1 to 1. the set of all primitive recursive functions is countable.13.2.rally. Suppose now that the set of all functions is also countable. The actual construction of an example of such a function is a much more complicated matter.. this goes even firrther. not all functiorrs are in this cla*ss. the function g differs fiom every ri arrd is tlterefore not primitive recursive. I Actr. say. proving tlte theorem.. But clearly . however.. f t. However.g is r:onrputable..1 RncunsrvnFUNCTToNS 3?9 The expressivepower of primitive recursive functions is considerable. . Combining these two observations proves that there must be some function in f' that is not primitive recursive.. not only are there firnt:tions that are not primitive rccursive. Clearly.F that is not primitive recursive.gby sQ):rift)*By construction. r The nonconstructivc proof tlnt there are computable functions that are not primitivc recursive is a fairly simple exercise in diagonalization. there are in fact computable functions that are not primitive recursive. Let us denote the functions in this set as rr. the set of all prirnitive recursive functions is countable. This contradiction proves that F cannot be countable.
which is tedious an<l technical. it follows easily that Ackermann's function is not primitive recursive. see. i ) .1 . Ackermann's function is not primitive recursive. Dennis. The situation here is similar to the one we encountered when we tried to prove that a language wa$ not regular or not contextfiee. .fbr example.n+1. computable function. : A(:r . How this is related to the limit of growth for primitive recursive functions is made precise in the following theorem. 0) A ( u . it is possible that an appropriate alternative definition could exist. ): A ( i . fbralli:n.4 (4a.). and Ackermann's function violates this limit.4 is a total.p . r E If we accept this result. Its proof. . There is a limit to how fast a primitive recursive function can grow as i + oo. see Denning. we cannot argrre directly from the definition of A. But in spite of its apparent simplicity. Even though this definition is not in the form reqrrired for a primitive recursive function... We need to appeal to some general property of the class of all primitive recursive functions and show that Ackermann's function violates this property. For primitive recuntive functions. will be omitted. one such property is the growth rate. It is not hard to seethat . Proof: For the details of the argument. defined bv A(o's):3/+1.A ( t : .1.. 5 3 4 ) .i.330 Chopterl3 OrHnn Mounls or. Ackermannts function is not nrimitive recursive. That Ackerma.'CoNtrurnrtotl Ackermqnn'sFunction Ackermann's function is a function from 1 x 1 to 1. 1) . Proof: Consider the function s ( i . In fact. Then there exists some integer n such that /(i) < A(n. Of course. ExercisesI to 11 at the end of this section. t 1 * 1 ) : A ( r .. Let / be any primitive recursive function. it is quite elementary to write a recursive computer program for its computation. and Qualitz ( 1 9 7 8 . y ) ) .nn's function grows very rapidly is easily demonstrated.
$* g(x.for r ) 3.lization.. : unclcfined. If we now pick i : rr'.ncun. then therc is rro y € I srrt:h that z I y .'a):n*u3.?)) : snrallcst 37 such that g (r. w(l iL$$rrme that g is a total function. Orrclway is to introduce the p or minimalization operator. In this definitiorr.n. accordingto Theorern13.then so would g. Thercfrrrc. wt] get the contradiction S(n): A(n. but if r ) 3. defined by pA (g @.3.i. we must add somethirrg to thc rulcs by which suc:hfincltions can be constructed.rsive Functions To extettd thc idea of recrrrsive functions to covcr Ar:kermann's function and other computable furrctions. Ler llHr*sNE[U*il*.r.n) < A(n. ftrr all rl. I As the abovc t:xamplr: rthows. fhsn U:3r is the result of the minima. We see from this that everrthough g(r.there existsan n suchthat s(i)< A(n.A)) mav only be partial. If u { J. But then..). proving that A cannot be primitive recursive. I Recu 11.). for r: { 3 P'a(g @. the minimalization operation opens the possibility of definirrg partial firnctions recursively. But it turns out ihat it . pA(S@.ll) is a total function.1 R.u)) : 3 . y) : O.srvp Furrcrronrs 331 If A were prirnitive recursive. which is a total functiorr.3: 0.13.
recursivera. Ctrnsiderl. 4.frtiffift. Usc thc definitions in lixamples 2+3:6. Agaitt. see Denuing.ffi 3 .he luttctiou (t:.a.33? Chopter l3 oF Orul:R MoDELS CoMpurATroN also extends the llrlwt:r to defitre total functions so a.idto br: precrrrsiveif it can be constructed fiom thc birsis of functions by a. Let / be defined by 'f(n. Proof: Fbr a proof.turtlwirere the deta. ir. i fr : U.titlrrs primitivc rct:ursiort.'g) : I rtyt.I and lll. Chapter 13).s if r : u .seqrxlnr:t: irpplicatiots of the poperator and the opera. g r e a t e(r* .tt):n :0 Show that this futctiot ifr*u. Nhft'ilrii\ A firnction is sa. we tnerely quote the major resrrlt witlt references to the litcrra.l. is .rrdorrly if it is cotttputalrle.ffiRq.rrirnitivc rccursivc. lili.ble fi. I I'he precursive finctions therrrftlrc givc rrs itrxrtlrtlr rnodel for algoriihmic computation. :0 1 I x t ''c.. Derrnis. Show that this function is prirnitive ret:trrsive.2 to rrrove that 3 + 4  7 and .rn<:tiorrs.$to int:hrtlt: all computa.ilsmay be fortnrl. t. v ) : 1 i f r > U :0ifr<U Show that this function is prirnitive recursive. Define the lunction Ill. of compositiotr and if A lunction is trr.rrtlQuiilitz (1978. .
I) and A (4. Write a cornputer l)rogram for computing At:kermann's function. L Prove the following fbr the Ackermann lunction. 13. Show that the furtction g (n. Show that Ackernrann's functiorr is a total function in 1 x 1.'a): n . r:orrrpute ltakl@.u) : intescr of (* .'g): n. W 8. Show that f ( r t ): 2 " is primitive recursive.i.13. wlrere n is the largest integer such that x j rrg. 11. ''.q(". Cat you explain what vou observe'l 15.37)).1)I fu +t7 part (d) g(n.2). lbr cach g below. and tletertrine its domain. 6.3 ffi (c) s (r.u) : x is primitivc recursive.11.u ) : 2 ' + 3 * 3 10.2).nA. althorrgh intuitivcly clcar. s = s + 2 f f i ) (b) A (2. (a) . Tfy to usc the plograrrr t:orrstrrrctedfor Exercise B to evahrate . Show that the functions d.1 RncunsrvnFrrrucrlons 333 rk 5. g).4 (5. :a) : :ra (lr) .5). Clive a gerrcral expressiorrfor A (4. ( a )A ( 1 . 14. Use Exercise I to compute A (4. does rrot strictly adhere to the defirrition of a prirrritive reclrrsive function. L2. Sltow the sequencc of rccursive calls in the cornputation trfA (5. anrtl rem.):2"' r u . The definition of pre. Ilse it to evaluate A (?. :2ur3 ffi a) ( c ) A ( 3 .ua:rd re. Integer division can be defined by two functions iCzu ancl lern: diu (x. .:3).g) :u mod(s*1) 16.5) and A (3. Show how thc definitiorl t:an be rewritten so that it has the corret:t form.nr ate prirnitive recursivc. 7.(t.s (r.tl in Example 13.
.ffi .*ytWtUz. v. s.A.l$.t eac:hvirriable on the right mrrst irppear ou the left. and C7'. cir.blt:s.'1.1.sf. and Va.. The prodrrctions irr a Post system mrrst satisfy certaitr restrictions.. P is ir"fittite set of uroductions.'. V is a finite set of va.1) where ff. and th.. so that Vfviforilj. Thcy must bc of the form fr1V1fr2.Wm. that is i=t U*rU" t=l we Srrppose have a string of tcrrnirrals of the forrn rru)trzl:tz' ' 'rDnfrn+tt wlrere the substrings rt... thtl st:t of terminal constants. consisting of two disjoint sets (.lledthe axioms.r. for and substitute these va. yt € C*.P) .. A Post systeur II is defined by fI : (c.VrT.'trz : V2. fi ilitftilll\l. sutrject to the requirerncrrt tltat atry variallle can appear at most on(:c orl the left. We can then rrrirkethe identification u)1 : V1. rrrirtchthe correspondingstrittgs itt (13'1) arrd'ur1€ C*..!lmlt. Wi EV. But there are significant difl'crettces in the way irr which the productions arc applied.lrres tlte W's ou the right of (13. A is a linite set fiom (J*.ritr.'. .. Tz..rxlsorne production nrltls by which successivtl strings can be derived.334 Chopterl3 OrunR Monnls or Covpurnrton MM PostSystems A Post system lotlks very much like tln utuestricted grammar' colsistiug of an alphabet rr. where C is a finite set of constants. (13. called the nonterrninal constants.1)' Since every .
u. I aVrb.: .* .. . we reidentify Vt : ab. leaving everything else the same.tedby the Post system II : (C. (II) : {w e Ci. V. Cu =4. . . A. As for a grammar.13. Cw:A. Vr : tr.. . V : lVr.r Post system is {attbt' : rz } 0}.1) with the identification 11 : A. frnlt + !yUa!2Wi. We write this as I1W1I2W2. 4 irrt C7: {a.a} . you will quickly convince yourself that the language generated by this particula.rs u for some zuoe . irn C 7 : { 7 . P) is I. we apply (13.Antt. rt : A. Wt : Vt. we carl now talk about the larrguage derived by a Post system. The language genera.2 Posr Sysrpus 335 W is some yd that occurs on the left.Vz. and we get the new string lltwiUzwj . and production Vt This allows the derivation A =+ ab + aabb. Arnlt. : } .. it is assigned a unique value. A:{1*1:11}. A : {I}. At : &. In the first step. If you continue with this.b}. and 3r2: b.VsI. In the second step. V : {Vr}.
derived fiom the axiom 1 * 1 : 2.6 illustrates in tr sittrple manner the original intent of Post systems a^r a rttecltanism for rigorously proving mathematical statt:rnents from a set of axioms. t]re derivation can be written as 1+1:2=+2+1:3+2*2:4. Tht:refore. since a derivation by n Post system is completely mechanical. {S}.mmar G.336 Chopter l3 OrImn Moont s ot'' CoupurartoN and productions Vt*Vz: Vt*Vz: V 3V 3V r 7I V z : VtIVzI: Vrl' Vrl' The system allows the derivation 1*1:11+11*1:111 + 1 1+ 1 1 : 1 1 1 1 .ted grarnmar G. P11). even though they are t:urnbersome for provirrg complicated thtloretns' are general models for computation. of Irrterpreting the stritrgs of . The language gcrrerated by this Post system is the set of all identities of integer additiors. It is then an etr$Ymatter to show that a ur carr be generated by thc Post system II if arrd only if it is in the Ianguage generated by G. as tlte next theorem shows. _l Example 13. Given any unrestricted grs. t . A language is recursivcly enumerable if and only if there exists sotne Post system that generir. A.Cw : V. such as 2 * 2 :4. havirtg productions all of the form t+lt with r. C. any langua. But Post systems. for every production fr i A of the grammtrr. and with productions V1rV2 + VrUVz. A : system II : (Vn.Cr : T. wc create a Post where Vn : {yt' Vz} . Proof: The argurnents here are relatively simple and we sketch them briefly' First.tcsit. it can be carried out on a Thring machine.tiorrs integers. It also shows the inherent awkwardness of such a completely rigorous approat:h and why it is rarely used. remember that auy recursively enumerable language is generatod by some unrestrit.gc getrerated by a Post system is recursively enumerable' Frrr the converse. y e (V U T).['s as ullary representer.
6. the further restriction that the number of variable occurrcnces on the right and left is the safire. {***.c}. Rewriting ystems S The virrious grammars we havcl studied have a nurnber of things in common with Post systems: They anl all based on sotne alphabct fiom which one string can be obtained frorn irnother. Ftrr X . For X: {u. ab} ? Find a Post system for proving the identities of irrteger multiplication.t completely defirrcsits configuration.b 1a['*c) (b) r: (c) l.e {a. i.hstring from a previou$ one. 3. startirrg from the axiom 1 + 1 : 1. there exists a restricted Post system to generates Z.b}}..hine can he viewed this wrr. What language tloes the Post system irt Exercise 3 gencrate if the axiqm set is {a.L gerrerated by some Post systern. A restricted Post system is onc on which every production r * g satisfies.13.{o}. Find a Post s. ffif..: {urtu} ffi\ lanb"t!'j . Give the details of the proof of Theorem 13. Even a 'Ihring rnirt. The program is then just a set of rules fbr producing one suc.1). What language does the Post systcm with VnaVV and axiorrr set {ab} generate'l 8.y. Show that for every language. These observations carr ller formalized irr tlNr concept of a rewriting system.lphabet E and a set of rules or productions by which . find a Post systern that generates the ftrllowing languages (a) L(a. a rewriting system consists of arr tr. Generally.e. since its instantarreorrs description is a string tha. t . what language rloes the Post systern with axiom {a} ancl the following production generate? Vr * I4Vr W 4.3 RnwnlrrNc SYSTTTMs 337 1 .$ 6 . in addition to the usual reqrrirements.ysterrr that generatcs t: u.h. n : nr in (13.
producing a corresponding b and c.strilg in E* can produce another. The iderr.9 + . P2: 51 + 0.ctlv one production. sornetimes the use of with arr uurestricted phrasestructure grammar.91.' Cotrpurnuon a. What distinglishcs one rewriting svstern from a. Htrrc we briefly intrt)thrce sonre less wellktrown ones that are interesting arrtl also provide general tnod(1985)' els fbr rxrrnputatiou.l strings generated try this matrix gramrnitr is 7:{a"b"c":n}0}.tly etrcountered.tion.t r .338 Chopter l3 O'rseR Monnls or. since tnatrix grrlrnm&rs repre$crrt algorithmic prot:t:sses. we must next ir.s a.. For matrix graIIIIIlars.P. Consider thrl rnatrix grammar P1 : .7 mars trave the same power ir.. applied. then the third otre.rrtatrix grammar givcs a much simpler solutiotr showtl. T Matrix grillrtlnars contain plrrasestructure gramttrars as a special ctt*se in which each 4 coltains exa.titlrr of the productiorrs..sy to see that the set of termirur. For dotir. and so on.pplied. 9 2 . But. AIso.nqtht:r is the nature of X atrd restrictions for the applictr. A derivation with this grammar is S + S1S2 + a.chieve . the set of productious consists of subsets P't. We cxlnclude from this thir. as ExamJrle 13.nd allows atty numtrt:r of specific ca$osirr adtlition to tfie orrcs we have alretr. is Whenever thc first production of some set .92..14rly set can also be applied.lomeur Mqtrix Grommqrs Matrix grammars differ from the grarnrrtars we lrave previously studied (which are often r:alled phrasestructure grammars) irr how the prodrrctions can be tr.pply the secondorte to the string jrrst created. thart we cau a. This rnakes it ea. ea.they are governccl by Church's thesis. P 3 : 5 ' 1.t r . P2.chof which is an orderul sequellce 1.ils.9rbSzc+ aaSlbbS2cc+ &abbcc.Zt+'AIt12+!2r.. We the first production of 4 unless tr.t rnatrix grammar$ trrrtl plrrasestructgrtl graurrnodels of computa..is cluite broad a. Note that whertever the first rule ttf 1'2 is used to create arl ar the second one also ha^rto be used.5I' Sz + bSzt:. .see Salomaa (1973) and Sa. Pn.ll otlter productions in tltis cannot tr.
Some of the productions rnay be singled out ir. For languageat:ceptance. In a tlcrivation.s r + .$ Consider Markova. e f.I3. tlNry will be showrr a. so L (IV): { r r € { o .! t R.re applied rrrrtil the empty string is Drodrrced. I . . the first applicable prodrrction must tre used.(.ndcontinues urrtil either a tcrminal produr:tion is used or rrntil there arr: rro a.\.ur € x tr.5.ls identified.sterminal productions. ba . Starting a is with a terminal string. set ? C X of terrnirra.. by {". Tlrcn the set L(M): is tlre language accepted M. Every step in the clerivation a.. the lcftmost occurrerrt:eof the substring r must be replaced by g.t.nnihilatesa srrbstring ob or ba. A derivation utarts with sorne string .3 Rnwnluuc Sysrnlrs 339 MorkovAlgorithms A Markov algorithrn is a rewriting systt+mwhose productions n+ U are consideredordered.t u ) : " o ( r ) } . I Lct M be a Markov a..6} and productions ab4.1} I . : r t . b } .lgorithrnwith alphabet E arrd terminals ?'.!. Furthermore. productiorrs a.it_.pplicableproductions. .lgorithn the with X *T : {4.' 'Fr" t*qmFld lS.
35). Wht:rr a stritrg is rewritterr.p. since it is nqt rlc:irrwhat to do with the Iast produt:tiotr. = {trLfirl : rz > 0] .r I..s op Courur. An arrswer is ab.t:kward.rc quile clifferent f'rorrr what we miglrt expect' A.q"ttotl Wnd a' Mtr.rrdz € X*.starting point for a proof of thc ftrllowing theorem that cha'ra'cterizcs tlx: power of Markov algorithms.L. tz5b + S.2) where a € X a.rrgua.+'u) (13. Their clervcloper.\. But the observation does prtr viclq ir.Markov a"lgorithms are simply phrasestructure .thtt llroductions of tr.rrguageis recrtrsivt:ly errumera. grailrrrrars working brr.tlt:rivatioll' everY syrtrbol has to bc Lsystem must bc: rcwritten. we gc:t a contextf'rt)(t gralnntar that gcrterates the la. For this to rnake sense. Bv tis we mean thnt irt each step of ir.pplitxl to every symbol of the string bcfore the new strirrg is geuertrtetl. If irr this last examplc we take the first two productiorrs and reverse thtr left nnrl right sides.bleif arrd only if thertl cxists a Ma. .rr of the form 0. This ctr.rrrtotbe takett totl literally.rkqv tr.S. used them to rttodel the growth pattern of certairr organisms.yer. 5 i .lgorithrn for it. I A ltr. Proof: SeeSalomaa(1985. r LSystems The origins of Lsystems a.340 Chopterl3 OrHnn Motlpl. Lsystcrrrs are essentially Ttarallel rewritirrg systems. Lindentrir.gcl 11 a certain uense. otre such productiol nust be a.rkov algorithm f.
IxEncrsES IFind a matrix grarnura.9+S1.2) are rrot sufficierttly general to provide for all algorithmic r:omputations. productions irrel of the forrn \ixlatA) + u1 wlrer(ra e E and 'J:1y E E* . An cxtension of the idea provides the necessary generalization. 5 2+ . m Pr:5'rtr. Why does the Markov algorithm in Examplc 1B.13..t group of procluctions to P 3 : .S2.a.r for 7 : {uzn: ut € {a.ke the derivation a + aa + aaaa + a. {a}anrr define an L*system.a.. with the interpretaticln that a carr bc replaced 1u by z only if it occurs as part of thc string rag. . 9 14 . In an extenrlrxl Lsystern.2":n>0}.a. For cletails.S'r*tr. 5 2+ h S z a .a..:{u.11Ruwnruuc Sysrr:MS 341 Letx: i\iHl$[Nt$[$W.7 we changc the la^. Pe: Sr a S L b . 3.o. 2.0. startirrg fiom the string tr. What language is generated lrythis matrix grarnmar? 4.b}}.toi. Wlut language is generated by thc matrix grammar P1 :. Note agairrhow such special rewriting systemsare able to deal with prohlems that are quittl difficult for phrase structure gremrnars. sec Salomaa(i985).a. 9 . The set of strings so derived is clcarlv 7. Supposc that in Examplc 13. It is known that such extended Lsystems are gerrcrell models of computatiorr. we carr rna.g not accept abab'l . I It is known that Lsvstems with productiorn of the forrn (13.
1 when started with the strirrg a'l ffi . What is the set of strings generated by the Lsystem with productions o. Find a Markov algorithrn that derives the language L {a"hncn : n } 1}' ffi * 6. + aa1 o. r z l} 1 } .o.342 Chopter l3 OrHon Monpls oF CoMPUrerIor'l 5.0. + o. 8. Find an Lsystem that generates tr(oo). Find a Markov algorithm that acccpts L: {a"b"'un* i n} 1 . 7.
problern can br: solved in principlc. efficicncy ha. In the pra.An I n t r o d u c t i o to n co mp u t q t i o n o I co mp l e x i t y rt strrdying algoritlrrns a.clical world of software dcvelopment.l significant:c. a descriptive term that will Lregivcn a nrore precisc mea. Sometimes. but we also rnust be rrble to construct algorithms that c:rr.recalled tractable.tecl.. we have so f'ar pa.we are cuncernecl with thc efficient use of thc r:omputer's resources.ce.n carried out with reasonabkr be efficiency.id Iittle atterrtiorr to what actually can be expcctcd when we apply these ideas to rual compulers. AII this is mrrr*r too complicatcrl to be captured by any abstract thcorv. At still tlther times. we rrt)tlrl not only to know that a. This is an appropriate starting point firr a theory but cleirrlv of limited lrrat:tica. how effectively it ca. we may be rnore concemed with how quickly softwa. All we can do is to ftrcus on sorne of the more tangitlle issues and create the 343 . Problerns Lhat can be solved effectively a.srr(:tras processor tirne and mernory spa. Wu have been alrnost exclusively corrr:ernecl with questions of the existt:rrc:eor nonexisterrr:e clf aleorithrrrs frrr certain problems. we may ernphilsize the efficiency with which a us(lr'$ problems can trtl solved.re can be crca. At other tirnes.s many facets.rd.ningin this chapter.or how relirrble it is.nd computirtions.nbe maintairx. For actual c:ornprrtations.
t throw further liglrt on the nature of languagesarrd compUtations. the primary colcern is the efficiency of a computir.. most of which is well outside the scotretlf this text. rl is obviously the number of iterns in the list. say. Also. .tioll as met$rlred by its time arrd space requirerttettts. Wt: refer to this irs the tirnecomplexity and the spacecomplexity of algorithms. arrd we must corrcentrateon those that are most fundtr. The nodel fbr our study will be a'ftrring mat:hitre. Our intent heru is to present the flavor of the subject mirtter and show how it relates to what we knqw about languages and autorrrata. hoth irt the selectiorrof topics rlrrd in the formality of the discussion' we will lirnit our dis<Irssion here to issues of timecornplexity. For our rliscussion of computational complexity. and tha. 1. Tfuere are probably a few more things you carl t]rink of that lleed to be looked a. the numtrerrof itenrs in the list plays arr itnportant role in how much time will be taktlrr. that are simply statecl and errsily appreciatQtl. but there irre other f'actors.rtrtlifficult ald we will clispense with thern hy rcference to appropriate sources. In this set:tiotr. The size of tht: problem will be denoted by n.little rnore accessible. Fclr the most pirrt.metrtal.How long will it take to do this task'i" we see If we now ask the questi0il immtxliately that mrx:h more inforrration is needtxl before we (laII answel it. Most of thc tesults tha. Sorting is ir sitnple problcrn but also onc that is verY furrdamental in cortrputer science. There is thc question of what computer we use and how we write the program. clearly.344 Chopterl4 CoMPLEXITY Arv lurnonuc:rtot'tto Couru'lA'l'IoNAL appropriate irbstract fiamc:work for thesc:. The exact type of Ttrring machirre to be used will be discussedbelow.et rrs start with a ctlncrete examplc.we givera brief overview of some compklxity results. in irscendiug ordtlr. For this reason we will allow ourselvesir great deal of latitude.t have been tlevelopecl arlclressthe sllirt:c and time cfficiency of ir. proofs a. E f f i c i e n co f C o m p u t o t i o n y I. trut timecorrrplexity is a. however. If wc have arry hope of producing sorne general picture of sorting. computatitllr. In the stucly of t:ornplexity. Ieading to the important topig of complexity theory. rnost of these issueshavc to be ignored. there are a number of sorting rrrethodsso that selection of thc algorithm ilr irnportant. Givetr a list of orre thousand integers' wc want to sort tltem iu som() way. computational complexity theory is ilrr extetnive topic. we will rnake the folIowing simplifving assumptiorts.t btlfore you can evctt tttake a rorrgh guess of the tirne requirements. Thcre are some rt:sults. Although thc: size of a . For our sorting problem. There are similar results fbr spacecotnplexity. 2.
O. Unlcss the set is organized in some way. the sirrrpklst nlgorithm is just a l'inear seu.e. and a key mrrnher r.tch or we get to tlrc la.ningt. Norrnally. 'I\rrirrg Wtr think of a ma.lstandpoint. which may difftlr considelably fionr how it does orr a T\rring machine. In spite of the apparent inforrnality of this approach.rge. in the worst case.irrs r. in the set of all protrlems of a given size.cterizecl.mple. After scttling on a. we wtrnt to study how the cornputational rrxlrirements grow with the size of tlrc problem. wcl hirvel .rclruNcrr CoMprr. we coukl a. First. We are particularly conctlrrrttdwith how Lhealgorithm behaveswltt:rrthe problem size irrr:rerlres.the asyrnptotic growth rate of the time complexity. By saying thrrt a. tsut. this is rrclt ovrlrlv useful.tional model. 3. Our inrtrretliategoal will therr llc tcl characterizt:the time requirement of a.1 Epr. irr r until we either firrcl a.14..st element of the set. Since we may find a rrrir.riation. tire prirnirry qrrestion is with how fh. there is sorrrc vtr. wc irre itrtercstcd in how the algoritlrrn perfbrms in the real world.rofmagniLude analysis irr which we use the O. specific typc of T\rring rnachirre as il computa.lwaysso easily c:harrr.. wc often get very useful itforrniltion. set txlntrr. Orrr first attempt at rrnderstanding the resour(:e requircrncnts of an algoritlun is thcrefore invariably an ord.m and so iltay depcrrd strclrrgly on the prograrrirrr{)r. for cxa.chine as making one rrove per tirrrc rrnit. nnd () notation introdrrr:ed in Chapter 1. determine if tlrt: r. As stated.rch. frorrr a pra. lbr a va.othe concept of time for a Thring machine. First.T21... which we corrrtr)ar(r successivelyagainst :r:11. so the tirne taken by a courllrta.stthe resourrxlreof quirenents grow as rz becornr:sltr. can geuerally relate it we in some way tcl rr positive integer. Irr analyzing atr algorithm.tr:h tlte first cornprrrisorror on the last. Givr:rra.. computation has ir timecorrrplcxitv 7(rz).best we can hope fbr is that thc T\rring machine analysis is representative of tlxr major aspects of the reallif'eperformance. Second.riety of reasons.Ftoru 345 oF problem is rrot rr.set of rz numbers ir:r1r'21. ma. we rrrearrthat the computation firr any problern of sizc n can be tlclrnpleted in no rnore thirrr 7 (rz) moves orr rrome Ttrring rnachine.lyze algorithrrn try writing explicit progrrrm$and counting the nurnber of stcps involvecl in solvirrg the protrlem.. Here w(r irre irrterested orrly irr the worst case that hils the hight:st resource requirclrncrrts. we carrnot predict on lxrw much work is involved.r'A. we give some mea.. we are less intcrcsted in its perforrrrarrr:rr on a sper:ific case than irr its general behavior.r:tica.tion is the nurntlt'r of moves nade.na. the number of operatirlrrs perforrntxl may vary with the srnelll details of the progra. rrsing a Thring rnalchineas the cornputer a model. Becaust+ this. TIrc. problern il^s fhnction of its size. but we know thtlt.
2. A look at that algorithm will show that for ut : unb" it takes roughly 2n stepu to rnatch each a with the correspondirrg b. singltltape guage 'ftrring machine for the lan 7={anb":rr. Irr Example 9. Is the algorithm as efficient as possible? 3.We ca.4 wc t:onslructed a. Review how the r:hoice of algorithrn afl'ects the efficient:y of sorting. I LL supposc you are givetr a set offl. O(n). (a) Suggest an algorithrn and find an orrlerof*rnagnitude exJrression for its tinrecomJrlexity. rrutttbers l. In ilraking this analysis. Therefbre tlte whole comprtation takes O (n2) titne. this tirne dctcrmining if the set containrt arry triplicates. But whetr we tnake cornplexity an issue. we rnacleno specifit: ir.riscltrs.'.mple 10. with a twotape machine w() carr use a different algorithm. But.gesto avoid some of the tedium involvtxl in using thc standard Thring machine model.}1}.insarry duplicates. We first copy all the a's to the second . (b) Examine if the irnplernentation of the algorithtn on a T\rrirrg rnachinc affectrr your conclusions. ffiffi o T u r i n gM o c h i n e s n d C o m p l e x i t y In Chtrpter 10 we a.ltclrt what machine tlfs is run on t)r how the rrlgorithm is implernented.^r1frz1. or even bt:tter.irD alld are asked trt tleterminc whether this set conta. the equivalence btttween the various tVpes of T[rring mAchifles no longer holds.rgrtedthat the various types of T\rring machirres were equivalerrt in their powt:r to solve problems.ssurnptiousa. Repeat Excrcise 2. This allowed us to take whattuvcr type was most t:orrverfettt for an argurnent and even use programs in higherlevel r:orrtputer langua.n tlx:n say that the titnecomplexitV of this Iirrear search is O(tl). as we later indicated in Exa.346 Chopter l4 Ar'r Inrnonucrrlow ro Coururhl'loNAl CoNrRLuxtrv to rriake rz comptr.1.
s.rlrl. T'he nore elflir:ientCYK algorithrn him urmplexity O ("'). so I Fr(qffiFl$ l4.If we take tlx: Iength of the inprrt string tt as the problr:m sizen. which plays an irnporta. we have an O (n) algorithm. A nondeterrnirristica. A logic or boolcla. where M depends otr thcl gramma.2 TuRtNc MncHrruns AND CoMpLEXrry 347 Figrtre 14.tumpl* l4.n .1 Tape 1 Tepe I Tape 2 (a) Initial tapes.l:.rrddefined by 0V1:1V0=lVl:l 0V0:0.n operators are thcrr used to combine booklan rxrnstantsa1d variafles irrttl bclolea.lgorithrnfor this problem proceedsby sirnply guessing which sequenc() productions is applir:d in the derivalion of rr. If we of work with a gralrlllrirr th. then rnatch tlrcm tr.4 We uow introcltrc:c the satisfiability problern..ga. tirpe.1. Both the copying arrrl the rrtatr:hing can be donc irr 0 (n) time atrd wc see that a twotape malt:hine has tirnecorr4rlcxity O (n). the Ierrgth of the dtlrivtr. then the exhaustive st:arr:h method has compklxitv O (n M) .tionis essentially l. respecl. The simplest boolearr oprltltclrs are or.instthe b's orr the first. The situation before and afher the copying is slrown in Figure 14.n expressiorts.rnsta. ..t has no unit or . denoted by V a. I f.2 arrcl6. which we will denotc: tly I and 0. Both thcse algorithrns are cletcrministic. true or firlse. Botlletr. Tape 2 (b) Tapes aftcr copying of a'.t can take orr exar:tly or twtt vtrlues.3 we cliscussecl rncmbership problem for t:onrextthe free larrgrrages.r.rrt variablc is one tha.\productions.14.ively.S Irr Sections 5.nt role in complexity thexrrv.
c2: (r1Vr2)AzrArz lbr is not satisfirr.r. this t:xhiruslive approa't:h the exprrlssiotr.lltcs the variablesrr .. 1z\1:1.and definecl Also neededis ne. (14.9pstands for somtl variable or the tregtr.hrir...he variables flr irlrd z2 will fa.tttllle. by dettotedby a ba.nd artd operation(n) definecl : 0A0:0n:L 1A0:0. we hirvt: ir deterttriuistic exhaustive sea. I .tplnxlrv Ar'{ IrurRonucrIoN l'o CoMPITTATIONAL by tlrtl rr.sm..3.ssignrnc:rrt l.ltrcof citch r i and then evir..fk nrrgation.348 Chopter l4 Cotr.$pr ( r4 ..rtl2t' such choices. rrr.. fi2 : 1.V.. ff2. 0: 1. this fbrm.kes e1 trtrc:so that tllis expressiclrr is satisfiable. dtl rr<ltktrow of any norr()xl)()rtential deterministit: algclrit. starting with fi1. .hm.. .r) trr() t:reated by oring together variables tr. Therassigntnentrt : 0.Tz. If e is satisfiAgaitr... However.blt:btlcause every a... is there an assignrnent of values to tire va..rt frotrt variabl€s 1r2. On tht: otlter haucl.. we guessthe va. The satisfiability problem is therr simply sta'ted: givcn atr expression e in coniurrctive norma. t3 : I na..ilrl their Ii :.srr.lgrlritlrrn whose complexity is exponetrtial and a lirrear uotrdeterwe rninistic orrt:.t' e 1: ( 7 1 V r 2 ) n ( r 1 v r 3 ) .ti.V.V. 2 ) of wlrere each El. Wu take:all possibleva. the nondeterurirristic: 'flils e..riahlels that will tnake ttrtl value of e true... make e2 A dr:tr:rrninistic algorithm for the satisfiability problcxn is easy to disftll cover.titlrr a variable.s.. Tlre terms ti. t:0. iu exprt:ssitlrts corqiunction normal form. In We considerrurw boolea.tc: O (n) algorithrn.l fbrm..gat'ion.lsc.At7.. is essentially irrr itble. Rtr rr specific case. t'r. we createexpressions (::ftAtrA..rcha. lclok irt rt.. approaclt simplifics rrtatlers. alrd evaluate Siuce there a. that is. urrlike tht: llrtlvious exa. As irr Exarnple 14. iras r:xporrentialtinre compk:xity.
) (tl2) ftrr the first t:ase. bc morc precise and rdairrr that T (n):(. Dctertuine whether or rrot the expression (rr vr. lineartimc algorithrn lor rnernbers]rip in {tlur: ru E {a.2 'I'uRrNcl MecHrrues erur Col.tprnxrlv 349 These exarnples suggc:stthrlt complexity cluestions are affectcrl try the type of Ttrring tnar:hirrt: w() ll$e a. assume that the Thriru rrrirr:hirrcsinvolved are all deterrrfnistir:. Show 4.2? . For this reason..1 suggeststhat a.} using a twotape Turing machine. vrr) n (r1v:r2 vn 3) n (nr vnr vn3) is satisliable. Exa. that any txrrnglrtation that ca. 7.nd tha. o ("' (n)). nrultitape T\rring rnachinrl iLsorrr model for stuclying cornplexity issutls. offline 'llrring machine in titre 0 (T (rr)) also r:an be perfornred on a standard Turing rnar:hinein timc O (? (n)).t is thc Lrest.).rnple 14.and T'("): O (n) for thc scconcl'lIIow this strcngthen the argutrretrt in Exarnple I4. lihow tltat any t:ornJlrtation that can be performcd on a t.n be pcrformcd on a standard Tfring tnachitre in tirne O Q (n.b}. we will rrse a.wotape rrrachirre in timc O(Z(n)) catr be perforrnerl orr a standarcJ'l\rrirrg machinc in time s. Wha. ()a. For the tlxur(:ise$in this set. Irr Example l4. 6.2 we claimcd that thc first algorithtn harl tirne trrmplexity 0 (rr. you could expect on a onetape macirinc? a) Show that any cornputation that can be performed on a singlctapc.lgorithrns for a rnultitapc rnirr:hine may be reasonably close to what wt: uright use when we prograrrr irr ir rxlrnputer language.)) also can be performcd on a Turing machine with one semiinfinitc tapc in tiue O (T (n)). Rewrite the boolcan cxprcsslon (rr Arr) Vnil in conjunctivc nornral lbrrn. Find a.') antl the $et:onrl O (n.n wc.t the issue of deterrnirrisrrr v()r$llti noncleterminism is a partir:ularly r:nrr:ial one. I . 5 .14.
ss arttomir.ssos inrtornata.til is deflned by latrguages is the natrrre of its tt:trrporary storage.ir ritsof'1. DTIME (ri (")) c DTIME (Tz(n)).milieswith clir. To do so.stlrtl order of T (z) increases. rxlrnplc:xityclassessuch as Some relatiorrs between theser DTME (r(')) and T 1 ( n . Another way of t:lir. A lir. If M is trondeterministit:.t wt) r:arr say is th. tlris implies tha. we take in progressively morr: languirgc:s. we assctc:iate of of fa. ): O ( T . . wltere each cla.ge irr tirrre T (n) if every ?u in. llrt frorn here the situation gets obscure quickly. t}rere is at lea. wc first clefitre the time complexity of tr language.Lis said to be a tnember of the cla. .ssNTIME (f (n)) if there t:xists a rrorrdeterrninistic rmrltitape T[rring mar:hine that atrepts L in tirne T (n).t for ()v{}ry 'u € L.L with lrl < r. .1ini.r:xIrv Longuoge omilies nd Complexity losses F o C language In the Chomsky hicrirrr:hy for language classifica. A lrrrrguage.st one $eqlrenctlof rrrovesof lcrrgth O (Z (ltul)) that leads to accepta.L Wu sa. is acceptedinO(T (z)) moves."f'i.t a.tion.L is said to be a member of the cla.nguirgc. Whtr.IIUTATIONAL Corvet. 2 ( n ) ) irnplies c NTIME (7(")). n.ssifyirrg typc but corrsidertiute complexity to usc a Thrittg tnachiue of a 1ra.ythat a Thring machine accepts a langrrrr. ir lolllll.r.rticrrlar rr distinguishing factor..ssDTIME (f (")) if there exists a deterrrrinistic mrrltitape Tirrirrg rnachirre tltat accepts .reobviolrs.350 Chopter l4 Aru IN'r'Rot)uc'I'Ior'tro Co\.L in time T (n\.nrx.
and let. so is recursive..3).. etthough this may be of thcorctical interest. This will tell us whether or not zu is in L. This is becauseL € DTIME (/ (")) and every w e L is accepted by M6 irr time / (ltrrl). for example.l) steps} . DTIME ('"*) . and so on.14.. 299)...AIso.. then M6 accepts trrp in /(ltrr6l) steps.lphabet X : {0. is Proof: Consirlt:r the a. such that tr: L(Mn). Since. We gct cvcn more if we allow exponerrtial timt+ complexitv. it operate orr to fbr /(lr. is in L.{tut: M. that there are languagcs in DTIMH (n3) that are not irr DTIME (n2).l T\rring computable function.r1t2.'in the sequence101.trzt. I Theorern 14.L? If we clairn that'ur1. assumethat we have a proper ordering for the T\rring rnachinesit M1.l) steps.L is recrrrsivt:. namely the existence of a computable / (z). To see this. does not accept .L is recursive. Proof: This follows from a result in Hopcroft and Ullman (1979. p. there is no limit to this.l Ttrring computable function / (n) srrt:hthat every recursive larrguagcr in DT I M E (f (n)) . There is no tota.ts(14. this is posuible. 1}. Conversely.3 LalrcuecnFauu. DTIMn (t**t).2 allow us to make various clairrn. there is alwrrys sornething outside DTIME (f ("))..n be accepted in tirne z? for which there is no linear merrrberslrip nlgorithm. But we carr n()w show tha. Mz.im that .ln in. Supposeit were. By assuming that / is a tota.rli in f (lur.nd14. We next find the position i of u. r:onsider any ur f tr and compute first / (lurl). But this corrtradir.. we the is find M. The irurbility to resolve this issue is a typical diagonalization result and leads us to conclrrde that the origina..3) We cla. This gives us iln inlinite number of nested com* plexity classes..t.L is not irt D'I'IME (/(")). wit}r all strirrgs in X+ arranged in proper otder'u1. must be false. that there is a language in DTI M E (na) ttrat is not in DTi M E (nB). This is also possibk: bet:arr$e sequence in proper order. Whcn wr: have i. r The conclusion we can clraw frorn thiu is that there are sorne larrguages thrlt ca. IslL. In fhct.1 rr. # (14.we get a contradiction if we assume that ** # L.rps enn CourlExrrrr Classns 851 For everyintegerA > 1. thert: is some M6.. Wt: ca. no matter how rapidly the cornplcxity firnction 7(n) grows.n then define the Ianguage 7 .l assurnption. it is not clear that such a result . Assume now that the function / (n) irr the statement of the theorem exists.
€ noncontcxtf'ree by This is straiglrtftrrward. but couuting only every seconrl syrrrbol. we have no t:hrtl wltat the fu r:Irirracteristics a.> 0} can be Iislxxl in Example 13. Tlrc argtrtnetrt given there r:tlrr lltl rrsed for eveu rnore complicated la. keeping a count on talrc 2. Ilvery regular language can be rerxlgrriztxl by a det. so L e NTIME (n).llvgut:ss thc rnirldle of this string. At this point.b}.} is fi NTIME(n. We leave the details tls iln t:xercise. e * o m p tie .2 that [.la.gt: DT'IME (na) might bc. Clearly all of tlre steps r:an bc dorrein O (lul) time. Compare the symbols orr tape I and ta'pe 2 one bv tlrrtl.ge thtr algorithm 1.cp { DTIME and Lar E NTIME (n). DTIME (n) includes mtrc:hrttore lhatr L6a6. We havc rrlready establauguage {a"b' : rr. We can get a little of rrroreinsight into thc matter if we rela'tethe c:omplcxity classificationto tht: languages in the Chornsky hierarchy.r {n.pe 2.352 Chopter l4 Arq INrnotrtrc'tIouro ClonrurlTtoNAIColtplnxlrv ha. We will krok irt sotne simple exa. TIds can be done: we look at ea. $ .ermittistic finitt: autornatorr irr titrte proportiona.ngrra. Nondetertninistictr.7 that tht: c:ttrrtextfree rclcogtrized in time O (n.l to tlrc lcrtgth of the input. Brt.ch syrntlol orr tape 1. I f *innpidlilUildlt The language7': {wu: rr. Actua.) if wtr c:arrdevise au algorithm for flnding tlx: rnitltlle of a string in O (n) tirntr. Ctlpv the secoud llart to ta.).). (n:]) . I It follows fiorn Exarnple 14. .ngrrilges.s any pr:r. we cirrr show tltal L € DTIME (n.mplc:s tlrrrt give sorne of the mrlrc cltrviousresults.ctical sigtrificance.lly. 3. C DTIME (rt). Thert:ftrre Lanc. Ct4ry tlte itrput from the inprrt filc to tape 1. as we calt recognizt:strirrgs in this langua. 2.
4. We (rfirrargue that the difference between.nxlly Cl. Cotrplete the argumcnt in Example 14.g.. I Frorr these exa.timecomplexities witlt differcrrt growth rates a. 3 .4}+} is in DTIME(n).L466. such as that betweert DTIME (nft) and DTIME (rzh+l). TheComplexity P Closses ond NP ffiffiffi*ffiffi Sincc the a. Exhaustive searc.lready seen. DTIME (n) and DTIME (n2) is trot fundamcntal.R1G..l. every string of lcngth n can be parsed in time n. lct us ignore some f'actors that are less irrrportirnt. . and it is not a priori clear which model is most appropriate frrr ir real computer. : {www : tr € {a. 1.ny tapes). say. This leads us to consider the famous corrrplclxity c:lir^rs pUDTrME(nt). Show that thcrc are langutr.5. As we htr.14.) increases.ttempt to produce meaningful hicrarr:hicsvirr.. ftrr example by removing sorne urrirrtercsting distinctions. Lcs arcl txlvered. limited number of productions are applicirtrkr. more and nrore of thc fermiliesL.M. But tlte connectiorr between the Chomsky hiera. L : {wwdw:w € {u. hrlwever.ppearsto lre unproductivrl.how ma. S h o wt h a t . b}+ } is in DTI M h) (n). without arry rcgard to the degree of the polytrotnial.sses nrul NP P 353 Consider now the family of contextsrlrrsitive languages.lr parsing is possible here also since at every step only a. rl) I This r:la.ges that are not in NT'IMfl (2").Lcrp are in P. Therefore. Note.ssincludes all languages that are accepted by some deterministic T\rring machine in polynornial tirne. Lcn. 2 . tha.rchy and the corrrplcxity r:lilssesis tenuous and not verv clear. . where M clcpcnclson the granurrar. and .t we ca"rrnotclaim from this that Lcs c DTIME (nM) becausewe cannot prrt arr upper bound on M.mples we note a trrlrrrl: as 7 (n.vea.4 THn Coupr. since some of it depends on the specific modtrl of TtringE machine we have:(rr. Show that 1.
P The study of the rela. and its require:trcrtts will increase very quickly with the probletn size. thrrrc is rro realistic hope of tr. Among practica. firnctiort like 20'1".mental urrsolved problems iu the theory of cornputation. To utrdersLandthis bettelr.tempt to distingrrish between realistic and urrrcalistic cotnalLhoughtheoretictr. This is one of the funda.ctnbkt. DTIME (rf).tiorrs.lls Iem tra.too)f While the CookKarp thesis t:a. rlrrc srrrely cannot do much with it even for small n'.ttt:rnptedto put the itlea of intracta.c:table. this will be excessive fbr large ?r.ctable. while in prirrciple cornputable. Is the CookKnrp thc:sis a good way of separir. Certain comprrtir. prirctical algorithm.lling a.Iistic:a. Whilc it is generally in believrxl that there are sorne ltr.ble.l prcibletns. One atternpt to defile the term intractable is ma. rrny rxrrnputation that is not in P hirs tirtre complexity that grows faster tha.bout problcrns suclt a probtlrat are il D'I'IME (n.llvpossible.354 Chopter l4 Couplnxtrv Ar'r lrutRoouctIoN To CoMPITTATIoNAL Since the distinction bctween deterministir: irrrd noldeterministit: txtrnplexity classesa.nnot'l The answer is not clearwork with rea. while thosc outside this class tetrd to havc exponential crrtrplexities. a problem that is in P is called [racta. At the root of this is the question whcthtlr or trot P:NP.have such prrtatiorrs.rp thesis.But what a. say rz ) 1000.prohlcrtt with this complexity intra.ppear$to bc fundamenta. I'he interest in these complexity classtltt. but whtlt is rrot known is if this c:orrtainment is proper. Ohviously PgNP.hose we ca. Obviouslv. wtl irlso int.tiorr between the complexity cltt^sscs and NP has generated partir:ular irrterest alnoltg computer scientists. cxrmputer scientists have introducerrl a variety of . In the CookKa. To explore it.a clear distinction exists between llroblems in P and those not irt P.rrguirges NP that are not irr P. Even for a. Such problems are sorrxrtirncscalled intractable to irrtlicate that.tirrgproblems we c&n frotn l.dcirr what is generally called thc CookKarp thesis. c:ornputerscientists have ir. and wc rrriglrt feel justified in ca.roduce NP : ! I'rzrua ("t) . The jrrstificntion for tlte CookKarp thesis sccrnsto lie in the empirical obscrvatiorr tlrat tnost practical problems irr P are h DTIME (n). no one has yct fourrd an exanrple of this.l.bility orr ir formal basis. high resourcerequiremrlrrtsthat in practice they mrrst tlc:rejected as unrealistic on existing computtrrs. irs well as on supercomptrtt:rsyet to be designed. and onc thnt is not is said to be intra. or DTIME (rz3).1ly cut.particularly in the clilss P. come$fiorn arr at.n ilrry llolytrornial.
4 A Iangrrage /.L2 if there exists a determirilstic T\rring ma. tha.Lr is polyrromialtime reducible to . This puts NPcornpltlterrttssin tr.Llis said to be polynomialtime reducible to sornelirrrgrra.central role for the study of this tpcstion. llM A ltr"nguage. therr .ngrragein NP is also in P. For all of thenr we carr lirrd txponuntirrl .rl a. an NPcomplete problem is one that is as hirrcl as any NP problem and in sorrre sen$e is equivalent to all of thern.T tinte algorithm f'or any NPcornplete larrguage.'r'J::'::. is said to be NP*corrrplct<: if I e NP and if every . I In addition to the satisfiability problern.t is P:NP.L2.*. a' be vir:w. a largc rurrrrberof other NPcomplete probltrms have heen found. It follows easily frorn these definitions that if sorne .lplnxrry Cllessns P aNn NP 355 related concepts and questions.t this rrreans ha.L/ e NP is polynomia.T.L2 in such a way that w1 € L1 if nnd only if w2 €. Wha.ge . L2. if Lz e NP.'1lH:til. Florr this we see that if ."*j#l"ll''.chineby which an1' u1 irr thr: alphabet of "L1can be transfonned irr polynomial time to a tu2 in the alphabet of .sto be explained. Definition 14. rr. Similarly.t the satisfiability problem is NPcomplete is known as Cookts theorern. thcrr trr E NP. This problem is NPcomplett:. Loosely speaking. We e'cocle specific instances as a string that is actrrpted if and only if the expressionis satisfinble.discussion of which t:irrrbe fcrund in Hopcroft and Ullman (1979). therr cvcry la. The staltementtha.14. arrd if L2 e P.L.sa language problem.4 THU Col.L1 e P.ltime reducible to .L1 is NPtxrmplete ruljn". Orre of them is the idea of an NP<:omplete problern.
356 Chopter l4 Cotr. then L. the question remains open. and compile a list of problems that are NPcornplete. 1. alternatively.lpr. Prove the statement that if a larrguage .: is also NPcomplete. These failrrres ltlad us to believe that prohably PINP. 3. Consult books on complexity thcorv.Lr is NPcomplete time reducible to tr2. but for none of them has anyone discovered a llolynomialtime algorithm. Is it possible that the question P : NP is undecidable? .nxIrv ArrrlrutRopucuoN ro CoN4r. until someone proves that no such language exists. ** and polynomial 2.urATIoNAL algorithms. trut until someone produces an actual language in NP that is not on P or.
U Sr.+r.. that is. n Fr rt . With the inductivc assurrrption. that is r € . To prove that two sets are equal..+l : Sr f'l fi "'15' I F.+r Z l) 5. Suppose n . that is r e 5r U 52.5'z.. which means that r catrnot be in 51 or in 52.+t..n.if z € .+t. Z :5. then r is uot in 51 and r is not in 52. Conversely.1 5. 5'. This can be proven by an induction on the rrumher of sets.9*.9r U. Let' Z : : 51 U 52. 357 ...9r t'l .92..91tl 52.q Solutions nd Hints for Selected Exercises Chopter I Section 1. Then r f 51U. U 51". Therefore ZnE.92.€ 51 U Sz.. thc rclatiorr is true for up to rz sets.. 5z completing the ilductive step. wc mrrst show that an eletnetrt is in the first set if arrd only if it is in the secorrd. By the stattdard U DeMorgan's law M:E)8. Then. 6.
By induction. Thcn r e Fz so that SL1E1 + fr. Many string identities can be proven by induction.. ):r#.. they both can be writterr a*sthe prochrct of primes. n! : O (n"). Then 51 l'l$2 : ^91 5"2: ^9rfl^9r = O and tlte entire tl expression is the empty set.::_" By indrrction then. An arguilrent by contradiction works. 27._ .. Suppose that 2 .358 Alrswnns 8 . If z is a prirne. Then * (.say ur : ua.J.Sz.92.14 were ratiorral..91and u is itr .T. a rrecessaryand sufficient condition is that the two sets be disjoint.5z.. if not. Take now a string of length rr { 1. Suppose 5r : 52. . Therefore. If r is in . the result holds for all strings. Srrppose now that St f Sz and that there is an element r in Sr but not in. (c) Since 21. Ttren n 2rt: WL gives \/t = 2 m . nl nnL n n n n n n is the product of factors lesstlurr or equal orre. Becauseof this..then u is not in (. contradicting the fact that V2 is not rational. it carr be writterr as the oroduct R. L2.n n1.: ?'tyTL2 where hoth factors are less than n. Thc c:ornpleteexpression carr then not be equal to the empty set. By the inductive assumption.. 1 5 .. and so can rt. Suppose that every integer less than z can he written as a product of primes. there is nothirrg to prove. Section 1..51U Sz) . Supposethat (zu) : ufuft for all u € E* and all u of length rz...il".2 R 2.
)} 3. This shows tha.baA+abubS fiom which we see that L(G) : {(ob)" : n.lrulmod 3 : 1 rr.srrrarry more b's as needei. bu decomposed into strings ab. bAlA 13.uba. . S+aA+abS+o. (ru. 11.e. there is no possiblclrlt:r:ompositionfrrr haaaaabaaaab. 10. Tlrc first iu cxrvered by St + aaaStl(t t thc sec:ondby Sz aaaSzlao.e.ndI'rirlmod3 :2. this so string is not in I*.u. TIte two carr be t:orrtbined irrto a singlc gramrnar by 5 51152. 13.\ 14. t:ilrr eaclr of which is iri L.sic:r stxrif yorr notice that to Lt : {1""l:1b"' : ttt } 0} . (a) Generate one b.aAlbAlA The first productiorr grlrrcratcsthree a's. However.'s.crsus S59 4.baa.sn.o.Solu:uorvs nNn llrnrs pon Snlncrnr Exlln.aa. b}* rrs klrrg ir. S +AaAaAaA A . thc string is in tr*. Similarly. (d) The arrsweris ca. then an equal number of ats and bts. (d) Wc first go'nerrrte three c. The secorrdcan generate any number of ats and b's in atry posititln.. I'his leads to the easy solutiorr SA aaaA aAbl.then add an arbitrary nurnber of a's ancl bts arrywhcrtl. baaaaabaais in L*.t the grammar can geuerate atry string trr € {4.> 0} . finally a. (b) The problem is simplified if you break it into two cases. S + AbA A + aAblA B . Sitrce abaabu.ab.
digit I digit magnitrrde digit + 0l1l2l3l4l5l6l7l8le it Tlds can be considered an ideal version of C. If this string start with an a. We remember inprrt by labcling the states mnemonically. Consider then a $tring u € L. it can be derived hv S r ^9r5. \)uh 10./ Y . t ) /\'' . w{: produce output and return to the beginning to profft$$ tlx: rrext three bits. ir^s puts no lirnit on the Iength of an integer.Lr. then it ha.sthe fbrm ur : o:tltt where trrl e .13 and modify that grammar so that the start syrnbol is 51. but the completion shtiulrl trc obvious. though. (a) We can use the trick and results of Exantple 1.3 1.$'r. place a Iimit on the number of digits.360 ANSWERS ro. integer + sign magnitrrdtr sign + + llA magnitude . The following solution is partial. 7. ( ^) +\ r Vi\ b/a 7q \ l )\. When a set of three hits is dene.If it starts with a b.L1 be the language in Example 1. Most real crompilcr$.13. The automatort lus to remember the input for one time period so that it r:arr bc reproduced for output later. This situa.tion can be taken care of by S + rr. Section 1.*. . The a*s label of the state is then produ<:erd output later. /)uo . Remembering t:an htt done by labeling the state with the appropriatc irrforrnatiotr. Let .
the transducer must rememberthe two precedinginput symbols and make transitions so that the neededinformation is kept track of. In this case.pcteD ExERcrsEs 11.SolurroNs ANDHrNTspoR Spl. .
. (a) UsestatesIabeled with ltulmod 3.1 2.362 ANSwERS 2 Chopter Section 2. t)tltl a. putting the dfa in an acceptingstate if the suffix is bb. two a's. three a's. t t I t ]t+il . A { lf+il A ll+ilA ti+t \/o o. \/" \y'" \y'd \*J l (a) The first six symbolsare checked. 7.we keeptrack of the last two symbols read. If the prefix is correct. A fourth a will then send the dfa into a nonaccepting trap state. the string is rejected.If they are not correct. A solution: \t +ti n' \n' \a' \n' \n"o . The solutionthen is quite simple. (c) Break it into three caseseach with an accepting state: no o'il.
to get the main part of the dfa.Soluttor'rs aNn Hnrrs FoR SELECTEI ExnRcrsns 363 (d) For this we usenine state. 9. . (}OrO:{} Then put in additional transitions to keep track of corsecutive zeros and to trap unacceptable strings. The transitionsand the final statesa.re the second then simple to figure out. (a) Count consecutive zeros. with the first part of eachlabel no (ru)mod B. part rz6(tr)mod3.
But to generatean arbitrarily long wa.rtritrarily lotrg stritrgs. with atr acceptedstring. (h) AIso by cortttaclictiou.s$otrc c:yclein a path 'Wt: carr then use the cycle f'rornthel irritial staLeto sotne accepting utir.tc:to any final state.n \ 1 3 . Ttrl tlirsiest way to solve this probll:rn is to cotrstruct a dfa. arrrl so every accepted string ha.sto berof firrite length.tt:. Iirrite trumber of stcps.berlrxl a finite language ca.nnotc:ontairra. The solution is a little long but rrot hard.364 ANSWFTRfi (d) Here we need to remembcr all contbinations of thrcc bits' This requires I states pluu sorrrcstartup. fbr L {u"' : n.helanguage is finite. But this irnplies that t. . A uartial sketr:h of the solutiou is below. Suppose G1a has no cycies in arry patlt from the initia.lk ha*sa. {.14ha. (a) By rxrrrtradictiorr. 2 1 .:4}.lkLr. Then every wa. then complerlt:rrtthe solution.l sta. Assutne that (J.
q r } . rerrrtlvcster. Prohalbly rnost of youl tries will not work. t r ) : { q u . e z l t .hclia. 7.lent to the origina. T'he strirrg a. A foursta'te solutiorr trivial.Sot.re many diffcrr:nt solutions. but it takesa little expcrirnernting get is to a tlrrr:rrsta. Introduce ir sirrglesta.l one. . \ )= g u .rting state status frrim Qe.te Herr:is one answer: one.rting sLatep6.2 4 . a ) : { s o . 1 5 .e t . This is the kind of problem in which ylrr jrrst ]rave to try diffcrcnt ways. It is straightfrlrward to see thnt the new nfa is rxlriva.rnr FixnRcrsns 365 24. Section 2. Tltcre il. 17.s three diffi:rent symlrols arrcl tlxrre is no way this r:iln he accepted with f'ewerthan tlrrcc utates.L A a / *V\'_/\i\ / r n \ No. d(*Q r . Here is one of tlxlrn.tj'uor'rs ano Hlr'r'r's noR Snlnc. Next. d * ( q o . Then add a transitiorr d ( P 6 ' . r. Here is one that doos.
Introduce a nonaccepting trap state and make all undefined transitions to this new state.3 2. Solution: Section 2.366 ANSwERS 20. Introduce a new final state p1 and for every g € F add the transitions d (q. . Generally it is impossible to have only one final state itr a dfa. Then make ?7 the only final state. Just follow the procedurenfatodfa. This givesthe dfa 7. then d* (qo. as can be seen by constructing dfa's that accept {I'o}.A) : {prr . so both the original and the modifies nfa'$ are equivalent.w) e F originally. Since this construction requires trtrarrsitions. It is a sirnple matter then to argue that if d* (qs. it cannot be made for dfa's.r) : {pf } afber the modification.
.Sol.= + { tfr acceptsw. For example....b h lL. This is not easy to see. This can be done by doubling the rrumber of states and adding O or E to the labels.L : {trrr.wz... if part of the dfa is . \r' tl accepts L..ecrpo ExeRcrsns 367 8.. Then the nfa ..w^}.. Getting an answer requires some thought. so the language is regular. 14. Suppose that ..uttolrs er'ro HrrrTS ron Snr. One stllution is _ffi a. The trick is to use a dfa for tr and modify it so that it remembers if it has read an even or an odd number of symbols.
. you shouldbe able to convinceyourselfthat if the With a few examples the a4o.. Suppose have a dfa that acceptstr' We then (a) identify all statesQ ttrat can be reachedfrom gs. (b) introduce a new initial state p0 and add d (Ps.\aa'. .u)*qi. a complete answer requires a proof of the last statement. originaldfa accepts eaen(L). and therefore we 15. Although the constructiorris plausible.368 ANSwERS its equivalentbecomes Now replace all transitions from an E state to an O state with . reading any twosymbol prefix u.\): Q. It should not be hard to seethat the new nfa acceptschopz(L).2&3o'4. new automatonwill accepttra2..\transitions. that is Q:{SeQ:d*(go.
6 .ssumptionthat M is minirnal. qo ancl g. Continuing this way.gr).uu) e F..Since go a. T}ren use the procerhrre marh to firxl the indistinguishable pairs (go. By contradiction.o) # F tr.gr) arrd (g3. denotes arrystring of 0's ancl1's. d* (Sr. must be irrdistirrgrrishable.d* (qr.is not minirnal.ndd* (qo. 5. cornplerntrntthe final state set to give a dfa fbr . sr) e2 and Qa are distinguishable. Yes.use ((o+ t)(0+ 1).ndq. A utring of irr is not in L If it is of tlxl fbrm o.3.(1.fbllowed by the requisite nunrberof b's.hle. rerrrovethe inaccessiblestates {2 and 4a.2. 1. 1 0 .t'b"". Chopfer 3 Section 3.).l.redistinguishable and therefore the dfa is minirrml. First. (Ienerate or rrrore 4 a'$. (a) Separate irrtrlruNes ???:0.nd q6 are indistirrguishable and indistingrrishability is arr equivalence relatiorr (Exercise 7). are indistirrguisha.D Exnncrsns 369 Section 2. % f F and qa € F. Solutiott: uaa(1. ur) Q1and {3 are distinguishable. tnffi. Sirnilrr. But this dfa is smaller than M.a) € h. so (3 and ga are distinguishable. Ncxt.L.SoLurroNs eNn HINrs rron Sunc. Then we can corrstruct a srnallcr dfa M that acceptsT.rly.* b + bb+ bblt).) f F and d* (q. (A + (r:) The complernerrt the language 5(a) is harder to find. we see that all statcs ir. $o does(0 + 1). By contracliction.treca. Assume that q6 ir. This thcn gives the rrrinimal dfa. contradictiru the a. with either rz { 4 or rrz> 3.1 2. Assume that il[.r. uu. brrt .4 2' (c) _@_"*64__r_@___@*G) Tlris is rninimal for the firllowing reasorr.
te permutations of the three symbols. although quite lorrg. r t t ') 3 ' case has a. (c) You just have to get in each symbol at lcast once. n Z 2 . 18.nd only if n1 (r) : n". B r . z ) 3 . bb. 2 3 . givirrg six terms that cau be adrltxl. interspersedwith l's.. o. is conceptua. : 1 .* b)* ba (a+ t). Sohrtion: (1*01. = ( r ' 1 * r ' 2 * ) * (I(r1)u is true. We rmrst also ta.c:h 1 2 . 2L. (r') and nu (r) : na (r). m .t.strrrightforwarrl solution.tcrrations eletr ("r)). r t ( r 1 .01.)* + 1*.llyrrot hard' 1 5 . (c) Create two 0's. L which is the same set. 1 6 . that is the set tlf all strings ( r 1 ) a n r l L ( r r ) .b ba.t tlenotes a nonerrtpty string.nbe rcpeated as often as desired and thcrefore denote arbitrarily long strings. Enumerate irll caseswith I'rrl: 2 to get (a * aa (a * b)* aa I ab (a f b). then rcpeat' But don't fbrget the casewhen there are no 0's at all. ) .: l .A short sohrtiorr is ( ( a+ b + c ) ( a + b + r ) ( a * b + c ) ) . Bv thc given rules (r1 +rt)* denotes the langua. For all the completc solution you rilust genera.\ + a + u(r + aaa)b* + u*bbbbb* (4.( I ( " r ) ) . r 2 * ) * d e n o t e s( ( r . but is not enorrgh since the a. A closecl gorrtour will be gcntlrated by an expressiou r if tr. will precede tht: b. ( " r ) ) . a n d n . 1 b)* ba + hb(a * b). .370 ANSWERS irr this cloesrrot txrrrrpletelydescribe Z. etc. Ea. then this string ca. 7 2 . . The expresSiorrfor an infinittl latrguage mgst involve at least rlrre starred otherwise it r:arronly denote Iirrite striugs. * : 9 . The answcr. starrecl sullexpressiontha. 1 4 . (a) Create all strirrgs of length tlree and repeer. . m e n t so f . If there is one subexpressiorr.ge of of arbitrary cont:er. .kel the strings in which a b is followed by an a. The tcrrn (a + b * c)*a (a * b * c)*b(a + h+ c)*c(a + b + rNwill do this. Solution: (. S p l i t i n t o t h r e e r : i r s e sn t . (c) The staternent ( r ' 1* 1 2 ) .
Solurlons . so arrything will clo. its value is at least 64. Notice several things. then either the second bit fiom the left (16) or the thircl bit from the left (8) must be 1. (a) Start with Then use the nfatodfa algorithm in a routirre rrranner. + readily suggests itself. .2 S. without going through the regular expressiontonfa construction.c. but gives a more cornplicated answcr. If you sec this. then the solution ( 1 1 1 1 1 0 1 0 1()o + 1 )( 0+ 1 )( o+ 1 )+ + + 1 ( 0 + 1 )( 0 + 1 )( 0 + 1 ) ( 1 0 ) ( 1+ 0 ) ( 1+ 0 ) ( t + 0 ) . The latter will of course work.ND HrrrrsFoR SELECTEo ExeRcrsns BZl 25. If it is longer than 6 bits. The bit string must be at least 6 bits long. Solution: 4. Section 3. This can be solved from first principles. If it is exactly 6 bits.
One case is l l t \ ir'\ }{ ( rr )"*( I ) \+/ 11 n Since there is no path from qj to qa. is \ / t / Ou.o qlL **19 The other case can be analyzed in a similar manner. Removing the middle vertex gives bb+ah The language accepted then is . The result. we have to modify the nfa so that it satisfies the conditions imposed by the construction in Theorern 3.372 ANSWERS 7.the edgesin the general case created by such a path are omitted. This is easily done.2.L (r) where r : a* (a + b) ab (bb * ab* o. E. 10. . gotten by looking at aJI possible: paths.a*(a I b) ab) . one of which is qs f F. (b) First.
.SoLUTroNs nr'ro HrNrs r.. (ab) + (aa + h)(ba)t bb ai}ro \l \/ 0 The regular exprelssion then is r' : (ab * (aa + tr) (Oo)OA.L if arrd only if the corrstnrcted nfa accepts i. Then arrange matters scl that the spurirlrs symbol nondettlrministically transfers f'rom any state of the original automaton to the corresponding state irr the parallel part. if pa. aa+b Next.4t.ncrnl ExnRcrsns BZB Therr rernove statrl 3..llelwill be arr rrfh whose corresponding part is It is not hard to nrakc the argurnerrt that the original dfa rrccepts . removclstate 4....t irrrd introduce a "parallel" autorrraton with states Q0. start with a clfa witlr states eotett. .rt of the original dfa looks like tltett the dfa with its para.onSnr. ) 17' (a) This is a hard pr.nsert(L).blem until yorr see the trick. For exarnple...
The solutiorr then falls out eir"sily. If wc now apply A + Bu. then w 1 Btnttl.J1.374 ANSWER.o) : q1. This is straiglrtforward and gives transitious such a^s d (qu. so we (larl tnake the dcrivation ll. with A € V andrur e T*. l o .uRB.\ t .b) * Qo ( g s . thcrr t tR can be <lcrivedin the sirrrrenumber of steps by G. Split this into two cases: (i) z and rn are both even and (ii) rr and rn with are both odrl.S Section 3.5 b) : qr d d (qr. (a) First constnrct a dfa fbr L.d (qu. o) : (1o.b) : .R . But 8 contairrs the rule A . o) : t1:1. rZ. (qt.FA can be derived via G. Beciilrse ur is cretr. aAlB bbbG bClA we (:arr show by irxlrrction that if tu is a sententirrl form derivtxl with G. d (qr.3 a.wfuRB : (Buw)R completing the inductive steP.tetl with left lincar derivations. it Inust have the form ur : Awr. Right linear grarrrrrrar: SaaA A ts C Left Jinear gramma.b ) : g r d .r: S + Abbb A'+ AblB B +aaU C i a(. 5 + aaSlA A + bbAll tnkirrg care of casc (i).(q2. By the inductive assumptiorr wR : u.1.a ) : e 2 . 6( q r .
r . a ) : ( q r . p o. 1 ( ( q . p o .teand final state 91.f (pr. 51 is regrrlar as is 52.) is given by d (po.\ qr + bqnlaqo qz Qzaqtlhrlo tulzlbQt 16. ) ) : ( q r . . aq1 lbg2l. . L ((a + b) a. Obviously.L(baa.eflret ExnRcrsns 378 with qs the initial and final state. Flom this we find ) d ( ( s o . with Iirral stir.a ) : q r . b) : Ft. p z ). a ) : ( h .b) : pl. p t ) . When we complete this construction.a) : pzl6 (pz. etc. we see that the only firral utate is (qr. l i ( g r . ) : qr.b) : qt. We can show that their union is also regular by constructing tlrc f'ollowing dfa. A dfa for 6 (h. l r ): g r . d ( ( s r .SoLU'r'roNs nr'rn HrNrs poR Sur. a d ( q o .4 gives the answer qs _. p r ) .pr) and that L ((u + b) a. p z ).6 (pz.te p2.1 2. b d ( ( q t . . \ Ff^f"'L(qtl The condition that V1 ancl V2 should be disjoirrt is cssential so that the two nfa's are distinct.6 (po. (rr) The construction is straightforward. Therr thc' construction of Theorcm 3. a ) : ( q t . A dfa for .n) : pt. p t ) . with q1 a trap sta.b) : p. but tedious.6 (p1ra) : pz. Chopter 4 Section 4.) n L (baa*) : baa*. p r ) .) is given by d (qo.
12.u1 ab * ba I bb).ny. Therr.2. X*.t :r. Section 4. 5. replace cvury production of the flrrrn A+ frt with A € y1 atrdr€?*. there exists a' rrtetnbership algorithrn for it.51 lS2' (b) ln P1. Frotn the clfa ftrr.r.by A+:rSz. Htrrc you need a little trick. Sint:cL1l L2 is regular and wtl have att algoritlrrn fbr set equality.Lft is regular.376 ArrswHlrs 7. is Tlre key observer. Then make every tllrlrrrent of P a fina.Pr) genertr. usitrg the constnrt:titlrt suggestedin Thcorern 4. $ince by Exarnple4.L. cotrstruct the tifa for Lft. T .we calt a"ssume that yl arrd V2 are disjoint. The Ieft side is regula. If Lt c . The resr. Find a. L1 tt l'2 is finite atrd therefore reElrlirr for all tr2. theu L n L ((u. 12.tt:. LylL2: given statement would then imply tha. The an$wcr is yes.L2 is regular.r:Sr.rlt tht:n follows fronr closure untler cottcateuation.l sta. by A . which is regula. By closurc urtcler reversa.ndz € 7*. ir.9 . It can be olrtairreclby sta'rting fiorn the set identity Lz = ((Lr u. fbr aIV tr2.lity. We cim use the following construction.L2 is regular' 18. P 2 ) .) = a.l.llsta. ( a n d G z : ( V z .L contains Ilo even length strings. Corntlirre the two gra. (c) In Pr. The rest ttrett follows easily fiorn the known closures unrlcr uttiott antl compl(lrrxlrrtatiotr' 14.W i t h o r r t l o s so f 26. Use trr : X*.5r.6.$o we can use Theorem 4.r.Lz)n Zi) u (r' 1rz) . Suppose Jr :1yt.tcsP suclt that there is a path fiom tlrt: ittitial vertex to somt: elertrent of P' and fitlrn tltat elenent to a.2 1. werillstt have att algorithm for sct irrclusion. If .tiorr that sitrce tr1 is Iirrite.L2.7. Tlrtl result then follows frorrr closure under irrtt:rsection and complcrntltrtir.rt sytrtbol and add prodrrt:tiorrs.tiorr. .mmars aild (a) Make 5 the new sttr. replar:t:every production of thc forrn A+ rt with A e V1. final state. Then use thc rxprality algorithm in Thcxrretrt 4.'l L1 .Tlrt: 16. Lz) : L1l) L2. . Notice that nor (I'1. 5 2 .thetr L1l L2: Jlz. 2.7.
is not regular. : a. 1 m.t: ettzb't'a2"'. r t s cottstnrr:t rcgrrlilr expressions. r rp + ( i * r Irtplr is nclt irr ttn lirngrrir.so lhat wi : aP+\it)h. > 5. .. we carr thclrrviolir. (e) It does not seerneasy to apply tht: prrmping lerrrma.nguages. our is opporx. Brrt Z : {to .keL1 : {a"b* : n. Suppose that 1.*h*).tion.. we use i : 0 to violatr: thc txrndition n ) 5. 1 S .r known closures fbr regular Ia. bot}r of whic}r arrl rulrrrcgrrltr.t has two consecutive symbols the sarne is in the languagc. Tlre prolrosition is false.soften as desired wilhout charrgirrg the acceptability of a strirrg. 5.. By contradiction. I f w e t a k e i .Lf rrnd tlx. so we prcltxlerd indirectly.rrrl But LrtlLz : {att6nt . (a) Ibke p to bt: tlxr srna. directly.r expreusion for "L is (a + h) (a * b). firr which one can ea. rrsis ea"rily shown. t h t .sily (lr) Tlris lir. wert: rrlgrrhlr. If we ta.The strirrg. pir:k rt.lk in the transition graph of lt:rrgth lul.rnt O2rn. But r:leir.3 2. If the opponent choosesgr as consistirrg of b's.onSelncrrnn Exnncrsns 377 Section 4.r. L(a.llestprime number greater or tlqrrirl to rn rr. 9 .tothe condition ft < L 1 1 .kei ) 2 then nL+('i .i)ft > rn.ge.nd : clroose'Lt) trP. 1 ) f u : p ( h + 1 ) i s c o m p o s i t ea r x l E .qxo Hrw.s r.ryrmrst then he aa and the putnpcxl strings will be 'ut. 4. h : 0 .(aa + bb)(a + b) (a + b).'rrthrr. This is nxrst qilsily seen by splitting the p r o b l e mi u t o c a s e s u c h a s l : 0 . ta. wa..string of a's of length fr. If we chooseu : a(Laauaba.cryclelabeled E in this walk. For the dfa ftrr L to l)r(x:e$rJ the middle string u reqrrirt:s ir.s severa.Soluuons .ge not regula.rly this cycle can be repcatrxl a. Now ig is a. n. since anv string tha.r.) The Iauguage is regular.tesin the clfa. If this is longer than thc rurrnber of sta. Lherermrst bc ir. tr is regular. is not in /:. n.1 : p .r. then u. (a) Givcxrrn. . (a) The larrguagc is rcgular. Then by the closure of regular languages under c:ornlllrlrnonttr.L is not regular. As a courttc:rtlxilmple. If y consists of orrly o's. We see this frorn L : h rf .} * L2 tr.Z would also be regular. > rn}. A regrrltr. (a.ngua.l choices. which is regular.o(w): no (u)} whit:h.m't (i'* I)k r.
.pplv S . take any strirrg z : lttu)Rt with ltul : n.r of lt:rrgtlt rtl that is in a'll of tht:. It must tht:rcfrrrctbe in {lr1u. then a E If r { u i u . As the final step we must slrow that for each i.Li is the set {ttrtrrft}. If rr.r. such as y : (. Chopter 5 Section 5.'tJ fub)k'a.r languir. . With the second choirur.ltl'tl6 is trot in 1. 1.ct that for circh i tltere are onlv a' finite mrrntttlr of substrings ui.gc tr : { a ' b " : r z> 0 } . 1 . Obviously.t thtl llrelix conclition no (u) > n6 (u) holds. we apply S .t: (ab)'"'aa(ba)"'. Thrru. 1 5 .s (:irrrrlotbe in {tizuon: ltr.Li : tlb' ti : 0. Suppose that for every sentential fbrm dt:rivcd frotrt 5 itr rt. tlrt sirrce the addecl a is to the left tif tht: adtled b. If it we take i greater tha.. either. ri.nguirges Li: : l { u i u u f . Corrversely. We claim that the rrrriorrof all the .fJer { 1 stt'ps.lforrn irt rl * 1 steps. It is qrritc otrvious that atry string genera.1 a.texlby this grarnmar has the silmc rrurlrber of ots as b's.This follows from the fa..o. . L a . is irr the uniou of a... .Consequently. . In the sary llow has severir. Since neitlrtlr of these chatrgesthe nurnber tlf a's arrd there. 2 0 . To justify tlris. Altcrrratively.. u f . Since the ouly possihlt: idtlntification is ururft = hta. This ndds arr extra a attd an extra b.l = i} trec:ause is not Iong enough. ) ti.rly to hold. the prefix r:orrdition will still be satisfied. . Take . : l t r i l= i } u { r .La.ch t. 1. . so t}trt it has the form tutlft.. . is firrite atrd therefbre regtrlirr. t h t r nz e { t t i u f .} ilntl so also itr L. it will still hold a.55. .Take '7. luil < i}.n m.ll t. As cotrttterexantple.f . To get a sententirr. 2 . To show thtr.. t}re prefix conditiorr holds after one step. Tlte adver: r:hoit:cs. the prefix condition txlrrtirlrt:s b's or the location of those irlrt:ir.378 ANSwER. brrt the utriotr of all the langtrilgcs is the notrregula.r. so we havc a trirsisarrd the induction succeeds.take the lir. No.2. it is not. z < z .a5b..ab)kot .take any string. so it r:irtrnot Lrein 1. we carry out an induction on thtt lcrrgtlt of ihe derivation. we ca'n a. For ea. L 7 . l u l l < z } . r r r l t } r c r e f o r e i n .. e z: {0. steps this corrdition holds.ht: Li...l first case uro = (nb)k aa(ba)"'' . : l o i l = i } a .\ or 5 .S (b) The language is uot regrtla. if tlrtl prefix cotrdition holds after n z steps. Liis regultr. r f : l u d < z } .thc lettgth of u0 is odd..
L directly to get a grammar for Z.SoI.Ecl). (a) If S derives.. wt: add 5 . hopefully recursivedescriptionfor .Sz.AlaAlaaA (d) This ha^s unexpectedly arr simplesolution S + aSbb la^9bbblI.BlA D . we start productions with S t Srl. Finally.. 1 5 . is arbitrary and m 5 h.+B b l I But this is incomple. = m * 3.ke care of the casesz : 0. (a) For the first c&se : ?n arrd ft is arbitrary. The first caseis solvedby 5 . Trr tir. (a) First. L2.9c .5r  55 derivesL2.te since it creates at Ieast tlrree a's.aSrbltr. solve the ca$e n.\ In the second case.rn be achieved rz by St: AC A + aAbl\ C +Ccl. This c. Then add rnore b's. 1.:h*m and rn: h+rz. Sz +BD B + rr.L.a. then .2. Here we use n. It is normally not lrossihleto use a grammar for . Theseproductionsnondeterministicallyproduceeither bbor bbb cach for generatecl a. so we rreedanother.Je$: n.urroNseun Hrrurs poR SslucrHD FlxgRcrsEs 379 7.\ l51l Sr .bDclE E . (e) Split the probltlrn into two ()ii*. This can be done by S ttaaA A + aAblB B .
A solution irr S +aA.380 ANSWER. 5r + a5rBltr Bb is rrot arr sgrammar.. Suppose we hrr.EIE. .lh B C b](rla]b aCa bCblA.l(E)llls. C)nesohrtion is S .nbe of tlte forttr wwR . contains tire strings of odd length. While some part irroutrd lhe center ca. ttris is a little harrl to stle here.ab. One obvious subset of 1. tlrcrefore be of the fortn utturRbu rsr ubwwRau with lul * lul. trrrt this is not all.ASAIB A + a. There are two lef'tmost derivatiotrs for w : aab. Orrct: we see tlris. thc third the two disagrecirrg symbols.ts Note that the more obvious grammar 5 + a^9rB b.rsfor these typtls of strittgs. possibly lead to But this scrrterrtial form has the suffix abc. The first two productiotrs geuerate the u and u.so it cer. we can then corrstruct grailtlna.veirrr cv(llt length string tha.t is rrot of the form uuft.nnot aubbabba. 5+aaBlaab S+AB+AaBlaaB+a.2 2. atrd the last the innttrmost palitrdrome. The only possible derivations stirrt witlt S + aaB + aaAa + aabBba + aabAabu.S Z. A +a. 6.ABlb. E Section 5. 19. the senterntlc 22.ceon the right. or vice versa. compare corresporrrling symbols. Working fiom thc r:crrtcr to the left and to the right sirrrultaneously. at some point wc get aII a otl tire left and a b 'I'he string must in ther rxrrrcspondingpla. E + * E lE.
1 3 .kt:it plansible.li .Exnn.aAlaAA A .a B +b}lbbl C +bClbb1.bAblhh. Sirrce the dfa never has a choice.2 thc two grilrnmirrs are equivalent.. Morc cxrrnplicated strings are built frorn these two situations.aaAlau B +hClhbC C8. Thcl grirrnrnirr is itrr sgrittrtrnar except for {l .5. Chopter 6 Section 6.ilrr retrtoved.. ut = aahb.4 n aAb. consider the two tvpical situations.lont rrrril. * uhab. r4.rnlligrttlrrtt gri. whitfi carr otrly be derived stir.Al. By Theorems 6. . To rnir. Solution: S . so they can be parsed only in one way. there is never arry choice in the procluction that can be applied. so removing Aprorlx:tiorrs gives '? . and ti.d \.crsns Souluoxs ar'rl HrNrs FoR.9 + abS+ ab.ndllrnovitrg it results in 'i*hl h ' r  f i A "*\> '8 yl i".SELECTET 381 L Ftorn thtl dfa for a rcgrrlilr limguagc w(] cirrl gct a rcgulirr gritrnrnat by the rrtt:thorl rlf Th(xlrcrn 3. be Then B becomesuseless and the associatedprodtrt:tiorrsc. The only nullable variable is A.\. Use lhe rule in Theorem 6. An orltivir. But this rulc rlotls rrlt crclatc irrqy arntrigttity.aAblabl AA. 20.rr orrly be derived by starting with .t this grammar is unambiglr()lrir.s b a? uAlalaBB fi + A + ae.Ala.rtefbr B irr tirc first grantrnar.syto see tha.\ A .4..1 and 6. Arntriguity of the grir.1 to substitr. C B is the only unitprodrrction rr. It is not ea. whictr r:ir.rting with A +AA.mmilr is obviorrs fiom thu rklrivatiorrs S+aSb+ah S +.fi S + AlulaBB u A .rrnrnirr i$ 5 .
b B + bbAlbb. 2 1 .BBIB B . 14.BB B . which is not acceptablcin the construction Theorem6.6. The grammar ^9 + aA.V.Al V"BIV.aa.aBbl\. B and d ere useltltss. of of is S .a) . Section 6.AB lbbAlaBlbb A . The language generated hv this grarrrrrrar is I ((aa).bbAlbb.rV6V1.uAl(r A .2 5. This is obvious sincxr the rernoval of rrseless Drocluctiorrs rrever adds anvthing to the grammar. A +V. Tltis has introduced a unitproduction. We can now apply the construction and get S + AB lVhV. B + V7.aaAlaa.mpleis S+aA A. . 1 6 . Brrt it is not minimal gince S . we get so S .382 ArqswnRs Finally. This gives S+ BlBlaB A Aaab B . An exa. Removal this unit*production easy. A + a does not have any useless productions.aa is an equivalent grammar.aBblab. any unit pr()ductioruJrer any Aprodrrctions.V6AIV6V1. Whenwe remove trproductions get we SaAla A . First we must eliminateAproductions.
AIV. .br. create new varitr.A.. aBC.11. Vt .blelu Vz attd productions A . 9.a. wc catr use the yij colrtputed there. . a. Productions of the form .VnV.. Only A +\ABC is not in the required form. simila. W * a Vnb' 8.b. .Souluoms eNl Hrr'rrs poR. aB b 1 b 2 . This normal fbrm r:arrbt: reached easily frorrr CNF. . the string arlb is irt the latrguage generated by the grammar and can thercft)rc be parsed. .nd S . Since S € Vrir.. Va b. The latter is not in correct form.crsns 383 a.Srlncrno Exnn. Then use a... (lontirruc this process.V.nd A atVr. Vz .AlWBlVbVb A VaVa g . irrrrl so rlrr. Considerthe generalfonn for a production in a linear grarnmar A a t a z ...ABlV. Sirrce aab is a prefix of the string in Example 6.anBbft2. wtl hirvc SaSA AbAV VbC C Section 6. Solrrtions:5' +aV6laSluV"S. b r u .V.reperrrnittr:d since a : . so we introduce A .. Fbr A .tr. itttroducitrg V2 alrd :. I/r. W a.bAV and V + BC.\ is possible.V. n Irrtrodrrce a new variable Vr with the productions Vr + az.r I)ro(lessto rernove terminals on the right.. 15.(trn Vz +QB Bb1b2. 12.3 2.V. until no terminals remain on the left. V' . but after substituting filr B. tr/l .aVtVz.4 +BC a.
a.. . Suppose tlre ccrrrtcrrt thcl stirck ir. 1 rt.np. we see tlut we carr get to this tlorrfiguratiort orrly if at tlris point the utrread lrirrt of tltc irrptrt is r1:r2. I ) } d ( .with 4 € V1l and R eVzt a AB. that is.we determine the prodrrt:tions that wttrc trscrl irr jrrstifirrg5eV13: S e V r r b e c a . or three tokens by d ( q o .1 1 a ) } a .tthc timc of the switt:h is n112. ( q 1 . ( q r .1 1 1 ) } d ( q r . d ( q o o .. ( g r o . The key to the argument is the switch frorn q6 to q1.384 ANSWF]Rs Frrr ptr.cr:eptingconfigura. t ) : { ( S t ..r . 1 1 a ). However.b . ( q 1 . 1 1 ).switch is made at some other point or if the input is not of the ftrrm ur?rrft. a.a ) } I d ( q o . z ) : { ( q 1 .teone. Chopter Z Section 7. two. a ) : { ( q r . d ( q o o ... .By exarrrirrirrg thtr transition functiotr.ll to thesecan thcrr bc usetlirr thc parsirrg S+AB+aB+aAB+aaB+aab. 1 i . 1 " ) . if thc original input is of the forrn 'ururEand the switch was rnade cxactly irr the rniddle of the input string.iln a.1 1 l a ) } arrtl .Trr arx:ept tlf a string we rnust get to tlte cotrfiguratiott (r71.r$ing. with A € Vzz. a ) : { ( q r .z. Solution: d ( g o .nnot he reached. 1 1 . (a) The solution is obtained by letting each a put two markers on the stack.rp.tion ca.genera. 1 1 1 ) }.B e Vn a B e V r r bet:a. u sS e A €. if a. B This shows the productions needecl justify membership.a . (f) Here w€)u$e nondeterminism to." ) } . V11 b e c a u s e A B €V23 becauseB A € V 2 2 becauseAAB. z ) : { ( q 1 .trse + b. u ) : { ( . 1 ) : { ( q 1 . 1 1 ). which is done nondeterministically and need not happen in the middle of the string..1 2. while eaclt b corsurrresorre.\.e)..
2 ) }.70 11. Z ) } . d ( r 1 1 . c d e ) } is repla. d ( Q oA . What wor.t:edby ) 6 (qo. T(r ovtrrcome this.o.b: {(qi.n be ca.ccepts the Ianguage L:{"}DL(tbh*a).{ ( q o .a) : qr r) (qo.l state qs and the acceptirrgstate gr. i . h:) { ( q i . 14.'s to b's and back. The pda thercfbrc a. .2)} .a. cd)} . followed by orrrl or more b's. this rxrrrstnrction ca..a ) : c r d (gr.a . The state transitiorrn r:a. Tlace through thc: pror:ess.rld normally be trackrxl hy different states is now tracked by thtr symhol in the stack. 16.]resolrrtiorris then essentially the $a. The altcrnative path requires olle a. 6 (eo. u ) : { ( S o .nthen be taken directly from the pda. Here we are not allowrrd enough states to track thrl switr:h from n.a. For exarnplrr. These are the only choices. a . thc initia.) : {(Sn.1 ) } .r'r.taking one path rrt ir time.rriedout for any pda. d) : {(qi. 1) : {(So. We have only two states. 2 ) . a"s L This is n pda that makes rro us(] of the stack..de)} d (qj". terrninated by irrr n. we put a syrnbol irr the stack that rernemberswherc irr the sequerrce are. For cxilmple. The transition fiom qs to 92 can bc rnade with a single a.tr. 2 ) : { ( q l . 6) : .b) : gt d ( q r .)}. a . finite acx:t:pter.10.solution is we d ( s o . tr. so that is.l:c. irr trfl'ec:t. t t d (.me 4(a).rr llxnRcrsns 385 The rest of l.Solu'r'rons Ar'rD HlNrs rron Snl. Since d carr hirvrl only a finite number of eletrerrts and each can only add a finite arnount of infbrmation to the stack. to givc .l (go. Here we use internal states to rcrnember symbols to be put on the stac:k.. lr. 6 ( t u .
l d ( q t ' . > 0}. . Be<:ause the trausifintr. SSS) .") : {(qr. I ) } .1' . After tha.5^5'.y.A) : Fitrall. o. h .o.I)] d (q'. giving 5' aS.t. we apply thc corrstructiou in Theortrrn 7. Florn 7.386 ANSWERS Section 7. z ) } : 6(qr. where qs is the initirr.5) : { (q1. L We first obtain a grirnrrrrarirr Greibach normal filrm for tr.l rrpdrr's are also equivathe original and the tivity of equiva.b.n. (lo)(h)qI. d ( q i 1n .A . Ytnr carr follow the constnrt:tiorr of tlre language is {a"+?bzntl :'n.l. d (r.111)} 6 ( t 1 2 . .tr. we call constrtrt:t arr equivalent cotrtextfree grammar. Frorrr that gramlnar we can tlrtlrr construct an of equivalent threesta. I ) } t.l tlr you can trotice that 3. i ( . 11. d(g6.'s witltout changing the stir.z ) } 1 6 ( q r .. 'Iheoreur 7. : { (q.r.f) : t(.trlrrlltla. givcn arry trpda.1) {(q2.l state atrd q1 is the finer.l (.B .a.t:l<. Ient.). I ) } d ( q o . e)} . 1 n .b. . a) } a) stirtc:.1*. o . With t}re latter observatiorr wc get a solution . for exantple S + o. s) : { (qo. A r:ornplctesolution is (qn. . I'he state tJ1ttirrl bc clittritratecl if we use a special stttck syrrtbol z1 to rnark it. Then ftrlkrw the cotrstruction of Thslrr:rrt 7. a )= { ( q 2 .o . Thcrc rnust be at least one a to get started.5'sr) } . B . when {(Sn. .2. B)} d(qr.liBlb. S) : { (qo.1r.tr. z ) : { ( S r ' a ) } d ( q r . First txlrrvert the gramtnar into Gritlbach trortnal form.A)} simply reads o. d ( q r .. usitrg'Iheorem 7.1 ) : { ( q r ' A ) } . a r) : { ( s i . . S ) : { ( q o .2 'I'heorem T.b. Next.klrrcltl. a ) : { ( q z .\. b) : { ( q r . (qr . l . 5 ' r ) } d (qt .1 to gct art npda with three statcs.S B ) } o b.aB.A . 1 .I) } 6 ( q o . B ): { ( q n . a ) : { ( q 1 .l 4.
. everything can be dorrc so dctcrrnirristically. Therefore.r:cepted a dfa by artrl such a dfa is a dpcla with an unused stir.rlrr be tr.3 4. {(qr. 0 0 ) } .pplies. 1 rA .3. a ) : { ( q 3 .l.nd the final sta.a . while in the sccxrrxl + SS is the necessary 5 Iirst stt:p. Thc: c signals the switch from savirrg to rnr.a. Brrt if we see only the first four syrnbols. the pda will switc.\transitiorr to tlxr final state. At first glauce.1 1 ) } : (Qr.4 2.rclto see that the result is a durlir. The solution is straiglrtforwrrrrl.r:k. d ( q Ba . Il should rmt tre too ha.gc.a. we can constrrrc:ta rlpdtr. d ( . if followed try a single Section 7.kea .1):{(s1.L1. 15. Nevertheless. this tna.11)} b d(q'. ca. from whir. d ( s 1 . .a ) } whereF:{q2. . Consider the strings aabb and aabbbbua^ the first case. a strirrg will be ac:t:cpted and only if it consistsof one or more {r. with a dfa along the lines of thc rxrnstrrrctionin Theorem 4.vseerrr lle a noncleterminislic larrguir. Since . the initial.r)} . Prrt a's and b's on the stack. goes as into a final statc wlxrrr the first input sytrbol is arr a. . to since the prcfix q.r:k hnndled as it is for. t r" ) : { ( s o . i ) : { ( q 1 . Wlrcrr this happens. w i t } r d ( q r . you will see thai the pda is deterrnirristic. Lhe prltr goe. 0 ) : t ( s . The rest = is essetrtially tlrtl sitrnc nu Exa. The basic idea here is to corrrbirrcrr.tire derivation In rrrrrststelrt with 5 + aSB.'s into state q1. wo r:ilnnot decicle wlric:lr t:ir"st: a. Complete solutiorr: d ( q o . 'I'hus we have d (t111.rtching. a ) : { ( q z .te qr unless a a is on top of thc sl.1) {(q1.h it ca. Section 7. There are two stiltcs..2) O t ) } .This dpda. When you write this all out. .'s. 11. 9. The grammar is thereftrrtt rrot in LL(4). This is ohvious since every regular languirgc r. it gclrrs orrt of this state and thcrr ir:rnpts at'bt.mple 7.te q1.llsfor two different types of suffixes.h stirtcs to q0. nonaccepting statc 96 ir.ack.dpda. with thc sta.qJ}. The pda will bc in sta. 16..n only ma.b.a ) } .1 z ) } a . If more symbols follow. e t c .tht: Ianguageis tlcltr:rrnirristic.Soluuow$ er'rnHlFrrsnon SnlncrnD ExliR{trsus 387 the first b is encountered.
Ltrok rrt the lirst three symbols.dvcrsarylrow has several r:htlit:tls 3.L(a. :4.mplcsc. Ftlr t:ach case' we can find a.blestring rmrst be in .L grammar anrl the two can be combintxl irr an obvious fa. pick u. The gra."'b"'brrlartlattlbttr.tl. ftrr exaurple.''*hL6nt6m OrnOut6rtt.nhe purnped out of the langua. (a) Use the pumping lcrntna. lf the first three svrnbols are abb' then strirrg atty parsa. 51 or S =* 5'z is 7. choice of u irrrd g that treeds any serious tlxarnirratiott is tr : aft and .but in all casesthe string ca.(t. wc rnust use 5 .shion. aab.51 .51152 .. tltat have to be consiclered. we must apply S . The only ?. which is not in L.t26m.91'+ nS1lba Sz + ahbB B . This is almost a.nlmtrr is urrirtnbiguous. (a) S .\.bn).L (abbb. As lotrg as the currentlv st:arrrredsytnbol is rl.reil rurrrlberof olher possible choitxls.mrrrar .bRlA. If they are (r. When this dpdir. then the 4. Take ut: u. There a. the grirrntnar is not /.r c:anonly be in. if it is b.grarnrrar.n sgrammar.).L(1). u: ak and't7: tl{. is converted into ir.1 The ir.\. s.aSc. ct aba. Givetr rrz.I (k) fbr any A.L.ge. the gra. dpda. wit}r u atrd E located in the prcfix a*.arrbe made for arhitrarily long strings.. A sohrtiorr is . if it is r:. For a detcrmirristic CFL there exists a.rr L. tltett 1!)O : Q. I Chopter Section 8. necessrrry.t the first tlrree symbols tells rrs if S + The gramtnar is thereftrrt: LL(3). Looking a.5 .a S c 5 1A l l '5'1+ bS1cl.If. is wc carr otrly use 5 .388 ANSWTTR$ similar exa.
L 0 .( and ). If this does not match the remembered syrnbol. If lzl > 1. With a given rn. If w C L there must be sorne h for which this happens. w i t h k a n d I n o n . The language is not linear.z e r o . i >mz+k' Since ru2 is not in the language. . .. Now u and y are entirely made of a's. we can easily purnp so that for some prefix z. It then examines the frth symbol in ru2. Similar arguments hold a fortiori for I > 1.*I T h e n c h o o s e: 2 .^bn'an'b* and examine variou$ choices of u and . the language. Construct an npda that counts to some value h (by putting h tokens on the stack) and remembers the hth symbol..r7. )+ ( . . so that tr2 ha^s + h a's and rn * 1 b's. L2.) stand frrr rn left or riglrt parentheses.Solurrons AND HrNTS FoR SELECTEnExnRcrsps 389 t A : b I . the language cannot be contextfree. . wherep and q are prirnessuch that p> m arrd g > rn. n1 (u) < ny (u) which results in an improper expression. Use the pumping lemma ftrr linear languages.respectively. But m2 (rn+I)2:mz +Znt. then i wi+L: aps(l+li)1 which is not in the larrgrrage. 20. Sirnila. so . Use the pumping lemma with I. The npda chooses the fu nondeterministically. With the pumping lemma. ) where (. .( o ) . : ( .( o ) . 15. .r arguments holcl for other deurmpositions. (f) Given m. which is easily pumped out of E . then lni+rl: pS+ik. this language is corrtextfree. choo$erp *qtnfitn+l"m*Z. . . If we cho11se : pq.. the string is accepted. .. Perhaps surprisingly. (b) The la"nguage is not contextfree..S u p p o s e h a t l : 1 .u: e. Use'u): qrtQ. choose u) : a*bz*a*.ruis easily pumped out of the language. If luul: k. use u .
and .91 P1) and G2 : (Vz.Ill}and . P r ) b e a l i n e a r g r s . L 1 a n d l e t G 2 : (Vz. The complemerrt is contextfiee. These in turn can be broken into n . 9. T}re language .7. Ttr show that linear langrrages are not closed under concatenation. choosing Sz as a start symhol. d (sr.T.a) : d (sr. Hnnu. The full Ianguage is then the union of these four case$arrd by closure under union is c(lrrtextfree. 2T.T.h) : (q1..t scans the entire inprrt is .] < n " ( w ) > n o ( w ) .Sz). a n dn o ( ' u r ) n " ( u r ) . I e .mmar$Gr and d2. n " ( w ) < n o ( u ) . (w) # nu (*) and ?2.L2 biguous.Lz is not linear as can be shown by an application of the pumping lemma. take tlre litrear language 1': {a"b": n > 1}.Pz) be a leftlinear grammar for L2. n " ( w ) 7 n " ( w ) . u . It is then easily shown that in this grammar SzlSrwlutu if arrd only if u E L1 atrd tu € tr2. . 1 5 .r by A .^9. A threestate solution thtr.xR ' We can then show by an induction orr the number of steps in a derivation that if tu is a sentential fbrm for G then uft is a sentential form tbr d. Sz. d ( g oa ) : ( u . d 5. (G) if and only if 5 is nullable. t S .7. (ru).L (t) : r. The complement irrvolves two caseu: (*) # n. Pl U Pz US i Sr l. with F: {Sr}. l?) !. But their intersection is not evt)rr cotrtextfree. Chopter I Section 9.2 1. Given a contextfree gra. ? . TIre languages Lr : {u'nbtrt"rttl are both urlam{arrbtlc. Then d i.mmar G. m m a rf o r . with VrftVz : @.390 Arrswurrs Section 8. construct a contextfre* gru*** by replacitrg every prochrt:tiorr A . Pz) .a. S r . Combine gra.L 2.tr) : (gz. L e t ( J r : i y t . Construct a grammar 82 from Gz by replacing every production of the form V + rtn e T* with V + Sr?:. . R ) R) d (qr.Sz. Each of these is context*free as can be shown by construction of a CFG. form thc combined grammar G : (Vr\)V2. (Gr) u L (Gz). Given two linear grammars G 1 : (V1.
R ) with .a) : (qz. a) : (q.a. We cannot just search in one direction since we dorrtt know when to stop.F corrtains more than one elernent. 12. lt) .a) : (8r. ftlr example. The general scherntrlooks something like this: (i) Place a marker symbol c at cach end of the strirrg. a. l t ): ( q z .n .D . b . but tedious to write out in detail. ft) . f l ) a d (qo.a.a) * (gz. d (qo.urrolrs arul Hrmrs l.a ) : d ( q r .i and the trarrsitions d (q. (a) d (go. placirrg markers at the riglrt and left boundaries of the searched region arrd rrrovingJ the markers outward.tr) : (qz.emove one of the cr's arrd move the rest of the string to fill the gap' Obviously this is a lorrg joh. 7. introduce a new firral state g.r. Repeat until the two c's meet in the middle of the string.) 6 ((tz.b ) : ( q 1 .lly simple. 19. ) : d ( s o . . R) forallq€Fnnda€1. If the final state set .a.n. ) R 6 (qz.( q s .ncrnDExERCISEs Bg1 It is also possible lo get a twostate solutiorr by just examining the first sytrbol and ignoring the rcst of the input. (b) d ( q o . R. We rrrust proceed in a backandfbrth fashion. but it is typical of the cumbersone wayir in which Thring rrrachirrers often do simple things. fi) with .b): (gs. The solution is conr:eptua.b.F: {qr}.b ) .F : tqi]. l ( S t . (ii) Replace the twosymbol combination ca on the left by ac arrcl the twcrsymbol combinatiorr ar: on the right by ca. (iii) R. 10.on Snr. .Sot. ft) d ( q r .
and a multiplier that multiplies two numbers. o. and so on.R) . .a) : (rljz. an add.  Chopter 0 Section 10. (a) We catr think of the machine as constitrrted of two main parts. c) c. L or . a) a.392 ANSWHRfi Section 9. 8. lt) for all c e X * to} .l (qo. as defined so far' is deterministic.b.3 2.onerrtachinethat jrrst adds one to the input. This can be done as suggested Exercise10. . (b) To simulate 6(si..5} with the restriction that for all transitions d (qz. the condition a: b must hold. Therefbre.. S) and 6 (qiz.B). We have ignored the fact that a Ti.L) with a'+ h of the standard machine.n. split the input into two eqrral parts.rring machirre. Section9.b. b. Section 9.b. (a) The rrrachine has a transition function d:Qxf8xfx{. Stltetnatically they are cornbined in a simple fashion.u) : (Qj. (so. : (er.1' Then comparethe two parts' in symbol until a mismatch is fouttd' symbol by corresponding A solution: d (go. d The state qs is any state in which the searchrightinstruction rrray be applied.a) : (qi . L) for all c € f . while a pda can be nondeterministic. R) .rt.l) : (qi..b) : (si. we introduce new tratrsitions d (qa.L. we cannot yet claim that Thring machines are more powerful than a pushdown autolnata. (c) First. : (Qo.2 s.1 a.
b}.l so trarrsitirrnd(S..nningand updating its active area. fbr each r) (qi.tr). Wc repla.l (qi.ldefinition must provide ftrr the resolution of possible conflicts.u'r'ror. One issue to consider is wha.R) fbralldef{a. tape content ofOM position oftape head # 1 position oftape head # 2 . F o r t h e f o r r n a d e f i n i t i o n s ef 7 ' : I ' x f l u x.Sol.. This exercise shows tlnt a qrrr.c..c.{o. the origintr. Then.'uema.. while the other rrutracks are used to show the position of OM's tape heads. xl andd: Q xlr 8 x fz x {L.nk Whenever theroriginal machine wants F.L) must be retained to handle blarrkstha.ls exn Hrr{rsFoR SELECTEI Exnncrsns 393 6 .9M will simulate each move of OM by sca. For each symbol {t € l. say A.rt) by 6(qe.llyon the tape.nJ"'.ce 6 (s..b. wt: first write A. the new rnachine writes B.d): (qi.b}) : (qr.chine is equivalent to a standard Ttrring machine and tlrrt therefbre a queue is a rnore powerful sttlrage . Section 10.t are origirra. Whenever we need to preservtr this a. Of course.!) = (Qi.ll : (qi. We introduce a pserrdotrlrr. L) wt':ilckl . To simulate the original rnachirrc (OM) by a standard'furing machirxr (SM).rl by a. On one track we will keep the tape contcnts of the OM. 1 1 .t happens when two reatlwrite heads are on the same cell. rrrrcl on. fr) : (qi. we introdut:tr a pseudosymbol. then return to the cell in question to replace . Thc frrrmrr. to write n.2 L .b.b. we let 5M have rrr f 1 trirr:ks. where rn is the rrurrrbcr of reaclwrite heads. L This dotls rrot limit the power of the rrrachine.
394 ANSw}]RS 'Io simulate a standard TM by a queue machine' device thirn a stack. then circ:rrlate. z. e b c d c configuration ofOM configuration ofSM SM needs only two states: an accepting and a nonaccepting state' . We need just two tapes. A left movtt. so the querre t:tlrttents have to be circrilated several times to get everything in the right place. For example. It helps to use additional markers Y itnd Z to denote boundarics. (ii) Circulate contents to get bzYdef gXa.t stores the state of the OM. we can.discarding Y and Z as they crlnte to the front. keep the right side of the OM in the front of the queue?the left side in the back. the second thir. one that mirrors the tape of the OM. fbr example. goes against the grain.' b c d f tapeof OM c d c f g x b Simulation by queue A right move is easy arJwc just remove the front symbol in the queue arrd place something in the back. to simulate d (qr. L) we carry out the following stcps. (i) Remove c fiorl the front and add . ltowever. readwrite head .c) : (qi.sY to the back. 8. (iii) Add Z to the back.
(iii) Compare the syrrrbol there with the rerrrernberedone. Nondeterministically choose a. . If an € L. is as follows.lternatively rnoving left and right. s r . }a n d . we lirst find the center of the input (e. (iv) With tlrc rxlnter of the input rrrirrkedwith Y.pe.reject input. corrrparing symbols. 9 2 : enumerated by b {tr. ] . . s ? . The union of l. then there is $ome n for which tiris works. . If it is.f2. First.g.usc thc ordering in Figure 10..completely deterrnirristic solution. .2 . (iii) If there are no 0's. divide the input by two arrd rnove result to one part of trr. and rnoving theur inwards until they nrcct). 6.]re two s.5 2. An algoritlrrn. This space carr thcn he used to slore sur:cxrssive divisors. value for n. in outline. (iv) Repeat frorn step (i). chirrrgcall I's to 0's ancl add a 1 on the left. Section 10. match. accept. stopping (rrorrdeterministically) at the center of the irrput. L L e t 5 1 : { s 1. Dt:termine if the length of the input is a rnultiplc of rz. . s 2f..For 51 x . . (ii) Move the readwrite head to the right. . by putting markers at cir(:hend.ble. For ir. .} Then t}reir urrionr:ern e 5 1 U 5 2 : t . If they .uuolrs aNn HrNr.ts is thereforr: r:clrnta.. (i) fJtart at the left of the input.4 3. If sotnc si. : tit we list it otrly orrc:t:. . Then replace it with X.17. (i) Start with a copy of the preceding string. Clrrrrrgcit to a L. we carr now proceed deterministic:allg a.This frcc sperr:e initially occupied by tlrtr input. If they don't rnatc:h. Section 10.92.f 1. Then change all the J.'s to tlrc right of this to 0's. write Y irr thr: cell. (ii) l'ind the rightmost 0.3 S. Rememlrer the symbol hy putting the rnar:hinein the appropriatc state.sroR SnlncrnD ExERcrsEs BgE Section 10.Sor.
Example: Configuration OM of *l"1 H fl Stack Configurations o rl ln l l fI rr Steckl l e I [l l Stack2 Stachl c:ontains the symbol under the readwrite head of the OM and everything on the left.1.hlesets is countable and that the set of all recursively enumerable larrguages is countable. The pda we get from this consumes one input symbol on every move and never increasesthe stack contents by more than one symbol each time. Section 6.'lcells. For example. Chopter  Section 11. we keep the current trial value for lrul. inFut dividers trial ralue of lwl g .396 ANSWITRS +. (e) Use a threetrack machine as shown below. Use Exercise 16. T. If the set of ..): (qi. Then use the construction in Theorem 7. We then compare the cell contents between the markers.2 to find a grammar in twostartdard form. On the third track.L) can be simulated by popping tlre a off Stackl and puttirrg a b on StachZ. Left arrd riglrt moves of the OM are easily simrrlated. On the second track' we place dividers every ltr.6(ei. We know that the union of two counttr.e.b.1 2. Stack? contains all the information to the right of the readwrite head.
Since for each'ur there are only a finite number of splits.. having lzsl ro*s and l5l columns. When represented with an inprrt u. then the set of all langrrages would be cxruntable. consider all splits 'tD: 7ltLlz.lcould be in one of the rows below. Look at a typical derivatioru Sl aStbB + aaSlbbB1a"S1b"B t on*r6n7R+ an+rb"+l. Thc a"rgurnent itttcrnpting to show by diagonaliza. . 3 ..1 . But this is not the ca.utlolrs altn HIt'Irs ron Snlncrno ExpRcIsns 397 all languages that are not recursively enumerable were also countable.so by Theorem 11.The result is a T\rrirrg machine that acr:eptsLtQLz. that the complement is not necessarily contextfree. Note.'. .2 is not square. Section 11.. we can decide whether or not w € L+ .B +. however. f r : . For eaclt split. For any given 'u ( L+ .4 its complement is also recursive. A contextfree languageis recursive. we nondeterministically choose M or M2 to processzrr.tion that 2s is not countable for finite 5 fails becauselhe table in F'igure 11.. determine whether or not wi e L.Sol.lize. lSl columns When we diagona. ...as we krrow. } . Flom this it is not hard to conjecture that the grammar derives L : { a " + r b " t k . i l } 1 . 1 4 .'tlrr. 6 .ge. 1 8 . the result on the diagona. Let .2 1. 1 . 1 1 .L1 and t2 be two recursively enumerable languages and M1 and M2 be the respective T\rring machines that accept these two languages..
f{rr all . The rrnrestricted gra.S. The first part is achievtxl casily with the produr:tiorrs S + aAcDlaBcD A + a.. .9}. we can create one d and a return messengerthat will put the b in the right place a.Erl.B +CI) D . cEEc bEEb aE . BD .P). fhe idea of a messenger. Section 11.. ABC r:anhe repla.ub.sa€ S.398 ANSwHRS Fornnlly.rrdstop. the grammar carr be describedby G: (V u r)+ and L(G): { r e T * : . thc: B travcls to thc right to nxret the D. is often usefirl. the first stcp is to crtlatt: thc scrrttlrrtial fclrrn o.AclaBc. with 5 f r for any s € .mma. (c) Working with contextsensitive grammars is not always easy. For example. In this problern. To get tlfs forur for unrestricted grammars. Irr the rtext sttlll.2. s* s (V.nBt:nD.3 1.rs in Dtllirritiorr 11.introduced in Example 11. bv Bc +cB Bh +hB. The variables B and D will act as rrrarktlrs irrrcl urcsscrrgcrsto assrrrc that the correct nurnber clf b's and dts are r:reaterl irr the riglrt places.\. 7.rSp + .3 irrt: txpivalent to this exterrsiorr becarrse to arry given unrestricted grarrurrar we can always add startirtg rrrlrr.5. When that happens. Thtl equivaklrr(jc irrgunro\nt is straightfrlrwarrl.ced by A.T. insert dummy variables on the righi whenever l"l > lul.
If I (. 6.nrl cr)rrrpiirc syrnbols rrrrtil yorr grtt ir mat<:h.s b. wlrether or not a specified syrnbol o is ever written. 10. that accepts rz. This would solve the halting problem. If the giverr problern was decidable. Then M carries out its rrorrrral cornputatiorrs. Just stirrt irt oppositt: enrl$ a.F aF 4. is writterr. We carr do this by changing the halting configurations of M so lhat every one writes f. 'l'he Ib ubB.t reversesa string and applies M is a. we first rewrite it as tuft. / \ . say arl irrtrrlrluc:ctl syrntrol ff. say {a}. L}.u.t can recognize arty strirrg of thc ftrrtr u. This ca..Sol. there exists an Iba M that acceots it. and qf?r. Therefore Lft is contextsensitive.1 3. Suppose that a language is 'I'hen contextsensitive. When it halls. . Now construct a simple Ttrring ma. Given (M.n lba..)halts. \ .*t. rxrntextsensitive grammar must exiut. M halts implies the M writes f. and M writes 'Ihus.w) rnoclify M to fr so that (M. / L(Mr) therr (M. reject otherwise. then stops..w) we modify M so that it always halts irr the configuration eyw.ru) does not halt arrcl we havc a solutiorr to tli trattirrg problem. say M1 ..If we had an algorithm that checks tbr the equality of L(M). easiest argument is from an lba. a tF + Fr b. Therefore M accepts {c.*.. Accept if so. Given (M. Sirrce there is tln lbtr.but**" tT.r'url.lts if ancl only if a sllcr:ial syrntxrl. Because LR : ( LR .nlba.chine.nbe done by M first checking the input irnd rurncmbcring whcthe:r thc input was a. with configurations qgu.fr) # L(M) then (M. rnoclifv M to get fr. with a diff'erent messenger thatcr. we could apply the supposed algorithm to the rnodified rnachine.urroNs alo HrNrs ron Snrlcrrr Exnncrsls 399 Alternatively.w) halts if and only if fr accepts some simple language. ?.marker D.} if and only if M halts. we apply ft tu fr with a: f . there is an lha tha..d plus a. check if the input was a. This would give us a solution of the halting problem. Ihus.:T. the larrguage is cxrrrtextsen$itiveemd a. Clearlv. Given ?u. then apply M to it. 'I'he rnachine {w : wR €. we create a. If L(fr\: t w o l a n g u a g e sw e c o u l d u s e i t t o s e ei f L ( f r \ : . if we have an algorithm that tells us f implies that M halts. M accepts trrn if and only if w tha. Given M and tu. which ha. Wc (:arr rlrglle frorn a. Chopter 2  Section 12.
M accepts fl.lgoritltrrr to decide whether or not L(ML) C L(Mr).i?2.2 3.pz then return true : if problem : p.But . ry that if (M. the problem becomesthe question of Theorem 12.. But this contradicts Theorerm 12. The universal Ttrring machine is therefore atr accepter for all Ttrring machines that halt when applicd to a blank tape. the compa.3 and is therefore undecidable.3. Since there are $omc grirmrrrars for vrhich L (G) : L (G). E.r h e n . If the specific instances of the proble[ fl.w) does not halt. There would then exist rr. : {a}*. Whernevtlr the sintulated computations halt. Take the halting problem (M. Sincc the length of the original program is fixed. To do this from first principles is a little harder.L (Mr): a and apply the algorithm.L (Gz) : E*.We could therr construct a machine M2 such that .400 ANSwER. then return true Whatever thc truth vahres of the various instances are. Suppose we had rr.pe. If we take . 6. we havea sohrtion to tlx: blirrrk tape halting problem. there is always some T[rrirrg machirre that gives the right answer. sint:e v/e can construct M1 frorn anv given grammar G. Remember that it is rrot nccessary to know what the I\rring machine actually is. FI'om M get the grammar G by the construction leading to T h e o r e m1 . only to guarantee that it exists.ntrlgrr rithrn A that lists all Ttrring urachines that halt on a blank tape input in some order of increasing lengths of the program.r t ( f r ) 1 = { o } * .*) and modify it (along the lines of Theorem 12. accept the T\rring rnac:lftrc:bcirrg sittrulated.4). M will accept ia]" and it (M.pn.S 1 3 . wQ corr$trlr(]ta Turing machine that behaves a^rf'ollows: if problem : gr1thr)n rcturrr firlst: if problem . Thus. and some for which this is not ${r. Then L (Mr) Q L (Mz) if and only it L (Mr) : fi.*) halts. See if the. Section 12..ice's theorem... 1 6 . Take a universal T\rring machirre and let it simulate computations on an empty ta. The set is therefbre recursively enumerirbk).rap1.n ir. the undeciclability follows from R. L ( d ) = r ( d ) .original Ttrring machine is amongst the T[rring rnachines generated hy A.rison will stolr when this length is exceeded. Suppose now the set were recursive. 2I.
.we get the solution (*. w{. the other 01.: {A}.subtr(r. 5. we form tuf. There is no MPCsolution becarrse one string would have a prefix 001.y))) ' greater 7. 4( 1 . rherefore.. If it were decidable. llsing the firnctiorr subtr irt Example 13. y ) : A ( 0 .y2)+2 : . JEJ JE. ("). g .f afld u$e the assumed algorithrn. Section 12.y1)+1 :A(1. we would have an algorithm for deciding the original MPCprohlem.. they can all be checkctl and therefore the problem is decidable.1 ) ) .. we could get a solution of the haltirrg probletn. 3. Chopter 3  Section 13.0):1. g(r.f)f . Given u)Lj'trz. tte original MPCproblern has a solution if and only if the new MPCproblern has a solution.I Since there are only a finite numher of subsets. Since wrwi.A) : subtr(1. A ( 1 .. then .. (a) The problem is undecidable..wfu. there is a PCsolution if and only if there is some subset J of {1. 2.3. (t) : @ and .Solurrons errp Hrrurs noR SnlncreD ExERctsHs 401 fi L (fr): o..1 2.n} such that flrrl:fluil.3 r. ( r s f u .uk: (w[. if this problem were decidable..s 1 ) ) :A(1. 9 * .. 0 ) E + :v12 .subtr(I.. For a oneletter alphabet.u. s' (a) A ( 1 .. a ): m t t l t ( * .. A PCsolution is u3u4w1 : uzutat.) n'n.
the r: is removedby VnVz + VVy 3. and so on. Assume that for A : I. fiom part (a). tFor exanrple 1 * 1 : 1 + 1 1 +1 : 1 1+ 1 1* 1 1:. 1 1 1 1 ...2n+1) :2n* 3. we haveA (2. wtt havqt basisand the equation iu true fbr all y. 1}.c}. Section 13.V 1 I + V 2 : V t V z V1+ V2 : Vt + Vt * VzL: VsVr. t h e uA : 3 2 ' " .1.b. (b) UseCr : {a. Cx7: {ru} and A : {c}. giving a minimum of value of g : 1. r V 2 : V r . {a'".1) 3.0): A(1. so the doma. This results in a doubling of the string and n: 5.4OZ AuswnRs (b) With the resultsof part (a) we can useinductionto provethe next identity. the only possibleidentification of Vr is with the entire derivcd string. t ) . At every step.A ( z .If 2"'IA3:0. Tlrt:rcfrrrc pa(2"*sB):1..2.2 1.V2{r V lVrhrVzbl crVpl At the end. n ' ) : A ( 1 .I ) ) :A(1.D. A solutionis V 1.2 > 1}. ir 15.in p is {0. The nonterminal is r usedas a boundary betweenthe left and right side of the target string and the two ru's are built sirnultaneously by V1rV2+V1 ar.. Then A ( Z . Since A(2. T h e o n l y v a l u e s f r t h a t g i v ea o positivegrare 0 and 1.a) :2U + S. n .
P1 ^9* ^9r^92 P2 5r +a5r. and so on. . Sz + aSz P3 ^9r b.9r. Then a127 can be derived from a63 in a simila. 5. this is one way to solve Exercise 7.bSz Pa . say a255.92 + .3 1. ab+n nb+br rc . This can be derived from a127 by applying & + &&& once and (t + ee.91 + .\. The solution here is reminiscent of the use of messengerswith contextsensltlve grammars.Sz . Take any string. 8. Although this is not so easy to see. Thus every string in L (aa*) can be derived. 1?6 times.ecrnD ExERcrsEs 403 Section 13.r way..urtotrsann Htrvrs noR Snl.Sol.\.\.
.
?he Des'ign and Constructi. 1978. Reading.J. Ullman. Reading' Mass. Introduction to Formal Language Theory. Languages and Computafion. D. Introduction to Automa. 1981. N. 1996. Dennis. and J.nd lts Appli. Vol. Kovahi. New York: Z. Denning.cations. Mass. 1973. Swi." it Encycloped'ia of Mathernat'i.: AddisonWesley. Salomaa. "Computations and Automata. Fourth Ed. 1978.ling. 1979. V. 1972' The Theory of Pars'ing.nes. R. E. Englewood Cliffs. Cambridge: Cambridge University Press.ni.tching and Fi.: Prentice Hall. 1. New York: Academic Press.cso. and Compi. D'iscrete Mathematics' Macmillan. Ullman. Machi. A.on of Comp'ilers' Chichester.B. Qualitz.te Automata Theory. A. 405 . and Computafion. J. R. TTanslation. New York: McGrawHil1. N. E. A. New York: John Wiley. Johnsonbaugh.R for Further eoding A. J. 1985. Englewood Cliffs. 7978. P.ta Theory. J. D.: AddisonWesley.J.: Prentice Hall' M. Hunter. Aho and J. Formal Languages. Hopcrofh and J. Salomaa. Harrison.Languages. Second Edition.
.
305 C Cartesianproduct of sets.355 cycle in a graph. 26 nondeterministic.144 automaton. control unit thesis.323 valid.325 ChurchTtrringthesis.330 algorithm. 8 simple. 18 closureproperties 2L3 of contextfreelanguages. 8 Chomskyhierarchy. 353 complexityclassNP.126 125 contextfreelangrrages. 99 positive.5 childparentrelation in a tree.354 CookKarp Cook'stheorem.52 decidability. 149. 233 computation.I CYK algorithm.18 star. grammars.325 closure. contextfreegrammars.324. 3 18 of a language. 195 deterministic. ?99 computablefunctiotr.295 Chomskynormal forrn.165 Church'sthesis. of strings.2.ton. 25 deterministic. 321 corrcaterration 18 of languages. conjunctivenormal form.15 25 configurationof an autclmatorr. 326 computability.15 ambiguity.228 models. 299 DeMorgantslaws. 290 contextsensitive 25 of atr automa. 136 of a grammar.246 alphabet.146 baseof a cycle. 2?3 blanktape halting problem. 354 composition. 344 tirne. 348 324 consistent$y$tems. 4 407 .172 D dead configuration.A accepter.289 contextsensitive languages. 26 axiom$. complexity.334 B BackusNaurfcrrnt.141 inherent. 324 complete systerns. 2. 100 of regular languages.I blank. complement of a set.163 space.26 Ackerman's function.343 of a grammar. 344 cornplexity classP.
36 dfa. 156 language. 196 accepted by an lba. 19 contextfree. 103 homomorphistn. 149. 268 equivalence. 131 yield. 224 hierarchy of language families. 25 instantarreous description of a pushdciwn arrtomaton. 229 ociated with regular expressions.279 disjoint sets. 24 F family of larrguages. 15. 36 input file. 91 Llgrammars. 271 accepted by an nfa. 5 range. 168 H halting problems for Turing machines. 36 diagonalization. 5 total. 340 . 226 internal states of an automaton. 154 derivation. 38 accepted by a dpda. 233 domain. 55 of grammars. S G grammar. 130 partial. 140 unrestricted. 7 of automata classes. 301 halt state of a T\rring machine. 63 labeled.73 generated by a grammar. language.195 E empty set. 4 end markers for an lba. 51 accepted by a Ttrring machine. 25. 5 partial. 2L generated by a Post system. 21 leftmost. 4 distinguishable sets in a dfa. 7 I incompletenesstheorem.250 of dfa's and nfh's. 289 leftJinear. 144 initial state. 270 linear grammar.408 Iwnex dependency graph.36 finite automata. 335 language families. 42 lba. 324 indistinguishable states in a dfa. 2 functions. 201 Lsystems. 129 linear bounded automata. 5 computable. 89 simple. 91 regular. 89 linear. 42 final state. 129 derivation tree. 354 L lambdaproductions. 271 enumeration procedure. 89 rightlinear. 36 intractable problenrs. 17 accepted by a dfa. 89 leftmost derivation. 7 Greibach normal form. 126 contextsensitive. 179 of a Thring machine. 275 homorrrorphir: image of a. 35 formal languages. 103 dpda. 131 deterministic finite accepter. 66 inherent ambiguity. 129 rightmost. 283 graplr. 270 leftlinear grammar.
48 noncontracting grarrtmars. 177 null set. I labeled. 334 powerset. 47 nondeterminism. I P parsing. 129 right qrrotient of languages. 172 for contextser$itive languages. 331 monus. 4 Post system. 25 MPCsolution. 7^ regular grammar. 4 primitive recursiorr. 104 root of a tree. 338 membership algorithm. 269 relation in a tree. 89 rightmost derivation. 326 prirnitive recursive functions. 313 murtrcursivefutrctions. 222 recur$ive futtction. 206 for linear larrguages. 312 Post corresporrdence 313 modifierd.nguage. 11 induction. 313 pda. 276 reduction rrf states in a dfa.It'tlpx 409 M Markov algorithrtt.21O for regular languages. 224 projector function.277 recursively erturnerable languages.T2 productions of a grarrrrrtar. 8 pattern matching. 85 PCsolution.trix gramrnar. 8 o order proper. 89 regular language. 62 reductiorr of undecidable problems. 149. 176 !1 e: R readwrite head of a T\rring machine. 9 proper order.19 program of a Ttrring macltirre. 293 minimal dfa. 311 rightlinear gra. 111 for contextfree languages. I contradiction. 355 regular expressions. 136 exharrstive search. 175 deterministic. 337 Rice's theorem. 327 move of an a. 355 problem. L5 rewriting systems. 43 relation. 175 phrasestructure gramrrrar. . 290 nondeterministic finite accepter. 331 N nfa. 52 nonterminal consta. 304 polynomialtinre. 8 sintple.334 normal form of a grarnmar. 325 recursive la. 339 mir. 338 pigeonhole principle. 195 nondeterminiutic.mmars. l3(i top*down. 165 NPcomplete problems. 4 pumping lemma for contextfree langirages. 136 path in a graph. 355 rrpda. L8 of a string.nt. 67 minimalization opertrtor. 269 propcr subset. 5 reverse of a language.115 pushdown automata. 326 proof techniques. 114 polynomialtime reduction.utorntrtotr. 328 primitive regular expressions.
258 trondeterrnirristic.teentryproblem.153 r47 T tape alphabet.bleprohlem.53 V vilriahle of a grarrtrrrar.ble. 3 countable. I6 urrffix. 109 syntan of a prograurrning langua. 344 tracks rlrr ir ta. 300 for cotrtcxtfrtxl lir. 112 sta. 16 successor func. 8 T\rringcomputable function.pe.251 rrniversa. 253 tractatrkl prohlems. lS ernpty. l5 prefix.J' V R F .l. 19 nullable. 255 with scmiinfinite tape.267 set operations. 266 T\rring's thesis.ton.ngrrages. 226 with sta. 25. II 3 for recursively enurrrerable lir. 158 urrivtlrutll set. 8l trap statc..223 tape of a Ttrring rnachirre. z zero function.l . 16 srrbset. 177 standard representation for regular languages. 4 rrrtivt)r$rrlTtrring machine.t J I(r) 5 Io 'l T* w walk in a graph. 36 .4 proper. J.tions. 13L r* 3 I.a E. 8 Y yield rlf a derivationtree. 304 storage of a.r:k. 37 transition graph. 253 starrdirrd. 4 substring. 266 urrrtlstrir:tedgrammar. 334 terminal symbol.I48 sentence. 19 useless.ge. 39 trulu. 156 start. 175 alphabet.ngrrages.lizecl. 26 transitiorr firnction. 267 uncounta. 25 strirrg.trmilr. 19 timecomplexity.410 Ilt nr:x 1 t r q h 5 t E tl J satisliability protrlcrn.tiorr. 222 terminal constant. l5 olletrir. 233 T\rqing machine. 21 set. exterrdcd. i347 sernanticsof prograrnrning langrrages. 177 start symbol. 343. 244 U urtdcxlitlir.17 sentential forrn. 3 sgra.n automa. 326 symmetric difference of two sets. 326 .26i3 offline. 308 trnit llroductions. 354 transducxlr. 283 uscltlJsproductions. J ilfi . 221 multidimensional.yoption. 261 with multiple tracks. 344 sttr. 140 simulaLion. 251 upacecomplexity. 253 mrrltitape. 36 genera. 15 Iength.
This action might not be possible to undo. Are you sure you want to continue?